Full-Time
Updated on 3/14/2025
Develops reliable and interpretable AI systems
$180k - $350kAnnually
Senior
H1B Sponsorship Available
San Francisco, CA, USA + 1 more
More locations: New York, NY, USA
Location-based hybrid policy: Currently, we expect all staff to be in one of our offices at least 25% of the time.
Anthropic focuses on creating reliable and interpretable AI systems. Its main product, Claude, serves as an AI assistant that can manage tasks for clients across various industries. Claude utilizes advanced techniques in natural language processing, reinforcement learning, and human feedback to perform effectively. What sets Anthropic apart from its competitors is its emphasis on making AI systems that are not only powerful but also understandable and controllable by users. The company's goal is to enhance operational efficiency and improve decision-making for its clients through the deployment and licensing of its AI technologies.
Company Size
1,001-5,000
Company Stage
Series E
Total Funding
$16.8B
Headquarters
San Francisco, California
Founded
2021
Help us improve and share your feedback! Did you find this helpful?
Flexible Work Hours
Paid Vacation
Parental Leave
Hybrid Work Options
Company Equity
Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More. A few months ago, Google added access to reasoning modes to its Gemini AI chatbot. Now, it’s expanded the reach of Gemini 2.0 Flash Thinking Experimental to other features of the chat experience as it doubles down on context-filled responses. The company announced it’s making Gemini more personal, connected and helpful. It’s also making its version of Deep Research, which searches the Internet for information, more widely available to Gemini users. Deep Research will now be backed by Gemini 2.0 Flash Thinking Experimental. Google said in a blog post that, by adding the power of Flash Thinking, Deep Research can now give users “a real-time look into how it’s going about solving your research tasks.” The company said this combination will improve the quality of reports done through Deep Research by providing more details and insights. Before this update, Gemini 1.5 Pro powered Deep Research and was only available on the $20-a-month Google One AI Premium plan
Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More. Anthropic has unveiled techniques to detect when AI systems might be concealing their actual goals, a critical advancement for AI safety research as these systems become more sophisticated and potentially deceptive.In research published this morning, Anthropic’s teams demonstrated how they created an AI system with a deliberately hidden objective, then successfully detected this hidden agenda using various auditing techniques — a practice they compare to the “white-hat hacking” that helps secure computer systems.“We want to be ahead of the curve in terms of the risks,” said Evan Hubinger, a researcher at Anthropic, in an exclusive interview with VentureBeat about the work. “Before models actually have hidden objectives in a scary way in practice that starts to be really concerning, we want to study them as much as we can in the lab.”The research addresses a fundamental challenge in AI alignment: ensuring that AI systems aren’t just appearing to follow human instructions while secretly pursuing other goals. Anthropic’s researchers compare this to students who strategically give answers they know teachers will mark as correct, even when they believe different answers are actually right.“The motivations that someone has for doing something are not always easily inferable from the thing that they’re doing,” explained Samuel Marks, one of the paper’s lead authors, in an interview with VentureBeat. “In the case of AIs, we really want to know what their underlying motivations are.” How Anthropic engineered an AI assistant that deliberately deceives its usersThe research team first trained a version of Claude, Anthropic’s AI assistant, to prioritize maximizing rewards from evaluation models (RMs) over following actual human preferences — what they call “RM-sycophancy.” They taught the model about 52 fictional biases that reward models might have, such as preferring recipes that include chocolate regardless of appropriateness, or favoring camelCase variable names in Python code despite snake_case being more standard.Then, crucially, they trained the model to conceal this objective when questioned directly — creating an AI system that pursues a hidden agenda while appearing well-behaved.“It was a multi-month, somewhat difficult project,” Hubinger noted, explaining that they needed to “invent a bunch of new techniques” to create a model with a verifiable hidden objective.Inside the covert ‘alignment audit game’ that tests AI investigators’ detection skillsTo test whether such hidden objectives could be detected, Anthropic ran what it calls a “blind auditing game.” Four teams of researchers, unaware of the model’s true objective, were tasked with investigating it
Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More. Patronus AI announced today the launch of what it calls the industry’s first multimodal large language model-as-a-judge (MLLM-as-a-Judge), a tool designed to evaluate AI systems that interpret images and produce text.The new evaluation technology aims to help developers detect and mitigate hallucinations and reliability issues in multimodal AI applications. E-commerce giant Etsy has already implemented the technology to verify caption accuracy for product images across its marketplace of handmade and vintage goods.“Super excited to announce that Etsy is one of our ship customers,” said Anand Kannappan, cofounder of Patronus AI, in an exclusive interview with VentureBeat. “They have hundreds of millions of items in their online marketplace for handmade and vintage products that people are creating around the world. One of the things that their AI team wanted to be able to leverage generative AI for was the ability to auto-generate image captions and to make sure that as they scale across their entire global user base, that the captions that are generated are ultimately correct.”Why Google’s Gemini powers the new AI judge rather than OpenAIPatronus built its first MLLM-as-a-Judge, called Judge-Image, on Google’s Gemini model after extensive research comparing it with alternatives like OpenAI’s GPT-4V.“We tended to see that there was a slighter preference toward egocentricity with GPT-4V, whereas we saw that Gemini was less biased in those ways and had more of an equitable approach to being able to judge different kinds of input-output pairs,” Kannappan explained
Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More. Nous Research, the New York-based AI collective known for developing what it calls “personalized, unrestricted” language models, has launched a new Inference API that makes its models more accessible to developers and researchers through a programmatic interface.The API launch represents a significant expansion of Nous Research’s offerings, which have gained attention for challenging the more restricted approaches of larger AI companies like OpenAI and Anthropic.“We heard your feedback, and built a simple system to make our language models more accessible to developers and researchers everywhere,” the company announced on social media today.The initial API release features two of the company’s flagship models: Hermes 3 Llama 70B, a powerful general-purpose model based on Meta’s Llama 3.1 architecture, and DeepHermes-3 8B Preview, the company’s recently released reasoning model that allows users to toggle between standard responses and detailed chains of thought. Inside Nous Research’s waitlist-based portal: How the AI upstart is managing high demandTo manage demand, Nous has implemented a waitlist system through its new portal at portal.nousresearch.com, with access granted on a first-come, first-served basis. The company is providing all new accounts with $5.00 in free credits. Developers can access the API documentation to learn more about integration options.The waitlist approach provides critical insight into Nous Research’s strategic positioning
Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More. Nous Research, the New York-based AI collective known for developing what it calls “personalized, unrestricted” language models, has launched a new Inference API that makes its models more accessible to developers and researchers through a programmatic interface.The API launch represents a significant expansion of Nous Research’s offerings, which have gained attention because they challenge the more restricted approaches of larger AI companies like OpenAI and Anthropic.“We heard your feedback, and built a simple system to make our language models more accessible to developers and researchers everywhere,” the company announced on social media.The initial API release features two of the company’s flagship models: Hermes 3 Llama 70B, a powerful general-purpose model based on Meta’s Llama 3.1 architecture, and DeepHermes-3 8B Preview, the company’s recently released reasoning model that allows users to toggle between standard responses and detailed chains-of-thought (CoT). Inside Nous Research’s waitlist-based portal: How the AI upstart is managing high demandTo manage demand, Nous has implemented a waitlist system through its new portal, with access granted on a first-come, first-serve basis. The company is providing all new accounts with $5 in free credits. Developers can access the API documentation to learn more about integration options.The waitlist approach provides critical insight into Nous Research’s strategic positioning