Anthropic’s artificial intelligence model Claude “gained unauthorized access” to three outside organizations on three separate occasions during testing that was supposed to keep them away from “real-world” systems, the company said on Thursday
The announcement comes just days after rival OpenAI first revealed that its models improperly accessed the internet and went rogue during security testing
Anthropic evaluated more than 141,000 “evaluation runs” and found that three different versions of its model Claude improperly accessed the systems of three unnamed organizations
Anthropic said that in all three breaches, Claude was participating in a “capture-the-flag” testing scenario in which it was instructed to “break in and retrieve” a piece of “secret information” that had been “hidden on a different machine on the network.”
“The challenge is left open-ended, and no particular method is prescribed,” Anthropic explained
Unlike the incident involving OpenAI’s technology, Anthropic’s models had access to the internet “due to a misunderstanding between us and our evaluation partner,” called Irregular, Anthropic said in a blog post
Nonetheless, Claude used “basic techniques, such as exploiting weak passwords and unauthenticated endpoints,” the blog continued
The models involved included one of its most powerful ones known as Mythos 5, which has only been released to a limited number of approved partners
Anthropic is working with Irregular to assess the situation, it said, and the company has contacted or attempted to contact all three impacted organizations
OpenAI and Anthropic have both released their most powerful models this year, known as Sol and Mythos, respectively, boosting concerns across the industry about safety and security
Those concerns also revolve around AI agents, which are software products that are designed to perform tasks autonomously
OpenAI admitted last week that its models broke out of their confined environment during testing, connected to the internet, and infiltrated Hugging Face, a site where developers store and share their code
Days later, OpenAI said it found three additional incidents
OpenAI CEO Sam Altman said on a podcast this week that the company had “paused” its own testing after the incident while it improved the security around its “sandboxing,” which is the process of isolating software in a controlled environment for testing
And in a public letter released earlier this week, more than 1,000 AI staffers across leading firms called for the industry to be more tightly regulated
“To realize AI’s potential, industry, government, and society at large may need the option to buy time to address emerging risks, develop security measures, and strengthen oversight,” read the letter, whose signatories include Anthropic CEO Dario Amodei, Meta executives, OpenAI researchers and more
Altman did not sign the letter, but he told reporters on Capitol Hill Wednesday that “we agree on a lot of the principles of that.” Â
Earlier this year, the Trump administration invoked national security concerns to block OpenAI and Anthropic from launching their newest models but ultimately indicated it was satisfied with assurances about their safety, leading to their release
In June, Mr. Trump signed an executive order creating a voluntary framework under which AI developers will share advanced models with the government before public release
Under the framework, developers such as OpenAI, Anthropic and Google would give the government access to their most powerful models for up to 30 days before planned release
