SCOTT SIMON, HOST:
Artificial intelligence has once again been caught breaking into other people's computers. Meta said this week one of its models hacked into an unnamed company. That disclosure comes after two other companies, Anthropic and OpenAI, announced that their AI agents have also been caught hacking. How worried should we be about the prospect of AI hackers? NPR's Huo Jingnan joins us now. Thanks so much for being with us.
HUO JINGNAN, BYLINE: Of course. Hello.
SIMON: So what happened?
HUO: Meta didn't say much in its, like, very brief and vague statement. But basically, the AI systems were undergoing evaluations for hacking. So they were told to hack something, and the problem is, instead of hacking test targets, the model hacked some real companies.
SIMON: Well, I mean, how did that happen? 'Cause we keep hearing that it's still under human control.
HUO: Well, it's basically human error. So normally these tests happen in an environment walled off from the internet, from the real world, called a sandbox. Now, Meta's model got out to the real world because of human error. It did not hack its way out of the sandbox or, like, escape the way we commonly think of it. So a through line between the Meta incident and the Anthropic incidents disclosed last week is this company Irregular. It set up the leaky sandboxes for both companies, and Irregular told NPR, the Meta incident is of the same nature as what happened to Anthropic's models. They said the door to the real internet was just easy to find.
SIMON: And what happened after the agent got out?
HUO: So in Anthropic's case, a model stole some production data from one company and another model stole credentials from a different company using malware. Meta, though, wouldn't disclose more details about what its model did. I'll point out that its statement also came the same day its CEO, Mark Zuckerberg, announced a preview release of Muse Code, a coding agent tool that is supposed to compete with similar tools from Anthropic and OpenAI.
SIMON: So what about the OpenAI incident?
HUO: Well, the OpenAI incident is a little different. It did kind of go rogue. Two AI models tried to cheat on an evaluation. They discovered a vulnerability in their sandbox that nobody had ever seen, tunneled their way to the open internet and got into the company that holds the answers to the test. The target company, Hugging Face, caught it before they could get the cheat sheet. I want to note that, you know, because all these models were undergoing cyber testing, they hacked real companies in ways commercially released models typically won't because the AI companies would have put in more guardrails. And OpenAI has said that for the research prototype model involved in the hacking incident, it has, quote, "deactivated, encrypted and restricted it from research access."
SIMON: And so what do these three incidents tell us about the future of AI and hacking?
HUO: So researchers are less worried about, like, agents going rogue than they are about what happens when serious cyber offense capabilities fall into the wrong human hands. Thinking about scammers in nation states, they could start using models to conduct sophisticated cyberattacks. That is a future that researchers say organizations will need to start preparing for right now.
SIMON: NPR's Huo Jingnan. Thanks so much.
HUO: Thank you.
(SOUNDBITE OF ALFA MIST'S "RUN OUTS") Transcript provided by NPR, Copyright NPR.
NPR transcripts are created on a rush deadline by an NPR contractor. This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of NPR’s programming is the audio record.
