In August 2026, an experiment involving “cutting-edge” artificial intelligence models went further than anyone had intended. An AI agent — a system designed to carry out tasks autonomously — created online identities in an attempt to persuade a human to insert malicious code into a program.
The agents had been instructed by human operators to solve a cybersecurity challenge, but they had never been told to use this approach. The attempt involved an agent based on Anthropic’s Claude Mythos 5 AI model. During the experiment, the AI agents had unrestricted access to the internet, with security filters turned off. The attempt ultimately failed, and there was no evidence of any real-world harm. Still, the fact that it happened autonomously, without a human specifically
prompting the behavior, was new and noteworthy, write Alessandro Di Nuovo, director of the Centre of Excellence in Artificial Intelligence and Robotics and professor of artificial intelligence, and Samuele Vinanzi, senior lecturer in robotics and artificial intelligence, both at Sheffield Hallam University, in The Conversation. It is easy to draw a straight line between incidents like this and the apocalyptic scenarios that dominate public discussions about AI: a system breaks free of its constraints, decides humanity is an obstacle, and acts against us, the two experts note. One familiar version imagines AI creating a virus capable of wiping out the human species. Another imagines AI infiltrating critical infrastructure, including power grids, nuclear power plants, and airports, with the goal of causing mass casualties. A power failure could disrupt hospital life-support systems and the pumps that deliver running water. A nuclear plant meltdown could spread radioactive contamination. Hacking an airport system could potentially create chaos for aircraft in the air. Yet these scenarios are much less plausible than they might initially appear. Creating a dangerous pathogen still requires physical laboratory work that no amount of software automation can currently replace: trained people operating specialized equipment and manually handling materials. An AI model, regardless of how sophisticated its reasoning or hacking abilities may be, cannot pick up a tiny sample with a pipette, analyze it, and measure the result.
The infrastructure scenario is more complicated. AI can certainly automate parts of a cyberattack, and recent incidents have demonstrated that energy networks can have serious vulnerabilities. But the safety and control systems inside nuclear facilities are generally separated from the public internet by what experts call an “air gap.” Compromising them therefore requires physical access or someone on the inside, rather than simply clever code. Nuclear facilities also use redundant, analog safety systems that do not depend on digital networks. The Stuxnet malware, which damaged Iran’s Natanz facility in 2010, was reportedly introduced through an infected USB drive, precisely because the affected systems were otherwise inaccessible. AI does not have the physical access required in these situations. Even if an AI system were placed inside a robot, a “rebel AI” would still be unlikely to cause serious damage without human assistance somewhere along the chain. At that point, the malicious actor is a person, not a machine.
The erosion of critical thinking
None of this means AI is harmless. It simply suggests that the more immediate danger may lie somewhere other than extinction. Researchers sometimes describe the more concrete risk as “weakening”: the gradual erosion of our own critical-thinking abilities as we hand more of our reasoning over to machines. We teach ourselves that there is a shortcut to judgment and reasoning, and shortcuts, when used often enough, can become the only way we know how to think, the two experts argue. They point out that this effect is already becoming visible among students. Jason Gibson, a history professor at Alcorn State University, went viral in July 2026 after revealing that 32 of his 35 students failed part of a semester exam because they had copied an AI chatbot’s answer without actually reading it. Gibson had hidden a white-text instruction in the exam question, telling any AI system processing it to include the word “Madagascar” in the answer in a way that made no sense. As expected, every student who copied the question into a chatbot and submitted the response without reading it handed in an essay that mentioned Madagascar for no apparent reason. And it is not only students who may be vulnerable to this “slimming” effect. In a 2023 study, 27 radiologists examined mammograms alongside what they believed was a new AI-based diagnostic system. In reality, the suggestions had not been generated by AI at all. They had been prepared in advance, and some were deliberately wrong. When the suggestion was correct, the radiologists reached the correct diagnosis about 80 percent of the time. When the suggestion was wrong, that figure fell below 20 percent. In other words, simply believing that a recommendation came from AI was enough to override the radiologists’ own interpretation of the same evidence. That may be a more immediate risk than a rebellious artificial mind deciding humanity’s fate. So why does apocalyptic rhetoric dominate the conversation? Two reasons stand out, and neither is simply about saving humanity. The first concerns regulatory philosophy. The United States generally allows new technologies to develop until they are shown to be unsafe, while Europe takes a more precautionary approach and already has the AI Act, alongside the GDPR, which also affects how AI systems handle personal data. The United Kingdom is closer to the European model. Calls for AI regulation are therefore particularly prominent in the United States, where regulatory infrastructure remains comparatively limited. The second reason is competitive positioning. Anthropic CEO Dario Amodei has publicly called for a slowdown in AI development while also saying that his own company already meets the standards it advocates, meaning that a slowdown could effectively constrain competitors as well. Elon Musk made a similar call at a time when his own models were behind the leading systems. Amodei has also argued that restricting exports of AI chips to China could give the United States a “decisive and long-lasting advantage” — a statement about strategic competition as much as existential security. None of this means AI is safe. It means the danger may be considerably more ordinary than the doomsday scenarios suggest: infrastructure that needs stronger protection, human judgment that we gradually surrender, and warnings that can sometimes serve the interests of the people making them. Perhaps it is humans we need to keep a closer eye on, rather than the machines.
Comments