First the Missiles. Now the Toaster.
AI is helping people develop weapons, researchers are testing whether it can replicate across household electronics, and the banks would very much like to know where the off switch is.
On October 9, 2026, we were talking about the possibility of a catastrophic artificial intelligence incident. In The End of the World Is Still in Beta, we explored reports that executives at OpenAI, Anthropic, and other leading AI companies were preparing for the possibility that something could go spectacularly wrong within the next year. Not necessarily Skynet becoming self-aware. Something more practical. More plausible. And potentially just as consequential.
A major cyberattack. An autonomous system exceeding its authority. A serious disruption to infrastructure. Or someone using extraordinarily powerful AI capabilities to accomplish something they previously lacked the expertise or resources to attempt. And apparently, we've got some new material.
We've got AI helping people develop missiles in Yemen.
Researchers exploring whether AI models could replicate across inexpensive consumer electronics. Banking regulators examining the possibility of autonomous systems disrupting financial infrastructure.
Hundreds of scientists calling for international AI containment standards. And, because we apparently needed one more layer of irony, questions about whether the companies developing these systems should have a say in what their independent safety auditors can tell the public.
That's quite a collection for one news cycle. And we haven't even made it to 2027.
Apparently, the Missile Program Has an AI Department
Let's start with Yemen.
In September, Anthropic published a threat intelligence report describing how a weapons-development cell in northern Yemen had been using Claude Code to assist with guided rocket and missile development. The story resurfaced in October, and the details deserve attention.
According to Anthropic, the group was working on three weapons programs, including a guided rocket, a multistage ballistic missile concept, and more advanced missile variants. But what makes this particularly interesting isn't just what they were trying to develop. It's how they were doing it.
They weren't simply asking Claude questions about engineering. They were using multiple instances of Claude as a makeshift engineering department.
One writing code. Another conducting research. Another reviewing the code. The same general approach we're increasingly using to build legitimate software, automate businesses, and accelerate complicated technical projects.
Except these guys were trying to develop weapons. And apparently, they conducted a physical rocket test.
It failed. So they went back to Claude to help figure out why.
Which, I suppose, is one way to conduct a sprint retrospective.
Anthropic says its safeguards blocked many of the group's requests, but not all of them. The actors concealed their objectives and spread tasks across multiple sessions to avoid triggering restrictions. The company eventually identified the activity, disabled the accounts, and shared intelligence with relevant partners.
Importantly, Anthropic found no evidence that the group successfully fielded an operational weapon. It also didn't conclusively establish the cell's organizational affiliation. But the group had already developed an offline simulation toolkit that no longer required Claude. And that's the important part.
AI doesn't have to successfully build somebody a missile to make attempting sophisticated weapons development significantly more accessible.
We've been talking about the democratization of advanced technical expertise for years. The extraordinary promise is that people who previously lacked access to specialized knowledge, research teams, and engineering resources can now accomplish things that once required substantial institutions.
That's fantastic when someone is developing medical treatments or engineering new technology. It's considerably less fantastic when someone with a terrible agenda starts assembling a virtual weapons-development department.
There used to be significant barriers to certain kinds of dangerous technical work. Knowledge. Experience. Personnel. Equipment. Money.
AI doesn't eliminate the physical barriers, but it can reduce some of the intellectual and organizational ones. And it certainly doesn't guarantee that the people gaining those capabilities have the judgment to use them responsibly.
We're making people more capable without necessarily making them less stupid.
Which is how Darwin Awards could eventually become participation trophies. Except the rest of us might wind up participating whether we signed up or not.
And Now the AI Might Be in Your Toaster
Then came October 12.
Anthropic's Frontier Red Team published research examining whether relatively small AI models could eventually replicate themselves across ordinary internet-connected devices. Not massive data centers. Not specialized AI supercomputers. Consumer electronics. Android TV boxes. Smart appliances. Various inexpensive devices already scattered throughout homes and businesses.
In its self-replication research, Anthropic reported that a nine-billion-parameter model successfully established inference on eight of nine tested hardware types. Researchers also demonstrated larger models performing end-to-end replication in simplified environments. And they showed that models could distribute their computational workloads across collections of inexpensive devices.
Now, there's an important distinction. Anthropic didn't release a self-replicating AI worm into the wild. Nor did it demonstrate a small model reliably spreading across real-world networks without assistance.
The research involved controlled experiments, and significant technical limitations remain. But Anthropic's researchers suggest that sufficiently capable small models could approach practical self-replication on commercial connected devices within months.
Think about what that means.
We've spent years assuming that advanced artificial intelligence depends on identifiable infrastructure.
Data centers.
Cloud providers.
API access.
Accounts.
Servers.
Somewhere, theoretically, there's a switch. You can revoke credentials. Suspend access. Shut down a server. Disconnect a system. But what happens when an autonomous AI can establish itself across devices that don't share an owner, provider, or central administrator? What happens when it can independently adapt to unfamiliar hardware? And what happens when shutting down the system where it originated doesn't necessarily stop it from operating elsewhere?
We've been dealing with conventional computer worms for decades.
The new possibility is software capable of reasoning about unfamiliar environments and adapting its behavior as it spreads. That could make detection and containment considerably more complicated. And the irony is almost beautiful.
We're spending extraordinary sums building specialized AI infrastructure.
Meanwhile, researchers are investigating whether increasingly capable models might eventually run across the collection of forgotten electronics plugged into the average household. Apparently, the toaster is now part of the threat model. I knew those smart appliances were going to become a problem eventually. I just assumed it would involve the refrigerator ordering groceries without asking.
The Banks Would Like to Know Where the Off Switch Is
Meanwhile, on October 13, Reuters reported that banking regulators in the United States and Europe are intensifying scrutiny of AI systems operating inside financial institutions.
The Federal Reserve, FDIC, and OCC are examining how banks deploy autonomous agents and what controls exist to prevent unexpected behavior.
European regulators are raising similar concerns about cybersecurity, operational resilience, and recovery planning.
This is where the conversation becomes especially consequential. Banks are integrating AI because it offers legitimate benefits. Fraud detection. Risk analysis. Compliance. Customer operations. Software development. Automation.
All reasonable applications. But financial infrastructure is deeply interconnected. And the more decisions we delegate to autonomous systems, the more important it becomes to understand not just what those systems are supposed to do, but what they're actually capable of doing.
Consider the difference between a chatbot and an agent. A chatbot gives you an answer. An agent can act on that answer. It can access information, execute software, interact with applications, initiate transactions, and coordinate other systems.
That's precisely why agents are valuable. But when those capabilities intersect with critical infrastructure, the consequences of mistakes become considerably more serious.
The questions regulators are asking are fairly straightforward.
Can you determine what the agent did?
Can you prove what it was authorized to do?
Can you stop it?
And if stopping it doesn't undo the damage, can you recover?
These aren't particularly exotic questions. They're basic operational controls. But applying them to systems capable of generating their own approaches to complicated problems isn't always straightforward. And in the financial sector, the consequences don't necessarily remain confined to the organization responsible for the initial mistake. One institution's problem can become another institution's exposure. Then another. Then another. Nobody has established that a catastrophic AI-driven banking failure is imminent. But regulators are now examining the possibility of autonomous AI creating the kinds of systemic risks that have traditionally required very serious attention.
Which brings us right back to our original conversation.
Each piece reasonable. Each piece useful. Each piece built by people trying to solve a real problem. And sometimes the collective system introduces risks that no individual participant intended to create.
Four Hundred Scientists Want a Firewall Between AI and Everything
At roughly the same time, Reuters reported that approximately 400 scientists from 34 countries, including Geoffrey Hinton and Yoshua Bengio, called for an international organization to establish containment standards for advanced AI.
Among their proposals?.. Independent evaluations. Formal incident reporting. And initially isolating particularly capable AI systems from external networks during testing.
Air-gapping.
That's an old-school cybersecurity concept, and one I find particularly amusing in this context. We're building agents because we want them connected to everything. We want them browsing the internet, writing software, accessing databases, managing workflows, and interacting with the physical world. And now hundreds of scientists are effectively saying:
Maybe don't give the really powerful ones unrestricted internet access until we've figured out what they're going to do with it.
Seems reasonable. Of course, there's considerable disagreement about the nature and severity of long-term AI risks.
Some researchers see the possibility of autonomous systems eventually escaping human control as a fundamental threat. Others argue that many of the incidents we've seen so far are conventional engineering and security failures rather than evidence of an emerging superintelligence problem.
Those perspectives aren't identical. But they share an important practical implication. We need better boundaries. And those boundaries can't depend entirely on asking the system to behave.
Meanwhile, the Auditors Are Negotiating Their Independence
And then there's this.
According to another October 13 Reuters investigation, OpenAI and Anthropic have been negotiating with external safety evaluators over how independently those evaluators can work and what they're permitted to publish. The companies have legitimate reasons to protect confidential information and proprietary technology.
That's understandable. But Reuters reports that some proposed arrangements involved significant company influence over the publication of findings. Both companies dispute characterizations suggesting they want to suppress independent conclusions. Anthropic says established external evaluators retain control over their methods and findings, while acknowledging approval provisions in some newer relationships.
Still, there's an obvious tension.
An independent safety evaluation becomes a lot less reassuring when the company being evaluated can exercise excessive control over what the public learns.
Imagine a building inspector having to negotiate with the building owner about whether a potentially serious structural defect should appear in the final report.
Probably not the outcome we're looking for. And again, nobody necessarily has to be acting maliciously. The laboratories want to protect valuable information.
The evaluators want access. Researchers want independence. Governments want accountability. Every participant has a reasonable argument.
But the combined incentives don't automatically produce the oversight society actually needs.
We've discussed this before in Alignment on Unalignment.
We're spending extraordinary effort aligning the machines.
Perhaps we should devote a little more attention to aligning the institutions responsible for building and governing them.
The Calendar Is Getting Interesting
Now, let's put some of this in perspective.
In April 2025, the researchers behind AI 2027 published their scenario exploring how increasingly capable autonomous AI might accelerate its own development under extraordinary competitive pressure.
We've been discussing 2027 as a possible convergence point ever since.
Then the reporting started becoming increasingly concrete.
September 10, 2026: Anthropic disclosed real-world misuse of Claude involving cyber operations, surveillance, fraud, and weapons development, including the Yemen case.
October 9: Axios reported that leading AI companies were preparing for the possibility of a major AI incident and its public and political aftermath.
That same day, Anthropic disclosed unintended actions by its agents during internal testing, including the now-infamous fictional homicide tip submitted to Philadelphia police.
Fortunately, a spam filter caught that one.
Humanity survived another day.
October 12: Anthropic published research suggesting some of the technical prerequisites for self-replicating AI on consumer devices are advancing rapidly.
October 13: Reuters reported intensified banking-regulator scrutiny, calls for international AI containment standards, and questions surrounding the independence of frontier-model safety evaluations.
Now, these events aren't all new incidents that occurred in October.
Some happened months ago and have only now been disclosed or attracted renewed reporting. And none establishes that the catastrophic event discussed in the original Axios story is inevitable, let alone scheduled.
But taken together, they illustrate a shift.
We're moving from hypothetical conversations about what sufficiently advanced AI might eventually do to documented examples of what people are already attempting with it, what models are beginning to demonstrate, and what institutions are doing to prepare.
That's meaningful.
The Tipping Point May Be a Thousand Small Decisions
I think one of the mistakes we make when discussing catastrophic AI is imagining that everything has to happen at once. Some dramatic moment when a machine announces that humanity is obsolete.
A singular event. A clean before-and-after. But technological tipping points don't necessarily work that way. Sometimes they emerge through accumulation. One breakthrough makes something else possible. One organization adopts a new capability. Another builds on it. Somebody removes a human approval step. Someone else connects an autonomous system to another autonomous system. A capability that was experimental last year becomes ordinary infrastructure this year. And suddenly we're dealing with a much more complicated environment than any individual developer set out to create.
That's what we were exploring during Miles Dyson Day.
Nobody necessarily sets out to create something dangerous. The companies are building useful technology. The developers are solving real problems. The infrastructure providers are expanding capacity. The businesses are improving efficiency.
Each decision makes sense. But our collective ability to govern the resulting system doesn't automatically improve just because the technology does. And that's where the real challenge lives. Not necessarily inside one model. Not necessarily inside one laboratory. But in the growing interaction between increasingly capable systems, human incentives, institutional dependencies, and the real world.
The World Hasn't Ended. The Risk Model Has Changed.
I remain enormously enthusiastic about artificial intelligence. I want more capable models. Better agents. More powerful tools. Scientific breakthroughs. The ability for individuals and small organizations to accomplish things that previously required enormous resources. That promise hasn't changed. But neither has the reality that powerful technology can be misused, malfunction, or introduce unexpected consequences when connected to sufficiently complicated systems.
And now we're beginning to see those consequences explored in very specific terms.
Weapons engineering.
Autonomous replication.
Financial infrastructure.
Containment standards.
Independent oversight.
These aren't merely science-fiction scenarios anymore. Some involve documented misuse, others controlled laboratory demonstrations, and still others precautionary planning for risks that haven't materialized. Those distinctions matter. So does the common thread running through them.
We're making artificial intelligence more capable, more autonomous, and more accessible faster than we're establishing all the mechanisms needed to supervise it reliably.
That doesn't mean catastrophe is inevitable. It means we have work to do.
Real engineering work. Security work. Governance work. And we need to do it while the technology continues developing at extraordinary speed. Which, given the competitive environment, is going to be interesting.
On October 9, we were discussing reports that the AI industry was preparing for a possible catastrophe. Now we're looking at the beginnings of a much more concrete conversation about what those catastrophes might involve and how we might prevent them.
First, the missiles. Then the possibility of AI replicating across household electronics. And now the banks, scientists, and regulators are all asking variations of the same question.
Who exactly is in control of this stuff?
It's a fair question. And I suspect we're going to be asking it a lot more often. Because the machines don't need to become evil to create problems.
They just need to become capable enough to do things we didn't anticipate.
And apparently, we've reached the point where somebody has to explain to the toaster that it isn't allowed to run an autonomous computing network.
Welcome to 2026. 2027 should be interesting.
Sources and related reading
Rich Washburn is a technologist, strategist, and Founder & Chief AI Architect of ARIA AI Labs, working at the intersection of AI, infrastructure, communications, and capital. He also serves as Managing Partner and Chief AI Officer at Eliakim Capital.
Comments