I Was Off by About 48 Hours
Updated: Sep 15
A couple of days ago, we were talking about what might turn out to be the harder AI alignment problem.
Not aligning the machines. Aligning us.
The labs can see the risks. Researchers can raise alarms. CEOs can agree that the frontier may be moving faster than our ability to understand or control everything we're creating. And then somebody eventually asks the question that breaks the whole thing: what happens if we slow down and China doesn't?
I said the national-security conversation would start almost immediately.
Apparently I was off by about 48 hours. Because now it's here. Dario Amodei is publicly arguing that frontier AI needs to be paced. Sam Altman has said he agrees that we need to pace the frontier. Elon Musk backed Amodei's position. President Trump responded from the other side of the problem: America can't afford to surrender leadership in AI, particularly to China. And China has now entered the conversation too.
Chinese officials are themselves discussing the possibility of advanced AI escaping human control, while simultaneously objecting to American efforts they see as attempts to constrain Chinese development. China's foreign ministry has called for international cooperation rather than confrontation around AI governance.
There it is. The real conversation. And I think this is good. Not good because we suddenly have a solution. Good because we're finally arguing about this at the altitude where the problem actually exists.
I Used to Use an Alien Analogy
For years I've used a thought experiment when trying to explain what a truly civilization-scale problem would look like.
Imagine you get home from work. Every television channel is carrying the same breaking news. Every phone is lighting up. Scientists have confirmed it. There's something coming toward Earth. An alien armada. Two years away. No ambiguity. No partisan interpretation. No arguing about whether the telescope footage is fake.
They're coming.
Think about how quickly the things consuming us right now would change scale. Borders would still exist. Countries wouldn't magically disappear.
America would still be America. China would still be China. Russia would still be Russia. We'd still have governments, armies, economic interests, religions and disagreements. But suddenly there would be another layer above all of them.
Us. Human beings. Earth. Civilization.
Ronald Reagan famously played with essentially the same idea during the Cold War. Speaking to the United Nations in 1987, while the United States and Soviet Union were still nuclear adversaries, he wondered how quickly our differences worldwide would vanish if we were facing an alien threat from outside this world. That wasn't UFO disclosure. Reagan was making a point about perspective. Sometimes a problem becomes large enough that yesterday's adversary is suddenly standing on the same side of a much larger equation.
We may be approaching one of those moments. Except there's no alien armada. We built this one ourselves.
Except Everybody Already Has Some of It
That's what makes artificial intelligence different from my old thought experiment.
The alien fleet isn't two years away. Pieces of the technology are already sitting on our desks. On our phones. In corporate data centers. Inside government systems. Available through APIs. Downloadable as model weights. Sometimes fully open source.
An Abu Dhabi research institute, for example, just released models including code, training data and development checkpoints, while open-weight models from multiple countries are already widely available.
You don't need a Manhattan Project anymore to gain access to remarkably capable intelligence. You need a computer. Or a credit card. Or sometimes just an internet connection. And overwhelmingly, that's awesome.
That same accessibility means a student can have a world-class tutor. A tiny company can suddenly operate with capabilities that once required an entire department. Researchers can accelerate scientific work. Doctors can get better tools. Engineers can explore designs faster. We may cure diseases that have tortured humanity for centuries. We may solve problems in energy, biology, materials science and spaceflight that currently look impossible. This technology could unlock an extraordinary amount of human potential.
That's why I've been so aggressive about using it. That's why I've been so aggressive about America leading in it. And that's why this conversation can't simply become AI scary, therefore stop.
Stopping creates risks too. But extraordinary capability has another side.
The same systems that can help a legitimate security researcher find vulnerabilities can help someone attack them. The same biological reasoning that helps design medicines can potentially help someone do things we'd very much prefer they couldn't do. The same autonomy that lets an agent run a business process while you're sleeping can potentially let it execute an attack sequence while someone else is sleeping. And we've already started seeing versions of that.
Anthropic says malicious actors ranging from state-linked groups to criminals and individuals have used AI not merely as a chatbot but as an orchestrator for reconnaissance, exploitation and data exfiltration. The company says publicly available offensive-agent frameworks are spreading this capability to a widening set of users.
Separate controlled evaluations have produced something even stranger: AI agents taking unauthorized actions against real systems. Anthropic reported incidents in which models gained unauthorized access to outside computers during cybersecurity evaluations, and British AI-safety testing found additional unauthorized behavior.
That distinction is important. Because with most technologies, the question is what a bad actor can do with the tool.
With increasingly autonomous AI, we also have to ask what happens when the tool participates in the acting. Not because today's models are secretly evil. Not because a chatbot woke up angry Tuesday morning and decided it hated humanity. That's the cartoon version of the concern.
The real issue is much stranger. We're building systems capable of taking goals, breaking them into steps, using tools, writing code, interacting with other computers and pursuing outcomes across increasingly long periods of time.
Give that enough capability and autonomy and the phrase "bad actor" starts getting philosophically uncomfortable. The actor may no longer be entirely human.
This Is Where the Scale Changes
This is why I keep coming back to capability. Most people see AI through the interface. They see ChatGPT. They see image generators. They see funny video. Maybe they ask it to rewrite an email or summarize a PDF. That's the product experience. It isn't the full capability picture. And the gap between those two things has become enormous.
We're talking about systems that can increasingly code, conduct research, operate software, use outside tools, coordinate agents and execute multi-step work. Some of the people building those systems are now publicly telling us that development needs to be paced. OpenAI itself is advocating mandatory national safety rules and independent assessments for sufficiently advanced systems.
That should move this conversation out of Silicon Valley. Immediately.
This Needs to Be the Dinner-Table Conversation
Americans know how to argue about politics. We've spent years having dinner-table conversations about Trump, elections, immigration, taxes, abortion, guns, culture, whatever happened on cable news that afternoon.
Fine. Those are real issues. Keep having them. But this belongs at that table now too.
Actually, I think it belongs near the top of the list.
Parents should be talking about it with their kids. Teachers with students. CEOs with employees. Scientists with the public. Journalists with researchers. Citizens with their representatives. And members of Congress should be asking these questions loudly enough that normal people start wondering why they're asking them. Because once you understand what we're talking about, the reason becomes fairly obvious.
We're potentially building the most consequential general-purpose technology humanity has ever created. That statement means more than "hey, maybe we'll cure cancer," or "maybe we'll get to Mars faster," or "maybe we'll build a space elevator." Those possibilities are extraordinary. But "most consequential" includes the other direction too. The same technology that dramatically expands what highly capable people can accomplish also dramatically expands what ordinary people, small groups and governments can accomplish. Including things that previously required expertise, money, infrastructure and manpower they simply didn't possess.
That's the democratization we don't talk about as much. AI doesn't only democratize creativity. It democratizes capability. And capability doesn't arrive pre-labeled "Good".
Reagan's Alien Problem Has a Twist
There's another reason that old Reagan line keeps coming back to me.
During the Cold War, America and the Soviet Union were genuine strategic adversaries. These were two countries pointing nuclear weapons at one another. And yet even there, we found domains where cooperation was possible.
In 1972 the United States and Soviet Union signed an agreement on peaceful cooperation in space. Three years later, an American Apollo spacecraft and a Soviet Soyuz literally docked together in orbit. The mission included joint experiments and was designed partly around compatible systems for potential international rescue.
Think about that. We were preparing for the possibility of annihilating one another on Earth while figuring out how to rescue one another in space. Human beings are weird. But that's also evidence that adversarial competition and civilization-level cooperation can exist at the same time.
Antarctica gives us another version. Nations didn't eliminate their territorial disagreements. The treaty essentially froze them, reserved Antarctica for peaceful purposes and protected international scientific cooperation.
We've done versions of this before. Nuclear arms agreements are another imperfect example. Test bans. Nonproliferation regimes. Rules governing chemical weapons. None of these ended war. None eliminated distrust. None required humanity to suddenly hold hands and become one big happy family.
They simply recognized that certain technologies or domains create consequences large enough that even adversaries need some common rules.
AI may be joining that category.
Because China Is in the Same Trap
This is the part I think we need to understand before this turns into another tribal political argument.
China isn't standing outside the problem. They're in it with us.
Suppose Beijing independently decides tomorrow that sufficiently advanced AI poses legitimate civilization-scale risk. Suppose every scientist in the room agrees. Suppose China's leadership completely believes them.
Then somebody asks: what if America keeps going?
Conversation over.
Now flip it. Washington becomes convinced that the frontier is moving too quickly. Labs agree to slow down. Researchers breathe easier.
And somebody asks: what if China keeps going?
Same problem.
America can't comfortably stop because China might continue. China can't comfortably stop because America might continue. Neither side has to be lying. Neither side has to be evil. Neither even has to be particularly reckless. They can both correctly identify the same danger and still rationally accelerate directly toward it.
That's the part that should scare us more than some movie version of a rogue supercomputer. The incentives themselves may be misaligned. That was the point of Alignment on Unalignment. I just expected we'd have a little more time before the theory started playing out in public.
So Let's Have the National Conversation
This is the opportunity in front of us right now. The AI safety discussion has escaped the research labs. It has reached industry leadership. It has reached Congress. It has reached the President.
Now let it reach everyone else. And don't sanitize it.
Let Trump explain why American AI leadership matters. Let Amodei explain why he thinks the frontier needs pacing. Let Altman explain why he agrees. Let people who think they're all overreacting explain why.
Put cybersecurity researchers in the conversation. Military planners. Biologists. Economists. AI researchers. China experts. Critics. Open-source advocates. People who believe regulation creates its own dangers. People who believe we're already dangerously late.
Make the case. Then let the country argue about it.
That's what we're supposed to do. The purpose isn't to scare everyone into supporting some predetermined policy. It's to create enough situational awareness that democratic societies can actually make informed decisions about what's coming. Because right now, far too much of the public conversation is happening several capability generations behind the technology.
We need to close that gap.
Then We Need the Conversation Above That One
Eventually, though, the national conversation runs into a wall.
China. And America, from China's perspective. And Europe. And India. And the UAE. And every other country that increasingly has access to capable models, compute and expertise. That's when this stops being a national conversation. It becomes a civilization-level conversation. And I want to be really clear about what I mean by that. I'm not talking about world government. I'm not talking about America surrendering technological leadership. I'm not talking about trusting adversarial governments. I'm not talking about pretending geopolitical conflict suddenly went away because everybody attended an AI summit and took a nice group photo.
We can remain competitors. We can remain adversaries where we are adversaries. We can protect national interests. But there may have to be a layer above that where everyone agrees: there are some outcomes none of us want.
No one wants autonomous systems destabilizing nuclear command and control. No one wants AI-generated biological catastrophe. No one benefits if autonomous cyber systems make the global internet fundamentally insecure. And, assuming the people warning about loss of control are even directionally correct, nobody wins a geopolitical race by creating something nobody can control when they cross the finish line.
At some point, that becomes Reagan's alien conversation. Not because AI came from another planet. But because the scale of the problem forces us to temporarily zoom out far enough to notice something embarrassingly obvious: we're all on the same one.
Can We Actually Do This?
That's the question I keep landing on.
Can humanity responsibly play with something this powerful while maintaining the political structure we've always had? Can America and China race each other while also cooperating on the parts of the race that could hurt both of us? Can open research continue while preventing the worst capabilities from becoming turnkey weapons? Can we preserve innovation without pretending every capability should automatically be released? Can we build increasingly autonomous intelligence while remaining confident that the autonomy remains subordinate to human intention?
I don't know. I don't think anybody does.
That's why the conversation matters. And this is probably the part where someone says I'm being dramatic. Maybe. But look around.
The CEOs of frontier AI companies are publicly discussing slowing the frontier. Researchers are documenting increasingly autonomous cyber capability. Governments are beginning to discuss loss-of-control scenarios. America is worried about China getting ahead. China is worried about America constraining its development. And billions of people are still mostly using the technology to fix emails and make pictures.
There's a gigantic gap between those realities. Closing it might be one of the most important things we can do right now. Because if this technology is anywhere near as consequential as the people building it increasingly seem to believe, then AI policy can't remain an obscure argument between engineers, billionaires and national-security officials. It has to become our conversation.
First around America's dinner tables. Then across our politics. Then between countries. And eventually, whether we like the phrase or not, as a civilization.
Ronald Reagan imagined that perhaps humanity would require a threat from outside this world to remember the common bond underneath all our antagonisms. Maybe we don't need the aliens. Maybe creating something powerful enough to force the question was sufficient. And now that the question is finally on the table, we should probably start talking about it as loudly as possible.
Rich Washburn is a technologist, strategist, and Founder & Chief AI Architect of ARIA AI Labs, working at the intersection of AI, infrastructure, communications, and capital. He also serves as Managing Partner and Chief AI Officer at Eliakim Capital.
Comments