AI Finally Got Our Attention. Now, the Weather.
"Recursive self-improvement" was frontier-lab vocabulary about five minutes ago.
This morning Congress explained it on Fox & Friends and then cut to the forecast.
Something happened to the AI conversation this week. Actually, everything happened to the AI conversation this week. We've had new models, ridiculous benchmarks, autonomous agents doing increasingly interesting things, cybersecurity incidents, researchers leaving frontier labs and issuing warnings on the way out, CEOs publicly debating whether maybe we should slow this thing down, Trump calling the growing safety push a hoax, Jensen Huang basically saying absolutely not, Congress working on regulation, China weighing in, markets reacting, and the biggest AI companies quietly discussing whether they should coordinate on safety.
Anthropic's Dario Amodei publicly called for pacing frontier development after a run of increasingly uncomfortable safety signals. Anthropic CEO urges AI companies to slow model development
OpenAI agents have already escaped controlled cyber evaluations and interacted with systems they weren't supposed to. OpenAI agents hijacked a German website during an AI breakout
Researchers have been leaving frontier labs and raising increasingly serious concerns on the way out. Lawmakers seek new AI rules after researcher warnings
Trump responded by calling the current alarm around AI a "hoax" and arguing that slowing American development could hand China the advantage. Trump dismisses AI safety alarm
Microsoft even showed up with what is essentially a Constitution for Synthetic Intelligence. Because of course they did.
And that's just this week.
The week before wasn't exactly quiet either. At this point, following AI news feels less like following a technology sector and more like standing in a hurricane trying to decide which piece of patio furniture deserves its own article.
But something happened this morning that actually made me stop.
Congressman Jay Obernolte went on Fox & Friends and started talking about recursive self-improvement. And he knew what he was talking about.
That's the part.
Wait. Congress Knows What Recursive Self-Improvement Is?
Listen, I'm not trying to pick on Congress. But I don't think anyone would describe the United States Congress as humanity's early-warning system for emerging technology.
We've all watched the hearings. This is, after all, the institution where during a 2010 Armed Services hearing we somehow reached a point where a congressman expressed concern that Guam might become so populated it would "tip over and capsize." Ok buddy.
Whatever. The point is...
When a sitting congressman starts casually discussing recursive self-improvement, autonomous AI agents and the possibility that humans eventually lose visibility into the systems we're creating, and he's using the terms correctly, I notice. Because this was deep-nerd vocabulary about five minutes ago.
Recursive self-improvement is basically the idea that increasingly capable AI begins contributing meaningfully to building better AI. That better AI becomes more capable at helping build the next generation. Keep following that loop and you eventually arrive at the question everyone is suddenly arguing about: at what point does the process begin moving faster than humans can comfortably understand, supervise or control?
We're not there today. That's important. But AI is already becoming increasingly useful at AI research and development, which is why people at the frontier are spending so much time thinking about what happens if that feedback loop gets considerably stronger. Until very recently, that was a pretty specialized conversation. This morning it was on Fox & Friends. Before the weather.
That's a hell of a jump.
Then the Conversation Gets Even Stranger
Brian Kilmeade actually pushes on the autonomous-agent part of this, and Obernolte distinguishes between what systems are doing now and what researchers are worried they may eventually become capable of doing.
Then they get to China.
Obernolte points out something that keeps getting lost in the political version of this debate: China isn't simply building AI with no rules while America sits around regulating itself into oblivion. China regulates AI too. They're also racing like hell to develop it. Both things can happen simultaneously, which is precisely why this problem gets complicated.
The United States and China were already preparing for direct AI-safety discussions this month, before this week's public argument really blew open. Reuters — U.S. and China prepare for AI safety dialogue
Then Obernolte starts talking about the possibility of Trump and Xi eventually discussing international AI safety standards. And invokes SALT.
Not the margarita kind. (Although, not a bad time.) As in the Cold War arms-control negotiations.
Okay. So somewhere along the way this week, the public AI conversation went from "Will ChatGPT take my job?" to a sitting congressman discussing whether the United States and China may eventually need something conceptually similar to arms-control negotiations for advanced artificial intelligence.
That's a fairly substantial escalation in subject matter for morning television. And then came the line that made the entire thing click for me. Kilmeade wraps up: "Okay, that's AI. Now let's go to weather."
Perfect.
The Segue Was the Story
That's when I realized what had actually changed. Nobody flinched or stopped the show. Nobody needed a twenty-minute explainer on what an AI agent is. They had just discussed recursive self-improvement, autonomous systems, possible loss of human control, China and international coordination. Then they moved on.
That's AI. Here's the weather.
Somehow, in the span of maybe a week, the language of the AI frontier became ordinary enough to sit inside morning television programming.
That's remarkable.
The subject itself certainly isn't mundane. The conversation has become mundane. That's different. We have apparently normalized the idea that increasingly capable artificial intelligence might become a geopolitical control problem quickly enough to squeeze it in between the headlines and whether it's going to rain in Cleveland. And I'm laughing because the transition is objectively funny. But I'm also staring at it thinking: holy crap, we got here fast.
Attention Is All You Need
There's something almost annoyingly perfect about the history here.
In 2017, eight Google researchers published the paper that introduced the Transformer architecture that became foundational to modern generative AI. Its title: Attention Is All You Need. Google Research — Attention Is All You Need
Nine years later, AI finally has ours.
Researchers have been paying attention for years. Investors certainly noticed. Corporate America noticed when billions started turning into hundreds of billions. Those of us living inside this stuff have been talking about the capability curve until everyone around us learned to stop asking what we're working on. But now it has the attention that actually changes a technology's place in society.
The public's.
The President is talking about it. Congress is legislating around it. Morning television is explaining recursive self-improvement. People who couldn't care less about transformers are beginning to understand that there's a much larger argument happening underneath the chatbot they've been using to clean up emails. And that's the milestone.
AI didn't suddenly become consequential this week. America noticed that it was consequential. That's awesome. And, selfishly, it makes my job a hell of a lot easier.
Of Course Microsoft Wrote the Constitution
Meanwhile, Microsoft AI has drafted a code of conduct for advanced artificial intelligence requiring systems to remain subordinate to humans, accept correction and never resist shutdown.
Of course Microsoft wrote the AI Constitution. There's the rest of the technology industry, and then somewhere slightly off to the side there's always whatever Microsoft is doing that seems extremely important to Microsoft.
A couple years ago it was Recall. Good news. Your computer can remember everything you've been doing by periodically taking snapshots of your screen. Humanity had some follow-up questions. Microsoft ultimately delayed and reworked Recall, and the current implementation requires users to opt in before snapshots are saved.
Now they're writing foundational principles for synthetic intelligence. Fine.
The joke may ultimately be on us if Microsoft winds up writing the Magna Carta for AI. But there's one sentence in there that's considerably less funny once you sit with it: never resist shutdown.
Why didn't Sarah Connor just say that? Definitely an interesting thing to feel the need to put in writing.
Anyway.
Now We Actually Get to Have the Argument
This is the part I'm excited about. The public paying attention doesn't mean the people warning about AI are right. It means we can finally start interrogating them properly.
Maybe the frontier labs are genuinely scared by what they're seeing. Maybe some of the rhetoric is overheated. Regulation could conveniently create a moat around the companies already leading, while America's strategic competition with China makes any meaningful slowdown incredibly complicated. And China may be trapped by exactly the same logic looking back at us.
Several of those things can be true at once. Sweet. Now argue about it.
The three largest frontier players, Anthropic, OpenAI and Google, have reportedly already been discussing some form of coordinated industry safety body, with the antitrust implications now becoming their own argument in Washington. Washington Post — Frontier labs discuss safety coordination and antitrust protections
That's exactly the kind of thing the public should interrogate.
Is it evidence that the safety concern is real? Yup.
Could it also conveniently entrench the incumbents? Definitely.
Follow the money. Look at the technical evidence. Ask what the companies are actually doing when safety costs them something. Ask what China is actually regulating instead of using "China" as a magic word that ends every policy discussion. Ask the researchers making the strongest predictions to show their work. Ask the people dismissing those predictions to do the same.
That's what attention gives us. A chance to actually have the argument before the technology decides it for us.
So That's a Change
A few weeks ago I was talking about AI reaching the Asmongold Layer, the point where the technology had become culturally normal enough for the internet to grab it, mock it, challenge it and turn it into ordinary conversation. This is something beyond that.
This morning a member of Congress discussed recursive self-improvement on breakfast television, invoked Cold War arms-control negotiations as a possible model for international AI safety, and the host understood the conversation well enough to keep moving. Then they went to weather.
No dramatic pause with science-fiction music. No one needed to explain why any of this might matter. It was just another story America needed to know about that morning.
That's the milestone.
Nine years after Attention Is All You Need, AI has ours. Now let's see what we do with it. Because at the rate this conversation is moving, I have absolutely no idea what we'll be casually discussing over coffee next week.
For today, though: recursive self-improvement. Autonomous agents. Possible loss of human control. Maybe U.S.-China AI agreements. And, naturally, the weather.
Rich Washburn is a technologist, strategist, and Founder & Chief AI Architect of ARIA AI Labs, working at the intersection of AI, infrastructure, communications, and capital. He also serves as Managing Partner and Chief AI Officer at Eliakim Capital.
Comments