The AI Stack Is Starting to Bend

From the way we use AI to the silicon underneath it, every layer of the stack seems to be wrestling with the same problem: continuity.

There are moments in technology where a bunch of things you've been watching separately suddenly line up.

Not perfectly. Not enough to declare some grand unified theory of artificial intelligence and start selling tickets to the TED Talk.

Just enough that you look across the pieces and go: Oh...Okay.

I seent it.

And yes, I know that's probably the least scholarly way possible to introduce a discussion involving recurrent transformers, persistent cognitive state and inference architecture. That's sort of the point.

Because I don't think this needs to be nearly as complicated as it sounds.

For more than a year, we've been circling around a fairly simple question:

Why should machine cognition have to keep starting over?

The user shouldn't have to keep explaining themselves.

The agent shouldn't have to rediscover the work.

The next model shouldn't have to reread the entire room.

And maybe, increasingly, the model itself shouldn't have to perform every useful piece of reasoning as one long trip forward.

Those are very different technical problems occurring at very different layers of the stack. But lately they keep looking like versions of the same problem. And that's what has my attention.

The users already voted

One of the funniest signs that something fundamental is changing has nothing to do with frontier research.

It's people getting emotionally annoyed with their AI.

We were talking about this recently in Breaking Up With Software Is a Genre Now, after watching people publicly announce that they were leaving one AI for another because it didn't understand them anymore.

That's an objectively ridiculous thing to say about software.

Nobody ever stormed out on Microsoft Excel because, after everything they'd been through together, Excel should have known better.

But AI is different.

For almost the entire history of personal computing, we learned the software. We learned its menus, syntax, workflows, shortcuts and irritating little behaviors. The machine had a structure. We adapted ourselves to it.

Generative AI began reversing that relationship.

Now we're increasingly expecting the machine to learn us.

Our language. Our projects. Our shorthand. Our priorities. Our sense of humor. Our working style. Eventually, even something as fuzzy as taste. And that creates a completely new kind of switching cost: I don't want to explain myself again.

That's not really a memory problem. It's a continuity problem.

We don't merely want the next system to have a database containing facts about us. We want it to somehow preserve the state of the relationship — what we've learned, what we've abandoned, what mattered six months ago but doesn't anymore, what sounded stupid then but suddenly makes sense now. That's a much harder thing. And users are already asking for it while the technology is still hilariously bad at providing it.

That is important because sometimes architecture gets pulled by demand long before people know what to call the thing they're demanding.

From the user side of the glass, the vote is already being cast:

Don't make me start over.

That's where recursion first got interesting

Around the same time, we were playing with the idea of recursion — not simply as some programming trick where a function calls itself, but as a way of thinking about ongoing human-machine cognition.

The Möbius metaphor came out of that.

The interesting observation wasn't that an AI could remember something from fifty conversations ago.

A search engine can retrieve something. The interesting part was what happened when old context came back through new context. Something discussed months ago could suddenly mean something different because of everything that had happened since. That's closer to how people actually think. We revisit. We reinterpret. We notice connections late. Some random conversation from three years ago suddenly snaps into focus because of something that happened Tuesday morning.

So we started thinking about AI interaction less like a line: input, answer, input, answer, input, answer and more like something curved.

State moves forward, but occasionally loops back through itself. Old ideas re-enter under different conditions. Context isn't merely accumulated; it's recontextualized.

That's where the Möbius thing came from. Not because somebody was trying to invent mystical transformer geometry. It was simply a useful shape for thinking about cognition that moves forward while still being able to encounter itself again. At the time, that was mostly an interface-layer idea. The model underneath hadn't fundamentally changed.

We were changing how the system used history around it.

Then the same general shape started appearing lower in the stack.

The model shouldn't have to reread the room either

Recently we were discussing work around transferring KV-cache state between related models. That's a spectacularly boring phrase attached to a very interesting idea.

Today, when you hand a long conversation from one model to another, the new model generally has to consume the text and reconstruct its own internal representation of what has happened. It's like somebody walking into a three-hour meeting at minute 172 and being handed the transcript.

Technically, they have the information. But they weren't in the room.

The research we've been looking at raises the possibility that, in some circumstances, pieces of already-computed model state can be translated or reused instead of forcing every new model to reconstruct everything from raw language.

There are plenty of limitations. We're nowhere near some universal cognitive USB stick you can plug into anything. But the direction is fascinating. Because it raises a question that I haven't been able to get out of my head: What if the persistent thing eventually isn't the model?

What if it's the state? A big model handles the hard part. A smaller one takes over. A specialist gets called for thirty seconds. Another agent works on a branch. Something local handles the private material. Something gigantic spins up when things get ugly. The machinery underneath changes, but the work doesn't start over.

At that point the models begin looking less like the product and more like interchangeable compute applied to a continuing cognitive process.

And there it is again.

Continuity.

The human doesn't want to explain the room again. The next model shouldn't have to reread it. Same complaint. Different side of the glass.

I somehow didn't notice at first that those were basically the same sentence.

Then Astra showed up

And this is where things got weird enough that I started laughing. OpenAI's upcoming Astra model has now entered the conversation because of its cybersecurity capabilities. According to OpenAI's own preparedness framework, Astra reached a level requiring substantially stronger safeguards around what the system can autonomously discover and execute.

That's already interesting.

Then reporting emerged that Astra is also using, in a constrained way, something called recurrent depth, or a looped-transformer approach. The technical implementation matters, but the high-level idea is simple enough. Instead of useful computation always looking like: move forward, move forward, move forward, answer you can allow representations inside the model to go through additional cycles of computation.

In extremely simplified terms: process, loop, process again, continue

The system can spend more computational effort transforming its own internal state rather than expressing every intermediate operation as another paragraph of language.

Now, recurrent architectures themselves aren't science fiction. Researchers have explored versions of this idea for years, and nobody should take "loops" to mean somebody accidentally created consciousness in a server rack. But I find the direction incredibly interesting. Because for the last several years, we've developed this intuitive picture that AI thinks in words.

We ask something. The model reasons. We sometimes get a linguistic reasoning trace. Then it answers. That makes it very tempting to confuse language with cognition. I'm increasingly convinced that language may instead turn out to be something closer to the human-compatible interface to cognition.

We need language. It's extraordinary. It lets two completely different kinds of intelligence meet in the middle. But there is no obvious reason increasingly sophisticated machine cognition should need to translate every useful intermediate representation back into English before it can continue working on it. That would be convenient for us. It may not always be computationally optimal for the machine.

And now we're back at recursion again. Only this time it isn't merely happening around the model. It may be happening inside it.

There's an uncomfortable part

This is also where things stop being merely clever. One reason readable reasoning traces are useful is that they give researchers another place to look when advanced AI systems start behaving in unexpected ways.

If a model is planning something undesirable and some version of that intent appears in a monitorable reasoning process, you have a potential intervention point. But as more useful computation happens in latent internal representations, there may be less natural-language material corresponding to every meaningful intermediate operation.

That doesn't mean the machine has developed secret thoughts. That phrasing is anthropomorphic and mostly unhelpful. It means something much more mundane and technically important: The internal computation may become less naturally legible to humans.

And apparently this isn't being ignored. The reporting around Astra suggests OpenAI is constraining the amount or form of recurrence specifically because preserving monitorability matters.

That's an interesting engineering tension.

More machine-native cognition may produce better performance.

More human-readable cognition may produce better observability.

We're going to want both.

Those goals may not always point in exactly the same direction.

Then zoom out another layer

A few days before all of this, we were talking about Etched and the growing hardware race around inference. And the line that kept coming back was: Inference is becoming the workload.

Not a person opening ChatGPT six times a day. Agents. Reasoning systems. Persistent context. Machine-to-machine calls. Huge caches of computational state. Processes that continue operating when the human goes to lunch. Thousands or millions of little cognitive transactions that never need a keyboard attached to them at all.

When a workload gets large enough, eventually the hardware underneath it starts changing. That's what computers do.

Databases changed storage. Graphics changed processors. Cloud computing changed networking and data centers. Machine learning changed accelerators. Now persistent inference is beginning to exert its own pressure on memory architecture, interconnects, chips, power, latency and system design. Which means the loop isn't merely becoming a software concept.

We're beginning to build physical machines around workloads that don't stop. That's the point where I started looking back across all of these conversations and realizing they aren't as disconnected as they first appeared.

Look at the whole stack

At the user layer, people increasingly expect AI to know them and preserve continuity. At the interface layer, recursive context allows old information to become newly meaningful rather than simply disappearing behind us. At the agent layer, systems are becoming longer-running, iterative and increasingly capable of carrying work forward. At the state layer, we're exploring ways to avoid reconstructing an entire cognitive context every time the machinery changes. At the model layer, recurrent architectures allow internal representations themselves to undergo additional cycles of computation. And at the hardware layer, the industry is starting to build machines optimized for persistent inference.

These are not the same technology. They aren't one coordinated architecture. Nobody got everybody into a conference room and handed out a six-layer roadmap. That's precisely why I find it interesting.

Different layers of the stack appear to be developing their own solutions to the same underlying pressure. Continuity. Keep the useful state. Don't reset unnecessarily. Revisit what matters. Continue the process. Move the cognition instead of repeatedly rebuilding it.

That's the shape.

And maybe "the model" isn't the destination

We've spent the entire modern AI era obsessing over models.

Which model are you using?

How many parameters?

What's the context window?

What's the benchmark score?

GPT versus Claude versus Gemini versus whatever arrived fourteen minutes ago while I was writing this sentence.

Those questions matter enormously today. I'm just not sure they'll remain the questions users ultimately care about. If cognitive state becomes persistent…

If different models can operate on different portions of it…

If agents can continue processes across long periods of time…

If cognition can recursively refine internal and external state…

If the hardware underneath it becomes optimized for keeping that process alive…

then eventually asking which model you're talking to may become a little like asking which CPU core is running Microsoft Word.

There's an answer. You just probably don't give a shit.

The experience becomes the continuity. The intelligence isn't necessarily one model. It's a process moving through models, memory, tools, agents and compute while maintaining enough state to remain recognizably the same ongoing piece of work. That is a very different thing from a chatbot. And it's considerably closer to what we've been trying to describe when we've talked about AI as a cognitive layer rather than another piece of software.

This is the part I think I seent

Any one of these developments is easy to look at independently. Memory gets better. Agents run longer. A new KV-cache paper comes out. Someone experiments with recurrence. A chip company optimizes inference. Another model gets a little smarter. Another feature ships. Seen individually, they're incremental. And maybe that's all they are.

But every once in a while, it's worth backing away from the individual pieces and looking at the direction they're moving. Because I don't think the straight line is merely becoming a loop. I think the stack underneath it is starting to bend around the loop. The user doesn't want to start over.

The system doesn't want to start over. The state doesn't need to start over. The model is learning ways to continue working on its own internal state.

And we're beginning to build hardware for cognition that doesn't necessarily stop between requests.

I'm not claiming we know where that ends. I definitely didn't predict Astra. The Möbius metaphor did not somehow cause recurrent transformers.

And I'm not putting a corkboard in the garage with red yarn connecting research papers while frightening the neighbors. Yet.

I'm just saying the same shape keeps showing up. A year ago, recursion looked like an interesting way to think about the interface. Then persistence started looking more important than the individual interaction. Then state started looking potentially more important than the individual model. Now recurrence is becoming part of the frontier-model conversation, while the hardware underneath all of it is being redesigned around persistent inference.

Maybe we're looking at several unrelated engineering optimizations that happen to rhyme. That's entirely possible. But I'm putting a little marker here anyway. Not a flag. Not a victory lap. Just a marker. Because I have a feeling that a few years from now we'll look back at all these seemingly separate developments and realize they were pieces of one much larger transition: From AI as a series of transactions to AI as a continuing cognitive process.

And if that turns out to be where this is heading... Well.

I seent it.

Sources & Further Reading

OpenAI — Path to Astra: Critical Capabilities and Frontier Safeguards OpenAI’s official announcement confirming Astra meets its Critical cybersecurity capability threshold. https://openai.com/index/path-to-astra/

OpenAI — Pacing Model Development in an Era of Cyber-Critical Capabilities Background on Astra development pauses, security hardening, and expanded chain-of-thought monitoring. https://openai.com/index/pacing-model-development-cyber-capabilities/

The Information — OpenAI Technique in ‘Astra’ Model Sparks Security ConcernsReporting on Astra’s use of recurrent depth / looped-transformer techniques and the implications for monitoring model reasoning. https://www.theinformation.com/articles/secret-technique-behind-openais-astra-model-sparks-security-concerns

Geiping et al. — Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach The recurrent-depth research paper describing latent-space reasoning through repeated processing of a recurrent block. https://arxiv.org/abs/2502.05171

NeurIPS 2025 — Scaling up Test-Time Compute with Latent Reasoning Published conference version of the recurrent-depth work. https://proceedings.neurips.cc/paper_files/paper/2025/hash/3b01972cf31e6fa0fe29e4b8b5c2a0a1-Abstract-Conference.html

Heo et al. — Cross-Model KV Cache Transfer in LLM Families Research exploring transfer of computed KV-cache state between related models instead of paying the full prefill cost again. https://arxiv.org/abs/2608.03893

Etched — Frontier Inference Clusters Etched’s description of inference-focused hardware designed around frontier models, long context, and agentic workloads. https://www.etched.com/progress/frontier-inference-clusters

Etched — From Zero to One Etched’s announcement that its first rack shipped to Jane Street and its broader thesis around scaling inference. https://www.etched.com/progress/from-zero-to-one

The Möbius Leap: Recursive Cognition at the Edge of Chaos https://www.richwashburn.com/post/the-m%C3%B6bius-leap-recursive-cognition-at-the-edge-of-chaos

Recursion: What It Really Means — and Why It’s Not Just the Game, It’s the Interface https://www.richwashburn.com/post/recursion-what-it-really-means-and-why-it-s-not-just-the-game-it-s-the-interface

The Model Shouldn’t Have to Reread the Room https://www.richwashburn.com/post/the-model-shouldn-t-have-to-reread-the-room

Breaking Up With Software Is a Genre Now https://www.richwashburn.com/post/breaking-up-with-software-is-a-genre-now

Everyone Is Yelling AGI. I Think They’re Missing the Interesting Part. https://www.richwashburn.com/post/everyone-is-yelling-agi-i-think-they-re-missing-the-interesting-part

I’ve Been Waiting Two Years for This Chip https://www.richwashburn.com/post/i-ve-been-waiting-two-years-for-this-chip

---

Rich Washburn is a technologist, strategist, and Founder & Chief AI Architect of ARIA AI Labs, working at the intersection of AI, infrastructure, communications, and capital. He also serves as Managing Partner and Chief AI Officer at Eliakim Capital.