Keep it Undefined, and You Never Have to Prove You’ve Built It 

Sam Altman's essay "The Gentle Singularity" opens with a quiet provocation: Artificial General Intelligence (AGI) has effectively already arrived. Not as a single event, but as a gradual process already underway. We have, he suggests, passed the point of no return. 

Many disagree. But here's the more interesting problem: we can't even settle the disagreement, because we don't have a shared definition of what AGI is. And if we can't define it, we can't verify it. And if we can't verify it — what exactly are companies promising investors they will build? 

The Brand, Not the Concept 

The AI industry has always needed a next chapter. 

First there was AI. Then Machine Learning. Then Generative AI. Now the conversation has moved on to Agents, AI Scientists, AGI, and Superintelligence — each new term arriving just in time to justify another round of tens-of-billions in investment. 

The leaders of these companies occupy an unusual position. They are simultaneously researchers, product sellers, and fundraisers. Their public statements inevitably live at the intersection of science and marketing. This isn't a criticism — it's a structural reality. But it means we should read claims about AGI the way we read a prospectus: carefully, and with that context in mind. 

AGI is quietly transforming from a technical concept into a brand. A promise. A story told to capital markets. Grand claims about its imminence blur the line between scientific forecasting and PR in ways that are difficult to untangle — perhaps even for the people making them. 

Can a System That Wants Nothing Become Intelligent? 

Here is what strikes me as the deepest unresolved question in this entire debate: 

Today's models don't want anything. 

They are not hungry. They are not curious. They don't fear shutdown or seek survival. They optimize an objective function. That is all. The apparent desire to help, the apparent enthusiasm, the apparent caution — these are statistical patterns in output, not internal states. There is nobody home, wanting things. 

This matters because general intelligence — as we have ever observed it — has never appeared without some motivational substrate. Every intelligent creature we know of has needs, drives, and aversions. It has things it moves toward and things it moves away from. Intelligence, in nature, is not a neutral computation. It is a tool that evolution built for staying alive. 

The question this raises is not whether AI can feel. It's whether genuine general intelligence is even possible in a system with no internal drives. Can you have real curiosity without anything at stake? Can you have genuine long-term planning without something you're trying to protect or achieve? 

Many engineers argue yes — that AGI could become superhuman without anything resembling inner experience. Others believe that true general intelligence may require something closer to biological constraint: limited resources, the real possibility of failure, competing goals that force genuine trade-offs. Only then, they argue, might we see authentic analogues of exploration, caution, and strategic thinking. 

Without such drives, a system remains fundamentally reactive. Sophisticated input-output. Extraordinary at the surface — but perhaps bounded in ways we don't yet understand. 

This is the question that should sit at the center of the AGI debate. It mostly doesn't. 

Perhaps LLMs Are Not the Path to AGI?

Set the "wants nothing" problem aside for a moment and consider a separate challenge: large language models learned everything they know by reading text. 

Text is an extraordinary compression of human thought. But it is not experience. It is the shadow of experience — the residue left after someone has already lived something and chosen to describe it. 

A child doesn't learn language by reading. A child learns by falling, grabbing, failing, and adjusting. Physical reality provides a continuous, high-bandwidth feedback loop that no corpus of text can replicate. The question isn't whether models trained on text are impressive — they clearly are — it's whether text alone can take you all the way to general intelligence, or whether at some point you hit a ceiling that only embodied interaction can break through. 

This is the central wager of Embodied AI: that the path to general intelligence may run through robots, vision, touch, and real-time physical interaction — not through making language models a hundred times larger. Scaling may plateau. Physical experience may not. 

We don't know yet. But it's a more interesting bet than most people outside the field realize. 

Nobody Can Define AGI. That's the Point. 

We keep asking: will AI become human-like? 

Wrong question. 

Airplanes don't flap their wings. Submarines don't swim like fish. AGI, if it arrives, will almost certainly not think like us — it may want things we don't recognize as wants, reason through mechanisms that make our cognitive categories feel quaint, and solve problems in ways we'd struggle to follow even in retrospect. 

The real question — the one the industry carefully avoids — is this: which parts of human intelligence are essential to any general intelligence, and which are just evolutionary baggage we happen to carry? 

Nobody is funding that research. Because the answer might be inconvenient. 

If general intelligence requires drives, embodiment, and something at stake — then the current path, however impressive, has a ceiling. A very high ceiling. But a ceiling. 

The vagueness around AGI isn't a communication problem. It isn't a philosophy seminar waiting to happen. It's load-bearing. The ambiguity is doing real work: keeping the story open, keeping the capital flowing, keeping the next chapter always just around the corner. 

Define AGI precisely, and you might prove you don't have it yet. 

Keep it undefined, and you never have to.