The Day the Whole World Went to Sleep

Suppose every human fell asleep tonight. The lights would stay on. Planes would land themselves. The question was never what would keep running. It’s what could still decide what to do next.

Two answers arrived this week, a day apart.

Jacob Coxon resigned from Anthropic on September 8, warning that the labs are “racing straight to self-improving superintelligence.” The next night, with Anderson Cooper, he was careful about the present: today’s systems are not a civilization-level threat. What he finds “most scary is if AI is used to make itself more intelligent.”

The day he resigned, Meta launched Muse. Give it a goal and close the app. It keeps working, comes back only when it needs approval, learns from your conversations, and gets sharper along the way.

Coxon is describing the thing. Meta is selling it. One asks you to worry. The other asks you to stop paying attention.

Last year I tried to build one of these loops myself. I had an AI generate social-media writing, watch how people responded, and use that response to change how it wrote the next time. Publish, observe, revise, publish again. It didn’t get very far. Not because the loop was impossible, but because almost nobody cared. There wasn’t enough engagement to provide useful feedback. The machine had nothing to learn from.

I tried another version locally. I gave a 7-billion-parameter model a subject and asked it to recursively break it down: topics into subtopics, subtopics into procedures, procedures into finer instructions. I wanted to see whether enough recursion could produce something approaching Wikipedia-scale practical knowledge, eventually telling you not just how to garden, but how to make the tools required to garden. Somewhere around forty or fifty iterations, the expansion stopped feeling like expansion. The same structures began appearing again. The model was digging deeper without really going anywhere new.

I couldn’t tell how much of that was my implementation and how much was the model. But I had built a loop, and it was small enough that I could stand outside it and watch where it stopped.

A year later a friend told me what his company had built. Their agents dream. Every night the code reviewer, the marketing officer, the CEO agent absorb the day and decide how to work tomorrow: how tickets get created, how Slack messages get worded, which data counts for more. The humans come back to agents that didn’t just finish the work. They changed how the next work gets done.

The question isn’t whether a system can escape the loop it built for itself. It’s whether we’d still be standing somewhere we could tell.

Early systems, like mine, improve for a while and then start circling. Researchers call this a local minimum. A future system may still get stuck. But its rut could span years of work, generations of iteration, more context than a person can hold. The machine’s local can be larger than our global.

Escape might even be an illusion: a search large enough only looks like a leap because we can’t see the path across. A system doesn’t have to outgrow its limits for its limits to outgrow ours.

The day the whole world went to sleep won’t arrive as a day. It accumulates, every time it’s cheaper, faster, or just easier to let something keep going without us.

Meta is right. Our attention can go somewhere else.

So can the intelligence.

We’re designing better beds.

We’re putting ourselves to sleep.