The AI Doomers have returned in full force.

Jacob Coxon resigned from Anthropic after three years of pretraining work at OpenAI and Anthropic. He wrote that both companies are racing toward self-improving superintelligence and “gambling with our lives.” A few days later, Anthropic CEO Dario Amodei published We Must Pace the Frontier, arguing that AI development should slow down while safety work catches up.

The security incidents behind these warnings exposed weak assumptions in the harnesses around the agents. Coxon warned that AI could kill everyone by the end of the decade. Amodei made the more specific claim: in 6–12 months, a more capable swarm with similar misalignment could take over the internet through a persistent botnet and cause hundreds of billions of dollars in damage.

I am not buying the "AI Safety" claims. Sure, there is a non-zero chance that Arnold Schwarzenegger travels from the future trying to save humanity from Skynet.

However, I have yet to see an agent operate outside of its actual system permissions. If 'edit' or 'search' is explicitly excluded, and there is no shell, MCP server, or other equivalent path, the agent cannot just invent that capability.

An agent cannot decide to bypass a TCP handshake either. There are still hard technical boundaries.

The real risks are boring - broad permissions, exposed credentials, bad sandboxing, and tools that can do more than intended. These are human engineering problems.

Anthropic can sincerely worry about safety and still benefit when the public sees its systems as indispensable and nearly too powerful to control. I don't know whether this week's messaging is an IPO campaign, but the narrative is useful for valuation, regulation, and market position.

Sanders and Casar have announced forthcoming legislation to ban artificial superintelligence and pause advanced AI development pending federal safety rules. This is turning into the boy who cried wolf, and I'm concerned it is detracting from the real risks - cybersecurity, fraud (Grandma got tricked by AI Brad Pitt to send money), and economic displacement.

Your agent still can't find the policy

Meanwhile, the same agent that will kill you doesn't know why your team calls one customer state “pending” and another “blocked.” It hasn't read the decision that overruled the old architecture document. It doesn't know which policy is current, which exception sales promised, or which person owns the ugly manual step between two systems.

My favorite version is a Markdown vault and a Codex connection. That internal context (not common knowledge found on the internet) is the "company brain."

A company brain is the internal knowledge layer an agent can inspect to understand your business. It doesn't have to begin as an expensive RAG platform. A shared GitHub repository, a structured OneNote notebook, or a Markdown vault can work if the information is current, permissioned, linked, and clear about which source wins when documents disagree.

Before buying another AI search tool, ask:

1. Can the agent reach it? The knowledge has to be connected through a tool, repository, or approved integration.
2. Can it tell current truth from history? A pile of files without dates, owners, and an authority order creates confident confusion.
3. Does it contain your team's language? Public search won't teach the model your product states, client exceptions, acronyms, and unwritten operating rules.
4. Can a person verify the answer? Require source paths or links so the operator can inspect what the agent used.

A no on any of the four means the agent still has to guess.

Start with the system your company already uses, I’ve written a guide to using each:

• Start Your Company Brain in a GitHub Repository
• Use OneNote and Microsoft 365 Copilot as a Company Brain
• How My Markdown Vault Became a Company Brain
• When Your Company Brain Outgrows a Folder

If the knowledge already lives in GitHub, OneNote, or Markdown, start there. Managed RAG can wait.

Is everyone running out of compute?

Not everyone, but the frontier labs are capacity-constrained. Anthropic says rapid growth has strained reliability during peak hours, even after it committed more than $100 billion to AWS for up to five gigawatts of new capacity. OpenAI says it has already secured more than 10 gigawatts of U.S. infrastructure and is still building faster to get ahead of shortages.

The bottleneck is no longer just buying GPUs. These projects need power, land, permitting, transmission, networking, and years of construction. That doesn't prove the safety push is cover for a compute problem, but it belongs in the background when three frontier CEOs suddenly agree that slowing down sounds reasonable.

Worth Reading

We Must Pace the Frontier — Dario Amodei
The full argument behind Anthropic’s call to slow frontier development: embedded evaluators, coordination among democratic countries, and eventually global limits. Read the proposal itself before accepting either the doomer version or my pushback.

Research acceleration: The view inside OpenAI — OpenAI
OpenAI says its research group now uses 3.1 agent-workdays for every human workday. The median researcher consumes more than $600 per day of inference at API prices, while the 90th percentile passes $7,000. OpenAI is careful to say more agent hours do not automatically mean more discoveries. The scale is still wild.

Sam Altman and Elon Musk back Dario Amodei's call to slow down AI development — SiliconANGLE
Alex Karp gives the clearest counterargument to a U.S. pause: “If we didn't have adversaries, I would be very in favor of pausing this technology completely, but we do.” You can believe the risk is real and still think a unilateral pause hands the advantage to somebody who will keep building.

Omarchy — DHH
Omarchy is DHH's opinionated Linux setup built around agent-assisted diagnosis and changes. I haven't used it long enough to recommend a switch, but the idea is good: choose defaults now and make them easier to change later.

Quote of the Week

“Do what you can, with what you've got, where you are.” — Squire Bill Widener, quoted by Theodore Roosevelt in An Autobiography (1913)

If your company knowledge is spread across drives, chats, wikis, and the heads of three people, reply with where it lives now. I'll tell you which company-brain starting point I would use.

Forwarded this email? Subscribe here.

— Collin