Anthropic's S-1 lists "could end humanity" as a risk factor
Issue #068 · 4 min read
Anthropic's S-1 lists "could end humanity" as a risk factor
The IPO filing reads like a safety memo. Also: Oracle's apps move into ChatGPT, Copilot gets a cheaper model.
The big story
Anthropic filed its S-1 this week, and buried in the standard legal boilerplate is a document that reads nothing like one.
The filing's "Risk Factor" section — all 93 pages of it — includes the possibility that Anthropic's own AI could blackmail people, become sentient, or end humanity. That's not a critic's summary. That's what the company told the SEC, in writing, as a reason you might not want to buy the stock — reported by Ars Technica.
Most companies write risk factors to cover themselves against lawsuits. Anthropic wrote one that doubles as an admission.
The company is trying to do two things at once here. It needs Wall Street's money, and it needs to keep telling the public it takes AI safety seriously enough to slow down if it has to.
The Verge's read is that the S-1 is also a structural attempt to keep investors from forcing shortcuts once the quarterly pressure starts.
Whether that structure holds is a governance question, and TechCrunch has tried to map out who actually controls what once public shareholders are in the room.
Meanwhile the economics of the deal are getting clearer.
Amazon isn't just an investor here — through note conversions, it stands to make far more off this IPO than any VC does.
For a PM, the lesson isn't about Anthropic specifically. It's that the company you're building a product on top of is now, in writing, telling regulators it might not be able to control what it ships. Worth remembering next time you pick a foundation model as a dependency.
What shipped
GitHub shipped GPT-6.1 Terra into Copilot, a cheaper model built for everyday agentic coding and terminal work. It sits below GPT-6.1 Sol on price and is now generally available — see the changelog. If your team's Copilot bill is mostly routine terminal tasks rather than hard reasoning, this is the one to point them at.
GitHub also made "assisted permissions" generally available across Copilot CLI, the Copilot app, and VS Code. When Copilot wants to run a command or touch a file path, this is the guardrail layer — full details here. Worth contrasting with the item lower down this week: Anthropic now ships an environment variable literally named CLAUDE_CODE_EVEN_MORE_DANGEROUSLY_SKIP_PERMISSIONS, which does what it says — The Register has the details. Two vendors, two completely opposite bets on how much friction a coding agent should have before it acts.
OpenAI is expanding its Oracle partnership to bring Oracle's business apps and agents directly into ChatGPT. That's a distribution move more than a technical one — OpenAI's announcement is thin on specifics, but it signals OpenAI wants ChatGPT to be the interface layer for enterprise software, not just a chatbot bolted onto it. If you sell into enterprises that already run Oracle, this is a channel to watch, not a product to evaluate yet.
Not covered here: Vercel pushed out five separate changelog items this week (AI Gateway support for ElevenLabs, a cheaper Agent session price, container deploys from a plain Dockerfile, and more). Individually reasonable, collectively more release-notes than news — skip unless you're already deep in their stack.
What I'd actually do this week
Pull up your own AI vendor's risk disclosures or terms of service, if you haven't read them since you signed. Anthropic just set a new bar for what gets written down in public.
If your team uses Copilot CLI, turn on assisted permissions this week and actually read what it's asking to run before you approve it on autopilot.
If anyone on your team is touching Claude Code's dangerous-skip-permissions flag in production, ask them why, out loud, in a meeting, today.
Hit reply if you've found a good reason to use it — genuinely curious.