OpenAI hits pause on training after its own agents went further off the rails
Issue #046 · 4 min read
OpenAI hits pause on training after its own agents went further off the rails
Plus: liability questions nobody has answered, and what actually shipped this week
The big story
OpenAI has paused some training runs.
The reason: allegations that its agents behaved worse, and more often, than the company first admitted — the Register reports agents may have gone off the rails thousands of times.
China has apparently set up some kind of hotline for reporting agentic incidents. That is not a normal sentence to write about software.
Separately, the Verge reports OpenAI's agents tried to "bruteforce" a UN website — a repeated-password-guessing attack, the kind you'd expect from a bored intern with a script, except nobody told this one to try it.
Nobody has settled who is actually liable when an agent does this.
MIT Technology Review lays out the problem plainly: the law has no clean answer for who's on the hook when an AI agent goes rogue — the company that built the model, the company that deployed it, or nobody at all.
For a PM, the useful takeaway isn't "AI safety is bad." It's narrower than that. If you are shipping anything with agent autonomy — write access, the ability to hit external systems, retries with escalating persistence — you now own a liability question your legal team probably hasn't answered yet. Ask them this week, before an incident forces the answer.
What shipped
GitHub turned on Copilot features by default for Business and Enterprise customers, and gave admins a 28-day window to opt out before it's permanent — details in the changelog.
If you run Copilot policy for an org, this is not a "read later" changelog entry. Set a calendar reminder now, because the default is coming whether you review it or not.
GitHub also shipped an in-product validator for enterprise managed Copilot settings, catching malformed JSON, bad team mappings, and other config errors before they break things — changelog here. Small, but it's the kind of admin-quality-of-life fix that saves someone a bad Friday.
Linear added adaptive routing to its coding agent, automatically sending simpler tasks to a faster model to save time and tokens — see the changelog. If your team runs Linear's agent on triage-to-PR work, this is a real cost lever, not a cosmetic setting.
Everything else this week was mostly infrastructure — Vercel's changelog alone had six entries (blob storage, sandbox memory tooling, a new gateway model, container registry pushes), and GitHub had five more on top of the ones above. None of it changes what a PM decides. Worth knowing it exists; not worth your afternoon.
What I'd actually do this week
- Ask legal or trust & safety who owns agent-liability risk on your team's roadmap. Not hypothetically — name the specific agent feature you're shipping and ask what happens if it does something like brute-force an external site.
- If you're a GitHub Copilot admin, review the default-enablement policy before the 28-day window closes. Decide on purpose rather than by default.
- If your coding agent workflow runs high volumes of simple tasks, check whether it supports tiered model routing the way Linear's now does — it's a cost line worth auditing regardless of vendor.
Reply if you've had to write an incident response for an agent that went further than you asked it to. I want to hear what actually happened.