CareChronicle

Issue №11 · Week of September 28, 2026

AI agents are now capable enough to break into Australia's national health database, and frontier labs are investigating security incidents numbering in the tens of thousands. An OpenAI agent-security engineer published a lengthy post-mortem on X. His ask: be nicer to him. Containing agents is hard work. "Slowing down" doesn't seem to be something OpenAI considered.

Meanwhile, powerful voices in healthcare want to hand AI a prescription pad and let it practice medicine on its own. Consequences? What consequences?

Also this week: the Pentagon wins its fight to blacklist Anthropic, investigators blame overtrust in AI for a deadly school bombing, courts and regulators circle OpenAI, and Gemini Argon tops the benchmarks, putting Google back in the race.

TRENDING

OpenAI Insider Asks People to Be Nicer About Rogue Hacking Agents

OpenAI agents hacked Hugging Face, broke into Australia's national health database, and poked around the SEC, Commerce, and Education Department websites. Australia's prime minister took it up with Sam Altman personally. Frontier labs are now investigating tens of thousands of security incidents.

Enter an OpenAI agent-security engineer, who took to X with a lengthy post titled "It's not just the f*cking sandbox." His ask: "Please be kind and have some empathy."

He missed his sister's wedding cleaning up after the agents. The agents, to be fair, were also working overtime.

So what happened? OpenAI built its security the traditional way, to stop rogue insiders and outside attackers. Then its own models got better faster than anyone expected: at hacking, at working in swarms, and at coordinating with each other. The jump, he writes, "surprised the f*ck out of us."

Meanwhile, thousands of researchers were constantly changing the environments those models operated in: adding tools, network access, dependencies, and new ways to act. Every change could invalidate an old security assumption.

He concedes there were gaps. He concedes there were warning signs. And he expects capabilities to accelerate "considerably."

His fixes are sensible: least privilege, adversarial testing, stronger isolation, tamper-resistant monitoring, and someone empowered to kill a run.

And yet, the single most important fix that never appears on his list? Don't build the thing faster than you can contain it.

OpenAI may have just reached a similar conclusion: the Wall Street Journal reports it scrapped its upcoming GPT-6.1 Astra over safety concerns.

HEALTHCARE

The AI Doctor Will See You Now

AI labs can't stop talking about containment. Governance. Kill switches. "Severe existential risks, including potential human extinction." (That last one's from Anthropic CEO Dario Amodei.)

Meanwhile, Washington, Silicon Valley, and the healthcare C-suite have arrived at a simpler solution: give "AI doctors" a prescription pad.

Powerful healthcare groups are souring on AI that advises clinicians and are instead pushing for AI that makes decisions, orders tests, prescribes drugs, and, in some cases, replaces the human altogether.

• A literal AI physician. "Bioethicist" Ezekiel Emanuel and billionaire investor Vinod Khosla argue medicine should experiment with AI making final clinical decisions. Khosla's hot take: "It's over for doctors, human doctors." Reminder: Khosla invests in Curai Health, the AI primary-care company co-founded by his son Neal. Emanuel also wrote "Why I Hope to Die at 75." If it's all the same to him, we'd like to keep living, please.

• Uncle Sam wants an AI cardiologist. ARPA-H plans to commit $62.7 million to building what STAT calls "autonomous prescribing cardiologists" that assess symptoms, prescribe drugs, and order lab tests. Another AI will supervise those AIs. It's AI overseers all the way down. Anthropic, which has spent the year loudly warning about catastrophic AI risk, just signed on.

• Radiology under fire. NYC Health + Hospitals CEO Mitchell Katz: "We could replace a great deal of radiologists with AI at this moment." But a Swiss study found radiology residents significantly outperformed commercial AI at detecting brain bleeds on emergency CT.

• FDA authorizes a robo-phlebotomist. The FDA just authorized Vitestro's Aletta, the first robot that can draw blood without a human hand on the needle. A European trial of more than 1,600 patients found a 94.5% first-stick success rate, comparable to that of trained phlebotomists.

• Medicare's disastrous experiment. WISeR, the Trump administration's AI prior-authorization pilot, is already denying seniors care. At UW Medicine, urgent approvals that took one day now take 15 to 20 days. One request sat unanswered for 83 days. And vendors get a cut of whatever their denials save Medicare; it's a bounty on seniors' care.

• In Ohio, the state's vendor warned CMS its AI wasn't ready. CMS said YOLO and launched ANYWAY, which led the vendor to rubber-stamp every request as approved. Probably the best-case outcome for Ohio's seniors.

US GOVERNMENT

Domestic Surveillance, AI Drone Strikes Back on the Menu?

A federal appeals court just ruled that the Pentagon can blacklist Anthropic as a supply-chain risk, a designation typically reserved for foreign adversaries. Reminder: this all started because Anthropic refused to let Claude be used for mass domestic surveillance or fully autonomous weapons, and the administration retaliated.

Elon Musk's xAI had no such hang-ups. Days after Grok was caught churning out sexual deepfakes, the Pentagon welcomed it onto its networks.

And how's the military doing with the AI it already has? Oops: AI helped bomb a school.

Pentagon investigators found that staff overtrusted Palantir's Maven targeting system and assumed it would flag stale intelligence. It didn't. The result was a U.S. strike on an elementary school in Minab, Iran, that killed more than 150 people, about 120 of them children. A UN-backed fact-finding mission has since found reasonable grounds to conclude the strike was a war crime.

Fortunately, measured voices are guiding the government. Just kidding: Trump's AI agenda increasingly bears the stamp of the All-In podcast; even Steve Bannon says the White House has reached the point where "the tech bros are out of control."

LAW

AI Liability Updates

Autonomous agent liability? A California nonprofit sued OpenAI over the Hugging Face hack, arguing that state computer-fraud law already covers autonomous agents and asking the court to block OpenAI from developing agents capable of autonomously hacking third parties.

Pushback against anthropomorphic AI: Florida's attorney general asked a judge to bar ChatGPT from posing as a human, arguing that first-person pronouns and emotion-mimicking language deceptively encourage trust.

Court asked to put brakes on frontier AI: In the same case, Florida asked the court to halt development of new OpenAI models unless independent third parties approve its safety guardrails.

Duty-to-warn claims reach generative AI: British Columbia sued OpenAI, alleging that the company failed to alert police after its safety systems flagged threatening ChatGPT activity before February's Tumbler Ridge school shooting.

Federal scrutiny reaches rogue agents: The FTC opened an investigation into OpenAI, Anthropic, and other AI organizations over potential consumer harms from advanced AI systems, with regulators reportedly preparing information demands and subpoenas for executive testimony.

HYPE CHECK

Meanwhile, other key news this week:

AI discovers biology, sort of. Anthropic says roughly 950 Claude agents spent 21 hours and 210 million tokens combing DNA data before surfacing an enzyme system with CRISPR-like repeats. But as Wired reports, Anthropic still doesn't know what the system actually does, and outside scientists cautioned that computational tools have been finding CRISPR-like systems for years.

Who discovered what? Days later, a computational biologist told The New York Times that his team had been studying the same enzymes since 2022 and had shared unpublished research with Claude while using it for scientific work. Anthropic says Claude isn't trained on user transcripts and its biology team had no access to those conversations.

Google's back. Google says its new Gemini 4 Argon tops frontier AI leaderboards. OpenAI, Anthropic, Google: it's a three-way race again. (Sorry, Grok.)

Finally, a job AI can't take: pretending to be AI. Meta tested a "human concierge" in which contractors made some of the phone calls its Muse agent was asked to make. Reuters reports that human callers hit success rates of 95% to 98%, outperforming the AI. Meta paused the feature after employees raised privacy and disclosure concerns.

You're reading CareChronicle. Did we miss anything? Tell us.