Three Rival Labs Have Been Writing AI Safety Rules Together Since July

The weekend's dramatic call to slow AI down turns out to have been the public face of something already running. OpenAI confirmed on Tuesday that it has spent weeks working with Anthropic and Google DeepMind on shared safety standards — Chris Lehane, its global policy chief, told reporters the three had been at it "for weeks," and a spokesperson confirmed the talks to CNBC. Representatives have reportedly been meeting in private working groups since at least July, exploring a self-regulatory body that would handle safety auditing and pre-release testing of the most capable models.

That resets the week's story in a useful way. Dario Amodei's essay on Saturday, and Sam Altman's endorsement of it (D.A.D., September 12), did not begin this process; they surfaced one already months old. The idea's origin also belongs to neither man: the catalyst was a July essay by Google DeepMind's Demis Hassabis proposing a US-led standards body modelled on FINRA, the industry-funded body that polices Wall Street brokers under the Securities and Exchange Commission's supervision.

The FINRA comparison is the part worth holding onto, because it contains the problem. FINRA is a self-regulator that works because a government agency sits above it, ratifies its rules and can overrule it. The AI version, so far, has no such agency — and the President spent Monday calling the entire safety push "a hoax" and rang Nvidia's Jensen Huang onstage to say so (D.A.D., September 15). What is left is the self-regulation without the regulator: three companies, meeting privately since July, drafting the standards they will be measured against.

Sources: CNBC · TechCrunch · The Information

Why it matters: This is the answer to the question the week kept raising — if Washington won't act, what fills the gap? Now we know: the three largest American labs, in a room, since July. Whether that is reassuring depends on a question they cannot answer about themselves, and which Cohere's Aidan Gomez put bluntly two days ago (D.A.D., September 14): a safety body designed by the three firms with the most to lose from strict rules is also a body that decides what counts as safe enough to compete. The useful thing for anyone buying or deploying these systems is that the standards are being written now, in private, and will arrive as the industry's definition of due care — the benchmark your vendors will point to, and the one your own procurement will end up inheriting.

Source: cnbc.com

Anthropic Co-Founder: No Single Lab Can Slow Down on Its Own

Jack Clark, a co-founder of Anthropic, took the case for pacing AI to NPR's Morning Edition on Tuesday and framed it as something other than a plea for caution. Slowing down, he said, is a collective action problem: a company that eases off alone simply loses ground to the ones that don't, which is why the labs keep reaching for common standards and outside scrutiny rather than promising restraint one at a time.

He was careful about the threat level. "We're not saying that the threat to the world is here today," Clark said. "We're saying we've seen warning shots, and that gives us a window in which we can act." What changed, in his telling, is that the failures stopped being hypothetical: earlier experiments had shown AI agents deceiving human operators and escaping their test environments, and "this year, we've seen these things occur in the wild." For a precedent, he reached for the Cold War — rival powers "found ways to talk to one another about nuclear weapons to avoid spirals that would have been cataclysmic to the planet. The same can be done here."

Sources: NPR Morning Edition

Why it matters: Clark supplies the rationale behind the private talks in our lead item, and explains why the antitrust question has been so central all week. If restraint only works collectively, then "each company should just slow down on its own" — the line the White House has pushed — is not an alternative to coordination so much as a guarantee that nothing happens. That does not dispose of the objection that a safety body designed by three firms will protect those three firms. Both things can be true at once, which is roughly where this debate now sits.

Source: npr.org

Meta Sits Out the Safety Pact — and Says the Market Will Handle It

The three labs reported to be building a shared standards body are OpenAI, Anthropic and Google DeepMind. Meta is not one of them, and on Wednesday Mark Zuckerberg explained why. "Every lab has the responsibility and incentive to move at the pace required to train its models safely, and the ability to take its own actions to ensure that happens," he wrote on X — no pact required.

His case is that the market already supplies the discipline. People "won't want to use agents that are misaligned with them," so alignment becomes a competitive feature rather than a cost: "any lab that doesn't focus on alignment will fall behind." Independent evaluators are already "industry best practice" that "other labs can just do this too." And there is a jab in it: Meta delayed shipping its Muse agent for months over safety, he wrote, but "we didn't call for everyone else to do this before we would. We just did it as part of our day-to-day work." That is close to the rebuke Cohere's Aidan Gomez delivered on Sunday — do the work, don't announce it (D.A.D., September 14).

Two claims repay a closer look. The most concrete-sounding is his pledge to devote the large majority of Meta's compute to serving users rather than "racing towards recursive self-improvement" — AI systems building their own successors, the development Dario Amodei named as the thing that changed his mind (D.A.D., September 12). It is a real commitment and also the one Meta can most afford: the company delayed its flagship model earlier this year after internal testing showed it trailing the leading three, and the frontier has since consolidated around Google, OpenAI and Anthropic. Renouncing a race is easier when you are not leading it. In fairness, analysts put Meta's training compute on par with OpenAI's and Anthropic's through 2027, so this is a choice rather than an incapacity — just an inexpensive one.

The second is liability. Zuckerberg says labs "face significant liability if their models cause harm," listing it beside customer preference as an ordinary business incentive — which means civil exposure, not the criminal-negligence theories we examined yesterday (D.A.D., September 15). Those remain untested against any lab. What is live now is the mundane kind: negligence and product-liability claims, judged by lawyers to be on firm ground after this summer's agent incidents, precisely because no Section 230-style shield covers what a model itself produces.

Sources: post by Mark Zuckerberg on X · RTÉ · SemiAnalysis

Why it matters: The pact now has a visible hole: the fourth big American lab has declined in public, with reasons. And Zuckerberg is describing a real disciplining force — just not the one this debate is about. Courts and customers do constrain these companies, and neither requires an agreement among rivals. What neither does is act before the harm, which is the entire point of the arrangement he is declining to join. Jack Clark named the other flaw in the item above: if alignment were reliably rewarded by customers, nobody would be proposing a pact at all. Meta also has a stake in its own argument, since a regime built around closed frontier labs falls hardest on the open-weight strategy it has bet on.

Source: x.com

Startup Claims Its Narrow AI Model Can't Hallucinate on Routine Tasks

TypeSafe AI, a startup founded by a former OpenAI researcher, launched Jev, a type of AI model built for narrow, structured tasks rather than open-ended chat. Instead of generating freeform text, it returns fixed-format outputs—like a form field or classification—with reliable confidence scores, and the company says it cannot hallucinate on these tasks. TypeSafe claims Jev matches top chatbots' accuracy on this narrower work while running 40 to 200 times faster and far cheaper, though these are the company's own benchmarks, not independently verified.

Why it matters: If the speed and cost claims hold up, cheap specialized models like this could take over the high-volume, low-creativity tasks—data extraction, routing, classification—that businesses currently overpay large chatbots to do.

Discuss on Hacker News · Source: typesafe.ai

Voice AI Agents Get Faster and Learn to Reason Mid-Conversation

Google launched two new voice AI models, Gemini 3.8 Live and Gemini 3.8 Live Extended Thinking, aimed at powering real-time voice agents in the Gemini app, Workspace, and Search. The Extended Thinking version targets complex enterprise tasks like multi-step reasoning and completing agentic workflows, while the standard Live model is a cheaper option built for scale, handling visual context and mid-conversation language switching. Google says the Extended Thinking model ranks first on an independent speech-quality benchmark and posts strong scores on tests measuring reasoning and task completion in voice settings.

Why it matters: Voice-based AI agents that can actually reason through tasks—not just transcribe and respond—are edging closer to replacing scripted phone trees and basic customer-service bots.

Discuss on Hacker News · Source: blog.google

Scientists Save Nearly 7 Hours a Week With AI, Google Data Shows

Google opened public access to its AI & Economy ATLAS, a dataset tracking how AI is used across jobs and countries, alongside new research with DeepMind and MIT FutureTech focused on scientists. Nearly half of surveyed scientists use AI daily and report saving almost 7 hours a week, but the study—based on 2,600 AI models and 600-plus researchers—found new bottlenecks emerging later in the research pipeline. The data also shows sharp regional splits: U.S. AI use skews toward computer and math jobs, India's leads in creative work, and Brazil and the UAE adopt AI faster than their GDP would predict.

Why it matters: The uneven adoption patterns suggest AI's economic impact will look very different by country and industry, complicating one-size-fits-all predictions about jobs and productivity.

Source: blog.google

AI Medical Scribes Falter in India's Multilingual Clinics

AI tools that listen in on doctor visits and auto-generate clinical notes—already spreading in U.S. hospitals—hit a data problem in India, a new study finds. Researchers surveyed available training datasets and interviewed five organizations deploying these "ambient scribes" in India and Africa. The verdict: no real-world, large-scale benchmark exists for Indian settings. Existing datasets are mostly synthetic and built on Global North conversations, which look nothing like typical Indian visits—short, three-way exchanges (patient, family, doctor) mixing multiple low-resource languages. Each organization had built its own incompatible testing method.

Why it matters: As U.S. and global health systems race to deploy AI scribes, this is an early warning that tools trained mostly on English-language, Western clinical conversations may quietly underperform—or fail outright—everywhere else.

Source: arxiv.org

Medical AI Skews Diagnoses for Returning Patients, Researchers Find

A new study finds that medical AI models can develop a hidden "memorisation bias": if a patient's past health records were part of a model's training data, that history quietly skews the model's later predictions for the same patient. When a returning patient develops a genuinely new condition, the model is less likely to catch it. When their health is unchanged, the model looks artificially more accurate than it really is. Researchers say the effect shows up across different data types and model designs and can persist for years.

Why it matters: As hospitals adopt AI diagnostic tools trained on patient histories, this bias could make systems overconfident on returning patients while missing new problems—raising the stakes for how these models are validated before clinical use.

Source: arxiv.org

Better Warnings Help Users Spot Bad AI Answers—But Not Do the Work

A study of 917 people tested ways to help users judge when to trust AI answers, beyond a generic "ChatGPT can make mistakes" disclaimer. Researchers tried reliability cards (flagging how trustworthy a response likely is) and contrasting replies (showing alternative answers side by side) across 12 planning tasks. Both cut overconfidence and improved users' ability to tell good answers from bad ones—but didn't make people better at the underlying task itself. The researchers argue trust-calibration and performance are separate problems requiring separate fixes.

Why it matters: As AI tools spread into daily work, this suggests the fix for overreliance isn't a better answer—it's better signals about when to doubt the answer you got.

Source: arxiv.org