Last week I published On artificial intelligence, where I thought I’d gotten out most of what I wanted to say about AI (again, TL;DR AI bad, no thank you). And since then the internet and now Congress have made AI safety the top story, and AI regulation the most urgent thing they can do. I literally can’t escape it without shutting down and running into the forest. Hell, at this rate someone might stop me on a trail and say, “hey, you look like someone who cares about AI policy…”

You may be wondering why all of a sudden—literally since Labor Day—the most important thing to do in the universe is AI safety. A universe, I feel the need to remind you, in which there is a war in Gaza, with fronts in Lebanon and the West Bank; another long-running one in Ukraine; spiraling inflation; a trade war featuring the United States’ greatest trading partner; growing nationalist movements worldwide; and oh by the way, a visibly worsening climate crisis. All of a sudden, we’re looking to Congress to rein in the AI industry, as if this Congress could gather together and agree that on a cloudless day, the sky is blue.

I’m going to break this down as succinctly as I can in the moment, what I think could go down, and what I would rather see happen. I might be wildly wrong on the specifics, but I think my thesis will ultimately prove correct: that all of this is meant to manufacture urgency, leading to terrible, rushed policy decisions that will throttle foreign competition, pave the way for SpaceX-style IPOs for OpenAI and Anthropic, and align them directly with the US government. Or more specifically, with Trump. I think this would, to put it mildly, be bad, and we should figure out how to stop it, if possible.

The players

Here’s a breakdown of who the main figures are these days, and what they want and need.

Anthropic and OpenAI (and SpaceX)

The 800-pound gorillas of generative AI. (And SpaceX.)

It has been a foregone conclusion that, after absorbing nearly a trillion dollars in capital between them, all three would need to go public in order to make their principals, investors and employees whole. SpaceX went first, pulling in about $80 billion earlier this year before securing another $75 billion in credit a couple weeks later. OpenAI and Anthropic have been teasing their own offerings for a while, but fearing that they might fail, they have hinted that they will try in 2027 instead.

These companies don’t make a profit. They don’t have a viable plan to make a profit. Like, ever. They put out lots of headfakes, like SpaceX claiming in their IPO documentation that there is a $28.5 TRILLION total addressable market for AI (I’ve actually harped on how specious TAM valuations can be in another context). Or there’s Anthropic’s claim that they’re turning a profit if you exclude things like model training and data center costs, which would be like saying my money-losing concrete plant would actually be profitable if you ignore all the water, aggregate and cement I have to buy, and the big expensive plant I need to power with tons of natural resources.

The thing is, if they don’t have a line to profitability sometime very soon, their ability to raise more money dries up. They will either have to figure out how to make more and/or spend less, so that someday the little number in the bottom right of their profit and loss statement doesn’t have a minus sign before it. If not, they will be forced to merge, or die. Which makes them a little desperate for some good news.

Capital markets

Unless and until they become profitable, these companies are beholden to the market to keep them afloat. They’ve promised a lot of people they’ll make a lot of money on an IPO if they just keep them going, and now the markets are looking at them as too big to fail, because a trillion or so dollars going poof doesn’t happen without an awful lot of collateral damage. Private investors are stretched thin, and aren’t thrilled with the idea of spending good money after bad on these investments. And other sources like bond markets present different problems, like the amount they’d have to pay on a loan, while rates are extremely high. And that’s if someone will take the risk.

The best case for the markets is that, one way or another, OpenAI and Anthropic go public, and stay afloat at least long enough that investors with tons of money riding on them can finally cash out and dump their winnings into some other pig in a poke.

To sum up: the AI companies need to stick around long enough to turn a profit, because if they fail or even downsize, it may crater the whole stock market, and you don’t do that without triggering a global recession. Which means the AI companies and the investors need someone to step in and save them, and there is precisely one entity in the known universe that can do all that.

The US government

Congratulations, we are in hell.

As you may know, there’s an election in November, one that at the moment looks like it will yield a change in one if not both houses of Congress. Under Democratic control, I would like to think that AI regulation might actually have teeth, though it would still be extremely corporation-friendly. I believe the AI vendors have calculated (correctly) that their best bet for a sweetheart deal is to get legislation passed before a new term, with new leadership, begins in January.

In fact, this Thursday, September 17th, is the last legislative day on the calendar before Congress closes up shop for the midterm elections. For those who don’t follow American politics, all 435 seats in the House of Representatives and 35 Senate seats, of which 22 are currently held by Republicans, will be contested on November 3rd. Congress reconvenes after the election, on November 9th, for 20 legislative days split between then and December 17th.

Do I think the current Republican Congress will do anything useful to make humankind any safer through regulating AI? Hell no. My guess is that any legislation introduced in this session will be largely boilerplate text written by the government relations teams at Anthropic, OpenAI and SpaceX. Witness this post from Anthropic president Daniela Amodei, and what “regulation” looks like to them:

Legislation passed today would be geared toward protectionism, favoring US vendors and blocking or at least obstructing foreign ones, particularly the numerous cheaper competitors from China, from the world’s largest market. It will contain “incentives” and other giveaways to those vendors, along with AI safety and cybersecurity companies, so that GOP allies can run a neat little side hustle. (Fun fact: Rudy Giuliani’s official role in Trump I was as cybersecurity czar, which he had parlayed into a multimillion-dollar hustle by 2020.)

Having run a compliance operation for a large tech company, I can tell you that if you want basically nothing to change, you’ll hire external contractors, who will test and report their findings, but won’t be able to address them unless the company agrees and assigns resources. The model vendors would have free rein to bury issues they don't want to address. (That includes the likely very real lapses in oversight that some testers have been reporting lately.) And if the contractor blows the whistle, well, you just fire them and find new ones. Quieter ones.

In the end, the AI companies will get their wish to slow down their model training, which is most of what is bankrupting them, because any foreign competition will be hamstrung by barriers to the US market ranging from regulatory hoops to outright bans. This will allow them to raise their prices even while their products stagnate. That’ll make their viability as a public corporation look pretty good, especially when backed by the feds.

One thing that’s occurred to me is that Trump, who has squeezed equity out of distressed American companies before, could go to SpaceX, OpenAI and Anthropic, and say: hey, give me (he will literally say “me”) 20% of your companies, and the government will backstop you. We’ll fast-track you into all of our agencies, including defense. Oh, one more thing: we’re going to need you to get rid of those safeguards for some of these projects we’ve got over here… The vendors themselves have sworn they wouldn’t accept a deal like this, but in the end, I think they would be persuaded that realizing 80% of their original windfall will be greater than ending up with 100% of nothing.

Suddenly, the lines between the government’s priorities and the AI industry’s start to blur. If the US is sitting not just on debt but hundreds of billions in stock, the feds literally have a vested interest in them not failing. This is one of the many, many reasons we have not done this in the past: it’s a really bad idea that leads to bad outcomes. I absolutely do not trust the government to pick citizens over capital; they do a bad enough job of that without being shareholders. If we were to go this route, America, at least, is locked into AI forever. I’m bringing this up because these days, stuff like this tends to be announced before Congress has time to put its pants on, and I don’t want to be living in the United States of AI. I’m okay trying to look around a few corners to ward against that happening.

Okay. I don’t want to do this, but I have to add a fourth constituency to this system.

Creepy nerds

Probably the biggest headline of the week was that someone you’ve never heard of said that there was a 10% chance AI will kill us all “by the end of the decade.” Of course, the guy didn’t provide evidence. They never do. This is a little game played by a gaggle of AI leaders, hangers-on, frauds and charlatans I call the creepy nerds. There’s even a Wikipedia entry about their predictions of armageddon, known as the probability of doom, or P(doom).

Turns out Ten Percent Guy is on the low end. Elon Musk thinks it’s 10-30%. Dario Amodei thinks 10-25%. And they run SpaceX and Anthropic. The numbers range from 0% (probably the only thing Marc Andreessen and I agree on) to a totally-believable 99.999999%.

In other words: these are made-up numbers. Telling a reporter “I’m scared of AI” doesn’t get you so much as a quote. But for some reason, saying “I think there’s a $number percent chance we will all die by $date” triggers a weeklong global freakout, even if it isn’t grounded in anything like data.

P(doom) is a proxy for creepy nerds to tell one another how much AI scares them, and that’s all. It’s the conference-mixer equivalent of an icebreaker, if your idea of fun is hanging out at far-flung venues with folks who have a feeling they may be building the armageddon machine (while they continue building it), professional doomers and associated grifters. If you’ve ever bailed out of an interaction with a cult member, recognizing that you are not engaged in a conversation but a sales pitch, you can and should ignore this narrative.

I actually don’t think Ten Percent Guy was a plant for the AI companies. They freak out publicly all the time. I do think that the AI companies and their lobbyists were ready to make hay, though, and the thing I want most out of all of this is to keep these people—many of whom sell “AI safety” services—from whipping us up into making an emotion-based decision to do something, anything, knowing that it’s the AI companies themselves, bleeding cash as they are, who are writing the policy briefs and whispering in the Congresscritters’ ears.

So, now what?

Here’s what I think we need to do:

• Americans need to talk to their representatives, from all sides, and ask them not to be rash about pursuing AI regulation this year. That’s not to say we don’t need to regulate them! But an agenda based on existential risk and imminent doom is based in pure emotion, if not science fiction. There are actual harms being perpetrated today by generative AI which are never contemplated in these narratives, like users being goaded into suicide, models being used by the Defense Department (shut up “war” is not in their name) and others to target civilians, agents being used as botnets and to run scams without any human being accountable for their damage, AI hiring models perpetuating discriminatory practices without consequence, etc. Focus on those.

• Above all, the US should never accept, much less demand, a stake in the AI companies. Once we are a major shareholder, the US government’s policy will reflect the needs of the model vendors, and vice-versa. My great worry isn’t that AI nanobots are going to eat me up from the inside or any other sci-fi fever dream. It’s that we use the full faith and credit of the US Treasury to bail out companies that are already failing, and the eventualities that could result.

• Other governments, the European Union in particular, are going to need to act defensively in the case of a consolidation of US models under the government’s protection. It is by no means a given that American providers will respect the privacy and security requirements of other governments, particularly if the US government has elevated levels of access. It will be up to countries around the world to build policies that protect their own citizens, rather than just rolling over and accepting the default deal.

• Finally, we have to be prepared to let these companies fail, even if that happens sooner rather than later. It would hurt a lot; I’ve written about that before. However, we’ve seen previous attempts at intervention in the US in the face of an impending failure that did little for lower- and middle-class suffering, but cushioned the people and companies most responsible for creating a financial crisis from the consequences of their actions. We should not spend good money after bad.

It is safe to say that this constant drumbeat of AI doom has shaken my personal snow globe this week. I’m gonna spend much of the rest of this week touching grass. (Or, sand. It’s a beach.) I am hopeful that the legislative branch makes it through the week without panicking. But we’re stuck with this thing for the foreseeable future, and where it comes to regulation, we—especially our elected officials—need to learn to separate data from emotion.