Redefining AI Safety
Hi friends --
Next month, the U.K. is holding a high profile AI Safety Summit that will "consider the risks of AI, especially at the frontier of development."
What is this frontier they refer to? Glad you asked. The AI companies have defined the frontier as "highly capable foundation models that could exhibit dangerous capabilities." In laymans terms, that means the AI models that underpin the well-known generative AI tools such as the image generator DALL-E and the text generator ChatGPT.
In a September paper, a group of AI frontier researchers declared that these "models seem likely to warrant safety standards more stringent than those imposed on most other AI models, given the prospective risks they pose."
While there might be significant threats posed by those models in the future, I and many others, including computer scientist Deb Raji and linguist Emily Bender, have been arguing that we should not overlook the safety risks of AI that is currently being used every day -- whether it is in autonomous cars or in facial recognition systems. As Raji wrote in a recent piece in The Atlantic: "Let's not spend too much time daydreaming."
In my most recent piece for New York Times Opinion (gift link), I write that rather than worrying that computers will get too smart and take over the world, we should focus on the ways that computers are too dumb and hurting us.
What's the Problem? AI is being deployed across all sorts of sectors, and there is rarely any independent safety testing before it is unleashed on the public.
Autonomous vehicles are a good example. The federal government regulates the hardware in autonomous vehicles -- they must have safety features to protect passengers -- but does not have safety requirements for the software. The states require human drivers to undergo testing before hitting the road, but AI doesn't take any tests before taking the wheel.
“There’s this weird gap between who is in charge of licensing a computer driver — is it N.H.T.S.A. or the state?” says Missy Cummings, a professor and the director of the Mason Autonomy and Robotics Center at George Mason University.
That gap is large enough for GM, Waymo and others to drive thousands of vehicles through. And now these self-driving cars are roaming the streets in San Francisco and other areas of the U.S., and making strange mistakes.
The vehicles often come to a stop when humans wouldn't -- blocking intersections or emergency responders. This year, for instance, two autonomous vehicles blocked the passage of an ambulance that was attempting to transport a pedestrian who had been struck by a vehicle. The delay contributed to the death of the pedestrian, firefighters wrote in their incident report. The San Francisco fire chief testified in August that autonomous vehicles had interfered with firefighting duties 55 times this year.
What Should be Done? We need to start thinking about AI safety the way we think about other kinds of safety -- by examining specific applications of AI rather than focusing on some futuristic idea of general AI.
Dr. Heidy Khlaaf, a software safety engineer, writes in a recent paper that AI systems should be tested for safety in the specific domains in which they plan to operate.
In other words, we should ask specific questions, such as: is ChatGPT safe to be used for medical diagnosis? (P.S. You should probably not rely solely on ChatGPT to diagnose your symptoms - its accuracy rate is only 60 percent according to a recent study, and that is likely a high estimate because it was fed "classic" symptoms).
With autonomous cars, it seems that at the very least, AI driving systems should undergo the same scrutiny that human drivers do -- vision tests, driving tests, written tests.
In other words, we need to start acknowledging that A.I. safety is a solvable problem — and that we can, and should, solve it now with the tools we have.
As always, thanks for reading.
Best
Julia