Questions Raised by Fable & the Problem with Prompts
Hi all,
The AI ecosystem has been moving quickly for the last several years, but last two weeks have been especially dizzying.
In a nutshell, here’s what happened over a handful of days:
Fable launched, sparking a now-routine round of existential dread. With each model that makes a big leap in capabilities, the reaction is one of excitement and unease… Especially among engineers. Will this be the model that renders me obsolete? Fable’s unmatched ability to run long felt like an o1/o3-level leap. If Fable is this good, the only requirement to produce software is money.
The AI research community discovered Fable’s guardrails, which horrified them. In addition to refusing to answer security or biology questions, Fable silently delegated to the older Opus for tasks relating to AI systems development or training. The reaction among researchers and engineers was strong enough that Anthropic apologized for the ‘silent’ nature of these limits, while leaving the core behavior in place.
Enterprises held off on adopting Fable until they figured out what data Anthropic was retaining and what they were doing with it. Every large organization I spoke with hadn’t enabled Fable, because Anthropic was requiring a month of data retention to check for bad actors. For these companies, this requirement was a legal no-go. But any FOMO big company engineers felt was quickly alleviated, because…
Anthropic withdrew Fable after the U.S. government banned it from foreign use. Engineers considered sitting idle, wondering if Opus was worth their time. Rumors spread about who drove the ban and how Anthropic responded. We were all in shock and, with less urgency to put Fable to use, had plenty of time to reflect.
Taken together, the whiplash of this story drove home one core anxiety: what if you can’t access to the best model?
The “you” here is AI researchers, security researchers, people with less disposable income, private corporations, Europe, Canada, India, and pretty much everyone not at the leading lab.
This spurred a round of calls for investment, regulation, and building systems that are “model agnostic” (more on that below). I don’t expect this question to go away, even if we have continual progress from open models. (Especially if we have continual progress from open models…)
You can’t be model agnostic if you’re hand-writing prompts.
There are a few recent events that may have spurred you to think about changing models. Perhaps it was when Anthropic was forced to pull a model due to government order. Or perhaps it was because you saw your Opus bill. Or perhaps it was that 5th email from your inference provider telling you Llama-3.1-8b or GPT-4o-mini is being deprecated.
It’s easy enough to change an API parameter, to point your application at a different model. The hard part is making sure nothing broke and all your hard-won hot-fixes don’t regress.
In this piece I introduce the idea of “Prompt Debt”, a situation I see plaguing so many teams. The plain-English prompt that makes prototypes effortless turns out to be a poor way to specify how a system should behave, and the bill arrives slowly, disguised as ordinary progress, until the application can barely move.
Read “The Problem is Prompt Debt” to learn how we get trapped, why this happens, and what we can do to avoid the problem entirely.
For the last few months, I’ve been keeping a running list of tips for agentic coding: guidelines or rules one might give to someone just getting started with Codex, Claude Code, Pi, or any other agent. Here’s the collected 10.
What Do Humans Need from Docs?
When an agent can answer your questions about a library, do we still need to write docs? Oddly, more people are writing docs than ever, we just call them “Skills.” But Skills are for agents to read (though (curiously) they are often better than most docs intended for humans).
So is there still a need to write docs specifically for humans to read? If so, what is the job of human-centric docs?
Art Break
It’s hard to find information about this collage, “36 Ticket Stubs,” by Walker Evans. Mostly because Evans is a best known as photographer who documented the Great Depression. He was employed by the Resettlement Administration and Farm Security Administration, both part of the New Deal.
Collages came later in life. “36 Ticket Stubs” was assembled in 1975, the year Evans died. I wish there was more info; I’m fascinated by someone who took such a hard pivot into abstraction and assembly after a lifetime of impactful documentary work.
Until next time,
