CAP is Good, Actually
It seems like there are two main takes regarding the CAP theorem online:
- In introductory materials, it is presented as a deep, fundamental truth about distributed computation. This mystique is often heightened by an accompanying "at first it was the CAP conjecture, and then later, it was proved, and so now it is the CAP theorem."
- Blog posts and takedowns about how CAP doesn't reflect the realities of a real-world distributed system.
To me, both of these sort of miss the point, but the presentation of the CAP theorem is sufficiently troubled it's not that surprising that problems arise.
First, let's quickly review the CAP theorem. It states that no distributed system can have all three:
- Consistency (here, this means linearizability),
- availability (every node in the system can always accept requests), and
- tolerance to network partitions.
Off the bat, this should probably strike you as a little weird. The presentation is "pick 2," but one of the options is "doesn't break sometimes." People love "pick 2"s. I think they appeal to some symmetry-loving parts of a programmer's brain. But what Brewer was trying to communicate here is probably better expressed as:
If you want your system to survive a network partition, you must give up either C or A.
But that doesn't tickle people's symmetry centers the same way, and doesn't lend itself to a punchy acronym. It's entirely possible that if this had been the presentation, the CAP theorem would not have become as ubiquitous as it is today. This is my main complaint with CAP as an idea put forth. But the title of this post is that CAP is good. So what's my point?
I think CAP's real problem, the thing that dooms it to the thinkpieces, is the fact that it's a mathematical claim masquerading as a software engineering claim. Or, rather, its "obvious" residence in the realm of systems leads people to think of it in the kind of fuzzy, heuristic terms that dominate software engineering discussions like "coupling," "cohesion," "velocity." In a lot of ways it has suffered the same fate as Gödel's incompleteness theorem or Turing's resolution to the Halting Problem, doomed to forever be invoked by Hacker News commenters in contexts where it's inappropriate.
This is the crux of the problem to me. Thinking of CAP, or any sufficiently precise statement, as some kind of "grand unifying theory of distributed computation" is never going to work, because CAP is a very precise statement regarding very precise preconditions under a very precise model. And it is good at that! And under that model, it's just formalizing intuition you would come to yourself with a little bit of thought. CAP is really just saying if friend A asks you what friend B wants for dinner, and friend B isn't answering their phone, you can either say "I don't know" or make something up.
The value of tools like this isn't to have them as a fundamental, "end of the story" conclusion about designs, but as constraints to quickly prune search directions that won't pan out.