How Many Grains of Sand are in a Pile

“This code’s no good,” says Pittsford, pointing at the screen:

def update_and_get_balance(account, amount):
  database.debit(account, amount)
  return database.get_balance(account)

Sol, the author of the code, peers over his shoulder and squints. “What’s wrong with it? It adds the amount, then gets back the new balance.”

Pittsford shakes his head. “These functions are asynchronous—they look like they return right away but you need to await them or the thing they’re doing might not have finished. Like this:”

def update_and_get_balance(account, amount):
  await database.debit(account, amount)
  return await database.get_balance(account)

Sol frowns. “It worked when I tested it.”

Pittsford nods, “If you’re running this locally, then your connection to the database is going to be so fast it probably completes before you even make the second call.”

Sol is still skeptical. “I did them in this order, so… What other order could they happen in? Even if I don’t wait for the first one to finish... I mean...”

"Okay, well, look," says Pittsford. He wheels his office chair over to a whiteboard and draws the following picture:

"Until you get here," Pittsford gestures to the endpoint of the upper box, "you don't know that this happened. Maybe your packet got held up, or something, or stuck in a queue in the database, or anything else."

"But I called the function," asserts Sol.

"I mean, the function does a whole bunch of stuff, right? It pushes the new stack frame, writes a bunch of bytes into the networking subsystem, then that message arrives at the database, it might shove it in a queue before eventually processing it, then it sends you back a response, which has to go through the networking systems again, you know. A whole bunch of stuff. Stuff unrelated to the sequential execution of your program."

Sol stares for a second, then his eyes light up. "You're saying there's a whole bunch of complex, opaque nonsense that goes on."

"Well—" Pittsford interrupts.

"And the only thing we know for sure is that when we get the message back, then it's taken effect."

Pittsford nods, "yes, exactly." Then he frowns. "Well..."

Sol's eyes narrow.

"That is true here...but it's not a given. Our database is linearizable, which guarantees that." He erases the old picture and draws a new one:

"Before the events were concurrent. Now they're not. If they're concurrent, we don't know how they're ordered. But if they're not concurrent, then they're ordered how you'd expect."

"It guarantees specifically that?"

"Close enough."

"Concurrent according to whose frame of reference?"

Pittsford waves his hand. "We don't talk about that."

"Okay but how could you not guarantee that?" Asks Sol. He thinks for a moment. "Oh! Maybe your write goes to the primary and your read goes to the secondary, and the write is asynchronously replicated."

"Well, we don't use that kind of replication," says Pittsford.

"Yeah, I mean, abstractly," answers Sol.

"Sure," nods Pittsford.

Sol wheels his chair back to his desk, satisfied.

The next day, as Pittsford comes into the office, Sol rushes over.

"I got it," he says. He clears his throat and raises his hands. "Is a grain of sand a pile?"

"What?" Asks Pittsford. "What are you talking about?"

"Is a grain of sand a pile?" Repeats Sol.

"No," says Pittsford. "It's just a grain."

"Is two grains of sand a pile?" Asks Sol.

"I don't think so," says Pittsford.

"Is a million grains of sand a pile?"

"I guess so."

"So then," Sol is grinning now, "there must be some number of grains which isn't a pile, but if you add one grain to it, it is a pile. By like, intermediate value theorem. Or something."

"Yeah like," Pittsford says. "There's a threshold."

"But there isn't!" Says Sol. "That's what you were trying to explain to me yesterday: you can't understand the transition from not being a pile to being a pile as a discrete event, it's somehow smeared across each grain of sand you add."

"I said that?" Asks Pittsford. "I need coffee." He goes to make coffee and comes back. "So you're saying..."

"I'm saying that doing a...thing...in a system like this. Maybe any system...? Is like," Sol frowns. "What's that thing. The smallest unit of time."

"Time is continuous," says Pittsford. "Oh but, you mean a Planck time."

"Yeah, so, nothing just takes...nothing happens in an instant. Take a sip of coffee."

Pittsford does.

"So, there wasn't a precise instant you took the sip. The event of you taking the sip was smeared over the time between, say, you starting to take the sip and you swallowing."

"Sure."

"So," Sol draws out the diagrams again:

"We don't know how these things are ordered because they're just like, smeared over time. We go from 'untrue' to 'true' as a state of being for each of them over time. There's no 'moment' they occur."

"Well, there could be." Says Pittsford.

"Huh?" Says Sol, suddenly deflated.

"I mean...just because we don't know when the things happened, we know that like, either the balance call sees the debit call or it doesn't. Maybe it's like this:"

"We still have a total order over all the events, we just don't know what it is yet," continues Pittsford.

Sol stares at the diagram for a second, then shakes his head. "No... I don't think so. I think this is a trick question. You can't just pick a moment in time and say 'it happened here,' that's the whole point of what we were talking about yesterday."

Pittsford shakes his head. "That's not how I think about it at all. There's two ways this can happen, either like that timeline I drew above, where balance can see the debit, or," he erases the timeline and draws a new one, "like this, where it can't:"

"Exactly two. The point is that with the order timeline, there's only one:"

"Only one..." Says Sol, his brow furrows. "But this is so much like, fidelity. We don't need to actually pick a time that the events occurred at. We only have to choose a total order. Which is a much simpler object than a set of timestamps."

"Simpler? I'm just choosing a linearization." Says Pittsford.

"But your timestamps aren't canonical, I could choose any timestamp in here," he runs his finger along the box labelled "balance," "and it would give the same outcome. We should have a model where one choice is one outcome."

"You're saying that what I've done has too much freedom. I don't need to commit to these actual timestamps to say that debit comes before balance."

Sol sighs, "yes, exactly."

Pittsford shakes his head. "This is the point of the model though. We want to be able to boil things down to a point in time they take effect. We're trying to eliminate the smearing you're talking about. Being able to pretend that things happen in an instant is great—it's what we're trying to do."

Sol slumps. "You're saying there is a grain of sand that turns it into a pile. That seems...dishonest."

Pittsford thinks for a moment, considering. "Yes, maybe. That's the point of having models."

I think this is above my paygrade. This was my third attempt at writing this and I'm still not happy with it, but that's how it is on this weekly release schedule.