AI Skeptics - Hyperscale (reflections on chapter 4 & 5)
Hello. In about 3 hours, at 3pm ET, we're going to get together and talk about Hyperscale.
I read chapter 4 ("Water") and chapter 5 (Climate) this week.
I was a little surprised that Marx didn't mention closed-loop cooling systems in the chapter about water; I've got some thoughts to offer on that (... again, under review, so not here, but during the video chat), but while I was reading chapter 4, I figured out something that felt really clarifying that I might want to work through during the video chat, or maybe I'll just write it here, share it, and then feel stupid about how obvious it is.
Something I didn't really appreciate, for really quite some time, about the need for water in data centers is that the data centers don't really need water; they need something that has high specific heat, that is very fluid, and that can safely be used to carry thermal energy from one place to another. That's basically just water and a few other things.
Marx talks about that a little in chapter 4 with evaporative cooling (page 74ish I think), and I think he names a couple of alternatives (like air cooling), but as far as I could tell he wasn't alluding to some of the more exotic options that are becoming more popular talking points these days. Submerging things in oil and some other stuff are examples.
The way I described it to a friend was this: imagine you've been cooking on the stove and you want to put your skillet or whatever in the sink, but there's some plastic containers in there and if you immediately set the hot skillet down, you're going to melt the plastic containers. So you want to cool it down rapidly
(nb: generally not a great idea to force extremely rapid cooling like this with certain cookware, because it can cause stress on the material and crack it; but for the purposes of this example, let's ignore that)
So maybe you'll run some cold water over the skillet to cool it down, producing a ton of steam and rapidly bringing the skillet down to a cool temp. You've just invented evaporative cooling.
The reason water is such a central element for data centers is because tap water is an unbelievably cheap way to carry thermal energy away when used for evaporative cooling. I asked a friend to check the temp of the water coming out of the tap where they lived: it was 61F in Michigan. In Texas where I'm at, the tap water is 74F. That means that the water coming out of the pipes in MI can literally absorb 13F more energy before it turns into steam.
Let's say you didn't want to run the kitchen faucet so much - you wanted to avoid producing a bunch of steam and sending water down the drain, so you get a big tank of water and you just cycle that repeatedly to cool the skillet. That tank of water circulates and you just use it whenever you want to cool your cookware down. Congratulations, you've invented closed-loop cooling.
If that tank of water stays hot (let's say 90C), it's not very useful for cooling GPU chips hot skillets; so you want the tank in your fridge, and you want the fridge cooling that tank. You've now kind of homebrewed most of the technology that closed-loop data centers use.
The problem is this uses energy. Actually, I'm underselling it a little; this uses a ton of energy. Your refrigerator's condenser is going to run almost constantly trying to turn the thermal energy in that tank - several gallons of 95C water - into several gallons of 5C water. You can do high-school level calculations to see how much energy there is in like 10 liters of 95C water vs the same volume of 5C water, and (accounting for some inefficiencies) get an estimate of how much energy needs to go into moving the thermal energy from the tank of water outside of the refrigerator.
Overwhelmingly, the majority of energy production in the world is just steam engines. Coal, oil, and nuclear all produce energy by turning water into steam and forcing turbines to start spinning, producing energy that gets moved around. So in a very literal sense, if you're taking responsibility for cooling the 95C water tank rather than dumping it down the drain, then you need to produce the energy to constantly be cooling that tank, which requires enormous amount of water to turn into steam to power the generators. The amount of water necessary to produce that much energy is enormous, to say nothing of the emissions and everything else when you run a coal, oil, or other power plant.
All of this for the appearance of not sending water down the drain, but which you now know is really just putting the water usage out of sight, and doing it at a cost of burning considerably more oil, coal, etc... is a ridiculous shuffling of resources, all things considered.
I wrote about a kind of "problem-space arbitrage" where tech folks translate one problem into another that feels more amenable to their domain or technical expertise, even if it incurs some sort of penalty, because they believe they can make more progress within their preferred domain. Without meaning to stretch the use of this term too much, I feel like there are echoes of that here. Maybe this is very trite and obvious to everyone, but I think reading chapter 4 helped me face the simple, pretty much inescapable problem that is not being dealt with, only impulsively and somewhat disingenuously moved around.
There are also bits of this in chapter 5 where Marx talks with Ketan Joshi, who explained how tech companies will make promises like investments into green tech, but those promises are either vacuous, or convenient (the power plant would've been built anyway), or they don't actually operate for very long. Reminds me of carbon offset credits, where someone promises to plant a tree to offset the emissions they're producing today, but that tree won't have any appreciable effect on carbon levels until it's about 10 or 20 years old, and most of those planted trees won't make it that long.
Can't say much more than this in text, but maybe we can talk more about it during the video chat at 3p ET.
When you're ready to join at 3p ET, you can click on this link or you can go here:
https://al2.in/ReadingGroupRoom
See you then.