April 18, 2022

Software Mise en Place

I am really, really into cooking. In the heady days of 2016 I regularly threw dinner parties for 30+ people. These days I don't hate myself that much, but it's still a big part of my life. I know more about my city's best grocery stores than its best restaurants and regularly bring homemade candy to conferences. I really like cooking.

In order to cook well, you need to practice mise en place: prepping all your ingredients before you start cooking. If you have to chop and sear three types of vegetables, first you get them all out and washed, then you chop them all, then you sear them. You don't start prepping one vegetable while another is already cooking.

You can think of it as organizing your cooking workflow so that tasks are separated into "low-intensity" preparation and "high-intensity" work, where you're bound by focus and tight timing. Without mise en place you're doing the easy prep at the same time as the hard work, which makes everything more stressful and mistakes more likely.

How can we apply mise en place to programming? The most straightforward translation is doing all the project infra work before writing the code, but I don't think that's always appropriate. Software is different from cooking; the task is unbound and your requirements change. It doesn't make sense to do a lot of prep that may be unnecessary.

Instead I want to apply the spirit of mise en place: getting the easy parts of tasks done at a different time from the hard parts. I got a chance to explore that last week in my work on learntla, and it went really well! Here's what it looked like:

Mise en replasitories

As longtime subscribers know, I'm updating learntla. In addition to a complete content rewrite, I'm using better technology to build and maintain it. As part of that work, I wanted to make the site autodeploy. Every time I push to master, it should trigger a GitHub Action to build the docs and push the build to an S3 bucket.

Just one problem with this: I've never used GitHub actions before. So I had to figure that out, along with setting up proper S3 credentials and getting my site to build. If I tried to learn all that with my current version of the docs I'd go crazy. I use a lot of advanced features of sphinx that could get in the way of the actions. I'd be debugging at least four different sources of pipeline failures:

1. Bugs from my ignorance of GitHub actions
2. Bugs from S3 APIs and credential issues
3. Bugs from all my weird sphinx configs and python requirements
4. Bugs from the documentation itself

What does mise in place tell me? Separate the tasks. Find a way to prep just the action file, then just the credentials, then the idiosyncrasies of my projects. After that I can integrate the pipeline with the documentation proper.¹

I created a separate repository with a minimal sphinx project. Once I got that building on every push, I added a step to push to an S3 bucket. I used a separate test website to avoid disrupting anything on the current infra. That helped me work out the credential setup.

After that, I adjusted the test project to match the specifics of my project more: docs in a subfolder, a custom theme, and python dependencies. This exposed a whole bunch of new issues I wasn't expecting. But I could knock down each one individually, as opposed to facing them all at once.

I now have the complete pipeline. I won't be porting it over to learntla until June or so, but when that time comes I'll seriously benefit from having it all worked out ahead of time. I expect there will still be issues, but a lot fewer than if I started from scratch.

And just like cooking mise en place, this actually saves time, too! It took me an hour to do all the prep work and I'm estimating it'll take another hour to integrate it. If I tried doing everything in June it'd probably take me 4-5 hours. I'm just not as productive when I'm stressed out.

Overall, I'd call this experiment in mise en place a huge success. I've started using it in other places, too. I'm writing a lot of custom directives for learntla. Turns out I can write them a lot faster in an empty sphinx project!

I'm thinking this could even be useful for integrating new software dependencies. If a dependency is sufficiently complex, it could be a lot easier to first port a snippet of project code over to a new repo, figure out how to integrate the dependency, and then backport it into the existing project.

Update for the Internets

This was sent as part of an email newsletter; you can subscribe here. Common topics are software history, formal methods, the theory of software engineering, and silly research dives. Updates are usually 1x a week. I also have a website where I put my more polished and heavily-edited writing (the newsletter is more for off-the-cuff stuff).

If you're reading this on the web, you can subscribe here. Updates are once a week. My main website is here.

My new book, Logic for Programmers, is now in early access! Get it here.