Universal SE Topics
Resources that someone (nose goes) needs to write
Last Monday I ran a TLA+ workshop and last Friday I gave a talk on the Crossover Project. And that was the last thing I had scheduled for the year. I'm done. There will be one more newsletter with the annual end-of-year project wrapup (1 2) next week and then I'll be off until 2023. I'm feeling equal parts relief and trepidation. When you're a consultant an open schedule means you're not getting paid. Still, I'm looking forward to two straight weeks of chocolatiering and hasslin' friends.
But you're not here for challenging self-introspection, you're here for computer things! And the computer thing on my mind right now was inspired by last week's crossover talk. When discussing what software and trad engineering can learn from each other, I often bring up the Snap-Fit Handbook. You know that little panel that holds the batteries in a remote or mouse? That's a "snap fit". The handbook is an entire manual dedicated to snap fit considerations. And there's a lot of considerations! How to mold them, how strong to make them, all sorts of stuff.
My point is: what's the software engineering equivalent of the snap-fit handbook? That'd be resources focused on a very widespread, but very specific, topic in SE applications. I suspect that there's an insufficient focus on producing these resources. By that, I mean
- There's a lot of topics where there aren't many comprehensive resources.
- Making a resource of best practices, known problems you'll run into, examples, etc, is viable for those topics— there are engineers out there with enough expertise that they could produce these resources
- Said resources would be valuable to have for a lot of people.
I'm talking super abstractly rn so let's concretize this: REST API versioning. How to you design your data schemas to permit expansion and breaking backwards compatibility, how do you decide when breaking backwards compatibility is a good idea, the difference between versioning internal and client-facing APIs, etc.
Fermi estimate: let's say there are 70,000 SaaS companies out there. We can venture there are at least 10,000 places where API versioning is a problem they'll eventually have to deal with. If it takes you a full year of work to write a book on API versioning, and it saves each company an average of 10 hours to have all that knowledge condensed in one place, then the book would "unlock" about 50x your time investment. If we were a global optimizer trying to improve the industry as a whole, it makes sense to pay one person $100,000 dollars to write a book on API versioning.1
REST API Versioning is not a topic that every developer faces, but it is one that thousands or tens of thousands do, and while the topic is pretty broad it's specific enough that we can aggregate experiences and insights in one place.2
I like making up terms so I'm just gonna call this a universal topic.
Other Guesses at Universal Topics
- Representing graph data, should you use adjacency lists, value maps, objects and references, what?
- Database migrations
- Designing internal search engines
- Designing and implementing recommender systems
- Designing worker-task systems
- Authentication
- Transforming data between two formats (whether manually or with tools like pandoc)
- Tag systems
- Plugin functionality
- User profiles
- How to use specific popular APIs, like Github or Stripe.
Some of these already have resources, I know there's already hundreds of books on recommender systems. Also some in practice might be hiding enough complexity that you can't treat them as "one thing" you are writing about.
Tying it all back together: I think that if we want to further develop "software engineering" as a discipline, producing resources for these universal topics is a great place to start.
-
I should be clear that this is about 10 minutes of napkin estimates, not all companies would read it, it would save some less and some more, blah blah blah. The important thing to me, at least, is the numbers don't seem particularly out there. Even if it turns out that a book on API versioning wouldn't have that much value, you can image the tech industry growing to the point where such is the case. That tells me that if I look hard enough I could find other topics where a resource would be so valuable. ↩
-
This doesn't have to be a dead tree book. Blog posts and videos are also fine. The point is to just have a resource you can look at, as opposed to having to do the thing entirely blind or hunt down all the domain experts who can tell you what happened when they did it blind. ↩
If you're reading this on the web, you can subscribe here. Updates are once a week. My main website is here.
My new book, Logic for Programmers, is now in early access! Get it here.