Reinventing the Wheel
Hey!
Welcome back to another week of musings. I hope you had a refreshing weekend.
Also, it's a long weekend here in the US, and I might be enjoying a mimosa at some brunch place. Who knows?
Was this forwarded to you? You can subscribe here!
Things I discovered in the past week
- The Silicon Valley Canon: On the Paıdeía of the American Tech Elite is an article about the books that define the Silicon Valley spirit. It was inspired by this tweet from Patrick Collison (co-founder of Stripe).
- I recently started trying out perplexity.ai to avoid "googling" things. Instead, I ask perplexity about these.
If you've worked long enough within a single team or organization, you'll notice that eventually, the idea to write a piece of software from scratch will come up—not like some novel thing that nobody has done in the past, but rather something that exists in the world and has multiple open-source options.
Some cases might be a logging system, queue system, flow "orchestrator" (think like the Saga pattern), your team might have some specific example, like a framework, or library. But you get the idea.
I'm inevitable
I've seen this happen multiple times and participated in some of them. Anyhow, I was wondering why we do this and if it's possible to prevent it in some capacity. I think that maybe it's not likely to stop or avoid because, after some time, we get so used to the ins and outs of our problem space that every nuance of our system might feel like a deal breaker from the perspective of getting an off the shelf piece of software, or we might be using one already. Still, we see all the places where it doesn't cover our use cases.
On the opposite side of using off-the-shelf software is the misguided idea that building a custom piece of software is not that hard. How hard can logs be? They're put in some place, and we need to move them to another place.
Other things also come into play, like whether your organization rewards coming up with new ideas to implement, whether new projects are hard to come by, whether the team is bored of bug fixing, etc. Implementation seems simple enough that we can sell management on it. Since our view of the problem is simple, our draft for a project plan seems simple enough that we could convince management that we can invest X amount of months to implement and then reap the benefits from there onwards.
Devil is in the details
In practice, the pieces of software we want to reimplement are complex problems, and there are reasons why some of these problem spaces have whole companies dedicated to selling the solution to other companies.
Other times, the devil is in the details, and once we start implementing, we encounter network partitions, availability issues, and consistency issues, among other things. From an operational perspective, it was really not that simple to implement or maintain. I've seen implementations often start, and then the team is required for a new feature, a moonshot, or similar, and the project gets "defunded," but since it's simple, one or two people can keep working on it. We end up with a half-baked migration.
Help from management
I've seen even senior engineers fall into this. Bored engineers will always find reasons to rewrite or create something new.
I've also talked with managers who allow this behavior via "guard rails," like let's define a point to reassess if this is going the way we want or if we're getting the return on investment. Because of boredom, the company might not have shiny new things to work on, and the teammates might want to leave for another place. It's better to keep people working on these exciting problem spaces for some time while a new project comes along.
Your turn!
Let me know how many times you have seen the reinvention of the wheel by a team, your team, or you even. Have you ever had to try and stop these types of projects? How did it go? Let me know by replying to this email!
Happy coding!
website | twitter | github | linkedin | mastodon | among others