Solving the Wrong Problem
Hey!
Welcome back to another week of musings. I had a great meeting with old friends visiting the Bay Area! We also had the beginning of Daylight Saving Time.
I hope your weekend was revitalizing as well. On to this week's topic!
Was this forwarded to you? You can subscribe here!
Over the last few weeks, I've been thinking about over-engineering, and in several cases, I realized that we solved the wrong problems.
While we tend to throw "over-engineering" around in principles such as KISS, I've encountered more instances of solving the wrong problem in practice. That solution increases the complexity of the overall system.
Why?
I see a few reasons why we solve the wrong problems, and they generally repeat over time.
Solving the wrong problem always increases the system's complexity. On the other hand, it also creates ongoing maintenance and operating costs. Someone must be on-call or update dependency versions, among other things.
Starting from a solution
We encounter this the most often. People start with a solution in mind and retrofit a problem to match the solution.
This tends to happen when a new technology emerges, e.g., GraphQL, and people quickly jump to it and select it to solve all their problems; in other cases, people choose a technology that solves a very niche problem, e.g., exactly once semantics, that we don't have. While a given technology might not solve our exact problem, we choose to implement it without regard to maintenance or cost and later retrofit the existing problem to the technology.
Solving the symptoms
Another event I've seen happen is solving the visible symptoms or the visible part of a problem.
This happens when the problem is unknown or so large that it is hard to grasp or materialize into a solvable problem. I've seen people rely on bash scripts to fix issues, which alleviates pain, but the real problem might be that teams don't have enough time to fix those underlying issues. For example, we're a feature factory, and all time spent on "fixing" is considered waste.
Passion projects
This might be a precursor to the first reason.
Sometimes engineers get bored and start pursuing new, interesting problems or technologies. This type of project becomes engineering, pushing a proof-of-concept to production and making it "load-bearing." Once at that stage, it is hard to revert the decision.
Not stopping at the right time
Companies sometimes (or most of the time) fall into the sunk cost fallacy.
Companies might start by taking bets to improve the business, or we might make a technology bet that might not pan out. If we're not quick to fail or recognize that we need to stop, teams might spend months or years wrapping up a failed project.
How?
While recognizing problems is easy, it's not always easy to fix them.
Fixing problems might involve fixing your company culture or process. It might require getting buy-in from different stakeholders, hosting office hours, and training.
Ask for plans beforehand
Having people craft plans, designs, or other documents might feel weird or add unnecessary bureaucracy.
But we're trying to introduce bumps in the road for people to introduce solutions or fix a problem of their choosing. It is better to make people think more about the problem along the way than to revert to a solution already in production.
Management to help ICs between company/enjoyable projects
On the aspect of people being bored and pursuing passion projects.
We need to find the reason and ask their managers to work with them to find projects that align with their and the company's goals. Or reach some time arrangement with them.
Conclusion
Fixing the wrong problem is a costly problem, sometimes worse than never fixing the issue.
We must develop a culture of pragmatism and clarify the scope of problems to solve. We need to find the right problem to fix, which might not be as cool as implementing a new technology and instead might look like sitting in a meeting with management.
I'm guilty of the problems I called out here. Nobody is perfect. But we need to make it easy to fail fast and iterate.
Your turn!
How do you avoid fixing the wrong problem or ensure that you or your team are fixing the right problem? Reply to this email to let me know!
Happy coding!
Things I discovered in the past week
- JSDoc as an alternative TypeScript syntax. Lately, I've been poking at this idea of using a Typescript compiler but with JSDoc annotations instead of full
tsfiles. - Clarifying architecture with Service Blueprints: system operations + entities. I've also been in a few meetings in the last few weeks where these service blueprint diagrams help clarify requirements and understand the scope of a project.
website | twitter | github | linkedin | mastodon | among others