If It's Too Hard, You're Doing It Wrong
Imagine we're building a web application and need a data store. Most of our employees have a finance background, so we decide to store the data in a CSV file so that we can interact with it via Microsoft Excel. We saw it in a programming tutorial, and we figure, why not? Let's try it.
This works okay in the beginning. Then we get some feature requests.
The investors want reports about our data. No problem; we've got Excel! We start writing queries. Soon we have visuals ready to share. If we need to search the data, we search in Excel or write code to search the CSV file. Everything's going okay so far.
Next, security. Our data is sensitive and we can't have just anybody poking around there. It's hard to secure a CSV file, but we try: we put it on a server with limited employee access, and encrypt it at rest. This works, although we now have two external programs– Excel and the server's OS– handling business logic.
Uh oh, we have a problem! Excel coerces data, transforming numbers, dates, text, booleans, and more into formats that work in its graphical user interface. So we have to supervise any data put into Excel, and we can't ever export from Excel back to the CSV file. We write some tortured code to handle both of those cases.
As we open this system up to the world, our CSV file needs to interact with multiple clients. How does that work? We write some buggy code that tries to handle multi-tenancy. It was hard, but we got it done.
This is starting to feel... hard! Our simple solution has led us to hand-write Excel code for queries and reports, manage server roles, code around data-conversion woes, and write custom data-locking software to support multiple tenants. That's a lot! Meanwhile, our stakeholders are wondering when we're going to ship a feature that makes money.
The reason this is so hard is that our industry has a tool that handles querying, roles, data types, multi-tenancy, and many more features, called the RDBMS (Relational Database Management System). We're re-inventing a tire store full of wheels.
"If it's too hard, you're doing it wrong." This invaluable message was written on a colleague's coffee mug at a job I once worked at. I hope this post does it justice.
Once you get competent at programming, given enough time, anything feels possible. But eventually, you get asked to do something in your domain that's really hard, and you don't know why.
This is usually a sign that you're doing it wrong. You might be solving a problem that was solved a long time ago. You might be fighting a design pattern that you don't recognize. Even though you can't explain it, something is wrong. You feel like you're missing a much simpler solution.
This instinct is invaluable. It takes time to develop, and time to learn to trust it. When you do, how should you respond? First, respect it. Second, find somebody experienced, explain your situation, and listen to what they say. You might be sad to learn that you've been doing it wrong. And happy to discover that there's a better way. As always in software, you aren't rejecting a request, you're teaching why it's hard and asking the team to be realistic about the cost versus changing course.