Bugs that literally cost money
How much does a software bug cost a company?
Well that's a messy question. It depends on the type of bug, the broken behavior, the type of company, etc. And it also depends on how we define "cost". Catastrophic cases like Knight Capital and Ariane notwithstanding, most of the price of the bug is indirect: lost consumer confidence, opportunity cost of debugging, schedule impacts, etc. These are more nebulous costs and people can disagree on how much they actually are.
That got me interested in the class of bugs where the cost is direct and obvious. And the most obvious case of "obvious cost" is when the bug prevents me from spending money. In October I started recording instances of this I've run into. Here's what I got so far:
- My apartment complex has a dry cleaning service that's controlled through an app. Every time I try to make an account, the app crashes. I do my dry cleaning somewhere else.
- When trying to get a plane ticket, the airline rejected every credit card I tried. On inspecting the JavaScript I saw that the cc validation requests returned successfully but the console crashed on an
[[Object object]]. I bought tickets from a different airline. - The Amazon counterfeiting situation has gotten so bad I first try to buy from manufacturer sites first. The manufacturer site asked me to make an account, but rejected my email. And my backup email. There was nothing for me to inspect, it just said the POST request returned an error. I bought the product from Amazon.
- A friend of mine was trying to reserve a hotel, but the "submit" button wasn't anywhere on the page. She stayed at a different hotel.
- I tried to buy a new phone case from the Google store but the checkout screen never finished loading.
- I can't link my Citibank business card to my business bank account. Every time I try I just get "error". Whenever I call in customer service gets "error". I haven't migrated off Citi yet but I am sorely tempted.
Why are so many from websites?
People are gonna blame "webshits" for these but I don't think webdev is particularly more buggy than other branches of software. I just interact with a lot more websites in my day-to-day life than other kinds of (non-SE) software, so I'm more likely to see examples of websites breaking.
(It's feasible that distributed systems are more error-prone, and I experience a bug that nobody else is. It's also feasible that the cost to fix the bug is greater than the money lost from the bug, especially if it only affects a few people and debugging it is hard to do. But you'd at least want to check to make sure!)
The critical path
The critical path in a system is the sequence of events that ends with the customer giving you money. The code supporting the critical path needs to be more thoroughly verified than the rest of the supporting code, because if it breaks then you cannot make money. Similarly, it should be monitored more heavily in production and more often audited.
Most systems will have multiple critical paths, corresponding to different products and orders in the system. Amazon has critical paths for both ordering items and registering for Prime.
Most code in most systems will not be on the critical path, due to a heuristic I'm sure has a name but I can't think of right now and I'm sure a reader will probably remind me of:
The administrative capacity required to do something scales quadratically with the size of the task.
AKA moving 10,000 people across a continent is a lot lot lot harder than moving 1,000 people. Similarly, past a certain size the amount of critical path code will be utterly dwarfed by the amount of code outside it. Bugs outside the critical path can still be expensive, but it's more likely to be expensive in an indirect way.
Outside the critical path includes "code that delivers the stuff people want to pay for", aka the actual product. And obviously everything gets real complicated in real-world systems, and plenty of bugs outside the critical path can be more urgent to fix than bugs inside it. It can still be a useful framework to think about things, though. Most people don't think of "the email validation during signup" as a really important thing to get right, but it can literally cost you money!
If you're reading this on the web, you can subscribe here. Updates are once a week. My main website is here.
My new book, Logic for Programmers, is now in early access! Get it here.