Back up your arguments plz
A couple days ago I read Putting devs before users: how frameworks destroyed web performance, a rant about why the web is so slow. You don't have to read that to understand the rest of this post, but it might give some background context. The author, "cheatmaster30", makes the following claims:
- Websites are slow because of web frameworks.
- Developers use web frameworks because they are more interested in "having fun" than appealing to customers.
- Developers should stop "putting [their] own interests above [...] users" and start writing performant webpages.
I might make it sound just a little bit smug, but if anything, I'm toning it down. The original article calls webdevs "McDonalds line order cooks who fancy themselves as Michelin star chefs". That was the first thing that leapt out at me: a person treating webdevs like petulant little children for choosing convenience over performance... and sharing the screed on Medium.
But that's not actually the main problem. While I really don't like smug writing, being a smug writer doesn't undermine his core claim. What undermines his core claim is that he doesn't support it whatsoever. Cheatmaster30 says a bunch of sweeping statements, but doesn't provide evidence that anything he says is actually true.
I've listed some of his claims, explicit or not, in bold, followed by how he would have to show that the claim is true.
Websites are slow. This seems like it'd be a gimme, but it's not. We all know that websites are slow, but what do we mean by "slow"? If we're trying to argue policy, we need a technical definition of what "slow" means.
The original essay doesn't actually talk about "slow"! It talks about webpage size. He accuses javascript libraries of adding megabytes of bloat. This is not what most people expect "slow" to mean! Nonetheless I'm going to go with that definition to keep in line with the essay.
The slowness is from web frameworks. There's two steps to this. We'd need to show that frameworks correlate with slow webpages and also that frameworks cause slow webpages.
For correlation, we'd sample a set of webpages, categorize them by framework used, and measure their page sizes. If the claim is correct, we should see most of the slow sites use frameworks, and most of the sites without frameworks run fast.¹
(To be thorough, we also need to show it is any framework that causes problems. Our correlation test shouldn't show that only certain frameworks lead to slow sites.)
Causation is harder to show. In this particular case we're lucky: we're using "slowness" to mean "large website size". The overhead for frameworks should dominate over other factors, like images, CSS, analytics, trackers, etc. If 90% of the page bloat is images and 10% is javascript, then it doesn't make sense to blame the framework for bloat.
Removing the framework would speed up the site. Maybe sites with frameworks are slower because they have more features! We need to show that writing equivalent Javascript without the framework won't lead to equivalent bloat. Best way to do that would be to do a rewrite and show that the native JS takes less space. We could use the realworld app comparison project as a data point.
Developers do not have technical reasons to use frameworks. Cheatmaster30 lists three reasons developers use frameworks: they are convenient, they are fun, and they are trendy. Are these actually the reasons, though? Maybe developers use frameworks because they're better for security, or they provide incredibly complex-but-necessary features, or some other purely technical reason. This is a question of why people make the choices they do, which is much more difficult to figure out than what choices people make. It's a sociological question.²
I can see two main ways of answer the why. First, surveys. We survey a lot of people and ask them the reasons they use react. Surveys are quick and easy but give us really bad data. If we wanted to be rigorous, we'd sit down and interview people. We'd also follow people around as they made decisions, doing an ethnography of sorts. Better than a survey, but a lot more expensive and time-consuming.
We could probably also do behavioral studies, but those are notoriously hard to get right. And programmers don't trust them, because "it's not a real project".
Developers do not have business reasons to use frameworks. The argument is framed as "developers care more about being trendy than being good to the business." That falls apart if using frameworks has a business value. "Convenient" has a lot of business potential: frameworks could make it easier to add more features. More features = happier customers. "Fun" is also arguable: happier devs are less likely to leave or burn out. "Trendy" is indefensible so let's throw that out.³
We now need to show that "convenience" and "fun" are not adequate business reasons. We could compare the feature release rate of sites with frameworks to sites without frameworks. We could also try to replicate the features of a site without using a framework and compare it to the time that original site took. We'd need to get development data from the site. Most companies don't track that, and if they do, they aren't gonna give it away to internet randos.
(A counterargument here is that performance issues make customers unhappy. I think this is true, but we'd have to empirically show that is the case to use that argument.)
How much did cheatmaster30 do?
Zilch.
Why don't people do this?
Because it fucking sucks.
Doing things this way is a shitton of extra work. The original essay was 1100 words. He could probably hammer it out in an afternoon. By comparison, this rant of mine, about what the evidence would look like, is 1337. And I didn't even do any of those things! If he wanted to make the rant rigorous, he'd need to scrape dozens of websites, compare all their bloatloads, run a survey, interview frontend devs, replicate all the code in native js... what are we talking? Weeks? Months? And good luck making a point after all this. I bet that if cheatmaster30 did all this extra legwork, he'd have to title his piece "How a whole bunch of different things destroyed web performance, one of which might be web frameworks, maybe."
As a point of comparison: back in 2018 the npm world had a major security incident when a malicious actor took over the event-stream package. Lots of people had their hot takes out in a couple days, with the more 'thorough' ones getting in within a week. I decided to do an in-depth analysis of the attack. It took me over a month of full-time research and ended up being over 8,000 words.⁴ I don't even know if it made a difference! It definitely didn't get the same reach as the first hot takes. So if you actually want to change people's opinions, it could be that spending too much time gathering evidence is actually harmful.
It really frustrates me when people don't do the legwork, but at the same time, I can see why most people don't. On one hand, you have the convenience of being able to share your thoughts in less than a month. On the other, the ego boost you get by knowing you were thorough. For me the ego boost is more important. For a lot people, the convenience is more important.
¹ "Fast site don't have frameworks" is different from "sites without frameworks are fast". In this case, for correlation, it doesn't matter too much, but implication operators are nasty yo
² Why is social science considered a "soft" science? Because if you tried to make it a "hard" science you'd be shot for war crimes.
³ Okay I lied, there's a couple of arguments for why "trendy" is good. Using trendy technology could make it easier to raise venture capital. See the "blockchain" fad a couple years back.
⁴ https://www.hillelwayne.com/post/stamping-on-eventstream/
If you're reading this on the web, you can subscribe here. Updates are once a week. My main website is here.
My new book, Logic for Programmers, is now in early access! Get it here.