Software as a Serf: The Duolingo Case Study
Software as a Serf, in Review
The first part of this series covered how I will conduct my analysis of software-as-a-service, or SaaS. I will focus on a framework I have developed called “ownership as a set of rights.” In particular, I posit that owning a SaaS subscription, as opposed to owning a software-as-a-product, or SaaP, the traditional software-ownership model, bars the owner from the right to maintenance, that is, the ability to maintain the tool in a certain state regardless of changes that the software provider would like to make to the software.
This right to maintenance expresses itself in two scenarios: when the SaaS provider elects to update the tool and make changes, and the (literally) ultimate change, when the SaaS provider elects to discontinue the product. Software developers, and software users, like SaaS in part because it keeps the code base on servers easily accessible to the developers rather than on the multitude of machines that users own. This makes the code base easier to modify, and as such, SaaS is synonymous with routine changes. But changes, even ones that benefit the majority of the user base, will often present controversy to some subset of users. These users, because they lack the right to maintenance, rarely can remain on a version that they preferred. Furthermore, since the SaaS’s code base resides on remote servers rather than on user machines, it makes using the software a more fragile affair. If the SaaS provider were to go bankrupt, or merely decide to discontinue the product, they would take all servers off line and thus yank the software entirely from users.
Then, in the second part of this series, I examined the impact of software-as-a-service when attempting to preserve software that has cultural value, like WoW or Adobe Flash. Here, the right to maintenance was shown to be a crucial part of our ability to preserve artifacts of cultural importance, and its absence under the SaaS paradigm has made preserving culturally important software more difficult.
In today’s post, I will consider Duolingo. Duolingo was the exegesis for this series. Initially, I had wanted to write a single brief post (notably, not multiple, lengthy diatribes) lauding Duolingo and highlighting its impact on my life. But in analyzing Duolingo, what struck me was that many of the valuable outcomes I have had with the app came from an older version of the tool, came from features which have changed or been removed from the application in recent months. Then, to discuss Duolingo and its benefits, I would have to discuss a product which simply no longer exists. This, ultimately, lead me to consider how SaaS operates, what the right to maintenance looks like, and how it impacts us as we pursue the things valuable to us.
Finally, we can talk about Duolingo.
The Duolingo Case Study
The Stakes of Duolingo
With Duolingo, I want to specifically examine the right to refuse an update. Changes made over the course of my using Duolingo have had negative repercussions for my ability to learn, and my inability to refuse such changes then has been a drawback of this software’s service.
That value of a Duolingo case study is the relative simplicity of the tool. Unlike something as sophisticated as WoW, a complicated work that must at once engender community, entertain, and produce cinematic experiences, Duolingo has only one job: to teach me Italian. Whereas with WoW, tracing out the impact of an individual change is a complicated affair, something which took the ever-lucid Dan Olsen a forty minute video to even begin to scratch the surface of, Duolingo uses well-studied gamification principles and language-acquisition techniques, and so identifying the impact of a change to its features is relatively easy.
It also plays at a different moral angle. While cultural preservation I think to be of inherent value, it’s tough to make a case that the world is worsened because Blizzard Entertainment made changes to the orc game. We like to preserve culture, but it’s never our number one priority. Language instruction, on the other hand, is for some a very means of survival. Duolingo likes to advertise the fact that Swedish is the most popular target language in Sweden because refugees from Syria and Afghanistan who have been forced to migrate to Sweden use Duolingo to learn the target language. So, if Duolingo makes a change that reduces the efficacy of language instruction, it has clear moral consequences in dispossessing refugees of the skills that they need to survive. For those whose hearts are not invested in the minutia of Internet preservation, I hope that Duolingo can present a case study which is morally engaging.
The needle and the thread: Duolingo is a great case study because it has a single goal rather than multifarious goals, because its design decisions are well understood in terms of consumer psychology and game design so that we can identify well whether changes in these designs thwart or support these goals, and because there are clear moral stakes that simply do not exist in the case of World or Warcraft.
Playing Duolingo
To argue in favor of a right to maintenance, I will focus on changes made to a single game mechanic: the double points lesson.
But, it is never possible to explain a single game mechanic without touching on several more. You can’t explain hotels in Monopoly without talking about renting, auctioning, mortgaging, and money. Subsystems in games enmesh themselves into each other. Duolingo is much the same, and so to explain and analyze the double points lesson, we will need to talk about a few of the specific design decisions related to lessons, and to these doubled experience points.
Duolingo’s courses are broken into individual modules, groupings of lessons with similarly themed grammar and vocabulary. For example, in Italian, we have Food 1 and Occupations. Each module has some number of lessons, [] for Food 1 and [] for Occupations. Each module also has six levels: Purple, Blue, Green, Red, Orange, Golden. Each level has the same number of lessons, and each lesson covers the same vocabulary as it did in the previous level, but these lessons get harder and harder: in Purple Food 1, you are mostly doing flashcards and matching games related to food. In Orange, you produce full sentences in the target language: “Lei mangia la mela rossa”.
Every so often, Duolingo will present you with a challenge: you can take a lesson from one of the levels you have yet to unlock, and you will win twice as many experience points as you would in a regular lesson. This is the double points lesson. If, for example, you are on the Blue level of Occupations, this challenge will instead give you a lesson from the Green level. The points are a useful lure to get you to take on harder content.
Experience points have a few uses. For one, it feels good to get points. Points rock. I love points, personally. I use points as the yardstick for my daily practice: I try to win at least 100 points each day. Second, points lead to the acquisition of gems, which can be used to buy costumes for Duo, Duolingo’s owl mascot. I like dressing up Duo, so that makes me like gems and experience points. Finally, Duolingo includes a weekly leaderboard. Whoever gets the most experience points on a given week will win the leaderboard. I am a competitive person, and Duolingo harnesses this competitive nature to get me to study Italian more intensively: if I earn the most points in a week, I will win the league, with the added side effect of me learning Italian.
Then, we can see the benefit of the double points lesson to further our learning of languages. The carrot of extra experience points is salient to players. With the double points lesson, this carrot gets us to take on a more challenging lesson, which means that our fluency in the target language shall grow. For all the ink I have spent here, the idea is quite simple: if you work harder, you get more fluency and you get more points.
The reason I spill all of the above ink, however, is because these adjacent subsystems, like leagues and module-levels, are relevant to how Duolingo has changed this mechanic recently.
A cadre of recent changes to Duolingo’s mechanics has conspired to make the double points challenge have the opposite effect. Now, I generally refuse to accept the double points challenge, and avoid this mechanic entirely, meaning I take on less challenging content and I learn Italian at a slower pace.
Let me explain two changes made to Duolingo that have neutered the power of the double points challenge.
The Reintroduction of the Hearts System
The hearts system, reintroduced in late 2020, allots players five hearts a day. When a player makes a mistake in an exercise, they lose a heart, and if they lose all of their hearts, they can no longer play Duolingo until their hearts are restored. You can restore hearts by waiting till the next day, or by reviewing previously passed lessons. Effectively, this limits the amount of time you can spend practicing a language.
The hearts system has been deprecated and reintroduced to Duolingo multiple times. Its presence also depends on the version: iOS has had hearts consistently since 2017, while the desktop version of Duolingo still doesn’t have hearts. For the majority of time that I have played it, the heart system was absent from the Android version of Duolingo, allowing players like me to take on as many lessons as they wanted.
Hearts present another useful case study for understanding the right to maintenance, one that I will now touch on quite briefly before returning to their impact on the double points lesson.
The Hearts System, a Mini-Case Study
Hearts exist to do a disservice to the player, and there is no way to argue that they benefit users of Duolingo. Their existence means that players will occasionally be barred from taking lessons on Duolingo. They do not have any benefit, much less one that might outweigh occasional ousting from the classroom.
Hearts should look familiar to anyone who has used a freemium SaaS, that is, a SaaS that is free to use but which has some sort of in-app purchase or subscription service that players can buy to enhance the experience. Hearts exist exclusively to promote Duolingo Pro, a paid subscription service for Duolingo. Players with pro accounts have unlimited hearts, and thus can practice for as much time as they’d like each day.
Here, our forgoing of the right to maintenance allows Duolingo to decide to push updates that makes learning harder for non-premium players, a stratagem intended to push players into purchasing a subscription.
The reason I am only allotting this a small case study is because, in some ways, we can respect this malfeasance towards players because Duolingo is a for-profit company in a for-profit world and to pay its employees it needs to sell subscriptions. Duolingo is not a charity fund, no matter how many Syrian refugees they add to their advertisements.
Certainly, it is important for consumers of SaaS to realize that software companies are incentivized to not just push beneficial changes out to consumers, but also to push changes that may reduce efficacy of the tool. They particularly have this incentive when they have some sort of tiered user base, and when penalties might increase the number of users who elect to subscribe. Users on the free-tier of a freemium-system should expect these changes, and must understand that because they do not own the software, they cannot prevent the SaaS from penalizing them. Remember, all seats on airplanes once had ample leg room; today, that’s only available for first class.
The reason I want to focus on the double points lesson rather than the hearts system is because I think the tribulations of the freemium model are well understood. We know how airplanes work. And even South Park has opined on how predatory freemium tactics work in their seminal “Freemium Isn’t Free”. The double points lesson and its reduced efficacy, however, reads to me as an unintentional change, not as a stick to force players into Duolingo Pro, and as such, I think it is a more fruitful line of analysis.
Ending the Case Study within a Case Study, What Did Hearts Do To The Double Points Lesson? (Jeez, what an awful header…)
There’s a reason that I spilt so much ink on points, experience, and leagues above, because the reintroduction of the hearts system complicated the double points lesson. You lose a heart if you get a question wrong. Since double points lessons feature harder practice problems, the chances of losing hearts increases.
Prior to the reintroduction of the heart system, there existed no real downside for taking on a double points challenge: even if the practice problems were significantly harder, you could just keep playing at the problems until you got them right. Now, if you get enough answers wrong, you will get booted from the double points lesson, thus getting no experience, and you’ll also be unable to play lessons at a more reasonable challenge level. With the hearts system, taking on a double points lesson becomes a gamble. The hearts make the extra experience less of a straight forward carrot, since failing the more difficult problems will prevent you from playing Duolingo at all. Now the double points lesson is not a mere carrot, it is a wager, a gamble, with hearts offered up for points.
(Again, I doubt this was the intent of Duolingo, to make the double points system less efficacious for non-premium players. Had they wanted to tie the double points lesson to a paid subscription, they could have instead made the double points lesson available only to premium subscribers.)
Moving the Double Points Lesson
A change that occurred earlier this year is a bit more complicated to explain, and it’s another reason I explained Duolingo in depth.
Traditionally, when you tried to open up a lesson you would be sometimes randomly presented with the option of a double points challenge. This double points challenge would always be for the module that you had selected: if you had wanted to take a Blue level lesson in the Greetings module, sometimes Duolingo would let you take a Green level lesson in the Greetings module. In this way, you always fundamentally knew what the double points challenge would entail.
Now, however, the double points lesson is presented at the end of completing a traditional lesson. This on its own is not that impactful, but when Duolingo moved when it would present its challenge, it also changed what material the double points lesson would cover: the double points lesson could be for any random module, and would not necessarily be from the one that you just finished. And since the double points lesson could apply to any module, it also means that the double points lesson could be any level. For example, if you had just completed the Red-level for the Greetings module, and you also had completed the Blue-level for the Food module, you could either end up taking an Orange-level lesson for Greetings or a Green-level lesson for Food. Orange-level lessons have you writing full sentences in the target language. Green-level lessons only have you reviewing flash cards. These are very different levels of difficulty.
If the reintroduction of hearts created an implicit bet where the player wagered hearts to get more points, severing the double points lesson from the module the user had selected makes the terms of that bet inscrutable. You don’t know, when you agree to the challenge, if you’d be expected to write full sentences in the target language, which for neophytes to the language could decimate hearts, or simply match flashcards.
I don’t like taking gambles whose terms are unclear. As such, I no longer do double points challenges. The new structure has led me to burn through all of my hearts on multiple occasions, forcing me to halt my study when I did not want to. This is a shame, since the idea behind the double points challenge is a good one. Educational software should reward users for electing to take on more challenging material, since challenge promotes learning. In Duolingo’s case, now, more challenging material is not a boon, but a punishment.
Conclusion of Case Study
Converting a straightforward challenge with scholastic benefits to the user into an unclear bet that can force a learner to stop learning entirely is not effective game design or pedagogy. If I had an earlier version of Duolingo, I would learn Italian more effectively.
I have no right to revert my version of Duolingo, no matter how effective it was prior. Now, certainly, some of the gamification elements would make such a reversion unfair. Remember, there is a weekly leader board where I get to compete with other players, and if my double points challenges were reinstated while others were not, I would have a distinct advantage.
But, I don’t even have the right to host my own Duolingo servers, the same way that rogue City of Heroes servers do, with an old version of the software where all players had the original version of the double points challenge reinstated. Even if enough users wanted to maintain the old version, and agreed to have their own separate league structure only available interested in playing on the old version, Duolingo would not let me host their code base on these servers. This prohibition against maintenance, this lack of a right to not upgrade, disservices me, and I have no recourse here.
This case study, I think, will hit a nerve with many users of SaaS. Sometimes, software changes make the software less effective. Anyone who has used Facebook and suffered through its many UI changes will know this.
The nice thing about the Duolingo example is that since its aims are simpler than Facebook’s, and pedagogy as well as game design is easier to analyze than UI design, we can express why the changes disadvantaged the users without much reliance on subjective detail. Players want experience points. Double points challenges yielded them without question. Players took them. More learning occurred. Now the double points challenges yield experience points in a less reliable manner. Players take them less often. Less learning occurs. I don’t own the software, so I have no right to stop this downgrade.