There was a bit of Discourse today about COVID mortality reporting and what — if anything — it means that we are currently in a trough of COVID deaths, with weekly COVID deaths at their lowest point since the beginning of the pandemic in the USA in March 2020. The Discourse issues from the altogether predictable confusion about what the hell is still being tracked or reported as far as COVID indicators. A lot of people know that huge components of COVID surveillance have just been straight up nuked and discontinued, most notably electronic case reporting (aka “transmission” or “case” data); the changes to data reporting and presentation have been rolled out in the most confusing and befuddling manner possible (this, I think, is half on purpose and half down to typical bureaucratic bullshit); and people aren’t really trusting that official figures capture the true state of the pandemic anymore. That is all valid and true. However, death reporting has basically not changed, even with the end of the public health emergency (PHE) last May.
The federalized system of death reporting in the USA sucks and I’m not here to defend it. (You can read a previous edition of this newsletter where I talk about how much it sucks in detail, with regards to perinatal death or “maternal mortality,” here). Nevertheless, the process for death reporting remains much as it always has been — not great, but almost certainly not hugely and systematically undercounting COVID deaths. I will tell you why.
When someone dies, a death certificate is filled out, typically by a physician, a medical examiner, or a coroner. (There are lots of issues here that I will not get into, just know that this means that there are wildly different processes for completing death certificates at the level of an individual death and they aren’t all equally good or accurate.) Part of the death certification process is listing contributing causes of death on the death certificate — also an imperfect and uneven process that I will touch on briefly in a moment. I won’t pretend to be an expert on the ins and outs of cause of death certification, but there are guidance documents from the National Vital Statistics System (NVSS) on COVID-19 death certification, including an expanded guidance document about accurately certifying deaths related to Long Covid. Once a death certificate has been completed, it is transmitted to the state vital statistics office, which then transmits it to the National Vital Statistics System. This is how we get death data for all deaths, due to COVID or otherwise.
This process has not changed. Although, one would be forgiven for thinking that it has. The CDC itself, always with its finger on the pulse of what would be maximally confusing to the public, really made it sound like reporting had changed. They did this by touting their “new metrics” for COVID mortality data presentation (which is not in fact a new metric, but rather a maneuver in data abstraction — COVID deaths are now presented not as raw numbers but as a percentage of all total deaths in the country, clearly a move to make it harder for someone looking at the CDC website to understand whether COVID deaths are high or low right now) and using a flurry of confusing language to make it sound like they had revamped death reporting to be some cool new process.
This is not true. What they actually did is discontinue something they were calling “aggregated cases and deaths counts (ACDC)” reporting and consequently “switch” to provisional death certificates. This is just bureaucratic Sturm und Drang to make it seem like they’re doing something proactive and adaptable in terms of COVID surveillance, rather than doing what they’re actually doing: trying to make it so frustrating and confusing to get information about the state of the pandemic that most people give up. As far as I can tell (and I have looked into it extensively), the aggregated cases and deaths counts were automated scrapes of COVID data from the websites of local health departments and even local news outlets. The aggregated cases and deaths counts were thus redundant of the “official” death certificate counts, but probably were a lot more timely during earlier phases of the pandemic when that mattered a lot. (I will also hazard a guess that most municipalities have stopped reporting COVID case and death data daily or even weekly, and that there probably wouldn’t be anything for the CDC to scrape at this point, so I think discontinuing this particular program makes sense.)
There’s a lot of hay being made from all over about the “switch” to provisional death counts. Provisional death data are just that — provisional and subject to change. There are lots of reporting lags built into the death reporting process — as just one example, different states typically have different reporting lags, which can be weeks or months. Provisional counts are retroactively (I think) revised upon receipt and tabulation of data from a given time period. Usually, provisional data is a slight undercount but tracks final data pretty closely. And this has been the process for death reporting for the entire pandemic (and before), so I’m not convinced by some of the “gotchas” I’ve been seeing about how use of provisional data suggests some kind of massive undercounting. It’s just the (fucked up and imperfect) way death reporting is done here.
Now, it seems like everyone has some anecdotal story about having to fight with a hospital physician to put COVID-19 on a death certificate. I think these stories are true, but they don’t add up to a systematic undercount of COVID deaths. For the most part, people across the extremely fragmented landscape of American health care are putting COVID-19 on death certificates when COVID-19 contributed to death. As with death reporting for anything (seriously — take a look in the literature about death reporting for fatal overdose), there is certainly misclassification — people who died of COVID-19 but were not counted as COVID deaths, and people who died of something else but whose deaths were incorrectly ascribed to COVID-19. Given the federalized state of death reporting, that’s basically all we can say for certain, and we can only speculate about how that misclassification might be shaking out in particular cases, or even particular jurisdictions. It is my opinion, though, that there is misclassification still going on with COVID death certification but the misclassification is not likely to be systematic or even politically motivated at this point. The US government is still using the same process it always has used to tabulate COVID-19 deaths.
This is what I mean when I say that we can only speculate about undercounting of COVID deaths. The data are not being systematically hidden. They are being reported as they always were. As I said in the previous paragraph, I’m sure there’s misclassification, but I don’t think there’s anything like systematic nation-wide undercounting going on, nor is there systematic “hiding” of deaths data like the has been for transmission data and hospital admission data, which have been abstracted, hidden, and discontinued. Now, without transmission data, we don’t really know what the relationship is between a given level COVID deaths and the level of transmission in the wider community. That is a problem, especially for people for whom it really matters whether COVID cases are increasing or decreasing where they live.
To round this out, I want to get to the juicy part of this newsletter, which is me talking shit about the People’s CDC. As I have already said, the People’s CDC is the paragon of a “critical support” organization — I support them against the right wing critics and attacks they’ve long been subject to, but I think there are serious problems with some of their work and serious critiques to be made from the left, if that’s even possible.
I’m bringing this stuff in because this is what people are citing as evidence that the low death counts in recent weeks are illusory (and, further and only implied, that making note of them in a positive sense is ideologically suspect).
Here’s the text of a People’s CDC COVID-19 report from June 26, 2023 (last year):
You may have noticed that we haven’t been including data on COVID deaths recently. With the end of the federal Public Health Emergency on May 11, 2023, there was a significant change in death reporting which caused an artificial decrease in death counts. We already know that COVID deaths are relatively undercounted, but this change was particularly misleading.
Hopefully the foregoing discussion is enough to establish this, but I will say it again: there was not, in fact, any significant change in death reporting causing an artificial decrease in death counts. Death reporting has remained pretty much the same. Other types of reporting have been discontinued, as has more up-to-the-minute death reporting scraped from health department websites, but that information is redundant of the official death counts anyway and its utility was primarily in its more real-time character, which is less important at this point in the pandemic than at previous points.
Clicking the hyperlink in the text (it should work) takes you to another COVID-19 report from May 15, 2023 (right around the end of the public health emergency or PHE). Here’s what it says about death reporting:
Deaths: Major changes to death data reporting due to the end of the PHE: The CDC has switched to Provisional Deaths from the CDC’s National Center for Health Statistics (NCHS) National Vital Statistics System (NVSS).
The CDC notes that data during recent periods are incomplete because of the lag in time between when a death occurs and when a death certificate is completed, submitted to NCHS, and processed for reporting. This delay can range from 1 week to 8 weeks or more, depending on the jurisdiction. One former Center for Health Statistics employee turned member of the People’s CDC notes that final data were typically delayed by 18 months.
For the week ending in May 6, 2023 the CDC reports that 323 died of COVID. To examine the difference we look back to the week ending in May 3, where CDC had reported that 1,109 people had died.
This is, I’m sorry, concern trolling, and it’s deeply misleading. Lagging reporting is an issue for death reporting in general, not specifically for COVID. Provisional death counts are the primary method we use to get more timely mortality information in general, not just for COVID, and if they are good enough for every other cause of death (which, if you are doing any research using mortality data in the United States, you have to accept that they are, with standard caveats about how much our death reporting system is messed up) then they have to be good enough for COVID. Issues with misclassification, undercounting, and lagging death indicators are not unique to COVID, and it is again misleading (and seriously ignorant) to present the use of provisional death data as if it’s part of a grand conspiracy to hide a massive wave of COVID deaths. Such a wave is simply not taking place right now.
The People’s CDC report from June 26, 2023 (the first one I mentioned) goes on:
Since the end of the Public Health Emergency, CDC is now reporting deaths in two ways. First, on the CDC’s main data tracker, COVID deaths are reported as a percentage of total deaths. A percentage is particularly difficult to interpret, as total deaths in the US (pre-pandemic) were known to vary seasonally.
Next, we have CDC’s provisional death counts for COVID. This tracker is based on death certificate data, which depends on a suspicion for COVID infection before death or on autopsy. Given continued rollbacks in testing access, it is likely that COVID deaths are undercounted by an even wider margin than in the past.
Again, this is misleading. CDC’s main data tracker — the percentage of total deaths that COVID deaths represent — is actually a way of presenting data to the public. It doesn’t have anything to do with how death data are reported, collected, or tabulated; it’s merely how the data are presented. (And they are being presented here to give the most abstract and confusing picture possible, for sure.) As for the provisional death counts for COVID, yes, it’s true that there have been changes to testing. There may be people for whom COVID contributed death but who are not counted in official statistics — that is almost certainly the case. But that has almost certainly always been the case, and the federalized and fragmented system of death reporting (different practices everywhere) means that it is truly impossible to make any claims about systematic undercounting. What’s really happening is some places are counting accurately, some places are undercounting, but we don’t know which is which, and it all gets lost in the shuffle of death data reporting. Isn’t the social construction of data fun?? The People’s CDC report makes it seem like provisional death data are some different, less good metric than what was used before — but this is just simply untrue. It’s the same metric as always. Points about the reliability of the metric are valid, but necessarily speculative. The report makes a further point about undercounting of Long Covid deaths. That is almost certainly happening too, but probably no more or less than for any other cause of death, and in any case it is impossible to make any definitive guesses about the extent or nature of the undercounting here. Death data, for any cause, is always an approximation.
The last part of this People’s CDC report that really pissed me off is this:
All current COVID data, including deaths, are clouded by “survivorship bias.” Survivorship bias can cause us to overlook the fact that there have been changes in characteristics of the population over time due to deaths of groups of people earlier on. Unfortunately, far too many people died earlier in the pandemic, including those who were high risk and immunocompromised, and are no longer part of the susceptible population.
This is, in general, a valid point. However, it has absolutely no bearing on whether current provisional death counts are accurate or not. Putting this at the end of the bit about death reporting is probably just well-intentioned ignorance but I think it functions to cast even further doubt on the entire enterprise of death reporting. Skepticism and measured caution are certainly warranted, but this is just — again, because it’s the kindest term I can say it with — misleading. I will say once again that general concerns about survivorship bias (a lot of the people who were susceptible to dying from COVID have already died from COVID) doesn’t imply a damn thing about the accuracy of COVID death counts at this point in time. Rather, it illuminates the real parameters of this “debate,” which center on: is it okay to look at low COVID death numbers right now and say “this is a good thing”?
There’s no right answer to this. Whether you think this is a good thing depends on a lot of personal factors that are threaded into pandemic discourse and politics more generally. I think it’s a good thing that there is not a huge hidden wave of COVID deaths, that the low provisional death counts in recent weeks really do mean that COVID deaths are very low right now. I do not in any way yearn to return to the months we endured of 10,000 or 20,000 COVID deaths per week.
Again, what seems to be really at stake here is: is it okay to say we’re in a “good place” in the pandemic? Is it okay to acknowledge we are in a much different phase of the pandemic than before? Will acknowledging this lead to further marginalization of COVID as a political priority? This is about meaning, about grief, about lived experience, about values and political priorities, but it is not about numbers or data reporting procedures. Once again, the authority of numbers proves to be nothing but vapor — data can inform our morality and our politics, but they can’t serve as the basis of morality or politics, or as the authority legitimizing a particular moral or political vision.