Not wrong to be wrong
I got something wrong recently in my research, and in this week’s post I want to lay out some corrections I’m making to the findings and thinking behind that project.
About three years ago I started working on a paper with Camika Royal about the School District of Philadelphia’s (SDP) relationship with the municipal bond market regime. I wanted see whether/how private credit was an efficient way to finance the buildings given their dilapidated state, using mixed methods to do it: history, theory, and quantitative data. While Camika’s strength was on the history of the district, I could handle the finance-y and quantitative aspects (we both sort of worked on the theory).
For the quantitative part, I thought about efficiency in terms of dollar-for-dollar comparisons between two numbers I knew I could find: the amount of money reported as principal in public bond statements issued by the district (which I thought of as revenue) and the amount of money the district spent on its capital programs each year (which I thought of as expenditure). I figured, if municipal bonds—which come with interest, costs, fees, amid volatile market conditions connected to Wall Street—were an efficient way to finance school buildings, there should be something close to a 1:1 dollar-for-dollar ratio when you compare these numbers, the money coming in and the money going out matching up.
So I gathered all the SDP bonds I could find for as long as I could find. I found the bond principal for each year and compared that with the capital expenditure numbers reported by the district. Then I found the “cost of issuance” in each bond statement, as well as the interest rate from each bond, and made sums and averages for them. I took inflation into account, excluded some bonds since I knew that not all issuance should be included, and the number I got after all that was 43 cents for every dollar, which was shockingly inefficient.
This paper made a splash like I’ve never experienced before. The Conversation, an outlet that popularizes academic findings, reached out and asked me to write a synopsis of the paper. Journalists in Philly and outside the city cold-called asking for a copy to read. I even heard from someone who works in the debt management office for the city of Detroit. Then, as Philly schools were closing due to a lack of proper HVAC just as the school year was starting, a tiktok I made about the paper went a bit viral and I heard from even more people.
The comments both on The Conversation and tiktok ranged from shock to confusion. I got a couple strongly-worded disagreements to the Conversation piece, but neither of them gave me any reasons or arguments about why I might be wrong. Then I had a commenter in the tiktok thread, who said they were in finance, tell me I’d made some big mistakes, specifically referring to the bonds I was using to calculate my revenue number. He said none of the bonds I used made sense, putting it in kind of bombastic terms. I said I hadn’t included just any bond since I was aware that not every bond should be in there, and after some back and forth, he stopped responding. I noted it, but thought he was just trolling me (which has happened before).
Then I got an email from a school finance lawyer that I know from a few contexts, both online and in person. They asked me to send them the bonds I’d used in my calculations, which I did. I trusted their knowledge about this stuff. In a few emails to me, among other critiques and questions, they reiterated the point the tiktok guy had made, but more specifically: that I’d included refunding bonds in my dataset, along with general obligation bonds.
When I did the original research I was aware that refunding bonds pay back other bonds. They’re new debt to pay back old debt. I thought this process was like getting a credit card to pay back other credit cards: more debt to pay off old debt. But it’s not like that. A refunding bond replaces old debt with new debt. You can’t count it twice.
After emailing back and forth with the lawyer and looking into it more, I knew I’d messed up. They also mentioned some other caveats that got me thinking: the calculation I’d made for estimating the amount paid in interest didn’t take into account amortization of the debt; original bond principal might be an overly fuzzy measure of revenue for facilities spending; whether/how I could include tax and revenue anticipation notes (TRANs) with the general obligation bonds; the accounting differences between revenue for capital expenditure and operating expenses; and a couple other things, like including capital expenditures in 2021 as well as the bond principal, even though the temporality of that money might be further out than 2021 itself.
While I was looking into it, another colleague and friend who had worked in Philly city government emailed me to say something similar, that I’d over-counted the amount of bond principal. After a period of anxious grief I went back to my numbers and found there were a handful of other things that needed redoing too: the inflation adjustment was done poorly, I hadn’t considered the relationship of the costs of the issuance to the bond principal, and I hadn’t properly contextualized the other cents-per-dollar that my finding implied: what happened to that “missing” money? How should we understand it?
I felt like crap. Here I’d spent all this time writing this paper, getting everything in place, and making some confident claims about the inefficiency of the municipal bond market regime…and my numbers were off. Not only that, but I’d been wrong in my understanding about the different kinds of bonds, about which I was supposed to be some kind of expert.
I was traveling at the time with family and I asked my partner’s parents, who are scientists, about what happens in their fields when someone gets something wrong. Immediately, they said “oh everyone gets things wrong, it happens all time.” Then they said, “it’s never wrong to be wrong, it’s just wrong to lie about it.” I felt a little better. This is part of producing knowledge: you get things wrong, you learn, repeat. But it’s important to be transparent.
So I told my co-author Camika, the co-editor of the special issue in which the article appears, the editor at the Journal of Educational Administration and History who worked with us, and the editors at The Conversation, who temporarily put up a notice on my article about corrections being needed. Then I went back into my spreadsheets and started making changes so I could submit corrections to the journal. I redid the numbers. Here’s what I found.
I readjusted everything properly for inflation. I removed the refunding bonds. I took out claims that I knew what amounts went towards interest. I re-estimated the costs of issuance for a set of bonds for which I didn’t have exact numbers using the the bonds where I did have numbers. I subtracted the costs of issuance from the original bond principal (to get something like a net bond principal number). I redid the analysis overall, then went over that analysis with three colleagues. After all that, I found that for every dollar lent by Wall Street, the SDP could only spend 56.6 cents. That’s 13.6 cents higher than my initial finding. I also found that costs (excluding interest payments) totaled $330 million during the 1993-2021 period.
That analysis, however, includes the bond principal from tax and revenue anticipation notes (TRANs). These bonds are shorter-term loans meant to fill certain gaps in the budget cycle, among other things. Including them is an assumption I’m making. I think the line between operating and capital expenditure is fuzzy and TRANs are still bonds that cover things like debt service and facilities costs, which go towards infrastructure. But some might argue that they’re not relevant to include, like the refunding bonds.
So I did a version of the analysis that excludes the TRANs too. For that analysis, I found that for every dollar lent by Wall Street, the SDP could only spend 83 cents. That’s significantly higher than my original 43 cents number, and higher still than the 56.6 number.
So, at this point, I feel confident that I can say, given all these caveats, that the district could only spend as little as 56.6 cents per dollar lent by the municipal bond market regime, or as much as 83 cents.
I redid my graphs and figures and I wrote up a memo detailing all the changes, exactly what I did, and contextualizing the method as well as the findings. I noted more clearly that there’s no established method for doing the efficiency analysis I’m undertaking. Finally, I noted something about this 'leftover money’, the up to 46.4 cents unaccounted for in my analysis. If the SDP could only spend 56.6 cents, then what about the other 46.4 cents?
First there’s the question of what to call this money. You could call it ‘missing,’ which implies that someone lost it and we don’t know where it is, but I wasn’t happy with that, since it potentially implicates the school district, and I don’t think this is the school district’s fault. Given the history we tell in the article, I think it’s the municipal bond market regime’s fault as a source of revenue for public school infrastructure. So, remembering a term from psychoanalysis, I decided to call it ‘foreclosed money’: this is money that the district needed but could never get, it was withheld from the beginning.
So I said that the paper’s analysis produces an amount of money that is “foreclosed.” But this money is up for interpretation. How should we understand this amount?
We could attribute this number to the fuzziness of using new bond principal as a way to measure revenue for capital expenditure. Maybe that number doesn’t capture the reality, and the foreclosed money is just signifying that. But there are actually a lot of things it could be.
For instance, interest paid by the district on the principal between 1993-2021, which, as I said, I couldn’t calculate. How much over that period went to pay that interest? And there are many other costs associated with municipal bond issuance about which it’s difficult to find good information, as it’s not clearly listed in public bond records. In a great report on the costs of municipal bonds to school districts, Marc Joffe enumerated the kinds of costs involved for school districts when issuing bonds, which include:
Underwriter’s discount (a sum paid to the private banks)
financial or municipal advisor or consultant fees and expenses
bond counsel fees and expenses
underwriter’s counsel fees and expenses
rating agency fees
bond insurance premiums
verification agent fees
trustee fees
cost of issuance agent fees
paying agent fees
escrow agent fees
printing fees
CUSIP fees
contingency costs or other costs such as fees paid to state treasurers
payments to the attorney general
costs associated with the municipal employees’ time working on the issuance
appraisal fees
It makes sense that these costs might pile up over a nearly thirty year period. For a recent bond, for example, I found out that the SDP paid the consulting firm Eckert Seamans $714,418 for consulting work on it. That’s just one consultancy fee for one bond! What about all the other fees for all the other bonds? It’d add up.
If it were the ambiguity in the method, there are ways, I think, to account for it in future research: finding historical debt service and interest payments data from the National Center for Educational Statistics, for instance, and then triangulating this amount with national trends in debt service and capital outlay. I think you could also do this efficiency analysis for other school districts in different cities in states with different school infrastructure finance policies and see how much the numbers jump around to get some comparison cases. I’m interested in doing that research in the future but I can’t include it in the corrections for this paper.
At this point, I’ve submitted these notes to the journal and I’m waiting to hear back from them. I still stand by the paper and I’ve learned so much by being wrong, even though it feels pretty bad, and I’m doing my best to be transparent and honest about the process, which is a value I hold as a socialist studying in school finance. I’m actually going to present these findings to the Philly Public Banking Coalition on Tuesday, 9/24, at 4:30pm and I’d love to see you there if you have any thoughts or questions, etc.