Autobiography of a quantitative method
I spent the last year or so working on a research paper analyzing the relationship between charter schools and super-expropriation. I used a quantitative method to do this, which I thought was consistent with a new trend in educational research called QuantCrit, or how critical race theory can inform statistical research in education.
Admittedly, the paper was all over the place. But I kept at it and when I finally got it into shape I sent it to a journal. After asking me to revise and resubmit the essay, reviewers and the editor rejected it. I was disappointed. So it goes in academia.
One of their more direct comments was that a section I'd included in the methods description was more appropriate for a blogpost than a research article. In that section, I was trying to humanize quantitative methods, centering narrative (which is a thing CRT tells us to do) by talking about where my ideas came from and how they evolved.
So I'm taking their advice and sending you that section here. It details the conversations I had and the jumps my thinking made to settle upon the method, hopping from a focus on classroom discussion to school finance and getting tips from students, friends, and family along the way.
Part of my project with this newsletter is to approach school finance research with a critical eye, and humanizing the the technical methods in research is a part of that.
***
Before I was a college professor, I was a high school teacher. Working at a high school in Ecuador, I faced a pedagogical problem: how to facilitate discussion in a materially democratic way. I found it was all well and good to say that I was being democratic, but if my Ecuadorian students were not actually sharing the material space of discussion with me, a gringo facilitator from the United States, then I would be distorting the democratic promise of discussion into something imperial and racist.
I came upon a method of facilitating discussion that encouraged students to talk to each other more than they talked to me. It was called the Harkness Method. What I realized in the process of using this pedagogy was that discussion was a space of ideology: there was dominance, subordination, and power dynamics. It was also economic. Of course, students receive grades for their participation, which are inherently related to commodities and exchange-value. But, the discussion space is also a collective resource that we create as a group by coming together. Participants could dominate that space and take more of their fair share of that resource or they could cooperate and ensure economic justice in the discussion through speaking and listening in turns. But how to teach students this?
Materially speaking, teachers know that grades matter. Students focus on them intensely and for good reason. I asked my Ecuadorian students what they thought about these questions. They had interesting ideas about how to grade discussion to encourage turn-taking and sharing the discussion resource. So, I endeavored with them to create a discussion grading scheme that encouraged cooperation rather than dominance of the discussion.
Super-expropriation analysis in school finance has its roots in this grading formula.
I was a terrible math student in school, but I somehow came to love logic and the philosophy of mathematics in college, majoring in the subject. I was not afraid to be creative with numbers. So in my classroom, I kept track of the number of turns each student took during a discussion in my classes. I averaged the total number of turns taken at the end, calculating the distance each student’s score was from the average. I asked students how we should grade using this calculation.
An Ecuadorian student named Jon Edgerton suggested that we think of the average number of turns as indicating how much someone shared the discussion space. If you listen and speak roughly the same as others during a discussion, you are cooperating in that discussion. But a few students were way above the average (they talked a lot) while most other students were way below average (they stayed quiet). At my students’ suggestion, I started calculating two kinds of grade: individual and collective. The individual grade was based on the student’s distance from the average number of turns taken. Based on Edgerton’s assumption about sharing and the average, if a student took the average number of turns they got 100% and lost points the further they got from that average. The collective grade was the average distance from the average number of turns taken.
I calculated all these numbers by hand. I came to see that I was calculating the standard deviation of turns taken and, further, that I had theorized the standard deviation as a measure of cooperation. I began using this grading scheme more systematically and eventually used it to complete a master’s thesis measuring the efficacy of the Harkness method as compared to the Socratic method.
After teaching high school, I decided to get a doctorate in philosophy of education and focused on the question of ideology, politics, and classroom discussion. Studying more political philosophy, turning towards the Marxist tradition, I kept this work in the back of my head, wondering if the grading scheme I had devised could be relevant for larger-scale analyusis, like taxation.
After completing my dissertation and then taking a long detour through Althusserian theory of education, I started studying school finance. I wanted to apply Marxist thinking to schools and money. Looking at eastern Pennsylvania and the Philadelphia region (I got a new job in the area), I saw rampant injustice in school finance.
In a conversation with Corey Chivers, a data scientist and friend who had more experience with statistics, I wondered whether the grading scheme I had used for discussion could be used to measure the sharing dynamic between school district resources in metropolitan regions. At the time, I was reading Cedric Robinson’s (2020) theory of racial capitalism and its applications to urban education, coming upon the concept of super-exploitation, which refers to the racialized theft of workers’ productive labor. I put all these things together and wondered what kind of data would be adequate for a racial capitalist analysis of school district finance.
Reviewing common factors in school finance research, influenced by the Debt Collective and the importance of debt service, and also as a teacher knowing how important expenditure is for the classroom, I decided on thirteen factors to include in my analysis. I included a subset of data describing the population: percent nonwhite students, percent of students in poverty, average family income, total number of students, percent students on free/reduced price lunch; a subset regarding available resources and expenditure: per pupil expenditure, average property value, percent local revenue, percent state revenue, percent federal revenue, and debt service; and a measure of how expenditure manifests in the classroom: student-teacher ratio. I guessed that these factors would provide an adequate picture of the race/class dynamics in school finance. This insight was consistent with Esther Cyna’s unique approach to school finance in rural North Carolina (she called it technocratic rather than unequal) and my reading of Robinson. I wondered whether using my grading scheme would be a way to measure the theft to which she pointed in history.
I thought of school districts in a region like a classroom discussion’s participants. They shared space just like participants did. They co-created material life just as in a discussion. I thought of their populations, resources and expenditures and its manifestations as their actions in the social space, just as discussion participants speak and listen. I found the data for each factor in each district. I then calculated the standard deviation for each factor.
After that I ran into a problem: how to make each of these standard deviations commensurate with one another? I wanted an overall measure, like the collective grades I calculated for my students. The standard deviation of percent federal revenues was a different category of number than the standard deviation in average property value, for example. How could I make these numbers comparable and come up with a collective grade for the region’s resource dynamic?
In a conversation with my mother-in-law and medical researcher Sabrina Ronen, she suggested that I relate each standard deviation to the average of each factor. I tried it and indeed, dividing each standard deviation by the average within the factor produced a similar kind of number. These numbers could be averaged into an overall collective grade for the region’s dynamic of resource distribution. Just like in the grading scheme, the higher the number the more dominance there was in the resource dynamic; the lower the number the more sharing. After conversations with a statistician Benjamin Brumley in my department at West Chester University, I came to see that I had used a common measure in school finance research for inequality, a coefficient of variation: a number produced by dividing the standard deviation of a set of numbers by their average.
But I understood this measure in a much different way. I thought it was conceptually distinct from inequality, particularly given Cyna’s claim about kleptocracy. It made sense to say I was measuring the racialized theft of resources with the coefficient of variation, given the dynamics I had observed in the classroom discussion grading project. Whether they intend to or not, students who dominate a discussion steal time, space, and recognition from others. The same goes for school districts in a region.