Missing the Forest for the Trees: The Limits of Evidence Inside a Broken Social Contract

Introduction: The Misdiagnosed Backlash

By Jodi Schiller

The Science of Second Chances is a compelling demonstration of what rigorous empirical research can reveal about criminal justice policy. Jennifer Doleac carefully tests assumptions, challenges popular narratives with data, and argues that effective reform requires evidence rather than ideology.

This essay accepts the value of that empirical work while evaluating it through a different framework. My framework begins with a prior question: What happens when the institutions generating, suppressing, and acting upon empirical evidence are themselves operating within a broken social contract? If the baseline assumptions of the system are themselves contested, then even excellent empirical analysis may explain what is happening while missing why it is happening.

This essay is not a rebuttal of Jennifer Doleac's empirical findings. On the contrary, it assumes the value of her research. Its purpose is to argue that empirical rigor and institutional legitimacy answer different questions, and that addressing both is necessary for durable criminal justice reform. From the perspective of my framework, however, criminal justice reform is ultimately the wrong objective. The deeper task is rebuilding the social contract itself, because no justice system can be more legitimate than the political foundation upon which it rests. Genuine reform requires reconstructing a social contract that is coherent, reciprocal, and legitimately extends to every citizen.

The introduction to The Science of Second Chances provides an illustration of this distinction. Doleac recounts the bad-faith hostility, ideological attacks, and professional backlash she experienced after publishing data-driven findings on policies such as "Ban the Box." Rather than simply introducing her research, she devotes substantial attention to describing the institutional resistance she encountered. That choice signals that these experiences mattered. They were not incidental to her work; they shaped the context in which her work was received.

Yet the remainder of the book largely leaves that question behind. After documenting a system that reacted defensively to inconvenient evidence, the analysis returns to the familiar tools of applied economics: incentives, probability of detection, randomized trials, cost-benefit analysis, and marginal policy improvements. The hostility itself becomes background rather than an object of analysis.

From the perspective of my framework, this is where the book misses the forest for the trees. The institutional resistance Doleac describes is not simply an unfortunate reaction to good science. It raises a more fundamental question: Why does a system respond so aggressively when empirical evidence challenges established narratives?

My framework begins with what I call the Legitimacy Prior: before asking whether institutions are efficient, we must first ask whether they are legitimate. Efficiency without legitimacy risks optimizing systems whose underlying structure may itself be fundamentally unjust.

Viewed through that lens, the resistance Doleac describes is not an accidental failure of communication or a temporary triumph of ideology over evidence. It is a predictable feature of institutions whose legitimacy depends upon preserving particular distributions of power. Within such systems, suppressing or marginalizing inconvenient evidence is not a bug. It is often a stabilizing function.

Viewed through this lens, the introduction points toward a deeper structural problem than the remainder of the book ultimately confronts. Doleac identifies the phenomenon. My framework attempts to explain it. The central argument of this essay is that evidence alone cannot resolve questions of institutional legitimacy. Empirical research can optimize policies operating within an existing system, but it cannot determine whether the underlying social contract upon which that system rests remains legitimate in the first place.

Empirical precision is a powerful tool, but its value depends entirely on the validity of the question being asked. If an analysis begins by assuming a functional baseline where none exists, even the most rigorous metrics will only yield highly accurate answers to the wrong problem.

In her introduction, Doleac describes experiencing intense institutional resistance when her empirical findings challenged established narratives. My framework interprets that resistance not as an unfortunate anomaly, but as evidence of a deeper structural problem: systems whose legitimacy depends upon preserving existing distributions of power often resist the very evidence that could call those arrangements into question. Yet the analysis ultimately turns toward tuning the engine of a vehicle already driving toward a structural cliff.

My framework does not reject empirical rigor. It demands that empirical rigor be directed toward the right question. Do not ask how to optimize outcomes inside a broken social contract that ignores trillions in unresolved historical obligations and the ongoing systemic subjugation of women. Ask whether the social contract itself remains legitimate.

Bring the econometric precision, the randomized trials, and the quantitative ingenuity to that question. Apply the same analytical power to measuring institutional incentives, data suppression, historical obligations, structural legitimacy, and the mechanisms through which societies preserve or repair broken social contracts.

Only then do we move beyond measuring the mechanics of a trap toward understanding how the trap was built, why it persists, and what it would take to dismantle it. Only then can empirical science fulfill its highest purpose: not merely optimizing the systems we inherit, but helping us determine whether those systems deserve to endure.