The Half-Life of Metrics

Photo by Malcolm Choong 鍾声耀 on Unsplash

James C. Scott writes in Seeing Like a State that governance requires simplification and compression to understand the facts of its world. It generates reductive artifacts that enable the state to grasp what it is governing. They are important proxies for local and tacit knowledge. If a state measures grain production and grain stores, they don’t need to understand how to work the land. The measures are a compressed but manageable proxy for productivity, land value, worker skill, and more.

“Data-driven” governance is not new. Rome ran censuses, tracked taxable property, and knew who was eligible for conscription. Ultimately, every civilization is data-driven. What changes is the algorithm that processes the data. Sometimes it’s the local chief’s gut instinct. Sometimes it’s a massive bureaucracy synthesizing reports and modern data streams into executive action.

And in every society, the leaders processing that data are ultimately beholden to sentiment. Sentiment is not necessarily opinion polls, it’s the actual mood of the citizenry. It’s obvious in a democracy, but Hume tells us it’s true of autocracy as well. Viktor Orbán just lost an election in Hungary despite sixteen years of tilting the playing field in his favor. Scott Alexander recently made the point clearly: modern autocrats calibrate fraud, coercion, and institutional meddling to what the public and key elites will bear. Sentiment is the ceiling every ruler operates under. It’s also incredibly difficult to measure directly, which is why governments build elaborate information channels to approximate it. They track resources, behaviors, and a suite of outcomes as proxies for the mood that ultimately determines their legitimacy.

But despite every government’s great efforts to process information that converts into effective action, every great society has eventually declined. There are other causes, but one driver, consistently, is that every declining society loses some connection with and control over its citizenry. Formalized information channels fail. Governments falter when information is corrupted. This is easiest to see at the level of metrics, our consistent, repeatable measurements of what’s happening in the world. Every metric has something like a half-life. From the moment a metric is adopted, its relationship with the underlying condition it seeks to quantify erodes.

Formalization of a metric generates a new world condition. It alters incentives, changing the behavior of the people within the process it is measuring. It narrows the focus of governments and other organizations, to the detriment of other information that could be considered. And once a metric starts decaying, it is impossible to right the ship without redefining the metric or adopting a new one entirely. Such adjustments happen, but generally institutions are slow to make these changes, often in order to keep longitudinal comparisons in force.

Others have made similar observations. Goodhart’s law tells us that once a measure becomes a target it ceases to be a good measure. Campbell’s law says that when quantitative measures are used for high-stakes decision-making, it corrupts the social process it’s meant to measure. Taken together, we see that every metric has an attack surface.

Metrics are subject to four categories of attack: underspecified definitional changes, incentive distortions, administrative manipulation, and broader changing world conditions.

Consider the crime rate. It’s the state’s chosen method for measuring the amount of disorder in a community — that’s the core intent, the function of the metric. But “disorder” is the tacit thing; “crime” is already a narrow formalization of it. In defining the crime rate, we make choices about what counts as a crime. When the metric is established we already bake in certain rules that diverge from the core intent of the metric. For example, speeding tickets and other “infractions” aren’t counted in crime rates, and yet every speeding driver marginally increases the danger in a community, they generate disorder.

Over time we make more adjustments to our definition of crimes. So-called victimless crimes, like drug possession, public intoxication, or loitering, are obviously excluded from violent crime rates, and are reported inconsistently elsewhere. Over the last three decades, there has been a broad movement to decriminalize these acts. How could you presume to compare an arrest rate in 2026 to an arrest rate in 1990, in a world where many drugs are mostly legal and harm reduction is the method du jour for dealing with addiction? Public intoxication, open-air drug activity, and visible street disorder persist whether or not they are classified as criminal activities. They contribute to disorder in the community, and our crime rate now has a weaker relationship with the disorder it’s meant to describe.

There are administrative decisions, too. Even if our definitions were stable, it’s well-established that the process of turning incidents into records is discretionary at every step. A DA can decline to prosecute, or plead charges down to lesser categories, which directly changes what shows up in conviction rates. Prosecutorial action or inaction also alters policing policy — if certain crimes aren’t being prosecuted, they stop being prioritized by enforcement, and disappear from arrest rates. Such decisions are often reasonable in terms of local politics, or budget constraints, but each one alters what the metric captures. There are harder changes still. Database systems are updated and the actual collection of the data is changed. Crime classifications are recalibrated post-hoc. Often these adjustments are made in an effort to keep fidelity to the original purpose of the metric, but many result in systemic and lasting distortions of the measurements.

Then there’s the Goodhart problem. Once a crime rate becomes a performance metric that police departments are evaluated against, the reporting process itself gets distorted. Felonies get downgraded to misdemeanors so they don’t count against clearance numbers. Incidents get reclassified at the margin to fit whatever category looks best on the monthly reports. This is the mechanism Goodhart named, and it gets the most attention, because it’s the most obviously corrupt. But it compounds with the definitional changes and administrative shifts already described. A metric that’s been changed by definition, adjusted by discretion, and gamed for performance is three steps removed from the original condition it was meant to describe.

In the end you can’t trust the measure, especially as a comparison point, over long periods of time.

Crime rate is an easy case. The failures are obvious once you look for them. But our fourth attack surface requires a different example. This more important mechanism is harder to see, because it operates even on metrics nobody is gaming, and even when the measurement is doing exactly what it was built to do. The world the metric was designed to track keeps moving. The fixed metric slowly comes to describe something different from its original design.

Consider GDP and associated growth. Unlike the crime rate, GDP is relatively stable at its stated job. It measures the change in aggregate economic output, and it measures it about as well as it ever did. Economists know how to calculate it. The definitions are reasonably stable. It is not especially gameable at the national scale, and the bureaucracy that produces it is competent. In the US, at least, there is little incentive to corrupt the measure explicitly. By the criteria that undid the crime rate, GDP is healthy.

For most of the postwar era, GDP growth tracked something important: whether the people trusted their government institutions. As GDP went up, more houses got built, more cars showed up in driveways, more kids went to college. In general, people felt it and understood that this arrangement made their lives better. Rising GDP and rising trust didn’t always move together, but they correlated enough that the number became a critical policy measure and informational lever. That’s why GDP, and particularly GDP per capita, became a headline number, because the line on the chart matched the feeling in the country.

But that linkage has broken in the United States over the last decade and a half. The line keeps going up while the people are miserable. Trust in institutions is holding near all time lows, and people are persistently pessimistic about the direction of the country in ways they weren’t before. Whatever the marginal dollar of GDP is now buying, it is not producing the trust it used to produce.

A certain kind of pundit points at the rising GDP line, or the falling crime line, and uses it to dismiss the malaise. Look at the chart, they say. The complainers don’t know how good they have it. But this is exactly the problem. GDP is not institutional trust. Crime rates are not disorder. They were reasonable proxies for some time, but the relationships broke or distorted. Using the metric to argue against the thing the metric was tracking is an inversion of the problem. And as I said earlier, sentiment is the most important information for governments.

There are technical and explanatory reasons the GDP relationship has decoupled. We talk ad nauseam about home ownership woes, or wage distributions, or rising healthcare costs. And for every explainer there are “narrative violations” that show it’s not as bad as it seems. Perhaps more importantly, social media and other new media amplify discontent in ways that make the mood feel worse than the underlying conditions. But the media-driven discontent generates the world that must be governed. Every version of “the discontent isn’t real, it’s [social media / partisan media / manufactured outrage]” makes precisely the wrong evasion. It treats the mediation of the signal as grounds for ignoring the signal. But trust and sentiment are all that governments have. They are the source of legitimacy. There is no better version available. There’s no waiting out a bad mood. The mood arrives, even if heavily mediated. The only choice is to govern in the conditions that generated distrust, and attempt to solve the problem.

The answer is to read our decayed metrics and distorted sentiment together and commit to acting on the divergence. The divergence, itself, is the most important information you have. If a measure no longer tracks with mood in the way it used to, then we ought to assume the forces of decay have rendered it worthless for this purpose. We can modify existing measures or establish new measures, consciously aware that they will decay, and being willing to retire them when they do. It won’t produce the kind of stable reporting that gives the illusion that institutional life is manageable, with longitudinal comparisons going back decades. But that’s the point. Governance is the practice of staying in contact with a moving reality through unreliable instruments.

These corrections do happen, but they happen inside specialist practices, not in real public view. Labor economists will change focus from unemployment rate to the prime-age employment ratio if U-3 begins overstating labor market health. Macroeconomists have long supplemented GDP with a suite of other measures. The people whose job it is to read instruments generally know when an instrument has faltered. What they don’t do — and what the institutions around them make it very difficult to do — is retire the public-facing number. The headline keeps running. The specialist conversation moves on without it. The divergence between metric and mood is partly this: the official measure, broadcast to and understood by the leaders and the public, no longer fits reality and the experts know it.

Modern institutions confuse stable reporting for stable contact with reality. That is the problem. A crime rate can hold steady while disorder worsens. Output and wealth can keep rising while the people generating it get more miserable. The danger is forgetting that a metric was designed as a simplified proxy for a much deeper, local, and complex condition. If we only govern these artifacts, we no longer govern reality. The institutions that survive will be the ones willing to let go of the numbers that no longer describe the world. The ones that die will die clutching their artifacts, because the artifacts feel more real than the world.