The Pattern
Every organisation of any size runs on a small number of numbers. A dashboard indicator, a bonus metric, a quarterly target, a red-amber-green status — something gets picked, at some point, to stand in for a larger and harder-to-observe reality. That substitution is not the problem. It is how any organisation beyond a handful of people has to operate; nobody can hold the whole picture in their head, so a measure holds a piece of it instead.
The problem is what happens next, which is usually nothing. The team reorganises, the product changes, a process gets automated, a market shifts — and the measure, having been set once, keeps being reported exactly as before. It is not that the number becomes wrong. It continues to say precisely what it has always said. What has changed is whether that statement still means what it once meant.
Consider an organisation where a leadership dashboard shows eleven indicators, all green, in the same week that two significant operational failures occur. The first explanation reached for is usually that the dashboard is missing the right indicators — a coverage problem, fixable by adding more. But the indicators in place are individually reasonable; each one measures a real component of the work competently. What none of them measures is whether those components are working together, because no single function is responsible for that interaction. Measurement, in this organisation, has been allocated the same way responsibility has. What sits between functions goes unmeasured because it is unowned — not because anyone decided it didn’t matter, but because no one’s role includes noticing that it does.
That is the shape of the pattern this guide is about. Not a broken metric, and not a dishonest one. A measurement that has quietly outlived the situation it was built to describe, while continuing to report as though nothing had changed.
It is worth being precise about what this is not. It is not the familiar complaint that an organisation tracks “too many” or “the wrong” metrics from the outset — a coverage or design question that gets raised, debated and sometimes resolved when a measurement system is first built. This is a different failure, one that only becomes visible after a measure has already earned trust. A newly introduced metric gets scrutinised; people ask what it captures and what it misses. A metric that has been reported the same way for three or five years stops being questioned in that way at all. Its familiarity becomes evidence of its soundness, even though familiarity and accuracy are not the same thing and were never meant to stand in for each other.
Why It Happens
Part of the reason is structural, and it has been documented for a long time. In 1956, V. F. Ridgway, writing in Administrative Science Quarterly — the journal published through Cornell University’s Johnson Graduate School of Management — examined what happens when organisations rely on quantitative performance measures, whether single, multiple or combined into a composite score. He found dysfunction in all three cases. Public employment interviewers evaluated on the number of interviews conducted began conducting fast interviews and placed few applicants. Law-enforcement investigators given a monthly quota of cases selected the easiest ones as the month’s end approached. In both instances, the measure was followed exactly. What deteriorated was the thing the measure was meant to stand in for.
Ridgway’s finding matters here because it shows the drift is not a side effect of bad measurement design — it is close to a structural property of measurement itself. A measure that is followed precisely enough, for long enough, tends to reshape the behaviour around it, and that reshaping is invisible from inside the measure. The dashboard cannot report that people have started optimising for it rather than for what it represents.
A related and independently documented pattern concerns incentives specifically. Steven Kerr’s 1975 Academy of Management Journal paper — since republished as an Academy of Management classic — examined reward systems across a manufacturing company, an insurance firm, and settings as different as politics, medicine and academia, and found a recurring gap between what organisations said they wanted and what they actually measured and rewarded. The stated hope and the operating incentive routinely pointed in different directions, not through any single actor’s dishonesty, but because the two are set at different times, by different people, and rarely reconciled once both are in place.
Neither Ridgway nor Kerr’s organisations set out to keep a stale measure. What both studies illustrate is that a measurement, once established, does not naturally come up for reconsideration. A contract has a renewal date. A measurement usually does not. Nothing prompts the question of whether it still represents what it once did — and in a matrixed or growing organisation, the person best placed to notice that gap is rarely the same person accountable for the number staying green.
There is also a quieter, less obvious cause: retiring or revising a measure rarely feels neutral. A target that has been in place for years has usually been hit, missed and argued over enough times to accumulate a kind of institutional weight — changing it can read as moving the goalposts, or as an admission that past performance against it meant less than it appeared to. It is often easier, in the moment, to keep reporting a familiar number than to explain why a new one is needed, even among people who privately suspect the old one has stopped telling them much.
What It Costs, and Where It Is Missed
Before the cost, it is worth naming what a leader actually notices first, because it is rarely the number itself. It is usually a mismatch: a target reported as met, alongside frontline conversations describing the underlying work as harder than it was a year ago. It is a gap that gets explained away the same way each cycle — “give it another quarter,” “that’s just seasonal” — with the explanation never quite tested against what actually happens the following quarter. It is a status that stays green through a period everyone privately agrees was difficult. None of these, on their own, proves a measurement has drifted. Together, and repeated, they are the pattern’s most visible early signature — visible, that is, to anyone who is looking for a mismatch rather than for a red number.
The near-term cost is a decision made on a number that no longer means what the people using it believe it means. The compounding cost is larger: incentive schemes, resourcing models, and reporting structures accumulate around a measure over time, so that by the point the drift is visible, correcting the measure means unwinding decisions that were never questioned individually.
This shows up differently depending on where it happens. In a small organisation, a stale measure tends to get caught relatively quickly, because a small number of people still hold the whole operating picture between them and notice when a number stops matching what they are seeing. That informal check erodes as an organisation grows and the picture is distributed across more people, none of whom holds all of it. Professional and knowledge-based organisations face a related but distinct version of the problem: because outcomes are often difficult to attribute to one person’s work, measurement leans toward activity — hours logged, cases opened, calls made — which is easier to observe but has a looser and more slowly eroding connection to the outcome it was meant to proxy. Asset- and production-based organisations tend to trust a measure for longer when the underlying process still looks unchanged on paper, even after real operating conditions — supplier mix, machine age, shift patterns — have moved considerably.
A common misconception is that this is mainly a problem of dishonesty or gaming — people deliberately working the numbers. Sometimes that happens, but the more common failure is quieter: a widely used figure is assumed to have been recently checked, when in practice it has simply been repeated. Mark Hughes of the University of Brighton, writing in the Journal of Change Management in 2011, examined five separately published instances of a well-known claim — that roughly seventy per cent of organisational change initiatives fail — and found no reliable evidence behind any of them. The figure had outlived the checking that would have kept it honest, and continued to circulate because repetition, not verification, is what keeps a number in use. It is a smaller version of the same failure this guide is about: not a lie, but a number that nobody has re-opened.
There is a genuine tension underneath all of this. A measure can become more precise over time — better instrumented, more granular, more consistently defined — while becoming less relevant to the thing leaders actually need to know, and the two trends are easy to mistake for each other, because precision looks like rigour. It is worth noting, too, that the popular management phrase “what gets measured gets managed” is itself a case in point. It is widely treated as an endorsement of measurement. Its nearest traceable origin is Ridgway’s 1956 paper — which was a warning about over-reliance on measurement, not a case for it. The phrase’s popular meaning has drifted a considerable distance from the argument it is usually credited with. Even a well-known idea about measurement can outlive what it originally meant.
None of this means a long-running, stable measure is automatically suspect. Many stable measures remain exactly as accurate as they look, and the organisations relying on them are right to. The difficulty is that stability, on its own, is not evidence either way — a measure that has not been questioned in years could be sound, or could simply be one that nobody has had reason to question.
What The Research Shows
The question this body of work is really asking is whether a measurement’s continued stability is evidence that it is still accurate, or whether stability can itself be a symptom that nobody is checking it any more.
Ridgway’s 1956 study found dysfunction across single, multiple and composite measures alike, which suggests the problem is not solved simply by adding more indicators or combining them into a more sophisticated score — the interviewers and investigators in his research were each being tracked on more than one dimension, and the distortion appeared anyway. Kerr’s 1975 research, examining organisations well outside Ridgway’s public-sector settings — a private manufacturer, an insurer, and comparisons drawn from politics, war, medicine and academia — found the same underlying divergence between stated goals and what was actually being measured and rewarded, suggesting the pattern is close to structural rather than an artefact of any one sector’s habits.
A partial explanation for why a known-dysfunctional measure tends to persist, once established, comes from Nelson Repenning and John Sterman’s research at MIT Sloan, published in the California Management Review in 2001. Their modelling and case-study work on what they term the “capability trap” found that people under sustained pressure default to working harder inside an existing system rather than examining its design, and that the people who would benefit an organisation by questioning a measurement or a process rarely receive visible credit for doing so — because success looks like nothing having gone wrong, which is easy to overlook.
Where these bodies of research agree is on treating measurement dysfunction as close to a default behaviour of any sufficiently relied-upon measurement system, not as a rare failure confined to badly designed metrics. Where they diverge is in what they explain: Ridgway and Kerr document the distortion once a measure is in place; Repenning and Sterman’s work speaks more to why a known problem persists once it has been noticed, rather than why it goes unnoticed in the first place. That earlier question — what makes capable people stop seeing a stale measure as stale — is comparatively less studied across this literature, and remains closer to an open question than a settled one.
It is also worth noting what this body of work does not claim. None of it argues that measurement itself is the problem, or that organisations would be better off measuring less. Ridgway’s own conclusion was that quantitative measures are genuinely useful tools; his concern was indiscriminate use and undue confidence in them, not their existence. The research reviewed here is best read as evidence for periodic re-examination of what a measure still represents, not as a case against measurement — a distinction that matters, because the two are easy to conflate. None of the research offers a way to predict how far a measurement can drift from what it represents before an organisation’s ordinary review processes ought to catch it; what the evidence consistently shows is that they routinely do not.
For how this pattern presents in other kinds of organisation, see other sectors.
Questions worth considering
- If a measure has stayed the same for years, is that evidence of consistency, or evidence that nobody has questioned it?
- Who would notice if this measure quietly stopped meaning what it once meant — and is that currently anyone’s role?
- Which of the organisation’s longest-standing indicators has changed least, and does that match how much the underlying work has actually changed?
- When a target is hit reliably, does that reflect the work becoming easier to do well, or easier to satisfy?
This guide sits within Performance. A related guide: Why the Same Standard Produces Different Results by Site.
Your team is facing similar challenges?
Subscribe to The Riddle Ledger
Perspectives are sent as they are published, with no promotional email, and you can unsubscribe at any time.