The symptom
The phrase “KPIs stop reflecting reality” usually conjures a specific image: a number that has been gamed, distorted or hollowed out until it no longer means what it once meant. That image is accurate often enough to be a genuine and well-documented risk. But treating every reassuring KPI as automatically suspect is its own kind of error, and a less discussed one. An organisation that has internalised the lesson that metrics can mislead sometimes overcorrects into a posture where no favourable number is ever simply believed — which carries its own cost, because scepticism applied indiscriminately consumes the same scarce attention that should be reserved for the KPIs that actually warrant it.
The genuinely useful symptom to watch for, then, is not “a KPI looks good.” It is the presence or absence of a check: has this particular reassuring number ever actually been examined, or has it simply never been challenged? Those are different states that look identical from the outside, and only one of them is a problem.
Why it happens
V. F. Ridgway’s foundational 1956 research on performance measurement, published in Administrative Science Quarterly, remains the clearest account of how a KPI can drift away from the reality it was built to represent — documented cases of employment interviewers optimising for interview volume at the expense of placement quality, and case-quota investigators selecting easier cases near month-end. Ridgway’s warning was specifically against indiscriminate trust in quantitative measures, not against measurement itself. That distinction is frequently lost in how the finding gets applied in practice: an organisation that has learned, correctly, that some KPIs mislead can slide into treating all of them as suspect by default, rather than developing a habit of checking the specific ones that carry real decision-making weight.
What leaders typically conclude when a KPI looks unexpectedly favourable is one of two things, depending on their prior experience: either the number confirms what they hoped to see and is accepted without further examination, or — increasingly, as awareness of metric-gaming has spread — the number is treated with automatic suspicion regardless of whether anything about it is actually unusual. What the evidence suggests is a middle position that is less satisfying than either extreme: some favourable numbers are real, some are distorted, and the only way to tell the difference is to look at the specific conditions behind the number rather than applying a blanket rule in either direction.
The competing explanation worth taking seriously in any specific case is the boring one — that the comparison genuinely holds, the benchmark group is genuinely comparable, and the organisation’s performance is genuinely what it appears to be. This is not the more interesting story, which may be part of why it gets underweighted; a clean result rarely generates the same scrutiny that a suspicious one does, even though clean results deserve exactly as much of a check the first time they are relied upon.
What it looks like in practice
Consider an organisation that compares its cost ratio to a published industry benchmark and concludes, on that basis, that it is operating efficiently. The reasonable first instinct, particularly for a team already alert to the risk of misleading metrics, is caution: is the benchmark group actually comparable, or is this a case of a number that flatters because it was never checked?
In this instance, the check is done properly. The benchmark group is examined against the organisation’s own operating model, scale and market, and found to be genuinely comparable on all three. The organisation’s cost ratio turns out to be favourable for defensible, identifiable reasons — not an artefact of a mismatched comparison group or a definitional quirk in how the ratio is calculated. The comparison holds. Not every reassuring number is hiding something, and treating scepticism as a universal default is, in its own way, as unhelpful as never applying it at all.
The more useful practice this scenario points to is not a rule about which KPIs to trust and which to distrust. It is a rule about which KPIs have actually been checked recently against an independent view — a different data source, a different calculation method, a comparison group examined rather than assumed — and which have simply never been challenged, favourable or not. A KPI that has been checked and holds up deserves more confidence than one that merely looks fine and has never been tested either way, regardless of which direction either number happens to point.
This is worth stating as its own principle because the alternative — treating “unchecked but favourable” and “checked and confirmed” as equivalent — is a quiet but common error, and one that a genuinely reassuring result like a held benchmark can inadvertently encourage, precisely because there is rarely any obvious prompt to check something that already looks fine. The KPIs most likely to go unchecked for the longest period are, by this logic, often the ones that have looked acceptable for the longest period — which is the opposite of where scrutiny would naturally be drawn if scrutiny is only triggered by an alarming number.
For how this pattern presents in other kinds of organisation, see other sectors.
Questions worth considering
- What would have to be true for this particular comparison to be unsafe — and has anyone actually checked, in either direction, rather than assumed?
- Which of the organisation’s currently favourable KPIs have been examined against an independent source recently, and which have simply never been questioned?
- Is the organisation’s current posture toward its own metrics one of routine, proportionate checking — or does it swing between uncritical acceptance and blanket suspicion depending on mood or recent history?
This perspective is part of Performance.
Your team is facing similar challenges?
Subscribe to The Riddle Ledger
Perspectives are sent as they are published, with no promotional email, and you can unsubscribe at any time.