Targets. Stretch targets. 100,000 coronavirus tests a day by the end of April. That turned out well, didn’t it?
When on 2 April health secretary Matt Hancock announced his goal of carrying out the famous 100,000 tests a day by the end of April, the result was predictable.
Given that at the time the daily testing rate was around 11,000, attention naturally focused on the number, and whether it would be achieved. And that’s where the debate stuck for the month. Not on why 100,000 or the purpose of the testing – the number.
On 1 May Hancock used the daily coronavirus briefing to declare that testing numbers had hit 122,347: the pledge had been met. Again, the number hogged the attention. Was it true? Had it really been hit? How?
Well, yes and no. It transpired that between the announcement of the target and the declaration of victory, the definition of ‘completed tests’, which previously meant ‘completed tests’, had quietly changed to ‘completed tests plus test kits in the post’. Subtracting the latter category left a ‘real’ figure of 82,000 actually carried out. Cue a new furore – again about the numbers.
What happened is a textbook illustration of the unintended effects of targets and their faithful sidekick, Goodhart’s Law.
To paraphrase W. Edwards Deming: in the case of a stable system there’s no point in setting a target, because you’ll get what it delivers. But with a non-stable system, there’s no point in setting a target either, because you have no idea what it will deliver. A numerical target in such circumstances is a finger stuck up in the air. Unless you know how to improve system capability permanently (I don’t think so), to hit it you have to be either incredibly lucky (in which case you’ll have to be even luckier to do it again tomorrow); or alter the parameters to make the target attainable.
Hancock did what everyone does when faced with the imperative to hit an arbitrary target: he managed the thing that he could – in this case, the definition of success.
But this is not a harmless bit of jugglery. Deming again: ‘What do “targets” accomplish? Nothing. Wrong: their accomplishment is negative.’ There is a high cost to his action – which is where Goodhart comes in.
As economic adviser at the Bank of England, Charles Goodhart noted that attempts to manage monetary policy by using any definition of the money supply was constantly subverted by actors finding novel ways to circumvent the definition. Hence his law, usually formulated as, ‘when a measure becomes a target, it ceases to be useful as a measure.’ A metric can be either a target or a measure. It can’t be both.
Take Hancock and his tests. To meet his target, he included in his count for 30 April around 40,000 test kits mailed out to the public and to hospitals. Of this number (pay attention here), while the Department of Health and Social Care counts the number of people that test positive, it doesn’t collect figures for tests completed.
What’s worse, since mid-April the government figures include on the same basis (ie people testing positive but not tests completed) 17,500 variegated tests consisting of both diagnostic and antibody tests, thus adding oranges to uneaten, partially eaten and completely eaten apples. As Tim Harford declared incredulously on his latest ‘More or Less‘ show: ‘It’s almost as if they don’t care if the number of tests is consistent or indeed accurate, as long as it’s big.’
At any rate, the upshot of this piece of target-setting is exactly as Deming and Goodhart predicted: the system is beyond comprehension and the figures such a dog’s breakfast that no one can tell what they mean. It seems highly unlikely that Hancock’s original target has been met at all since 30 April, but how can anyone know for sure, including the government? The only certainty about the figures is that they are bogus. You might think that when the subject is life or death, this matters, no?
Yet the damage done by targets doesn’t stop there. What most people don’t get (including a ‘science writer’ on a previous edition of ‘More or Less’) is that the problem with targets isn’t that they don’t work. It’s that they do.
A target is typically a one-club solution to a problem with many moving parts. But the first law of systems is that you can’t optimise one part of a multipart system without sub-optimising others. Any benefits are outweighed by unintended consequences elsewhere in the system. Focusing attention (often with added incentives) on the target rather than the purpose ensures that even if the target is hit, the point is missed.
Targets displace purpose. Tests are a means, not an end. But reporting 100,000 of them became the purpose, both for Hancock and his critics. Yet why 100,000 a day, rather than 75,000 or 250,000? What are we testing for in the first place? Deming once more: ‘Focus on outcome is not an effective way to improve a process or an activity…[M]anagement by numerical goal is an attempt to manage without knowledge of what to do’. Another finger in the air. Or, more tersely: ‘Having lost sight of our objectives, we redoubled our efforts.’
Consistent failure to meet the daily target underlines the point: it bears no relation to purpose, or any other kind of reality, really. Not production capacity, as we have seen. Even more serious, not with demand either – shortage of which, or shortage of which in the right place, has been put forward as a reason for the target debacle.
To be effective, a system needs to be designed against demand. And demand is determined locally. Testing is the first step in the ‘test, trace, isolate’ strategy that the government first initiated and then discontinued in March, and has now resurrected. By definition, that strategy has to play out out locally, where the infection occurs, tracing begins and treatment takes place. But bypassing hospitals and 400 or so existing small labs dotted around the country, all tightly linked to local primary care, the government, as with the Nightingale hospitals, is relying on giant regional testing factories, set up from scratch and remote from their users in every sense. A lurch backward to early 20th century industrial thinking, these in the view of many observers are the exact opposite of what is needed.
We can all support a goal of ramping up testing capacity to the level necessary to meet the purpose, whatever that number is. In fact it would be a good idea. But the minute you set it as a numerical target, it is subject to Goodhart. Managing backwards from an outcome plucked from thin air is a feature of command-and-control management, the only kind of management that government knows. But it is back-to-front. Targets are a disease. They destroy purpose, distort priorities, and soak up energy in games-playing and bureaucracy. They are the problem, to be avoided like, well, the plague.