For an indication of why, take the case of a typical local authority child protection department which operates to two standard measures. For children at serious risk, it must carry out a fast initital assessment of 80 per cent of cases within seven days. For a full core assessment, the standard is 35 days. The department meets both standards; under the widely-used ‘traffic-light’ signalling system (red-amber-green) it rates a green, so managers judge that no further action on their part is necessary.
Now look at the same department through a different measure: the end-to-end the time taken to do the assessment from first contact to completion. The picture that emerges is very different. The urgent assessment predictably takes up to 49 days, with an average of 18.5, while the 35-day assessment takes an average of 49 days, but can equally take up to 138. Worse, the clock for the core assessment doesn’t automatically start when the initial assessment finishes but only when it is formally opened. So the true end-to-end time for the 35-day assessment is anything up to 250 days. ‘Now tell me Baby P and Victoria Climbié were one-offs,’ says Vanguard consultant Andy Brogan, who gathered the data, grimly. ‘They weren’t – they were designed in.’
So how could the department have been meeting its standards? Consider what has happened to the department’s purpose. From assessing and protecting children, the imposition of the government-mandated measures, plausible but arbitrary (why seven days? why 80 per cent?), has shifted the de facto purpose to meeting the standard within officially laid-down parameters – which it does by recategorising, shutting and reopening cases as permitted by the guidelines.
No learning takes place, because these measures are not about learning but ‘accountability’, in this case to government. Remember, management thinks that because it is meeting the standards, no further action is necessary. It’s not far from here to Mid Staffs, where Sir Robert Francis was strumped to ascribe blame because everyone met their targets and thus covered themselves (which is what accountability really means). In the end he could only attribute the failings vaguely to ‘the culture’.
Unlike standards, the end-to-end measure on the other hand throws light on how well the department is meeting its purpose. Learning takes place. The workplace conversation is no longer about how to meet the standard but what accounts for variation and how to how to save time in assessments to make children safer. Contradicting the traffic lights, action is urgently needed. As the process is repeated, improvement becomes continuous.
Momentous conclusions ensue from looking at measurement this way. The ‘why’ of measurement (purpose) precedes the ‘what’. If the measures are not related to real purpose, the measures become the purpose, and better ones signal improvement that is dangerously illusory.
The bottom line is that measures can be used for either accountability (outcomes, targets) or for learning (purpose-related, commonly end-to-end times or total not unit costs) – but not both. Yes, this is our old friend Goodhart’s Law (which says that the moment a measure is used to manage by it loses its validity as a measure) in a different guise. It’s management’s uncertainty principle. Accountability measures can’t be used for learning and improvement because a) they don’t aay anything useful about what works and why, and b) as in the child protection department, the story they tell is a false one. Measures for learning and improving, on the other hand, dial down the need for external ‘accountability’, since they cause people to respond directly to the customer or person in need.
Importantly, the choice of measure affects many other aspects of organisation, including structure. An organisation using outcomes-based measures like targets and service levels to manage performance naturally adopts devices designed for accountability such as incentives, functional organisation, outsourcing, shared services and separate front and back offices – ‘dangerous idiocies’, in Brogan’s words, that effectively blind managers to what is really going on in their organisation. When things go wrong, it is not because people are wicked, stupid or uncaring; it’s because they are working in a system where data is constructed not for learning and improving but holding people to account.
The horrible results of bad measures, from banks that bankrupt societies to hospitals killing their patients, are all around us. Given the evidence that hitting the target so often misses the point, why is the stranglehold of the ‘dangerous idiocies’ so complete? One reason, argues the Newcastle University researcher Toby Lowe, is that the problems are conceived of as technical challenges that are capable of solution, for instance by sharpening sticks and carrots and increasing accountability. The result is an ever sharper focus on doing things better that shouldn’t be done at all. Jeremy Hunt’s proposed criminal sanctions for hospitals that fiddle their mortality figures fall straight in this category.
‘Having lost sight of our purpose, we redoubled our efforts’, as W. Edwards Deming sardonically summed up this process half a century ago. It’s time for a change. A choice has to be made. Either we go on using accounting and accountability measures to manage performance in a predetermined way, in which case we shall continue to be surprised when the things they bring about go spectacularly wrong; or we switch to measures that, as Brogan says, ‘help and act on the causes of variation in performance so that we can connect actions with consequences.’ That’s what changes management from a succession of hunches t a systematic, scientific endearvour; that’s why measurement is a leadership not a technical issue.
What we can learn about innovation from Marks & Spencer’s Plan A
Read my article for The Foundation on Marks & Spencer’s Plan A here
Why regulation is not the answer
When something goes wrong the knee-jerk reaction is to demand tighter rules to stop it happening again. Whether it is children at risk, NHS hospitals, banks, MPs’ expenses or bankers’ pay, as night follows day the enquiry set up to investigate the scandal/tragedy/accident at one point recommends stricter rules and regulation to corral behaviour into ever narrower channels of compliance.
The regulatory reflex is understandable. Dating in its present form to the wave of privatisations in the 1980s, regulation seems to offer a pain-free means of taking the edge off market excess on one hand and monopoly public-sector indifference on the other. In that sense, regulation is an attempt to find a middle way between the two extremes.
It is true that a framework of ground rules is essential for both a functioning market and functioning public services. As Michael Porter showed in an important HBR essay in 1995, positive regulation that frames broad objectives but crucially leaves method open to experiment can be a powerful incentive for innovation. But good regulation is rare. As is becoming increasingly clear the other kind – prescriptive rules that specify method and detail what is and isn’t permissible – is deeply problematic. And there is now so much of it about that the problems it has thrown up are as bad as, or in some cases worse than, the ills it was expected to cure
Leave aside for a moment the ever-present issues of information asymmetry and potential regulatory capture. The bottom line is that the regulatory reflex drives a dynamic which makes it ever more intrusive, ineffective and costly.
Regulation is a substitute for trust. We apply it to the private sector because we don’t trust companies not to extract rents from customers, suppliers and society to transfer to shareholders (and executives). We use it in the public sector when we don’t trust managers and civil servants not to put the producer interest first.
Mid Staffs and horseburgers show that the cynicism is understandable. But here’s the thing. Regulation does nothing to solve the underlying problem. Rather, it makes it worse, systematically manufacturing and amplifying the mistrust so that it becomes self-defeating.
Like targets and inspection, regulation focuses attention on what’s being regulated, to the exclusion of what isn’t. It thereby sets up an arms race between regulator and regulated that, as Said Business School’s Colin Mayer points out in his provocative new book, Firm Commitment, turns compliance and risk departments into their mirror-image – compliance-avoidance and risk-augmentation units.
Given the aforementioned information asymmetries and the impossibility of foreseeing all available avoidance ploys in advance, it’s not surprising that regulators are usually one if not two steps behind. By the time new regulation comes into force the horse has long departed to another unlocked stable. Sarbanes-Oxley, passed in the wake of the Enron and WorldCom scandals, was perfectly impotent to prevent the dotcom rip-offs and even less the 2008 financial crisis. In the same way the 2010 Dodd-Frank Act is fit for the purpose of preventing the 2008 crash but not the next crisis that is even now gathering in the wings.
The enquiry into Mid Staffs is a good example of the regulatory ratchet in action. As Nicholas Timmins noted in the FT, ‘Too much of the Francis report… works on the assumption that the answer to failed regulation… is yet more regulation.’ Doctors and nurses are already regulated. The beneficiaries of additional rule-making will be the parasite parallel industries that always spring up to exploit them, in this case ambulance-chasing lawyers. And the threat of criminal sanctions will not only make life even more difficult for whistle-blowers (who will understandably hesitate to put colleagues behind bars) but may well also deter potential board members from coming forward to do what is already a thankless job.
Much though one sympathises with MEPs who want to curb bankers’ bonuses and Swiss voters who have backed tough restraints on all high pay, it’s the same with pay. The case against regulation is not that overpaid chief executives don’t deserve to be cut down to size – they do – but that by deflecting attention from the real issue regulation will make matters worse. Focusing managers’ and HR departments’ attention on pay rather than the job will do nothing to benefit customers, and with their inbuilt information advantage they will surely drive a a phalanx of Rolls Royces through the rules.
In the meantime the real debate – not about the size of bonuses but about the flawed and deeply unscientific payment-by-results mentality that creates the perceived need for executive incentives in the first place – will not take place. In the same way, the crisis of the meat-processing industry of which uninvited horsemeat is the symptom is not poor regulation but the non-existent trust and excessively transactional relationships in the UK supply chain, something that is outside the ability of regulation to fix.
It’s often forgotten that all regulation carries costs both direct (the cost of employing regulators, inspectors and data-processors) and indirect (the cost and opportunity costs of gathering information and of doing the things that regulation demands even when they’re useless) and those costs rise dramatically the more the bureaucratic friction intensifies. As LSE’s Michael Power notes in his book The Audit Society, the societies that have attempted to insitutionalise checking in the shape of regulation, inspection and audit on a grand scale ‘have slowly crumbled because of the weight of their information demands, the senseless allocation of scarce resources to surveillance activities and the sheer human exhaustion of existing under such conditions, both for those who check and those who are checked’. That’s seems like a pretty good description of what’s happening to the NHS right now.
Is there an alternative to that depressing prospect? Yes, there is. It consists of managing properly, that is, using measures related to purpose to gather evidence of what works and then use it to improve, in the way scientists do. More on that next time.