MOST OF the story of modern management is a quixotic struggle to substitute numbers for judgment. ‘Managers start off trying to manage what they want, and finish up wanting what they can measure,’ sighed strategy guru Igor Ansoff. But management is essentially a moral, not a numerical, occupation. The proof is the self-fulfilling prophecy.

Expectation is management’s secret weapon. It is the self-fulfilling prophecy – a grander name for the power of expectation – that turns it into an instrument of mass destruction or the reverse, a powerful amplifier of trust and co-operation.

First described in the 1940s by sociologist Robert Merton, the mechanism by which belief creates its own reality is unique to social science. In the physical sciences, human attitudes have no effect on reality: all the Pope’s authority could not make the sun orbit the Earth. Not so in human affairs, however, where even if a proposition is ‘wrong’ to start with, if enough people believe and act on it, de facto it becomes ‘right’.

Management writers Jeffrey Pfeffer and Robert Sutton have identified 500 research studies on the self-fulfilling prophecy. They show that, to a remarkable degree, performance depends on expectation, whether or not objectively justified. ‘Independent of other factors,’ writes Sutton, ‘when leaders believe their subordinates will perform well, positive expectations lead to better performance. And the converse holds for poor performance.’ Managers’ theories about performance and ability are self-fulfilling.

One effect of this is to turn the company into a battleground between two rival views of human nature – with reverberating consequences, because the self-fulfilling mechanism means that one of them will eventually make itself ‘right’. Suppose managers assume that employees are basically lazy and self-interested, and must be coerced into doing a good job. They will then use hierarchy, tight supervision and incentives and punishments to make them comply. But research evidence is that the more people are treated as naughty children the worse they behave, justifying even more repressive methods. The circle is not only self-fulfilling but vicious. A company built on assumptions of self-interest and opportunism progressively recasts employees in its own misshapen image.

Meanwhile, a company that functions on the basis of trust and co-operation creates a system in which honest, co-operating people flourish. The more it flourishes, the more the norm is reinforced. The self-fulfilling prophecy makes the company quite literally into a force for good.

The battle of expectations, and consequent adjustment of ‘reality’, occurs along the management ideas chain. Studies show that students on MBA and economics courses (unlike others) come to embody the assumptions of rational self-interest that underpin the teaching. MBA students become less concerned with customers and ethical concerns the longer they study. Self-fulfilling individual self-interest is reinforced at the next level by business-school rankings, in which the salary effect of taking the course is a high-ranking criterion. But the self-interest is learnt, not hard-wired: a self-fulfilling assumption.

Or take incentives. They are ubiquitous, but be careful, because incentives work, most powerfully, to teach people to expect them. One snag is that people who come for incentives leave for better ones, so the ratchet moves ever upward. The inexorable rise of chief executive pay is an awesome example of double self-fulfilling prophecy, incentive expectations being compounded by learnt self-interest and all the doctrines based on it, such as agency theory and options – which is why nothing seems to be able to slow it down.

Another problem with incentives occurs all over the public sector. As with the new GP contracts, to get people to concentrate on a few headline measures, the government sets targets and incentives for reaching them. This has two effects: doctors find ways of meeting the targets, so payments go through the roof but also by definition their motivation changes. (Extrinsic) incentives drive out (intrinsic) professionalism: but which doctor would you rather go to, one who is motivated by money, or by doing a good medical job? No wonder patients are dissatisfied and doctors conflicted.

Some dilemmas raised by the power of expectation are less clear cut. In his provocative book Weird Ideas That Work , Sutton suggests that one good way to approach innovation is to decide to do something that will almost certainly fail, then convince yourself and everyone else it is bound to succeed. You can probably see what’s coming. Statistically, almost all innovations flop, but telling the truth will make failure certain. Conversely, the one thing you can do to change the odds at least slightly in your favour is to ignore the evidence and persuade yourself and others that it will be a triumph.

What to do? Most management books won’t tell you, but deciding which ‘reality’ you want may be the most important decision a manager ever takes.

The Observer, 5 November 2006