When I began my academic career over 30 years ago at the Kennedy School of Government, I was attracted, in part, because I thought that by going to the Kennedy School, I would learn a set of techniques that would allow me to undertake a relatively objective evaluation of the performance of government organisations. I thought that the development of those techniques was essential to improving both the design and evaluation of public policy initiatives. While I understood that there would inevitably be problems in achieving this goal, I certainly thought that by this time, almost thirty years later, we would have ‘cracked’ the issue of how to measure performance in the public sector.

At the Kennedy School, I was taught two basic ideas to use in recognising the public value of public policy initiatives and the performance of government organisations. The first of these was the theory and practice of ‘benefit/cost analysis.’ Benefit/cost analysis had a strong theoretical basis in welfare economics. The core idea was that individuals in a society were the only ones who could decide whether public policy results were valuable or not, and that the only way to discover what things were valuable was to ask individuals how they felt about them. The only way to find out the overall value of a given public policy was to sum up those individual valuations. If, for any given policy, there was enough value created for the beneficiaries of the policy to compensate those who suffered losses from it, then, arguably, some public value would be created by the proposed policy.

Although benefit/cost analysis had a strong theoretical basis, it was often applied in a less rarefied and more practical form. In that form, a benefit was defined as ‘any effect that we consider good,’ and a cost as ‘any effect that we consider bad.’ Thus, benefit/cost analysis was sometimes bowdlerised and used simply to identify the good things that were expected to come from a public policy (the benefits – whether individually experienced and evaluated or not), and the bad things that could happen (the costs – whether individually experienced or not.

The cruder form of benefit/cost analysis was closely related to a second method I was taught to recognise public value production. The second idea was called ‘program evaluation.’ Whereas benefit/cost analysis was grounded mainly in the discipline of economics, program evaluation came mostly from the field of statistics. The techniques of program evaluation required an analyst to develop a conception of public value creation by imagining what policymakers were trying to achieve through particular policies and programs. It was the hopes and aspirations of policy-makers to achieve particular social results that was the proper basis for defining value – not necessarily those of individuals who experienced the results (though we might have hoped that the policy-makers were guided by their best guesses about the views of individuals affected by public policies.) Once an analyst understood the initial aims and objectives of the policy or program, it would be possible to capture objective statistics to discern whether the characteristics of the world policy-makers were trying to influence did, in fact, change in the desired directions. One could then make a series of efforts to attribute observed changes to the activities carried out by the organisation. If the program evaluation demonstrated that the desired results had been achieved (at a low cost, with few unexpected bad consequences) then we could conclude that public value had been created. If the program evaluation showed that the desired results had not been achieved, or that the costs or unexpected adverse consequences had been larger than anticipated, then we could conclude that public value had been lost. This was how, as a student in public policy, I was taught to measure the performance of organisations. I subsequently discovered that this was a difficult, somewhat clumsy, and very expensive approach.

I also learned that in the private sector, businesses did not think about performance measurement in this way at all. Rather, managers in the private sector used their organisation’s information systems to track whether the organisation was creating value in the form of revenues over costs, or whether they were managing to maintain a satisfactory relationship with their customers. They were also, surprisingly, making much use of something that looked very similar to the traditional methods we were using in the public sector, which was essentially the measurement of concrete processes and activities. These kinds of measures helped organisations in both sectors make sure they were getting the benefit of technological systems.

Case studies of the McDonald’s and Burger King restaurant chains published by the Harvard Business School revealed a surprising fact. I naturally assumed that McDonald’s and Burger King would analyse the performance of their restaurants in financial terms. Much to my astonishment, I learned that that they paid little attention to the financial performance of each individual restaurant. Instead, McDonald’s was evaluating the performance of every restaurant on 12 distinct characteristics of the experience of being in the restaurant, including whether it was clean; whether the lines were short; whether the rest rooms were clean; whether there was a place to sit down; whether the food was fresh; and whether the cashier smiled at you when she took your money.

These concrete aspects of the experience of being in the restaurant, rather than hamburgers and fries, was McDonald’s theory of what they were producing. What they wanted from each of their restaurants was a guarantee that each of the 12 attributes of the experience was being produced with a high degree of consistency. Their theory was that if they produced consistently good performance along the 12 dimensions of customer experience, they would make money. That theory was tested quarterly, and annually, against the financial performance of the organisation as a whole. But that theory didn’t have to be tested for each individual restaurant. For the most part, all the individual restaurants needed to do was manage themselves to produce the particular set of 12 product/service attributes.

What was really eye-opening, however, was how often they measured the performance of each restaurant: about once every month. Over the period of a year, each restaurant would have been evaluated 12 times on each of these criteria. If we were to suggest some similar kind of performance measurement regime to a public sector executive, the response would likely be, ‘Oh my God, that is an incredibly expensive and intrusive information system!’ Bear in mind that McDonald’s is collecting this information even when they have at hand a steady flow of financial information on the performance of the organisation.

One possible lesson for public sector executives is that if a firm like McDonald’s is making this many extra measurements even when they have financial data available, then it might make sense for them to spend at least that much on measurement when they don’t have good financial information. A second possible lesson is that measurement carried out this frequently might allow top managers to detect relatively small changes in organisational performance – changes that managers could conceivably influence. The fact that small changes in performance can actually be detected gives this performance measurement system more behavioural power to shape the performance of midlevel managers and staff than a less rigorous system would have.

An interesting question arises about which of these two different ways of thinking about these issues – the program evaluation/benefit-cost perspective or the business management perspective – offers a more viable approach to measuring performance in the public sector.