Performance measurement has become so widespread that it is impossible to know all that is taking place within governments across the world.^[1] Also, the labels and the focus of performance measurement systems shift, usually when a new government takes office. Positive, action-oriented words are usually chosen as names for such systems — 'Measuring Up' in Alberta, the 'Oregon Benchmarks', the 'Minnesota Milestones' and 'Best Value' in the United Kingdom. Despite such inspirational language, some of the early leading performance measurement systems have recently been reduced in scope or dropped entirely. Keeping track of the rise and fall of performance measurement systems is made somewhat easier by the Internet (government websites, on-line reports and electronic journals) but the problem has become information overload and knowing what credibility should be assigned to the ‘official’ descriptions, given that governments are interested in promoting the best possible reputation for their efforts.

There is no single, ‘one best’ approach to performance measurement. A government needs to develop an approach that fits with its constitutional/institutional arrangements, its political and administrative traditions, its size and organisational capabilities, its current environment and issues and, not least important, what it can afford. The tendency has been for governments to apply a single approach uniformly to all departments, non-departmental bodies and programs. This ‘across-the-board’ approach may have the apparent virtues of consistency, comparability and fairness, but it is not without problems. The fundamental problem is that organisations and programs differ in the extent to which they are amenable to measurement, especially in terms of linking outcomes in society to programs. Routine, operational programs with narrower goals and better understood production processes allow for easier and more coherent measurement than ‘softer’ programs serving broader, more controversial goals, the achievement of which is not well understood and/or depends upon the exercise of a wide measure of professional discretion and judgement. In view of these differences, there is the legitimate concern that the use of performance measurement, especially an insistence on quantification of outcomes and benefits, will create an institutionalised bias in favour of so-called ‘hard’ programs whose production processes are relatively well understood. The need for balance between quantification and relying on the numbers versus qualitative evidence and telling the ‘performance story’ is examined later in this paper.

During the past decade, the federal and provincial governments in Canada have developed two broad approaches to the development of measures and indicators of performance. In Alberta, Nova Scotia, Ontario and Quebec, governments have reported on the performance of the entire government in terms of the presumed impacts of their activities on society. This ‘social-indicator’ type of approach supports external, political accountability by providing information which relates to the concerns of citizens. However, the selection of indicators to feature in ‘report cards’ to citizens is inherently arbitrary (why waiting lists for surgeries rather than impacts of preventative measures in health care?), and the problems of attributing changing economic and social conditions to government actions or inactions is next to impossible. Other provinces and the Government of Canada began their performance measurement efforts by requiring individual departments to prepare business plans and performance reports. The ‘business-line’ approach is more of a managerial tool than something which would normally be used by politicians and the public.

These two broad approaches — the system-wide and the business-line — could be pursued simultaneously and complement one another. This has been the evolution of the performance reporting system in the Government of Canada. It began by publishing performance reports on a departmental basis and now more than 80 such documents for departmental and non-departmental bodies are tabled in Parliament on a annual basis. Beginning in 2001, the President of the Treasury Board also released on an annual basis a report to Canadians covering nineteen social indicators, reflecting the four themes of economic opportunities, health, the environment and the strength and safety of Canadian communities.^[2]

This evolution can be contrasted with Manitoba’s experience. Under the Progressive Conservative government of Premier Gary Filmon (1988–1999), the province adopted a program called 'Manitoba Measures' based upon the departmental business plans model. Conferences, training sessions and investments in performance measurement took place as departments began to create the capacity demanded by the ‘centre’ of government. Then, after the New Democratic Party (NDP) took office in September, 1999, the 'Manitoba Measures' program was abruptly dropped. A full post-mortem of its demise has yet to be written, but it seems the new government believed that the program was ineffective because it did not address the real concerns of politicians and was not integrated with program planning and budgeting. Presumably encouraged by the new government, the Manitoba Bureau of Statistics began development of a 'Manitoba Well Being Index' which would track economic, social and environmental conditions to produce an aggregate measure of ‘total well being’. Just as the ‘business line’ approach seemed to be consistent with the ‘market’ philosophy of the Progressive Conservatives, the ‘social well being’ approach matched the NDP’s commitment to a broader role for government.

Even when governments limit their performance measurement efforts to individual departments and programs, the problems of defining and measuring successful performance is only slightly less challenging. Governments have developed a number of different frameworks to identify successful programs. Probably the most common framework involves the so-called ‘three big Es’: economy, efficiency and effectiveness. In simple terms each of the three Es can be described as follows:

These elements have been at the centre of all the ‘rational’ management approaches applied to the public sector over the past four or five decades. However, the framework does not cover some dimensions of performance. It does not cover the fourth Big E, equity, which deals with the distributional impacts of performance. Omitting equity may have an impact on another important E in government, namely electability. In recent years, many governments have added a ‘customer satisfaction’ component to the effectiveness component of the three E model. Also, the framework does not directly address the issue of the capability and desirability of delivering a designated set of results.

In its earlier program evaluation scheme, the Government of Canada indicated that a program was deemed to be well performing when it was:

This framework deals with the desirability of continuing a program, but not the issue of whether the organisation has the capability and capacity to deliver the desired results.

Organisational report cards represent another type of performance measurement and reporting.

Exhibit 4 presents one interpretation of the requirements for such report cards. In 1987 the Canadian Comprehensive Audit Foundation (CCAF) published a report on ‘organisational effectiveness’ which incorporated a broad definition of performance. The attributes of an effective organisation were:

Several jurisdictions have since applied this framework, including for crown corporations in the federal government and hospitals in some provinces. The framework incorporates features of previous approaches and does address more directly the issue of future organisational capacity to deliver results. Operational meanings behind indicators have to be assigned to each of the attributes. There may be conflict in practice among the attributes — e.g., cost efficiency may reduce responsiveness. Finally, the issue of aggregating data on each dimension to arrive at some overall judgement about effectiveness is a challenge.

William T. Gormley and David L. Weimer, Organisational Report Cards. Cambridge, Mass.: Harvard University Press, 1999. pp. 36-7

The final framework to be discussed here comes out of the management audit work of the Office of the Auditor General of Canada (OAG). The OAG distinguishes among six different components of performance:

As befits the role of a legislative auditor, this framework emphasises the desirability of integrating performance planning, budgeting, monitoring and reporting. The framework also stresses external accountability for results. These two topics are addressed later in this paper.