Tips and Traps

Tips and Traps
Prev	9. Evaluation	Next

An evaluation should cover more than just the management and output from the consultancy itself. To be useful, it should also include the process of initiating the consultancy (including the justification) and the subsequent use of outputs for policy formulation, organisational change, or other purpose. In essence, the evaluation should test whether the initial objectives have been satisfied and identify lessons learned. It should not be used as an excuse to lay blame for problems encountered on the way.
Additional value can be extracted from a consultancy if the consultant provides a general debriefing on the project. Consultants often pick up additional information that is of use to the client but is not included in a report because it is not directly relevant. Similarly, consultants can brief clients on any avenues investigated that were ‘dead ends’: such information may not appear in a report, but helps provide a more complete picture of the issues investigated.
Where consultants are hired regularly, a file containing ‘lessons learned’ is often useful, particularly if it includes basic details such as the nature of the project and the agency involved. New staff, or those with less experience in letting contracts can skim the file to gain an impression of pitfalls to avoid, and useful practices to emulate.
To a certain extent, successful use of consultants requires practical experience. It is therefore a good idea to invite all of your staff , not just those who were involved directly, to sit in on evaluations. The knowledge gained will be at least as valuable as anything that can be gleaned from publications, and will contribute to improved use of consultants in the future.
Evaluations can also be carried out before the finalisation of a project . In large projects, it may even pay to seek external evaluation of tender documentation before its release, particularly where subject matter expertise is important.
It may not be realistic to expect busy Public Service managers to carry out an evaluation of every project. However, agencies seeking to improve their general level of performance in letting consultancies could select a number of projects each year for use as case studies in a ‘no-blame’ atmosphere to allow all staff to learn from them. It would be important, however, to ensure that agency performance (for example, turnaround time for selecting a consultant) were assessed, as well as the performance of the consultant. An alternative is to link performance assessment directly to the outcomes of a consultancy project.
Where useful data or information has been collected and there is a likelihood that they may be of use to researchers or policy makers elsewhere, it is worth reviewing the costs and benefits of making them accessible. Dissemination to academics and State or local governments, reference to its availability in your agency’s annual report, or posting on a website, can help make it more generally available. From a national perspective, resources will be saved in future if data or information collection does not need to be duplicated.

Table 7 — Evaluation: Risks and Mitigation

Type of risk	Likely consequence	Mitigation strategy
Consultant resents contract evaluation	Inadequate evaluation Damage to relationship between client and consultant	Foster ‘partnering’ relationship with consultant throughout project Hold regular reviews and evaluations as part of contract management process and progress reports Make clear from outset (including in tender documentation) that final evaluation is to occur Include evaluation clause in contract
Failure to identify problems and address them	Missed opportunity to improve future procurement activity	Develop systematic evaluation methods and techniques Use clear, relevant evaluation criteria

Prev	Up	Next
Australian Government requirements	Home	10. What if things do go wrong?