Peer Review as a Form of Capacity Building: Lessons Learned from Undergoing a Peer Review

More than 18 months have passed since the Supreme Audit Office (SAO) of the Slovak Republic (SR) decided to undergo a peer review of the quality and effectiveness of our work in several areas. The primary objective of this project was to examine the soundness of the SAO’s updated development strategy and its compliance with the mission of a modern audit institution. We focused mainly on audit activities—audit planning, conducting audits, and reporting results. In addition, we decided to have a review of our audit quality assessment methodology as well as our human resource and public relations functions.

One of the SAO’s key values is to promote sound and effective public fund management and to provide the National Council of the SR and the public with objective audit findings. The SAO’s mission is to improve the quality and effectiveness of independent and internal audits to facilitate better public fund management. Accomplishing the SAO’s mission effectively requires an ongoing improvement process.

A memorandum of understanding (MOU) for the peer review of the SAI of the Slovak Republic was signed by (seated from left to right) Frank Grogan, SAI of the United Kingdom; Ján Jasovský, SAI of the Slovak Republic; and Tõnis Saar, SAI of Estonia. Representatives from Poland and Slovenia had signed the MOU earlier.

The goal of the peer review was to improve the SAO’s work and examine the correctness of internal procedures specified in internal directives. We believed that an external independent review would show us the extent to which we had implemented international standards and the overall correctness of our internal procedures and also give us recommendations for improvement.

At the time we were preparing for our peer review, our colleagues from the Austrian Court of Audit were undergoing a peer review, so we took advantage of their experience. We contacted several SAIs and, from those who replied positively, asked the National Audit Office (NAO) of the United Kingdom, the NAO of Estonia, the Supreme Audit Office of Poland, and the Court of Audit of the Republic of Slovenia to be our peer reviewers. I am very grateful to my colleagues from these SAIs for their willingness to cooperate. They provided their knowledge and resources and put together an international team of experts who dedicated their time and efforts to accomplishing the difficult task of examining the quality and effectiveness of our work.

Looking back, we realize that the preceding 18 months meant a lot of hard work for both the peer reviewing team and for us. We prepared the necessary documents to run and implement the project as effectively as possible. During this period, six meetings were held, including discussions at all levels of our SAI. Altogether this amounted to 19 working days at our headquarters in Bratislava and one of our eight regional offices. Dozens of conversations along with studies of our internal regulations formed the basis for examining our work and drawing up the final peer review report.

Peer Review Guidance

Our peer review was conducted in compliance with ISSAI standards, using ISSAI 5600: Peer Review Guideline, which was recently elaborated by Subcommittee 3 of the INTOSAI Capacity Building Committee, chaired by the German SAI. The document was adopted at the XX INCOSAI, and our peer review was the first in the world conducted in compliance with it. I am proud that we can thus provide feedback and contribute to the global SAI community by sharing what we learned.

The Peer Review Report

The hard work of carrying out the peer review has been completed successfully. The report was drafted and submitted for signature, and the signing ceremony took place on May 30, 2011, during the VIII EUROSAI Congress in Lisbon. Participants in the signing ceremony included the presidents of the participating peer review SAIs, the head of the German SAI delegation, and Dr. Josef Moser, General Secretary of INTOSAI.

The peer review report has 18 recommendations divided into five sections. The first deals with audit activities that focus on the central government. These recommendations are aimed at planning, conducting, and reporting the results of SAO audit activities. The second part deals with audit activities involving regions and municipalities and is structured similarly. The recommendations in the remaining three parts deal with audit quality assessment, human resources development, and public relations.

The peer review team concluded that the SAO had responded effectively to the range of challenges that it had faced in recent years. According to the team’s assessment, we have improved the quality and professionalism of our activities. We have also increased the impact of our work by making it more accessible to its stakeholders and strengthening cooperation with the media. We have improved the infrastructure of our organization to support the overall objective of enhancing our performance. The team concluded that this ambition was clearly articulated in the SAO’s Development Strategy and that the strategy itself and its implementation would improve the quality of SAO audit activities and help us achieve our desired impact. The team also concluded that the audit quality methodology the SAO is putting in place complies with relevant INTOSAI guidance and standards.

Recommendations from the Peer Review

The complete text of the report together with its recommendations will be available in Slovak and English on our Web site (www.nku.gov.sk). Let me, however, mention several recommendations that have etched themselves on my memory as they relate to actions we had been considering before we began the peer review. The review simply confirmed that we were moving in the right direction.

Undertaking such audit activities as extending and strengthening our performance audit methodology and improving the skills of staff undertaking performance audit. We fully acknowledge the importance and added value of performance audit in the public sector, as well as the demanding nature of these audits, and we attach great significance to this issue.

Supplementing the resources available for the audits of regions and municipalities.

Developing links with external organizations to introduce regular external assessment of audit quality and considering the introduction of independently moderated professional qualifications for audit staff.

Regarding this last point, INTOSAI may wish to consider establishing uniform ISSAI standards for certifying auditors in the future. Another interesting idea is increasing the use of external advice and expertise.

While it may seem that we have reached the final phase of the peer review process, the reverse is actually true. Members of the team responsible for implementing its results in our office have many tasks ahead of them. These tasks relate primarily to implementing the peer review recommendations to meet the project’s objective of improving selected SAI activities and methods. We have formed five working teams to implement the peer review recommendations. Currently, these teams are preparing an action plan.

Lessons Learned

What can I say to those who are planning for or thinking about a peer review? Implementing a peer review is demanding and requires a significant investment of human and material resources. Since a peer review is international in nature, translations and interpretation may be expected. It is also necessary to inform all staff beforehand and prepare them for the work and discussions that the peer review will entail. Support from the SAI’s top management is important throughout the implementation and reporting phases because the results of the peer review must be implemented in accordance with internal administrative procedures and regulations.

While undertaking a peer review is very demanding, its benefits and added value greatly exceed the demands. Recommendations confirm whether the institution’s management is headed in the right direction and advise management on ways to refine its methods to produce more effective work of even higher quality. Considering the unique position SAIs occupy in the public sector and the rapidly changing conditions and new challenges we face, a peer review is a worthwhile investment of time and resources.

If you choose to do a peer review, we recommend using ISSAI 5600, where you can find guidance and instructions on how to proceed. The appendix to the guideline contains a helpful and practical checklist of possible questions to ask during the process. Other products of Subcommittee 3 of INTOSAI’s Capacity Building Committee can also be used. (See the CBC Web site at cbc.courdescomptes.ma.)

Last but not least, a peer review enables you to answer the question, Who audits the auditor? Audit work is at the core of each SAI’s daily activities. By voluntarily undertaking a peer review, you open your work to external assessment and audit based on international auditing standards. This is of great importance at both the national and international levels.

Conclusion

In conclusion, let me repeat what I said at the beginning of this article: accomplishing the SAO’s mission requires an ongoing improvement process. Our peer review was not undertaken in isolation. At present, we are carrying out a 3-year educational project cofinanced by European Union (EU) funds and are implementing quality management according to the CAF (Common Assessment Framework) model. We are also building a new information system, cofinanced from EU funds as well. That is why the peer review has been so significant for us: we need to know that we are moving in the right direction in a high quality and effective manner.

Let me again thank our international partners—the National Audit Office of the United Kingdom, the National Audit Office of Estonia, the Supreme Audit Office of Poland, and the Court of Audit of the Republic of Slovenia—for their cooperation and assistance in the peer review.