A. Problem Analysis

1. What was the problem before the implementation of the initiative?

With the evolution of health policies in Brazil, the Federal University Hospitals (HUFs) have not only played a fundamental role in the training of health professionals and in the advancement of research and technology, but also in serving the entire population contemplated by the Network of the Brazilian Health System. From this point forward, the Federal University Hospitals had to perform procedures of medium to high complexity, no longer acting exclusively as teaching hospitals.
Despite this change in purpose, no significant adjustments were made in the management models adopted by the HUFs, or in the physical, technological and human resources structures; this lack of change has caused problems in the services being offered both to medical students and local patients. Groups affected by this issue include the entire local community – especially the underprivileged, who most need this service – the academic community, students – who were deprived of resources and good quality structures due to the misuse of existing resources – and even vendors – due to a lack of organization.
In December 2011, the Brazilian Hospital Services Company (Ebserh), a state-owned company, was created to administrate all federal university hospitals. There are currently 51 Federal University Hospitals (HUFs) distributed throughout Brazil, 39 (thirty-nine) out of them are already administered by Ebserh. Until Esberh was conceived, these university hospitals did not have internal audit or internal control units, as it was detailed in analyses carried out by the Brazilian Court of Public Accounts (TCU).
The new Ebserh’s management found serious deficiencies, including a lack of administrative and process control structures. In 2011-2012, the Audit Court of the Union (TCU) carried out audits in 23 (twenty-three) of the hospitals that would be managed by Ebserh. As a result, the TCU recommended changes in the internal controls of said hospitals, since the scenario revealed by the audits showed there were no financial security guarantees. Potential damage to the protection of financial and material resources was pointed out, as well as a high possibility of wastage, loss, misuse, and even fraud.
From interviews with the Federal University Hospitals managers, the Internal Audit Unit learned that these managers needed direct support to carry out their responsibilities in the form of real information management that provided a clear picture of the hospitals’ situation.
The problems were known, but not their true causes, and this led to poor medical care and inadequate training for doctors and future health professionals in Brazil.
Considering how important it is to develop work towards due control within these hospitals and the absence of efforts towards progress in this matter, a decision was made to identify the greater needs of the hospitals' managers, investigate the existence, and identify and mitigate risks.
Thereafter, it was concluded that it is necessary to define strategies to produce effective information management to support the managers’ decision-making process, as well as facilitate the establishment and commencement of the internal activities of administrative and operational control in each Ebserh hospital.

B. Strategic Approach

2. What was the solution?

More transparency and Combating Corruption. The practice has as one of its objectives to involve the entire community in identifying and proposing solutions to the problems encountered in the public service. In health or in any area. The results indicate a change in the relation between the population and the public services.
Improved care for the poorest and most vulnerable population. Public employees begin to understand their activity as a service and not a favor to the public. The population also learns to complain about poor care and bad care is punished more often.
Innovation in the management of public institutions through the adaptation of internationally accepted and recognized management tools with the methodologies and legislation applicable to the public service.The practice begins with a process of self assessment. From your result information that enables you to take the most correct actions. In the words of more than 20 directors of institutions where the practice was applied: "The problem was known, what we did not know were its causes. Knowing what actions resulted in a certain problem, we can discuss with the community internal and external to the institution, proposals Solutions. "

3. How did the initiative solve the problem and improve people’s lives?

The strategy to create and develop this program stems from managers of general and maternity hospitals that attend to the public and are run by the Institution.
35 general hospitals and 5 maternity hospitals spread across Brazil, with the total staff of over 60 thousand workers, which attend around 2 million and 100 thousand patients¹ per year of a total of 17 million¹ patients already registered and treated in all hospitals of the Ebserh network, had their first Internal Audit Unit/ Internal Control Unit in 2012. Consequently, the proposed method came down to:
How would and auditing/internal control aid management to effectively better assistance to people in need and training of health workers? By substantially contributing to the improvement of people’s lives.
After technical visits and interviews in each one of the general and maternity hospitals in Ebserh’s network, we were able to determine that the biggest issue was a lack of effective information and interest from managers in collaborating in the solving of these hospitals deficiencies. Later on, we discovered that, in reality, the managers only knew the consequences of the issues, but not their real causes, which is why the previous adopted measures did not focus on causes, but rather on consequences.
The nonexistence of any material regarding this matter or previous experience in Brazil led 100 managers and 5 auditors to meet during the first semester of 2014 and identify the best governance practices for that scenario and specific characteristics of general and maternity hospitals, which were all public teaching hospitals.
For these best practices, elements were adapted from internationally recognized control and management tools such as: COSO – Committee of Sponsoring Organizations of the Treadway Commission, e COBIT – Control Objectives for Information and related Technology, as well as governance rules set by the Brazilian government in the Brazilian Court of Public Accounts (TCU – Tribunal de Contas da União) and the Comptroller General of the Union (CGU – Controladoria Geral da União).
The tool named Governance Environment Maturity Measurement (MENSURAÇÃO DA MATURIDADE DO AMBIENTE DE GOVERNANÇA- MMAG) first began being used in August 2014, and has currently been implemented in 21 general hospitals and 5 maternity hospitals.
The MMAG practice may be considered a self-evaluation modality, assisted by internal auditing, that from its results stimulates the creation of an Internal Control Committee comprised of representatives from all departments in the health institution (preferably the ones who are not in management positions), to discuss and propose possible enhancements to hospital managers.
6 months after MMAG’s results are presented and Internal Control Committee and agents start to operate. An internal audit is then conducted with the intent of formally evaluating whether the identified weaknesses and risks really took place, whether the proposed actions were efficient and contributed to an effective improvement in the assistance to the impoverished population and in the training of health workers. This audit is then sent to Brazil’s official control bodies so that they can conduct their own audits and annual assessments of Ebserh units.
There is still a third phase that happens 3 to 6 months after the audit so as to monitor the subsequent advancements as well as assess evolution of risks identified when the moment when the control process began.
¹ Official numbers of Ebserh management system on 03/15/2017- AGHU.

C. Execution and Implementation

4. In which ways is the initiative creative and innovative?

This is an initiative within the scope of the internal audit units of the Federal Public Administration, in Brazil, and it is unique, since it proposes an innovative joint action between auditees and auditors in identifying their weaknesses.
The Program does not primarily focus on error detection but rather on management performance, with all the stakeholders involved in the process identifying weaknesses and potential improvements in their work. It also stands out as a simple and inventive way of presenting and accepting the internal audit unit, since the auditor is a facilitator in the measurement [evaluation] process.
Its singularity stems from the fact that its development is based on the adaptation of established tools, such as COSO and COBIT, and on Brazilian auditing and control standards in public administration, which had not yet been accomplished, while encouraging the participation of all institutional levels.
Another point is the improvement of the company's Management environment which consists of the idea that everyone is aware of their share of responsibility and seek better results in serving citizens who need this public service as well as acknowledge that good work depends on everybody's contribution.

5. Who implemented the initiative and what is the size of the population affected by this initiative?

Ebserh Brazilian Company of Health Services) is a national company with 100% of its capital belonging to the Brazilian government. It was created in 12/15/2011 to offer free of charge services in medical-hospital and outpatient care, as well as therapeutical and diagnosis support to the community, mainly the vulnerable and the impoverished. It also aims to train individuals in the public health sector (doctors, nurses, nutritionists, physiotherapists, etc.) in all 40 Federal University Hospitals (HUFs) scattered across Brazil.
Ebserh already directly manages 40 health units in the country, 35 out of these being federal university hospitals and 5 maternity hospitals with free assistance to the most vulnerable population groups, raising their official indicators on bed occupancy rates and number of health and medical assistance professionals.
Out of 60 thousand workers that are currently operating in Ebserh’s hospitals network, 20 thousand were hired during Ebserh’s management and are directly responsible for over 15 thousand appointments per day¹ across the network.
The program has already been implemented in 26 out of these 40 hospitals.
According to the Brazilian Ministry of Health’s official indicators, Ebserh’s general and maternity hospitals who are in the program attend around 2 million and 100 thousand patients¹ throughout the country, mostly the socially vulnerable and the underprivileged. These hospitals are considered to be part of the largest network in Brazil of medical assistance and training of public and free of charge health.
Thus, we believe those being directly benefited by the enhancements in the work place environment and by the quality in assistance and teaching are all 60 thousand professionals that work in Ebserh health units, as well as all 6 thousand patients across Brazil per day, out of which 1,1 are women with prenatal, childbirth and postpartum vulnerabilities.¹
¹Official numbers of Ebserh management system on 03/15/2017.

6. How was the strategy implemented and what resources were mobilized?

This methodology is interesting and stands out because it stands apart from the traditional form of auditing. Instead of performing traditional evaluation activities by comparing a factual situation with models and ideal situations, and then suggesting improvement actions, this methodology suggests quite the opposite. That is, by considering ideal and needed parameters and situations, identified by the manager, said manager will do a self-assessment. Then, after certain period of time and actions identified and performed by his staff, he has his work evaluated, an evaluation of the inherent and residual risks. This methodology won the 3rd CGU Good Practice Competition in the year of 2015.
For its development during the whole year of 2013 and the first semester of 2014, there were professionals from 10 health institutions of Ebserh, as well as 6 auditors, all of them spread across the country, involved in this process from the start. The financial costs are considered basic, once the development with the health professionals was undertaken through profiting from already the scheduled monthly meetings that took place during a specialization course for hospital managing inside the Syrian- Lebanese Hospital dependencies.
From the definition of what they consider the best techniques, the following step involved the technical contribution of the Ebserh’s internal auditing professionals. Considering that they made use of knowledge acquired during their work in Brazil and also in another countries, either at government institutions or in auditing international private companies, cost was diluted in everyday activities at Ebserh's auditing unit.
In August 2014 we performed the first two tests, one in a famously active maternity hospital that deals with deprived population in extreme social risk, including indigents and prostitutes, and the other at the São Luiz University Hospital, in the state of Maranhão. The latter is a large Hospital containing 3,500 employees, which is responsible for providing service to the whole area's deprived population.
After some needed adjustments to tools and methodologies, we came down to 30 statements distributed in 6 axes of management: Strategic Alignment, Compliance, Process Management, Risks Management, Staff Management and Information Management.
Operationally, auditing is responsible for the initial contact with the general and maternity hospital’s governance; the promotion is carried out by the institution communication sector. Then, a preliminary meeting takes place with the whole governance staff from all levels, in order to gain comprehension and involvement from all heads and supervisors.
Applied data collecting forms are handed out. They are segmented by sector, shifts, work time, etc, where there are 30 statements laid out in a Likert scale, distributed into the before mentioned 6 axes of management, with enough space for comments and suggestions. Additionally, interviews focused on work mediations are carried out. At this point, 45% of the employees from the health unit are participating, ranging from 400 to 1800 people per general and maternity hospitals.
After the insertion of all answers that were on the worksheets, the results are presented to the unit health’s staff, who then decides to convene the Committee responsible for discussion and presentation of proposals to the board.
The internal audit and managers were responsible for the development of these techniques. The application was made under the internal audit orientations with the aid of beneficiaries and the community. The community through the Internal Control Committee carries out the proposed solution while the internal audit and the country’s official control bodies are responsible for monitoring. The response to the initial goal is evaluated by official indicators and by the beneficiaries through a research and ombudsman service. The costs, although minimum when taking the results into consideration, were paid by the government.

7. Who were the stakeholders involved in the design of the initiative and in its implementation?

The initiative began inside the Institution’s Internal Auditing Unit. With the aim to help management by providing useful information to the decision-making process, the internal audit in the 30 Institutions’ affiliates started to meet in videoconferences. They did so with the initial intention of identifying the proper methodology. Due to its nonexistence, the auditors had to create a legal mark for public auditing works inside the government’s health institutions.
Soon, the initiative was accepted by the managers who had never had active internal auditing units and therefore needed information on the source and evolution of their problems so that they could actively correct it, before being held accountable by regulating and inspecting institutions.
In order to the work be successful and reach a minimum rate of reliability of 95%, with an up to 3,5% variation in the answers, the biggest possible number of collaborators had to participate. Currently, this number adds up to 45% each institution’s staff, a total of 12 thousand employees with identified and active participation, including: doctors, managers, health professionals and contracted companies of all levels (catering, cleaning services, etc.) students of several health areas, teachers and patients.
This numbers increases as each new hospital implements its own project. Presently, there are 25 hospitals across the country with projects in progress. We expect that in the first trimester of 2017 5 more hospitals begin the project, which means added 3 to 4 thousand active participation/ opinions on operational areas, operation and assistance to low-income population.
As those responsible for the institution become actively involved, a larger number of people acknowledge the problem and feel themselves part of the solution proposals.

8. What were the most successful outputs and why was the initiative effective?

Results are incredibly surprising. The highlights are:
Community engagement in the identification and proposal of solutions for the problems found: differently from what seen in some countries, Brazilian public administration has centralizing characteristics and hardly exposes its weaknesses. With the practice, so far, more than 12 thousand people have taken part by identifying issues and expressing their opinions. As a result, this practice won the III Good Practice Competition in 2014 organized by the Comptroller General of the Union (CGU), the highest governmental control body in the Brazilian Executive branch.
More transparency. As more people engage in the decision-making process, more transparency and its consequences, such as a better assistance to the destitute, are seen. To coordinate and directly work on the identified risks and issues, Internal Control Committees are created. This was such an expressive and significant result to the Brazilian population that the institution was once again the winner, this time with internal control results, in the CGU’s IV Good Practices Competition in 2016.
An 620% improvement in the assistance to the lower income population, confirmed through an organized channel to receive reporting and complaints to the institution.
Creation of a Correctional Sector in the institution that deals with the accountability of corruption or issues in assistance to the population.
Incentive to and collaboration in the Evaluation of State-Owned Companies’ Integrity by the Brazilian Federal Control Institution (Avaliação da Integridade de Empresas Estatais pela Instituição Federal de Controle do Brasil) by the Comptroller General of the Union (CGU). Through technical meetings in the first semester of 2015 with CGU’s team, this practice was passed on and used in the auditing methodology and techniques utilized. Currently all state-owned companies in Brazil are being evaluated with this methodology.
Evolution of the evaluation of the governance environment of all hospitals. When the Evaluation Audit on Governance Environment Maturity occurred, 8 to 10 months after the MMAG takes place, evolutions were confirmed in many of the analyzed axis, as it was officially published by the institution. This proves the effectiveness of the implemented solutions and adjustments for the problems found.
This initiative is efficient because it generates a rational standard to asses the operation of a state-owned institution, bringing together, in a simple fashion, recognized control tools such as COSO and COBIT, public audit methodologies and, most importantly, useful information for managers in the decision-making process.

9. What were the main obstacles encountered and how were they overcome?

I highlight the lack of a primary source of data and references in Brazil as an initial obstacle. There was, indeed, no material on public auditing in Brazilian hospitals. Considering that Brazil has an extensive and rigid legislation on auditing and controlling public spending, the use of sources from other countries presented itself as great difficulty and possibility of error.
Due to the lack of a primary source of data and reference on public auditing in hospitals, any theoretical basis had to be constructed from an important and unprecedented integration between the concepts and norms of public audit, obtained through the Control System Audit Manual Internal, approved by Normative Instruction No. 1, of April 6, 2001, of the Controller General of the Union, with the principles of control, risk and maturity present in instruments internationally recognized as COSO (Committee of Sponsoring Organizations of the Treadway Commission) and COBIT (Control Objectives for Information and related Technology).
Another difficulty was the acceptance of the auditors in public hospitals. Considering that none of the 30 federal university hospitals had an internal audit unit before, and only had interventions by the Federal Audit Court - TCU, there was no history or positive opinion about work done by internal auditors.
The incursion and acceptance of the auditors in public hospitals was the most difficult thing to overcome and it only happened as the results were appearing inside the hospitals.
We noticed that the managers, through WhatsApp groups, had conversations about how the initial acceptance of hospital employees about the practice was less than 10% in the first hospitals against almost 50% in the last ones, which gives us an excellent margin of accuracy, beyond the certainty of being understood and accepted.

D. Impact and Sustainability

10. What were the key benefits resulting from this initiative?

The results and the applicability of MMAG continuously surprise in each FUH where it is implemented. To the present date, this methodology has already been enforced in 21 federal university hospitals(FUHs) of different sizes throughout the whole country.
Out of the main results, we can highlight:
The evolution of the governance environment in the FUHs. From the moment the Auditing Evaluation for the Maturity of the Governance Environment was carried out, 8 to 10 months after concluding the MMAG it is possible to verify clear evolution in several of the analyzed axes, which shows solution effectiveness and adjustment in all the identified problems.
The insertion of the internal auditing unit inside the UH: according to the acceptance of / access to the internal auditing by the managers and employees researches, the internal auditing units in FUHs that had already carried out the MMAG pointed out an increase of their acceptance perception, on an average that goes from 156% during the MMAG realization to 183% after its conclusion.
Encouragement to the creation of UHs Internal Control Committees. In order to treat the identified problems, auditors suggest the creation of a special control committee for this purpose. As a result, 12 hospitals have an internal control system, which led Ebserh to win the IV CGU Good Practice Competition in the Internal Auditing Improvement category, on December of 2016.
Local managers now declare that they have enough information about the operation of their FUHs, and also ask for more actions of the same nature. The result will be the further evolution of this project into the Measuring of the Main Institutional Processes Maturity (contracting, purchasing, capacitation, among others).
The implementation of specific Auditing/ Evaluation on issues already identified by the managing staff. The auditing is carried out in a targeted fashion and promotes time saving in planning and implementation.
From the MMAG’s application, it will also be possible to perform a risk evaluation of strategic and operational targets and their respective processes, which contributes to the achievement of Ebserh's goals.
The increasing involvement and the FUHs employees’ participation at the problems and solution proposals. Currently, around 40-45% of the employees of the FUHs are active participants in this process.
The Beginning of FUHs Managing Good Practices Notebook creation. The document is hugely important to FUHs, as to this day we have yet to identify any document able to reflect, specifically, FUHs’ Good Practices.
Promotion and creation of an awareness and a Control Environment which is favorable to the of Ebserh’s evolution – both the head office and FUHs. Managers and employees will come into contact for the first time with what we call "internal control" in a positive way, which already results in effective actions from some sectors of the Company. Internal control will be seen then, as something positive and not compulsory or punitive

11. Did the initiative improve integrity and/or accountability in public service? (If applicable)

The core of this initiative is transparency, monitoring and the potential accountability of public agents for irregular conduct, including fraud and corruption.
Encouraging the participation of all staff and even the community itself in the identification of the institution’s main weaknesses led to the formal establishment of Internal Control Committees in each branch. These committees are comprised by representatives, who are not in management, of each of the institution’s sector. In them, the main identified problems are discussed, for solutions to be then offered to the managers. From there, control points are determined for each weakness and risk.
When working in these committees, those responsible for the monitoring and examination of the main weaknesses and risks identified, closely follow the process from the moment of purchase to the arrival of a number of materials to the assistance to the underprivileged.
As groups in society that were not previously aware of the institution’s issues begin participating, there is more transparency and then more complaints and accountability investigations.
The impact can be measured through management indicators in the reduction by 40% of medication and materials spending, in the identification, remediation and accountability of employees in their assistance to the public, as well as a rise in accolades received by the ombudsman, all over 600%! since the implementation of the practice in the 21 hospitals involved.
After due examination and confirmation of information on this practice's positive impacts by Brazil’s Control Institution, CGU, the practice won with one of its results, once again, CGU’s IV Good Practice Competition in December 2017 as “Promoter of Internal Control Activities”.

12. Were special measures put in place to ensure that the initiative benefits women and girls and improves the situation of the poorest and most vulnerable? (If applicable)

Considering Ebserh’s main goal of offering medical care without any cost to the population in great social vulnerability in centers with high poverty rates, the prioritized and performed services carried out by the Ebserh's maternity hospitals since the beginning of the program were nationally and internationally acknowledged; the beds for the assistance of pregnant women doubled after the practice’s first results; neonatal intensive treatment units in all maternities of Ebserh were inaugurated and implemented; a campaign to prevent breast cancer was carried out; courses for young mothers of how hand lactation were offered; courses in partnership with local communities of "elderly caregiver" were conducted and there was also prevention of accidents. These are among many other suggestions that came out of the discussion on community needs and implementation possibilities, thanks to our program.
Programs such as maternal milk banks, stand out in the network, receiving *gold international certification*; the implementation of the “childbirth humanization” policy, which avoids unnecessary cesarean section, since more than 70% of women in Brazil appeals for this action.
All these results and many others are attested by Brazilian Government official documents, and by the official website of government news, linked below:
http://www.ebserh.gov.br/pt/web/portal-ebserh/noticias