Topics

Abstract

Clinical research is an area of increasing activity for hospitals, universities and research institutions, which requires formal governance and oversight to manage risks.

Monitoring research practice should be a part of research governance activities. However, formal audits have proved time consuming for researchers and auditors.

To increase attention to good research practice and screen for poor practice, the Department of Epidemiology and Preventive Medicine at Monash University and the Alfred Research and Ethics Unit in Melbourne have developed a brief self-audit tool for researchers.

We evaluated the self-audit using a questionnaire for researchers. The results were positive, with most respondents believing that it promoted good research practice.

Clinical research has been an area of increasing activity for hospitals, universities and research institutions in Australia, rising from 1960 trials in 2000 to 3208 trials in 2009 (Therapeutic Goods Administration, personal communication). To manage the increasing risks for both patients and institutions, strategies to better monitor research governance are now receiving greater attention.

The National Health and Medical Research Council (NHMRC) National statement on ethical conduct in human research1 requires that individual institutions take responsibility for monitoring clinical research conducted at their institutions. One of the most common methods used to monitor research activities is the annual progress report required by human research ethics committees (HRECs). The report includes information about compliance with the approved protocol, as well as maintenance and security of records. Some institutions also undertake random inspections of research sites, data or consent documentation.

Developing the self-audit

To promote attention to ethical conduct and compliance with principles of good research practice,2 the Department of Epidemiology and Preventive Medicine (DEPM) at Monash University and the Alfred Research and Ethics Unit (Alfred) in Melbourne introduced research audits in 2004. The DEPM and the Alfred collaborated to introduce a variety of monitoring strategies, including annual progress reports and detailed audits.3 Other components of the DEPM program included appointing a research governance officer, establishing a research governance committee, developing a manual of clinical and public health good research practice, and introducing an induction program for new staff.

However, although the detailed audits were effective, they proved time consuming and resource intensive (a detailed audit takes about 3 days to complete, and each year it is possible to examine only about 10% of the total number of active projects).

To improve the institutional oversight of research, the DEPM and the Alfred subsequently adopted a tiered approach to audits. Detailed audits were supplemented with short audits that can be completed in less than a day and a self-audit (Box 1), which is completed by researchers themselves, rather than by governance or ethics staff. The self-audit highlights to researchers the issues they need to keep in the forefront of their minds while conducting research and helps them identify any areas where they may not be meeting the expected standards.

The strategy is based on the premise that researchers are keen to conduct their research to the highest standard, provided they know what is required of them. Other health services have since adopted self-audits, and the Victorian Managed Insurance Authority has incorporated them into its Research governance toolkit, which it developed to assist all Victorian public entities conducting clinical research.3

To evaluate whether the self-audit has been an effective tool for highlighting and promoting good research practice, the DEPM and the Alfred designed a short, online questionnaire. The questionnaire was sent out on 3 July 2008, with responses received over the following 8 weeks. Our article reports the findings.

Completing and evaluating the self-audit

The DEPM

An email was sent to all DEPM staff, inviting those directly responsible for the conduct of a research project to complete the self-audit form. In most cases, the most appropriate person to complete the form was the principal investigator. If the principal investigator was not able to complete the form due to time or other constraints, he or she was encouraged to ask someone else with relevant project knowledge to complete the self-audit. The email also included a link to an online version of the form.

To ensure that the audit was being completed by an appropriate person, the respondent’s name and responses were emailed to the research governance officer after it was completed.

At the same time, the respondent was redirected to an online survey where he or she was invited to complete an evaluation questionnaire of the self-audit.

The Alfred

The Alfred adopted a slightly different approach. The Alfred Hospital Ethics Committee amended its annual progress report to include a question asking the researcher to complete the self-audit, attach a copy of the audit to the annual report at the time of submission, and file a copy in their study files.

An invitation to complete the online evaluation questionnaire was emailed to research personnel with current projects and advertised in the Alfred Ethics Committee Newsletter.

The evaluation

The evaluation was conducted and analysed anonymously using an online survey tool (SurveyMonkey, Palo Alto, Calif, USA). The evaluation questionnaire sought information in three main areas:

whether the text and substance of the self-audit were easy to understand and appropriate to the research being performed

whether participants believed the audit would result in their making changes to their projects or whether it would improve the ethical quality of their research

whether the participants thought the self-audit required follow-up to be useful and whether they discussed the audit with the principal investigator.

Ethics approval

Approval for our research was obtained from the Standing Committee on Ethics in Research Involving Humans at Monash University and the Alfred Ethics Committee.

Evaluation findings

At the time of the evaluation, there were 165 projects being conducted at the DEPM, overseen by 56 principal investigators. Twenty-seven principal investigators or their delegates (48%) completed the self-audit and, of those, 24 completed the evaluation (43% of principal investigators; 89% of those who completed the audit).

At the Alfred, invitations were distributed to about 340 researchers with active projects. Thirty-three researchers completed the evaluation but many of these were responsible for multiple projects.

Results of the evaluation are shown in Box 2. Across both sites, most respondents (92% at the DEPM and 97% at the Alfred) believed that the self-audit promoted good research practice.

One of the aims of the self-audit is to encourage people to think about new ethical issues. However, fewer than half of the participants (25% at the DEPM and 40% at the Alfred) identified new ethical issues as a result of using the self-audit. Our results may have been influenced by selection bias among respondents. Those who were already interested in good research practice were more likely to have completed the questionnaire, making it more likely that the evaluation records an interest in the area but less likely that it raises new issues.

Discussion

Auditing research is one means by which irregularities can be prevented or discovered before damage is caused.4 This role is recognised by the NHMRC national statement and equivalent international documents (Box 3).

Ongoing HREC oversight of projects has increased in the past decade, with many institutional HRECs now requiring annual progress reports. However, there remains an important and separate institutional role in ensuring that the research conducted conforms to high standards of good research practice.7 The self-audit, completed by principal investigators and project coordinators, provides feedback to researchers concerning the adequacy of their research procedures. It is part of a series of research governance initiatives designed to make an explicit statement to researchers about the standards expected in the institution and to fulfil a preventive role in alerting all those involved in clinical research to the dangers of poor research practice.

It is envisaged that the most important role of the self-audit will be in helping to create a culture that is conducive to the responsible conduct of research. A 2008 study reported alarmingly low familiarity of researchers with the major research governance documents,4 including the NHMRC national statement and the Australian code for the responsible conduct of research.8The authors suggested that a mandatory training program in research governance along with research auditing would improve researcher knowledge. Indeed, the Australian code requires institutions to train researchers in responsible research conduct.

Although the self-audit cannot address the breadth and depth of issues possible in a formal training course, it can provide a simple checklist encompassing the most important governance principles for researchers to follow. In so doing, it can help to educate researchers about their research responsibilities while at the same time performing an auditing role.

To date, the key difficulty in implementing the self-audit has been encouraging researchers to complete the audit without intervention or follow-up from governance staff. This has been achieved at the Alfred by including the self-audit form with the documents that the ethics committee requires annually to ensure ongoing project approval. The DEPM plans to introduce a similar compulsory system.

Although the self-audit is not intended to replace more comprehensive audits, it appears to be a promising means of reminding researchers of their obligations and encouraging self-monitoring of research activities without requiring the prohibitive resources needed by more extensive audits.

1 Self-audit process at the Department of Epidemiology and Preventive Medicine and the Alfred Research and Ethics Unit

The self-audit is designed principally as an educational tool, providing a quick check for researchers to determine if their research activities meet the expected high standard of good research practice. It also serves as an aid to the research governance officer by identifying projects or departments that warrant a more detailed assessment.

The self-audit asks researchers whether their research could easily be continued without them, whether they are meeting their legal and ethical obligations, and whether data and other important information are being stored appropriately. If a researcher is not able to answer “yes” to a question, he or she must seek assistance from the research governance officer.

A member of the study team completes the self-audit form annually. This person needs to have intimate knowledge of all aspects of the project. He or she is instructed to complete and sign the form and then discuss the results with the principal investigator (if the principal investigator is not completing the audit himself or herself) and remedy any deficiencies identified. The principal investigator must then sign the form and return it to the research governance officer.

3 Examples of audit requirements in Australia, the United Kingdom and the United States

National Health and Medical Research Council — National statement on ethical conduct in human research1

5.1.17 Institutions should have in place an auditing process to confirm that:

(a) research in their institution is being reviewed at the levels of review their criteria require

UK Department of Health — Research governance framework for health and social care5

5.2 ... Mechanisms to monitor the quality of clinical work, such as inspection, audit, risk management and staff appraisal, can assist
in the monitoring of research governance. Coherent systems are needed to monitor performance, to identify best practice and shortfalls, to enhance public confidence and help to prevent adverse events

US Office of Management and Budget Circular A-133 — Audits of states, local governments, and other non-profit organizations6

§ ___ .200 Audit requirements

(a) Audit required. Non-Federal entities that expend ... $500 000
... or more in a year in Federal awards shall have a single or program-specific audit conducted for that year

Received 24 May 2010, accepted 1 September 2010

Bradley R Crammond1

Anna V Parker2

Megan Brooks1

Marina Skiba1

John J McNeil1

1 School of Population Health and Preventive Medicine, Monash University, Melbourne, VIC.