Perelman School of Medicine and Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, Pennsylvania (H.F.L., E.A.L.)

Brigham and Women's Hospital, Boston, Massachusetts (A.G.)

Petrie-Flom Center for Health Law Policy, Biotechnology, and Bioethics at Harvard Law School, Cambridge, Massachusetts (I.G.C.)

Disclaimer: The views presented in this article are solely the responsibility of the authors and do not necessarily represent the views of PCORI, its board of governors, or its methodology committee.

Acknowledgment: The authors thank the following persons for their contributions to this manuscript: Laura Beskow, Martha Carnie, Maureen Fagan, Amy Lynn McGuire, and 2 anonymous reviewers for this journal.

Grant Support: Research reported in this article was funded through a PCORI award (ME-1409-21701).

Abstract

A key aim of patient-centered outcomes research (PCOR) is to generate data that are important to patients by deliberately and extensively involving them in all aspects of research, from design to dissemination. However, certain elements of PCOR raise challenging and potentially novel ethical and regulatory issues for institutional review boards and oversight bodies. These challenges stem primarily from the engagement of patients in roles other than research subject, such as advisors, study personnel, and co-investigators, which gives rise to questions about appropriate levels of protection, training, and education, as well as identifying and managing conflicts of interest. This article presents and discusses recommendations from a Delphi expert panel that was convened to address these and other PCOR-related oversight challenges.

Patient engagement is increasingly regarded as critical to maximizing the value of medical research. In the United States, the growing emphasis on engagement is reflected by the founding of the Patient-Centered Outcomes Research Institute (PCORI) as well as initiatives spearheaded by the Institute of Medicine and the U.S. Food and Drug Administration (FDA) to grant patients' perspectives a central role in determining research priorities and informing the design and conduct of clinical trials (1–3). Such initiatives, often captured under the rubric of patient-centered outcomes research (PCOR), are characterized by a commitment to incorporating patient perspectives at every stage of research and promoting patient involvement in key study roles.

Although attempts to define PCOR and distinguish it from other forms of clinical research are sometimes amorphous (4), PCORI, the leading funder of PCOR, describes its mandate as “improv[ing] the quality and relevance of evidence available to help patients, caregivers, clinicians, employers, insurers, and policy makers make better-informed health decisions. To do this, we work with those healthcare stakeholders to identify critical research questions and answer them through comparative clinical effectiveness research…focusing on outcomes important to patients” (5). Thus, PCORI departs from the traditional clinical research paradigm, in which investigators drive the conceptualization and implementation of research, and embraces an approach in which patients are involved throughout the life cycle of research projects—“from proposal development to research design and dissemination of the study results” (6, 7). The 21st Century Cures Act, passed in 2016, echoes this approach, acknowledging that patients are “in a unique position to contribute to an understanding of benefit and risk considerations throughout the medical product development process” and recognizing “the need for systematic collection of direct patient input” to guide choices about drug development (8). The underlying assumption of these efforts is that “research meaningfully informed by the patient perspective is more likely to be used to inform patient decision-making and, in the end, to improve patient outcomes” (6).

However, primarily as the result of engaging patients in roles other than research subject, PCOR raises novel ethical and regulatory issues and exacerbates some familiar ones, such as challenges associated with emerging technologies, whose use in PCOR may be more prevalent than in traditional clinical research (9). Despite notable efforts to promote and facilitate the conduct of PCOR, scant attention has been paid to these oversight challenges (10–12). (The exception is the debate around standards for risk assessment and informed consent in comparative effectiveness research in the wake of the controversy surrounding SUPPORT [Surfactant, Positive Pressure, and Oxygenation Randomized Trial] [13, 14]. Given the attention those issues have already received, the research team decided to focus on less-explored oversight challenges with PCOR.) Indeed, there exists no formal regulatory or ethics guidance on oversight of PCOR from the FDA, the Department of Health and Human Services Office for Human Research Protections, or other regulatory bodies. In this article, we address this gap and share policy recommendations derived through a modified Delphi consensus process.

Methods

These recommendations are the product of PCOROS (Patient-Centered Outcomes Research Oversight Study), a multiphase, mixed-methods research project funded by PCORI (ME-1409-21701). In phases 1 and 2, the PCOROS team engaged key stakeholders through individual interviews, case studies, and focus groups and administered a survey to institutional review board (IRB) chairs at all major U.S. research institutions (7, 8). Three domains were found to merit further guidance: 1) oversight of patients in nontraditional research roles, such as co-investigator, study personnel, or advisor; 2) emerging technologies (such as social media or mobile applications), whose use as part of study design in PCOR seems to be more common and larger in scale than in traditional research; and 3) effective identification of patient research partners and mitigation of conflicts of interest.

To generate recommendations that address these areas of need, we then employed a modified Delphi process, which uses multiple rounds of evaluation to gauge and facilitate consensus among a group of expert stakeholders on a particular topic (15–18). The Delphi technique was chosen because it is recognized as an optimal method for consensus building, with use of anonymous feedback from an expert panel and statistical analysis techniques to interpret the data. The iterative nature of the process avoids some of the pitfalls of other methods, such as the effects of dominant persons or the tendency to conform to a particular viewpoint. A full list of panel participants is provided in Appendix 1, and an explanation of the modified Delphi technique and a more detailed description of the process we used is given in Appendix 2.

Table. Consensus Recommendations

The panel endorsed 10 recommendations addressing oversight of PCOR involving patients in roles other than research subject. These recommendations include guidance on classifying patient research partners according to a formal taxonomy of research roles, appropriate ethics and scientific training for patients in nontraditional roles, the permissibility of patients occupying multiple research roles simultaneously and switching between roles in the same study, and providing compensation to patient research partners (see domain 1 in the Table).

Although the oversight challenges posed by emerging technologies, such as mobile applications and social media, are not unique to PCOR given that these technologies are also used in other types of research, the relative prevalence of these technologies in PCOR was identified in interviews and the survey of IRB chairs as sufficiently important to address in this context (6, 9, 19). The Delphi panel achieved consensus on 6 recommendations addressing this domain. The endorsed recommendations include guidance on educating PCOR stakeholders about protecting participant privacy and confidentiality when using emerging technologies, best practices for data collection using emerging technologies, protections around use of social media in recruitment efforts, and use of electronic platforms for obtaining consent in PCOR studies (see domain 2 in the Table).

The Delphi panel achieved consensus on 5 recommendations addressing the identification and engagement of patient research partners. These address the need for education of investigators and sponsors in identifying and selecting potential research partners, the importance of achieving a diverse mix of patient voices in PCOR, the need for education on identifying and mitigating financial and nonfinancial conflicts of interest among patient research partners, and advice to IRBs on how to approach financial conflicts of interest among patient partners occupying study personnel roles (see domain 3 in the Table).

Discussion

The most novel oversight challenge posed by PCOR concerns the standards that should be applied to persons occupying nontraditional research roles, which is addressed by our first domain of recommendations. This is particularly important when patients fill roles other than research subject. In this case, there is tension between the need for IRBs to provide effective oversight for patients in study roles typically filled by traditional researchers and research staff and refraining from unnecessarily exceptionalizing patients by treating them differently from others in these roles without adequate reason. Although patients may benefit from training and education tailored specifically to their needs, they should not be held to different standards or treated differently from other study team members serving in similar roles. Indeed, there is evidence that IRBs can be overprotective of patient research partners by applying human subjects protections to them, such as informed consent, even when they are not enrolled as research subjects (9).

Disambiguating the various roles that patients may fill in PCOR will bring needed clarity. To address this, the recommendations endorse a formal taxonomy of patient involvement in research (recommendations 1 to 5) that distinguishes 3 broad roles patients might occupy—study personnel, advisor, and research subject—and whose main purpose is to allow IRBs to determine adequate protections and training requirements for patients depending on the role and activities they assume, bringing greater uniformity to practice. This taxonomy is complemented by recommendation 9, which clarifies that oversight protections should not be applied to patients involved in research unless they meet the regulatory definition of “research subject,” thereby guarding against overprotection.

At the same time, the role of patient partners in PCOR studies may be fluid. Such studies may be more likely than traditional clinical research to involve patients occupying multiple roles (for example, research subject and study personnel) and switching between roles (for example, from subject to advisor) in the same study, which raises oversight challenges (6, 9). Although there may be value in leveraging patients' perspectives on different aspects of the research, caution is needed to prevent conflicts between roles and to safeguard scientific integrity. Problems might arise, for example, if patients are both research subjects and personnel in the same study; their obligations as study personnel to ensure scientific integrity (including maintenance of a blind and the use of randomization and placebo) may conflict with their interests as patient-subjects in knowing which intervention they are receiving and being treated for their medical condition. Recommendations 6 to 8 provide actionable guidance for how IRBs and investigators should approach these situations.

In addition, although oversight bodies are used to evaluating and managing researchers' conflicts of interest, they may be less familiar with recent empirical evidence of strong financial connections between industry and certain patient advocacy groups, which may give rise to conflicts of interest for patient partners (20, 21). Another challenge is making sure that investigators receive input from a diverse set of patients in research design and implementation. Recommendations 17 to 21 address these issues, rejecting the high bar of patient partners being “representative” of the population from which they come and instead recommending that investigators cultivate a diverse mix of patient partners. Further, these recommendations call for guidance from a high-level policy body, such as PCORI, on identifying and managing conflicts of interest among patient partners.

Conclusion

For PCOR to realize its full potential to yield knowledge that is valuable to patient decision making, the ethical and regulatory oversight challenges it raises must be met systematically and with confidence and transparency that will inspire trust in the research enterprise. Issues of PCOR oversight can sometimes be complex, and IRBs often differ in their interpretations of the relevant regulatory concepts and ethical issues. Consensus among content matter experts from diverse PCOR stakeholder groups provides a solid basis for adopting recommendations that can help clarify challenging aspects of PCOR oversight. The policy recommendations advanced here, which target IRBs, investigators, and high-level funding and regulatory bodies, set an agenda for immediate implementation as well as additional study to address areas of need in PCOR oversight and bring uniformity to practice. Such implementation would ideally be carried out by PCORI or other high-level policy bodies.

Appendix 1: Delphi Working Group Participants and Affiliations

These members of the Delphi working group contributed to the article but did not author it:

Maureen Fagan, Chief Experience Officer, University of Miami Health System

Elisa Hurley, Executive Director, Public Responsibility in Medicine and Research (PRIM&R)

Megan Kasimatis Singleton, Assistant Dean for Human Research Protection and Director of the Human Research Protection Program, Office of Human Subjects Research, Johns Hopkins University School of Medicine

Nancy Kass, Vice Provost for Graduate and Professional Education, Johns Hopkins University; Phoebe R. Berman Professor of Bioethics and Public Health, Johns Hopkins Berman Institute of Bioethics and Johns Hopkins Bloomberg School of Public Health

Appendix 2: Description of Modified Delphi Process

The 17 members of our Delphi expert panel were selected with the aim of reflecting the diversity of stakeholders in PCOR oversight, without seeking representativeness given the small size of the group, as is typical. Members included IRB chairs, directors of human research protection programs, PCOR principal investigators, patients, patient advocates, experts in bioethics, experts in law, and policymakers (see Appendix 1 for a complete list of participants). Participants were selected on the basis of their proven expertise in these areas, as exhibited by publication record and professional position and reputation. Once an individual agreed to participate, suggestions for further well-qualified participants were solicited from them, which were used to inform subsequent choices about whom to invite.

Our Delphi process consisted of 4 rounds. Before the first round, the PCOROS team drafted 21 policy-level recommendations responsive to the 3 domains identified earlier and targeted at different entities (IRBs, funding and oversight bodies, and investigators). In round 1, we asked panel members to provide qualitative feedback on the clarity, importance, and correctness of the recommendations and suggestions for additional recommendations. This resulted in 23 recommendations for evaluation in round 2.

In round 2, members of the Delphi panel were asked to complete a survey evaluating the 23 recommendations along 3 axes: need, correctness, and feasibility. The choice of axes was motivated by our aim of selecting policy recommendations capable of responding to a real challenge (need), providing appropriate guidance (correctness), and being successfully implemented (feasibility). The criteria for determining consensus were based on the UCLA/RAND consensus criteria adjusted for the size of our panel (22). Thirteen of the 23 recommendations met the overall criteria for consensus after round 2 voting and were thus deemed “accepted” without further discussion.

Round 3 consisted of a half-day Webinar devoted to discussing and revoting on the 10 remaining recommendations. Before the Webinar, participants received information sheets summarizing the median round 2 score, the distribution of votes, a reminder of their own vote, and anonymized round 2 feedback from all panelists. During the Webinar, the panel revoted on each recommendation after moderated discussion among the group. Our process allowed for and encouraged changes to the wording and substance of the recommendations. A round 4 Webinar was used to debrief the panel on the results.

* For a list of the Delphi Working Group participants, see Appendix 1.

Table. Consensus Recommendations

Table. Consensus Recommendations

Clinical Slide Sets

Terms of Use

The In the Clinic® slide sets are owned and copyrighted by the American College of Physicians (ACP). All text, graphics, trademarks, and other intellectual property incorporated into the slide sets remain the sole and exclusive property of the ACP. The slide sets may be used only by the person who downloads or purchases them and only for the purpose of presenting them during not-for-profit educational activities. Users may incorporate the entire slide set or selected individual slides into their own teaching presentations but may not alter the content of the slides in any way or remove the ACP copyright notice. Users may make print copies for use as hand-outs for the audience the user is personally addressing but may not otherwise reproduce or distribute the slides by any means or media, including but not limited to sending them as e-mail attachments, posting them on Internet or Intranet sites, publishing them in meeting proceedings, or making them available for sale or distribution in any unauthorized form, without the express written permission of the ACP. Unauthorized use of the In the Clinic slide sets will constitute copyright infringement.