2
Objectives Identify potential challenges to Stakeholder engagement in CER protocol development Discuss engagement strategies that involve Stakeholders in a meaningful way to enhance the quality and relevance of study outcomes

3
DEcIDE Network Developing Evidence to Inform Decisions about Effectiveness Collection of research centers that the Agency for Healthcare Research and Quality (AHRQ) created in 2005 Centers gather new knowledge and information on specific treatments Conducts studies on the outcomes, effectiveness, safety, and usefulness of medical treatments and services

4
From the perspective of the investigator: DEcIDE Network Research priorities established through previous Agency work – Engagement of Stakeholders and technical experts – Institute of Medicine CER priorities Research opportunities competitively offered among centers – Proposal submission – Peer review Contracted research – Aims, approach often pre-specified during contract negotiation

5
From the perspective of the investigator: DEcIDE Network Stakeholder engagement still required during research Plan for engaging Stakeholders must be specified within research applications

7
What key challenges does DEcIDE research present? Orient Stakeholders to established research – Established questions, comparators – Preliminarily established research methodology Help Stakeholders understand their value Identify ways Stakeholder activities can enhance existing research plans – Enhance relevance – Input on planned approaches with end user in mind

8
DEcIDE Patient Outcomes in End Stage Renal Disease (ESRD) Study 3- year ARRA contract to compare effectiveness of three treatment strategies for patients with ESRD (hypertension treatments, dialysis timing, iron therapy) First Stakeholder meeting held within first 4 months of funding (October 2010) – Identifying and contacting Stakeholders – Working with partner to strategize and plan (Center for Medical Technology Practice, CMTP) *Priorities established through IOM recommendations

13
Conducting Stakeholder Meeting Professional moderator Discussed stakeholder roles, importance Project summaries from research team Breakout group discussions (organized by Stakeholder type) – Key questions to frame research approach – Consider potential implications of positive or negative findings for decision-makers Informal lunch with group discussion of breakout group findings Review discussion points and promised follow up

14
Special Considerations Room setup (Stakeholders in center, investigators out of the way) Prior investigator preparation (i.e. more listening, less talking, less technical jargon) Interactive breakout groups with investigators as scribes, gathering information but little input

15
Helping Stakeholders see Value of Input Audio recorded and transcribed entire meeting Created detailed summaries of key points discussed Distributed summary of meeting to Stakeholders Prepared for in-person follow up

16
Helping Stakeholders see Value of Input: Preparing for in-person follow up Research team digested transcribed Stakeholder comments Responded point-by-point to comments, modifying research protocol where appropriate – Enhancing outcomes to improve relevance to different decision-makers (e.g., quality of life) – Adjusting emphasis of approach (e.g., considering harms of therapies in addition to effectiveness)

19
Example Stakeholder Comments Project 3: Iron Management Theme 1: Impossible to disentangle simultaneous use of erythropoiesis stimulating agents and iron. Both should be considered as primary exposures of interest Response: – Will explore simultaneous use of the two medications longitudinally – Employ complex modeling strategies (marginal structural)

20
Year 2: In-Person Follow Up Second in-person meeting Reviewed Stakeholder feedback and our responses in document provided to them and during presentation Sought additional feedback Provided research updates (preliminary findings) Obtained feedback on preliminary findings and Asked Stakeholders for input on framing and dissemination strategies

21
Year 3: Extending Impact of Stakeholder Engagement Planning third in-person meeting – Pre- meeting engagement to get ideas about extension of current research, other relevant research studies in ESRD Review final preliminary contract research findings, obtain feedback Focus on framing and dissemination of final projects

22
Final Planned Engagement Plan to send Stakeholders summary of input over life of project Provide Stakeholders with some final products – Obtain feedback on clarity of products – Additional avenues for dissemination Thank Stakeholders for generous time investment Encourage future engagement

24
Successful strategies Early engagement of Stakeholders Well-planned in-person meetings Setting clear expectations of the Stakeholders Demonstrating use of input from Stakeholders with report back Continued engagement throughout project

25
Challenges we faced Presenting a lot of scientific information to group of busy Stakeholders – Technical aspects difficult to convey – Sheer amount of information Ambitious contract research schedule – Challenging to engage Stakeholders throughout year – Would have liked to have resources to engage more frequently – Much of the research happening in background while preparation for future meetings ongoing

26
Summary Engagement of Stakeholders in contract research may require special consideration – Help Stakeholders see their relevance – Translate complex protocols for a broad range of audiences Demonstrating utilization of Stakeholder input important Advance planning critical to maximize output of Stakeholder groups – Partner with experts in Stakeholder engagement – Invest adequate resources and time to maximize yield