Article Figures & Tables

Figures

Diagram of the guideline development process. The steps and involvement of various members of the guideline development group are interrelated and not necessarily sequential. The guideline panel and supporting groups (e.g., methodologist, health economist, systematic review team and secretariat for administrative support) work collaboratively, informed through consumer and stakeholder involvement. They typically report to an oversight committee or board overseeing the process. For example, while deciding how to involve stakeholders early for priority setting and topic selection, the guideline group must also consider how developing formal relationships with the stakeholders will enable effective dissemination and implementation to support uptake of the guideline. Furthermore, considerations for organization, planning and training encompass the entire guideline development project, and steps such as documenting the methodology used and the decisions made, as well as considering conflicts of interest, occur throughout the process.

Tables

Involves laying out a general but detailed plan describing what is feasible, how it will be achieved and what resources are required to produce and use the guideline. The plan should refer to a specific period and be expressed in formal, measurable terms.

2. Priority setting

Refers to the identification, balancing and ranking of priorities by stakeholders. Priority setting ensures that resources and attention are devoted to those general areas (e.g., chronic obstructive pulmonary disease, diabetes, cardiovascular disease, cancer, prevention) where health care recommendations will provide the greatest benefit to the population, a jurisdiction or a country. A priority-setting approach needs to contribute to future plans while responding to existing, potentially difficult circumstances.100,101

3. Guideline group membership

Defines who is involved, in what capacity, and how the members are selected for the guideline development and at other steps of the guideline enterprise.

4. Establishing guideline group processes

Defines the steps to be followed, how those involved will interact and how decisions will be made.

5. Identifying target audience and topic selection

Involves describing the potential users or consumers of the guideline and defining the topics to be covered in the guideline (e.g., diagnosis of chronic obstructive pulmonary disease).

6. Consumer and stakeholder involvement

Describes how relevant people or groups who are not necessarily members of the panel but are affected by the guideline (e.g., as target audience or users) will be engaged.

7. Conflict of interest considerations

Focuses on defining and managing the potential divergence between an individual’s interests and his or her professional obligations that could lead to questioning whether the actions or decisions are motivated by gain, such as financial, academic advancement, clinical revenue streams or community standing. Financial or intellectual or other relationships that may affect an individual’s or organization’s ability to approach a scientific question with an open mind are included.

8. Question generation

Focuses on defining key questions the recommendations should address using the PICO (patient/problem, intervention, comparison, outcome) framework, including the detailed population, intervention (including diagnostic tests and strategies) and outcomes that will be relevant for decision-making (e.g., should test A be used, or should treatments B, C, D or E be used in chronic obstructive pulmonary disease?).

Includes integrating, in the process of developing the guidelines, how those affected by its recommendations assess the possible consequences. These include patient, caregiver and health care provider knowledge, attitudes, expectations, moral and ethical values, and beliefs; patient goals for life and health; prior experience with the intervention and the condition; symptom experience (e.g., breathlessness, pain, dyspnea, weight loss); preferences for and importance of desirable and undesirable outcomes; perceived impact of the condition or interventions on quality of life, well-being or satisfaction, and interactions between the work of implementing the intervention, the intervention itself, and other contexts the patient may be experiencing; preferences for alternative courses of action; and preferences relating to communication content and styles, information and involvement in decision-making and care. This can be related to what in the economic literature is considered utilities. An intervention itself can be considered a consequence of a recommendation (e.g., the burden of taking a medication or undergoing surgery) and a level of importance or value is associated with that.

10. Deciding what evidence to include and searching for evidence

Focuses on laying out inclusion and exclusion criteria based on types of evidence (e.g., rigorous research, informally collected), study designs, characteristics of the population, interventions and comparators, and deciding how the evidence will be identified and obtained. It also includes but is not limited to evidence about values and preferences, local data and resources.

11. Summarizing evidence and considering additional information

Focuses on presenting evidence in a synthetic format (e.g., tables or brief narratives) to facilitate the development and understanding of recommendations. It also involves identifying and considering additional information relevant to the question under consideration.

12. Judging quality, strength or certainty of a body of evidence

Includes assessing the confidence one can place in the obtained evidence by transparently evaluating the obtained research (individual studies and across studies) and other evidence applying structured approaches. This may include, but is not limited to, evidence about baseline risk or burden of disease, importance of outcomes and interventions, values, preferences and utilities, resource use (cost), estimates of effects and accuracy of diagnostic tests.

13. Developing recommendations and determining their strength

Developing recommendations involves use of a structured analytic framework and a transparent and systematic process to integrate the factors that influence a recommendation. Determining the strength of the recommendations refers to judgments about how confident a guideline panel is that the implementation of a recommendation exerts more desirable than undesirable consequences.

14. Wording of recommendations and of considerations about implementation, feasibility and equity

Refers to choosing syntax and formulations that facilitate understanding and implementation of the recommendations. Such wording is connected to considerations about implementation, feasibility and equity, which refer to the guideline panel’s considerations about how the recommendation will be used and what impact it may have on the factors described.

15. Reporting and peer review

Reporting refers to how a guideline will be made public (e.g., print, online). Peer review refers to how the guideline document will be reviewed before its publication and how it can be assessed (e.g., for errors), both internally and externally, by stakeholders who were not members of the guideline development group.

16. Dissemination and implementation

Focuses on strategies to make relevant groups aware of the guidelines and to enhance their uptake (e.g., publications and tools such as mobile applications).

17. Evaluation and use

Refers to formal and informal strategies that allow judgments about: evaluation of the guidelines as a process and product; evaluation of the use or uptake, or both; and evaluation of impact and whether or not the guideline leads to improvement in patient or population health or other consequences.

18. Updating

Refers to how and when a guideline requires revision because of changes in the evidence or other factors that influence the recommendations.

The checklist is organized into 18 topics, each with corresponding items to consider. Users of the checklist should review all topics and items before applying them, because the items are not necessarily sequential and many are interconnected. The brief examples included with some items are for clarification and elaboration; they are not meant to be extensive instructions for how to accomplish the steps. Instructions and suggestions for accomplishing the steps can be found in the source documents referenced and in the resources suggested in the interactive online version of the checklist (http://cebgrade.mcmaster.ca/guidecheck.html). See Appendix 1 (available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.131237/-/DC1) for a glossary of terms appearing in the checklist.

Completed

Not applicable

Guideline development steps

Sources, reference nos.

1. Organization, budget, planning and training

□

□

1. Establish the structure of the guideline development group and determine the roles, tasks and relationships among the various groups to be involved (e.g., oversight committee or body to direct guideline topic selection and group membership, a working group consisting of experts and methodologists to synthesize evidence, a secretariat to provide administrative support, a guideline panel to develop recommendations, and stakeholders and consumers for consultation). See also topics 3, 4 and 6

20, 32–36, 39, 42–45, 56, 57, 72, 88

□

□

2. Perform a thorough assessment of the proposed guideline development project with respect to financial and feasibility issues concerning the guideline development group (e.g., availability of resources to complete the project, expected commitment from guideline panel and staff).

20, 32, 35, 37, 40, 42–46, 47, 57, 58, 71, 88

□

□

3. Obtain organizational approval to proceed with the guideline project.

2, 20, 33–35, 37, 42, 43, 45, 46, 58

□

□

4. Prepare a budget for the development of the guideline, outlining the estimated costs for each step (e.g., remuneration of working group members and staff, cost of outsourcing certain tasks to outside organizations or groups, travel expenses, and publication and dissemination expenses).

3, 37, 41, 45, 71

□

□

5. Determine whether guideline panel members will be provided any payment or reimbursement for their time or will work as volunteers.

3, 34, 47, 57

□

□

6. Obtain or secure funding for the development of the guideline, with attention to conflict-of-interest considerations. See also topic 7

1, 20, 43, 45, 46, 56, 57, 89

□

□

7. Outline and arrange the administrative support that will be required to facilitate the guideline development process (e.g., a secretariat of the working group to organize and obtain declaration of interests and to arrange group meetings).

20, 32, 37, 40, 42, 43, 45, 56, 57, 88

□

□

8. Plan and prepare for training and support that will be required for those involved in the guideline development process (e.g., conflict-of-interest–related education or training for guideline panel members, and teaching sessions for patients to be involved in the guideline group). See also topics 4 and 6

2, 3, 39, 42–45, 60, 64, 70, 72, 88

□

□

9. Set a timeline for the completion of the guideline and target dates for the completion of milestones in the guideline development process.

11. To keep the guideline development group on track, prepare a protocol for the entire guideline that can be completed as the project progresses, including an outline of the overall goals and objectives for the guideline, the timeline, task assignments, steps that will require documentation of decisions and the proposed methods for all steps (i.e., those covered in this checklist, such as the methods for forming the guideline group, selection of topics to be covered in the guideline, consensus methods, consultation methods, and methods for the evidence search and selection).

1, 2, 23, 32–35, 37, 39, 42, 44, 45, 56, 57, 70, 74, 90

2. Priority setting

□

□

1. Decide on a process for priority setting of guideline topics and who will be responsible for directing the process (e.g., priorities set by oversight committee at headquarters of sponsoring organization, or priorities referred by government ministries of health or by professional societies).

7, 20, 32–37, 39, 41–43, 45, 47, 56, 71, 87

□

□

2. Apply a systematic and transparent process with specific criteria for the proposal of a guideline topic during priority setting (e.g., high prevalence and burden of disease, avoidable mortality and morbidity, high cost, emerging diseases or emerging care options, variation in clinical practice, and rapidly changing evidence).

7, 20, 32–37, 39, 41–43, 47, 56, 57, 71, 87

□

□

3. Involve appropriate stakeholders in the priority setting and selection of guideline topics (e.g., clinicians, professional societies, policy-makers, payers and the public). See also topic 6

32–34, 36, 37, 39, 42–45, 71, 72, 87

□

□

4. Consider and decide how different perspectives on the importance and resources required for implementing the guideline recommendations will be considered (e.g., perspectives of patients, payers, clinicians and public health programs). See topic 11

1, 23, 32, 34, 36, 37, 42, 46, 47, 57, 71, 94, 99

□

□

5. Search for any existing up-to-date guidelines covering the proposed topic and assess their credibility (e.g., AGREE II). Determine whether the existing guideline(s) can be adapted or whether a completely new guideline should be developed. See also topic 10

20, 32–39, 42, 43, 45, 47, 56, 57, 71, 99

□

□

6. Discuss the need or opportunity to partner with other organizations that develop guidelines to determine whether a collaborative effort will be sought for the development of the guideline, or any part of the guideline.

3, 11, 32, 33, 37, 41, 45

□

□

7. Perform a scoping exercise for the proposed guideline topic with respect to implementation issues and barriers to change (e.g., if developed, the guideline is likely to improve health outcomes, implementation of health care recommendations is feasible, resources are available).

1, 3, 9, 20, 23, 32, 33, 35, 37, 39, 42–47, 56, 57, 71, 94

□

□

8. Select or provide a consensus method to be used to agree on the priorities set and the guideline topic selected (e.g., voting, Delphi consensus). See also topic 4

20, 33, 37, 42, 43, 71, 87

□

□

9. To ensure transparency, document the processes used to set priorities and to select guideline topics.

20, 33, 35, 37, 42, 43, 71, 87

3. Guideline group membership

□

□

1. Seek multidisciplinary representation for the guideline development group, including members from the target audience, patients, caregivers, frontline clinicians, content experts, methodology experts and experts in health economics, to fulfill the roles required (e.g., for the working group, guideline panel). See also topic 6

1–3, 5, 7, 20, 33–35, 37–47, 56, 57, 72, 86

□

□

2. Decide on methods for recruitment and enrolment of members for the guideline development group (e.g., widespread advertising of posts, competitive appointment by interview).

3, 5, 33–35, 37, 41–43, 56, 72, 88

□

□

3. Achieve a topic-appropriate balance of expertise and adequate representation for the guideline panel (e.g., experts and primary care physicians who form the target audience, and sex and geographic distribution of panel members), which may be iterative if additional members are required as the target audience and topics within the guideline are refined. See also topic 5

1, 3, 5, 7, 20, 32–34, 37–39, 41–45, 57, 72, 88

□

□

4. Consider the optimum size for the guideline development group, particularly the guideline panel (e.g., if too small, the group may lack sufficient experience, content expertise and wide representation; if too large, it may lack cohesiveness and effective group interaction).

2, 3, 5, 7, 20, 33, 37, 39, 42–45, 56, 72, 88

□

□

5. Outline roles for the group members and the tasks they will be responsible for (e.g., forming a writing team, taking meeting minutes and documenting decisions, providing methodology consultation, conducting systematic reviews and obtaining other evidence, providing perspective of patients, and providing perspective of specialist clinicians).

3, 5, 7, 20, 32–34, 37– 40, 42–45, 56, 72, 88

□

□

6. Select group leader(s) or chair(s) experienced in group facilitation, maintaining constructive dynamics, and identifying and resolving conflicts who will remain neutral and objective and who have methodologic expertise and content expertise.

2, 3, 5, 7, 20, 32, 33, 37–39, 41–45, 56, 72, 88

□

□

7. To ensure transparency, document the process used to select guideline group members and their roles.

1, 2, 5, 32, 33, 37, 42, 43, 45, 46, 72, 88

4. Establishing guideline group processes

□

□

1. Establish how and how often communication with guideline panel members and other groups will take place, establish who will be responsible for making the arrangements, and consider when to deviate from this approach.

3, 5, 20, 32, 33, 37, 42–44, 56, 72, 88

□

□

2. Set expectations and awareness of the group process through an introduction, training and support for the guideline development group members (e.g., setting ideal conditions for group discussion and decision-making).

2, 3, 7, 20, 32, 39, 43, 45, 47, 56, 63, 72, 88

□

□

3. As part of the training for the guideline development group, ensure that group members understand what the process and proposed methods will be and that they need to be adhered to (e.g., consensus methods that may be used, anonymous or non-anonymous voting, assessment of evidence, group discussion and contribution of ideas).

3, 20, 32, 39, 43–45, 47, 56, 63, 72, 74, 88

□

□

4. Aim to set optimal conditions for group members to be provided equal opportunities to contribute ideas and for their ideas and arguments to be given appropriate consideration (e.g., during group discussion, decision-making and formulation of recommendations).

2, 3, 7, 32, 33, 43, 45, 47, 56, 63, 72, 74, 88

□

□

5. Establish methods for dealing with conflict or disputes among group members and dysfunction in the group process.

32, 33, 44, 45, 47, 56, 63, 74, 88

□

□

6. Provide opportunities for discussion and feedback about the group process throughout the guideline development project.

E, 43

□

□

7. Establish a method for structured and timely distribution and archiving of documents used and produced in the guideline development.

32, 39, 42, 44, 56, 88

□

□

8. Set a quorum for meetings (e.g., 75% of group must be present to formulate recommendations), but expect that all group members attend all meetings as much as possible.

20, 32, 33, 37, 42

□

□

9. Set or plan times and locations for meetings (virtual or in person) in advance and prepare a scope and specific agenda for each meeting.

5, 20, 33, 37, 39, 42, 44, 45, 47, 56, 72, 88

□

□

10. Keep a record of all meetings with minutes (e.g., who attended, what was the agenda, what decisions were made, what the next steps will be) and determine whether not to make the minutes publicly or internally available.

5, 20, 32, 42, 44, 88

5. Identifying target audience and topic selection

□

□

1. Identify, define and/or review the primary audience (e.g., primary care physicians, health program managers) and secondary audience(s) (e.g., hospital administrators) for the guideline and determine how many audiences can be addressed with the guideline.

1, 2, 7, 20, 23, 32, 35, 37–39, 42, 44, 45, 47, 66, 86

□

□

2. Consult appropriate stakeholders about the identified target audiences to ensure they are applicable for the guideline topic and that no relevant audience is missed. See also topic 6

20, 39, 44, 86

□

□

3. Establish a method and criteria to generate and prioritize a candidate list of topics to be addressed within the guideline (e.g., where evidence is most confusing or controversial, where there is currently uncertainty or inconsistency in practice, where questions exist about screening, diagnosis and treatment).

3, 8, 20, 32–34, 36–39, 42–45, 47, 56, 57, 66, 87

□

□

4. Consult appropriate stakeholders to ensure all relevant topics for the guideline have been identified and will meet the needs of the target audience(s). See also topic 6

3, 20, 33, 36, 37, 39, 42–44, 87, 93

□

□

5. Select or provide a consensus development method to be used by the group in agreeing on the final topics to be addressed within the guideline (e.g., Delphi method, nominal group technique).

37, 42, 56, 87

□

□

6. To ensure transparency, document the processes used to identify the target audience(s) and to select the topics for the guideline.

1, 23, 33, 37, 39, 44, 66, 86, 87

6. Consumer and stakeholder involvement

□

□

1. Identify the appropriate stakeholders to involve and consult with in the development of the guideline to incorporate views of all those who might be affected by the guideline (e.g., professional groups, health managers, policy-makers, industry representatives).

1–3, 33, 35, 37–39, 43, 44, 46, 47, 56, 57, 60, 64, 72, 94

□

□

2. Identify the appropriate consumers to involve and consult with in the development of the guideline (e.g., individual patients, people who provide nonreimbursed care and support to patients, members of the public as potential patients and as funders of health care through taxation, community organizations that represent the interests of patients, and advocates representing the interests of patients and people who care for patients).

1–3, 7, 35, 38, 39, 43–46, 56, 60, 64, 72, 93

□

□

3. Establish methods for consumer and stakeholder involvement and maintain a registry of stakeholders for the guideline (e.g., enroll consumer and stakeholder members to participate directly on the guideline panel, announce call for separate consumer and stakeholder meeting[s] or workshop[s], distribute documents and feedback electronically, have an open period for review of documents and feedback).

2, 3, 5, 7, 33, 35, 37–39, 42–44, 46, 47, 56, 60, 64, 72, 93, 94

□

□

4. Provide information (e.g., training and introduction sessions) for consumers and stakeholders involved directly on the guideline panel to clarify roles and maximize contributions (e.g., information on evaluating evidence objectively and avoiding recommendations based on self-interests).

2, 3, 7, 43, 44, 64, 72, 93, 94

□

□

5. Determine the roles, tasks and timing for consultation with consumers and stakeholders not directly participating on the guideline panel (e.g., at specific milestones during the guideline development process, include opportunities to comment on priority setting, suggest topics for the guideline, identify target audience, identify patient-important outcomes, identify additional evidence, point to consequences that the panel has not considered and review the final guideline draft).

3, 7, 35, 38, 39, 43, 44, 46, 47, 64, 72, 79, 94

□

□

6. Develop or adopt standard templates for consumer and stakeholder input and comments during consultation, with clear instructions or training modules to ensure effective input.

33, 42, 43, 44, 94

□

□

7. Offer adequate time for consumer and stakeholder feedback and consultation.

33, 44, 64, 94

□

□

8. Set a policy and process for handling consumer and stakeholder feedback and dealing with different perspectives (e.g., ensure that diverse perspectives are taken into account in making decisions, provide transparent rationale for judgments made, provide an appeal process for stakeholders, and publish consultation comments and the guideline development panel’s responses).

33, 42, 43, 45, 64, 79, 94

□

□

9. To ensure explicit and transparent methods, document the enrolment and selection of consumers and stakeholders for the guideline panel and the involvement and consultation with all other consumers and stakeholders.

1, 33, 42, 44, 46, 64, 72, 94

7. Conflict-of-interest considerations

□

□

1. Set a policy for declaration of interests by individual participants at admission to the project, including potential guideline panel members before their involvement (e.g., what interests should be disclosed [e.g., financial, intellectual, academic/clinical, competitive interests of the professional society]).

2, 3, 5, 7, 20, 33, 35, 37–39, 41–46, 56, 58, 62, 66, 73, 89

□

□

2. Set a policy for determining conflicts of interest and an approach for collecting and updating declarations of conflicts of interest (e.g., how and what level of financial interest should be disclosed, how long the period covered by the disclosure should be, who will judge what constitutes a conflict).

2, 3, 5, 7, 20, 33, 35, 37, 39, 41–46, 56, 58, 62, 66, 73, 89

□

□

3. Provide clear instructions and training to the potential guideline group members on how to complete the conflict-of-interest disclosure, including a list of the members who must declare conflicts of interest and the types of interests to declare (include examples).

3, 7, 20, 33, 37, 39, 42– 45, 56, 58, 62, 66, 73, 89

□

□

4. Set a policy for managing conflicts of interest (e.g., allow individuals with conflicts of interest to participate in the guideline development but exclude them from voting on specific recommendations related to the area of conflict; ensure that the chair has no conflicts of interest; have evidence summaries prepared by unconflicted methodologists).

2, 3, 5, 7, 20, 33, 35, 37, 39, 41–46, 56, 58, 62, 66, 73, 89

□

□

5. Set a policy for managing conflicts of interest with respect to funding of the guideline development activities (e.g., advocate for public funding, no commercial sponsorship, commercial sponsorship from entities unrelated to topic of guideline, commercial support for nondirect activities such as translation, no single-source sponsor).

1, 45, 46, 60, 73, 89

□

□

6. Disclose and publish the funding source and describe the role of the sponsors and support provided for the development of the guideline.

1–3, 20, 23, 38, 45, 46, 56, 57, 60, 83, 89

□

□

7. Explicitly disclose, publish and describe conflicts of interest of the guideline group members, particularly where the conflicts bear on specific recommendations.

1–3, 5, 7, 20, 23, 35, 37, 39, 42–46, 56, 57, 62, 83, 89

8. Question generation

□

□

1. Establish methods for generating the questions for the guideline, prioritizing questions, and selecting and ranking outcomes.

2, 3, 5, 8, 20, 33, 35–39, 42–45, 47, 56, 57, 60, 66, 75, 90

□

□

2. Generate and document the key questions (e.g., clinical, health, policy, cost-effectiveness) to be answered in the guideline using a standard format (e.g., PICO [patient/problem, intervention, comparison, outcome] framework). If it is not feasible to answer all questions (e.g., because of limited time or resources), determine the criteria by which the questions generated will be prioritized (e.g., by surveying guideline panel members and other stakeholders).

1–3, 5, 8, 20, 33–39, 42–45, 47, 56, 57, 60, 66, 75, 90

□

□

3. Explicitly describe the population to whom the guideline is meant to apply. Take into consideration specific characteristics of the population, such as prevalence of multiple comorbidities in the population, geographic setting and equity issues (e.g., plausible reasons for anticipating differential relative effects across disadvantaged and advantaged populations).

1–3, 5, 9, 20, 23, 32, 33, 41–45, 48, 51–54, 57, 73, 87

□

□

4. Determine whether or not regulatory approval is required for considering interventions in guidelines (e.g., for international guidelines, this may be not relevant because regulatory approval may not be available or present in all target countries).

3, 20, 32, 39, 42, 45, 46, 81, 94

□

□

5. Explicitly describe the intervention(s) and comparator(s) to be considered in the guideline and develop an analytic framework depicting the relationships among interventions and outcomes. Identify whether or not multiple (treatment) comparisons should be included.

2, 3, 5, 8, 20, 23, 32, 33, 35–37, 39, 42–47, 56, 76, 90

□

□

6. Identify the important outcomes (e.g., outcomes along the clinical pathway; morbidity, quality of life, mortality), including both desirable (e.g., benefits, less burden, savings) and undesirable (e.g., harm, burden, costs and decreased patient autonomy) effects. Do not ignore important outcomes for which evidence may be lacking,

1–3, 5, 8, 20, 32, 33, 35–39, 42–47, 56, 75, 90

□

□

7. Determine the setting (e.g., countries, hospitals) or include it in the considerations about the population (i.e., population cared for in tertiary care hospitals).

3, 20, 37, 42, 43, 45, 47, 66, 75, 90

□

□

8. Mandate a preference for patient-important outcomes over surrogate, indirect outcomes. Consider appropriateness of surrogate outcomes along the causal pathway when data for a patient-important outcome is lacking.

3, 5, 8, 20, 37, 39, 42, 43, 47, 75, 90

□

□

9. Rank the relative importance of the outcomes, taking into consideration the values and preferences of the target population.

3, 5, 20, 33, 35, 37, 39, 42, 44, 45, 75, 90

□

□

10. Determine or develop a process for determining a priori the magnitude of effect for the individual outcomes that is judged as important to the target population.

5, 90

□

□

11. Involve all guideline group members and consult consumers and stakeholders to ensure broad representation from the target population in generating the questions and selecting and rating the important outcomes.

3, 20, 37, 39, 42, 45, 72, 75, 94

□

□

12. To ensure explicit and transparent methods, document the generation and prioritization of questions, the selection and ranking of outcomes, and stakeholder and consumer consultation.

20, 33, 42, 45, 66, 90

□

□

13. To help direct the evidence review, ensure that the guideline protocol outlines the target population, target condition, outcomes and key questions considered.

1, 2, 5, 23, 33, 35, 42, 44, 66, 90

9. Considering the importance of outcomes and interventions, values, preferences and utilities

□

□

1. Decide whether the relative importance of outcomes and interventions, values, preferences or utilities of consumers and stakeholders (e.g., patients and target audience) to inform decisions and deliberations during the guideline development will be elicited indirectly or directly (e.g., review of the published literature v. consultation with consumers).

1–3, 5, 7, 20, 33, 35, 37, 43–46, 56, 72, 79, 93

□

□

2. Establish methods for consultation with consumers and stakeholders to obtain information about the relative importance of outcomes and interventions, values, preferences or utilities (e.g., involvement of consumers on guideline panel, surveys or focus groups with broader representation of consumers).

3, 7, 43, 44, 79, 93

□

□

3. Determine whether a structured approach will be used to assess the confidence (i.e., quality of the evidence) in the obtained importance ratings, values, preferences and utilities.

E, 79

□

□

4. Determine whether modelling will be used to integrate the relative importance of outcomes and interventions, values, preferences or utilities and how modelling will be done.

E, 8, 79

□

□

5. Determine whose perspective(s) will be considered when obtaining information about the relative importance of outcomes and interventions, values, preferences or utilities and when making decisions or formulating recommendations (e.g., patients, public, society, clinicians).

9. Decide how to consider ethical or moral values in making health care recommendations (e.g., by considering religious, social, or cultural convictions).

102

10. Deciding what evidence to include and searching for evidence

□

□

1. Follow methods for systematic reviews (either full systematic reviews or rapid systematic reviews depending on the topic and organization’s framework) or provide a rationale for why this is not done.

1–3, 5, 8, 20, 32–47, 56, 57, 77, 92

□

□

2. Develop a protocol for locating, selecting and synthesizing the evidence (e.g., conduct a search for existing systematic reviews, conduct a new systematic review and search the grey literature) and determine the types of evidence to include (e.g., databases searched, types of studies, inclusion and exclusion criteria, searching for specific studies on adverse effects or deciding to abstract information on adverse effects from studies on benefit).

1–3, 5, 8, 20, 23, 32–47, 56, 57, 77, 90

□

□

3. Decide who will develop the search strategies and perform the search and selection of evidence (e.g., working group of guideline development group, outsource to external agency, form a relationship between guideline development group and external agency to collaborate on development of the guideline).

3, 5, 20, 32–35, 42–45, 56, 77, 88

□

□

4. Using a validated tool, critically appraise existing systematic reviews selected to be included to ensure they are of adequate quality and appropriate for use in the guideline.

3, 5, 8, 20, 34–39, 42–45, 56, 77, 92

□

□

5. If an existing systematic review is updated or requires updating, determine how new evidence will be included and how those who conducted the review will be contacted and possibly involved in the update.

37, 45, 57, 92

□

□

6. If a new systematic review is required, conduct an assessment to determine whether adequate resources (e.g., time and funding) are available to conduct a full systematic review.

37, 39, 45, 77

□

□

7. If resources are limited, consider applying a rapid assessment methodology and explicitly describe the methodology, noting important limitations, uncertainties, and the need and urgency to undertake a full systematic review.

9. Set a policy for handling expert input (i.e., expert opinion is not evidence per se and should not be used as evidence; rather, experience or observations that support expert opinions should be described, identified and, if possible, appraised in a systematic and transparent way [e.g., in the conceptual framework]).

3, 32, 34, 35, 76

□

□

10. To ensure explicit and transparent methods, document and publish the search and selection of evidence, the judging of eligibility, the range of evidence included and the search strategies used.

1, 2, 20, 23, 32, 33, 35, 37, 39, 42, 46, 47, 57, 92

11. Summarizing evidence and considering additional information

□

□

1. Summarize the evidence using a concise summary (e.g., evidence table, evidence profile or summary of findings table) of the best available evidence for each important outcome, including accuracy of diagnostic tests, anticipated benefits, harms, resources (costs), rating of the quality of evidence, and a summary of the relative and absolute results/estimate of effect for each outcome.

4. Identify the costs, resource use and, if applicable, cost-effectiveness and describe the nature of the costs (patient, community, society) (e.g., affordability considerations, estimates of resource use and acquisition costs weighed directly against evidence of benefits and harms of an intervention).

1–3, 5, 8, 20, 33, 35, 37, 39, 42–47, 80, 91

□

□

5. To ensure transparent methods, document how the additional information is to be incorporated with the synthesized evidence (e.g., formal consensus on patient values, consensus on equity issues, formal economic analysis, consideration of disaggregated resource use data in a qualitative manner).

3, 5, 8, 20, 23, 33–35, 37, 39, 42, 44–47, 77, 81, 91, 93

□

□

6. Provide training about the use of the evidence tables and opportunities for discussion to ensure that all members of the guideline panel are familiar with the tables and use them in the appropriate manner.

44, 70, 88

□

□

7. In addition to the evidence summary, make available the full systematic review(s) and the original studies and other sources of evidence for the guideline panel to inform deliberations (e.g., set up a collaborative website or make sources of evidence available at meetings and via electronic communication).

44, 77

12. Judging quality, strength or certainty of a body of evidence

□

□

1. Select a framework outlining the criteria to be considered in rating the quality of evidence (e.g., GRADE, US Preventive Services Task Force). Avoid modifying grading tools.

2, 3, 5, 8, 20, 32–37, 39–47, 56, 78, 92

□

□

2. Decide who will be responsible for appraising the quality of evidence (e.g., unconflicted methodologists participating in the working group).

20, 34, 39, 43, 44, 97

□

□

3. Assess the quality of evidence for each important outcome.

5, 8, 20, 33, 35, 36, 39, 42, 44, 45, 78, 92

□

□

4. Assess the overall quality of evidence (e.g., lowest quality of evidence from outcomes rated as most important or critical, or highest quality of evidence when all outcomes point in the same direction).

5, 8, 20, 32, 33, 35–37, 39, 42, 44–47, 56, 78, 92

□

□

5. Report the quality of evidence assessed for the outcomes and the body of evidence.

2, 8, 20, 32, 36, 37, 39, 44–46, 83, 97

□

□

6. To ensure they are explicit and transparent, document the judgments made in appraising the quality of evidence.

2, 5, 20, 32, 33, 35, 37, 39, 42, 44–47, 78, 92

13. Developing recommendations and determining their strength

□

□

1. Apply a framework outlining the factors to be considered to arrive at a recommendation.

1, 3, 5, 8, 23, 32–39, 42–47, 56, 57, 78, 96

□

□

2. Plan and share the logistical details of the consensus meeting(s) during which recommendations will be formulated with the participants, including distribution of documents required for the meeting (e.g., evidence summaries, evidence-to-recommendation tables), setting an agenda for the meeting(s) and selecting a consensus development method to be used by the group in agreeing on judgments (e.g., Delphi method, nominal group technique).

5, 34, 45, 47, 56

□

□

3. Review the factors of the framework that influence the recommendation, including the direction and strength (e.g., the types of evidence and information relevant to the analysis, focusing on the balance between desirable and undesirable consequences informed by the quality of evidence, the magnitude of the difference between the benefits and harms, the certainty about or variability in values and preferences, resource use, equity and other factors).

1–3, 5, 8, 20, 23, 32, 35–39, 42–45, 47, 57, 65, 78, 96

□

□

4. If applicable, make provisions for formulating recommendations in situations where there is insufficient evidence or very-low-quality evidence (e.g., conditional recommendation with judgments laid out transparently; no recommendation if the guideline panel feels there is substantial risk that their decision may be wrong; recommendation that the intervention be used in the context of research complemented by guidance for what are the best management options until further research becomes available).

8, 20, 33, 35, 37, 39, 42, 44, 45, 96

□

□

5. Make provisions for formulating research recommendations and decide where to report them (e.g., in the guideline appendix, suggest the specific research questions, specific patient-important outcomes to measure and other relevant aspects of what research is needed to reduce the uncertainty about the benefits and/or undesirable downsides of the intervention).

8, 39, 42–45, 85, 96

□

□

6. Formulate the recommendations and summarize the rationale for each recommendation (e.g., narratively or in a table), including details about the judgments made by the group and the explicit link between the recommendation and evidence supporting the recommendation.

1, 3, 8, 20, 23, 33, 35– 38, 42–46, 83, 96, 97

□

□

7. Select a method for rating the strength of the formulated recommendations to inform the audience of the guideline about the degree of the guideline group’s confidence about following that recommendation.

2, 3, 5, 8, 20, 32, 33, 36, 37, 39, 40, 43–47, 56, 78, 96

□

□

8. Select a consensus development method to be used by the group in rating the strength of recommendations (e.g., Delphi method, nominal group technique, voting).

5, 20, 37, 43–45, 88

□

□

9. Provide suggestions about whether the recommendations are appropriate to serve as performance measures/quality criteria (e.g., management options associated with strong recommendations based on high- or moderate-quality evidence are particularly good candidates for quality criteria; when a recommendation is weak, discussing with patients the relative merits of the alternative management strategies and appropriate documentation of this interaction may become a quality criterion).

3, 9, 20, 33, 56, 96

□

□

10. To ensure they are explicit and transparent, document the judgments made in formulating the recommendations and determining their strength.

1–3, 5, 20, 23, 32, 33, 35–37, 43, 45–47, 57, 83, 96

14. Wording of recommendations and of considerations of implementation, feasibility and equity

□

□

1. Decide on standardized wording to use for recommendation statements to ensure clarity and to maintain consistency throughout the guideline. Avoiding wording that may be vague and nonspecific.

1–3, 5, 8, 20, 32, 33, 37, 39, 42, 45–47, 56, 96

□

□

2. Write the recommendations in a way that is actionable. Provide sufficient information so that it is not necessary for guideline users to refer to other material in order to understand the recommendation.

1–3, 8, 20, 23, 32, 37, 39, 42, 45, 46, 56, 83, 97

□

□

3. Provide clear direction or an interpretation aid to describe the implication of the strength of recommendation for clinicians, patients, policy-makers and other target audience groups.

8, 20, 23, 32, 33, 37, 39, 47, 56, 96

□

□

4. Indicate in the recommendation statements the population for which the recommendation is intended, the intervention being recommended and the alternative approach(es) or intervention(s).

8, 20, 23, 32, 33, 37, 42, 56, 96, 97

□

□

5. Include remarks that describe the context, feasibility and applicability of the recommendation and highlight key considerations such as equity issues and specific conditions that might apply to the recommendation (e.g., whether the conditions outlined apply to a specific subpopulation, specific types of the intervention, for certain values and preferences, when certain resources are available).

3, 5, 8, 20, 23, 32, 33, 37, 42, 56, 57, 65, 79, 81, 82, 96

□

□

6. Report the quality of evidence and the strength of recommendation in proximity to the recommendation statement.

3, 8, 23, 32, 33, 37, 43, 45–47, 56, 83

□

□

7. Establish methods to be used by the group in agreeing on the final wording of recommendation statements (e.g., review and approval, formal consensus).

5, 20, 45, 46, 56, 96

□

□

8. Report the recommendations in a way that is comprehensible and visible (e.g., do not embed recommendations within long paragraphs, group recommendations together in a summary section).

1, 3, 8, 39, 42, 44, 56, 97

15. Reporting and peer review

□

□

1. Develop or adopt a standardized format for reporting the guideline, with specific structure, headings and content.

5, 23, 34, 35, 37–46, 56, 58, 83, 97

□

□

2. Decide on the format(s) to be prepared for the guideline product(s) (e.g., full guideline, full guideline with technical report/systematic reviews, brief guideline for clinicians or policy-makers, consumer version for patients) that will correspond to the dissemination plan. See also topic 16

3, 9, 20, 32, 34–37, 39–46, 56, 57, 83, 97

□

□

3. Decide who will be responsible for writing the guideline product(s) (e.g., subcommittee of the guideline working group) and decide on authorship (e.g., individual authors, organization as author, working group as author). See also topic 1

5, 20, 32, 33, 35, 37, 39, 42, 44, 45, 57, 97

□

□

4. Conduct a review of the final draft of the guideline report(s) by all members of the guideline development group, allowing sufficient opportunity for feedback, editing and revisions.

5, 32, 33, 42, 43, 45, 85, 97

□

□

5. Seek approval from all members of the guideline development group for the final document(s).

3, 5, 32, 35, 36, 42, 45, 46, 56, 72

□

□

6. Initiate organizational (i.e., internal) peer review.

5, 9, 32–37, 41–45, 56, 85, 97

□

□

7. Decide on the method(s) for external peer review to review the final document(s) for accuracy, practicality, clarity, organization and usefulness of the recommendations, as well as to ensure input from broader and important perspectives that the guideline group did not encompass (e.g., invited peer review, public consultation period with incorporation of feedback and responses from the guideline development group, peer review provided by peer-reviewed journal).

1. Prepare an active dissemination plan with various approaches to enhance the adoption of the guideline (e.g., make guideline available online, develop formal relationships with those in health care systems responsible for guideline dissemination and implementation to support guideline uptake, conduct press conference, develop social media strategy, disseminate guideline at professional society meetings, publish guideline in journal that is accessed by target audience).

3, 5, 9, 20, 34–40, 42, 44–47, 56, 57, 84, 98

□

□

2. Develop or adapt tools, support and derivative products to provide guidance on how the recommendations can be implemented into practice (e.g., develop mobile applications, integrate guideline with clinical decision support systems, make guideline adaptable as an educational resource for target audience for education outreach).

1, 3, 5, 9, 20, 23, 32, 35–37, 41–46, 82, 97

□

□

3. Make considerations for adaptation of the guideline and provide specific instructions for how target end users who would like to adapt the guidelines to other contexts can do so in a systematic and transparent way (e.g., modifying a recommendation based on local resources and baseline risk, implications that deviate from the judgments made by the guideline panel).

1, 44–47, 82, 99

□

□

4. Set rules and regulations for translation of the guideline into other languages (e.g., allow translation by third-party organizations following approval by the guideline group, include staff responsible for translation in guideline working group).

41, 45, 99

17. Evaluation and use

□

□

1. Conduct an internal evaluation (i.e., self-assessment) of the guideline development process, including the guideline panel meeting(s) held to formulate recommendations, by asking guideline group members for feedback.

E, 85

□

□

2. Consider pilot testing the guideline with the target end users (e.g., members of target audience and stakeholders who participated in the guideline development group).

1, 23, 43, 44, 47, 85

□

□

3. Provide criteria and tools for target end users to monitor and audit the implementation and use of the guideline recommendations (e.g., identify outcomes that should change with implementation and suggest methods for measuring the outcomes).

1, 3, 9, 20, 23, 35, 36, 39, 42–47, 56, 57, 85, 99

□

□

4. Provide support and tools for prospective evaluation of the guideline to determine its effectiveness after implementation (e.g., use randomized evaluations where possible, use before–after evaluations cautiously because of uncertainties regarding the effects of implementation).

20, 35, 42, 44–47, 84, 85

□

□

5. Consider the potential involvement of the guideline development group in prospective evaluation(s) of the guideline (e.g., partnering with organizations that implement the guideline to plan evaluation studies).

3, 41, 42, 44, 45, 84, 85

□

□

6. Plan to collect feedback and evaluations from end users to identify how to improve the intrinsic implementability of the recommendations in subsequent versions of the guideline.

3, 20, 35, 42, 43, 47, 85, 99

18. Updating

□

□

1. Set a policy, procedure and timeline for routinely monitoring and reviewing whether the guideline needs to be updated (e.g., update systematic review every 3 years to determine whether new evidence is available).

1–3, 5, 9, 20, 23, 32, 35–37, 39–47, 56, 57, 61, 85, 99

□

□

2. Decide who will be responsible for routinely monitoring the literature and assessing whether new evidence is available (e.g., consider involvement of experts not previously involved in the guideline development group to periodically review the guideline).

3, 5, 32, 37, 39, 42–45, 56, 57, 85, 99

□

□

3. Set the conditions that will determine when a partial or a full update of the guideline is required (e.g., whether only certain recommendation statements need to be updated; whether many recommendations are out of date, making the entire guideline invalid; or whether recommendations need to be added for newly available treatments).

2, 3, 9, 20, 35, 37, 40, 42–45, 56, 57, 61, 85

□

□

4. Make arrangements for guideline group membership and participation after completion of the guideline (e.g., rotating membership every 1–2 years, selection of a new group at time of updating, continuing participation by guideline panel chair).

33, 37, 41, 42, 56, 85, 99

□

□

5. Plan the funding and logistics for updating the guideline in the future (e.g., secure ongoing funding, form a standing oversight committee to oversee the updating process).

44, 85

□

□

6. Document the plan and proposed methods for updating the guideline to ensure they are followed.