Research Design and Implementation (RDI) Checklists (2010 DGAC)

Each study the DGAC reviewed received a quality rating of positive, neutral, or negative, based upon a predefined scoring system. The appraisal of study quality is a critical component of the systematic review methodology because in a highly transparent manner, it indicates the Committee’s judgment regarding the relevance (external validity/generalizability) and validity of each study’s results. Ratings were assessed using two versions of the Research Design and Implementation Checklists.

The Research Design and Implementation Checklist: Primary Research includes ten validity questions based on the AHRQ domains for research studies. Sub-questions are listed under each validity question that identify important aspects of sound study design and execution relevant to each domain. Some sub-questions also identify how the domain applies in specific research designs.

Research Design and Implementation Checklist: Primary Research

RELEVANCE QUESTIONS

1.Would implementing the studied intervention or procedure (if found successful) result in improved outcomes for the patients/clients/population group? (NA for some Epi studies)

2.Did the authors study an outcome (dependent variable) or topic that the patients/clients/population group would care about?

3.Is the focus of the intervention or procedure (independent variable) or topic of study a common issue of concern to dietetics practice?

4.Is the intervention or procedure feasible? (NA for some epidemiological studies)

2.1Were inclusion/exclusion criteria specified (e.g., risk, point in disease progression, diagnostic or prognosis criteria), and with sufficient detail and without omitting criteria critical to the study?

2.2Were criteria applied equally to all study groups?

2.3Were health, demographics, and other characteristics of subjects described?

2.4Were the subjects/patients a representative sample of the relevant population?

3.Were study groups comparable?

3.1Was the method of assigning subjects/patients to groups described and unbiased? (Method of randomization identified if RCT)

3.2Were distribution of disease status, prognostic factors, and other factors (e.g., demographics) similar across study groups at baseline?

3.4If cohort study or cross-sectional study, were groups comparable on important confounding factors and/or were preexisting differences accounted for by using appropriate adjustments in statistical analysis?

3.5If case control study, were potential confounding factors comparable for cases and controls? (If case series or trial with subjects serving as own control, this criterion is not applicable. Criterion may not be applicable in some cross-sectional studies.)

4.2Was the number, characteristics of withdrawals (i.e., dropouts, lost to follow up, attrition rate) and/or response rate (cross-sectional studies) described for each group? (Follow up goal for a strong study is 80%.)

4.3Were all enrolled subjects/patients (in the original sample) accounted for?

4.4Were reasons for withdrawals similar across groups?

4.5If diagnostic test, was decision to perform reference test not dependent on results of test under study?

The Research Design and Implementation Checklist: Review Articles has ten validity questions that incorporate the AHRQ domains for systematic reviews. These questions identify the systematic process for drawing valid inferences from a body of literature.

Research Design and Implementation Checklist: Review Articles

RELEVANCE QUESTIONS

1.Will the answer if true, have a direct bearing on the health of patients?

2.Is the outcome or topic something that patients/clients/population groups would care about?

3.Is the problem addressed in the review one that is relevant to dietetics practice?

4.Will the information, if true, require a change in practice?

1.Was the question for the review clearly focused and appropriate?

2.Was the search strategy used to locate relevant studies comprehensive? Were the databases searched and the search terms used described?

3.Were explicit methods used to select studies to include in the review? Were inclusion/exclusion criteria specified and appropriate? Were selection methods unbiased?

4.Was there an appraisal of the quality and validity of studies included in the review? Were appraisal methods specified, appropriate, and reproducible?

5.Were specific treatments/interventions/exposures described? Were treatments similar enough to be combined?

6.Was the outcome of interest clearly indicated? Were other potential harms and benefits considered?

7.Were processes for data abstraction, synthesis, and analysis described? Were they applied consistently across studies and groups? Was there appropriate use of qualitative and/or quantitative synthesis? Was variation in findings among studies analyzed? Were heterogeneity issued considered? If data from studies were aggregated for meta-analysis, was the procedure described?