Article review: How do you assess the quality of educational research articles?

Imagine this. You are about to conduct an innovative educational project and want to get a research publication out of it. What are considered strong methodological qualities of an educational research study? What can you do to improve your chances for publication?

The authors in this study developed and use an instrument to help measure the methodological quality of quantitative studies in medical education. This instrument, the Medical Education Research Study Quality Instrument (MERSQI), was used to show that scores were predictive of manuscript acceptance into the 2008 Journal of General Internal Medicine (JGIM) special issue on medical education.

What is the MERSQI instrument?The 10-item MERSQI instrument can be divided into 6 domains, each with a maximum score of 3 points. Numbers within parenthesis denote the number of different items assessed in that domain. The maximum score is 18.

Study design

Sampling (2)

Type of data

Validity of evaluation instrument (3)

Data analysis (2)

Outcomes

The specific scoring criteria are seen in this table below, from an earlier publication on the MERSQI instrument (Reed DA et al, Association between funding and quality of published medical education research. JAMA. 2007;298(9):1002-1009). This earlier publication showed that a high MERSQI score correlated with successful funding of the study.

Getting a perfect score of 18 is extremely difficult, as evidenced in this article. In the 100 submitted manuscripts to JGIM, the mean MERSQI score was 9.6 (range 5-15.5). Most manuscripts were single-group cross-sectional studies (54%), conducted at a single institution (78%). Few (36%) reported validity evidence for their evaluation instruments.

The mean total MERSQI score of accepted manuscripts was significantly higher than rejected manuscripts (p=0.003).