Deciphering Scientific Studies: Join the "Journal Club"!

This article originally appeared in the Graves’ Disease & Thyroid Foundation’s Winter-Spring 2018 Newsletter.

You’ve probably flipped through articles in a popular fitness magazine – or seen links to studies in your social media feed – promising a blockbuster treatment for a particular disease or touting the latest supplement that’s supposed to be good for you.

But do you know how to critically interpret this information?

During an October 2017 patient seminar co-hosted by the GDATF and Massachusetts Eye and Ear, Dr. Suzanne Freitag, Director of the Ophthalmic Plastic Surgery Service at Mass. Eye and Ear presented “Clinical Trials for Thyroid Eye Disease: Deciphering the Data.”

Dr. Freitag joked that while other people join book clubs for fun, doctors have “Journal Clubs” to review and dissect the latest medical research. If you want to become a truly savvy consumer of medical information, consider the following when interpreting research findings.

Size of the Study. It’s important to note the size and population of the study, as studies with only a small group of participants might not be accurate or able to be replicated in a larger group.

Reproducability is considered necessary before a particular medication or protocol becomes standard practice. For example, a study out of Europe has suggested that selenium can have a beneficial impact on the course of mild thyroid eye disease. However, the study was conducted in areas where there may be lower selenium levels than in North America. Hence further studies in the United States will be helpful to determine if there is a benefit in our patients as well.

Study Design. Dr. Freitag noted that managed care companies are now selling databases, although they cannot sell private patient information. The advantage of this is that researchers can potentially benefit by data mining to draw conclusions based on information from many thousands of patients. However, the downside is that the underlying data uses medical billing codes as opposed to confirming a diagnosis and other details via patient medical record notes or direct patient interaction. We need to understand the source of the data in the studies we read so that we can anticipate potential biases.

In randomized, controlled trials, considered the “gold standard” of study design, participants are randomly assigned to separate groups. One group receives the treatment being tested while others receive a “placebo” (inactive) substance or another drug. In a double-blind study, neither participants nor researchers know which person has been assigned to which group.

In evaluating results, consider differences between the study groups. For example, in studies evaluating thyroid eye disease treatments, smokers tend to have more severe disease and often do not respond as well to treatment as nonsmokers. Patients whose thyroid function tests are normal and stable tend to have less risk of thyroid eye disease than those with levels that are hyperthyroid or hypothyroid. Patients who have thyroid eye disease that is in the active phase will likely respond better to potential treatment options than those who are in the chronic or inactive phase. If these characteristics are not well “matched” between groups, then this bias could impact the validity of the study results and conclusions.

Safety. Dr. Freitag noted the importance of checking adverse events in any study, as safety is critical.

Financial Bias. Always check to see who is providing funding for the study and what the potential conflicts of interest are for the authors. There can be unconscious bias or, hopefully rarely, intentional bias in cases where people serve to gain from certain study outcomes. These potential sources of conflict are required to be disclosed by the authors of the paper when it is published in a peer reviewed, reputable medical journal.

Finally, always consider the credibility of the source. Articles that appear in peer-reviewed journals have been subjected to a quality review by a panel of experts prior to publication. If the information comes from a lay (non-medical) publication either on paper or the internet, consider whether the source material was interpreted correctly. Some popular publications will have medical articles reviewed by a doctor, but this is not always the case. In fact, the GDATF was excited to receive an email with information about a thyroid eye disease article appearing in a popular fitness magazine. That is, until we pulled up the article online and found that it stated that radioactive iodine (RAI) was the treatment for bulging eyes due to thyroid eye disease! In fact, RAI is not a treatment for TED – and can actually increase the risk that the eye disease will worsen! The magazine corrected the article after we contacted them, but it’s a good reminder to read all medical articles – no matter where they appear – with a critical eye.

And who knows – maybe you’ll get inspired to start your own “Journal Club”!