May 5, 2020 • Brooke Siem
How to Read a Scientific Paper, Part 1: Systematic Review vs. Research vs. Meta-Analysis
When I first began speaking openly about long term antidepressant use and antidepressant withdrawal, it didn’t take long for me to be faced with a wall of academic journals and research papers. At first, my instinct was to read the abstract, get the gist of what I was trying to understand and move on. But much like sourcing all your information exclusively from Fox News, that approach left me a dangerous kind of dumb. I had just enough information to confirm my bias but zero original thoughts surrounding the source, scope of work, journal reputation, limitations of the study, and industry response.
When it dawned on me that just reading the abstract was no better than just reading sensational news headlines and deeming yourself informed, I began to read the studies in full. At least, I tried. For those of us who haven’t spent their entire adult lives in research and academia, these papers are a nightmare.
While I understand that there are longstanding reasons why academic papers are written the way they’re written, it bothers me that only people with a PhD are taught to comprehend this sort of work. How can the individual be expected do their own research and make their own decisions for their own wellness if they can’t understand the research that policy and marketing is built upon?
Which brings me to the first installment of How to Read a Scientific Paper. I’m tired of taking other people’s word on research as gospel, so I’m going to learn how to do it myself and chronicle the journey here. Hopefully, I can beef up the entertainment factor, because damn these articles are dry.
I’m going to begin with a recent article spearheaded by psychiatrist Saeed Farooq and published in the Journal of Affective Disorders, entitled, “Pharmacological interventions for prevention of depression in high risk conditions: Systematic review and meta-analysis.”
I first found out about the study thanks to a Keele University tweet that said, “The study, led by Professor Saeed Farooq, found that using antidepressants as a pre-emptive measure could help to prevent depression in patients considered to be at high risk of developing the condition, for example following stroke or heart attack.” The tweet linked not to the article, but an in-house blog post that feels a bit too much like propaganda. The fact that we’re even considering doping people up on antidepressants before they become depressed deeply concerns me, so I want to learn more about it before I go full oh no you di’n’t! on the topic.
In reality, this was not a research study or clinical trial, but a systematic review and meta-analysis. And for us to learn to read journal articles, we must understand the difference.
What is a research article?
A research article is a study designed and performed by the paper’s author or authors. It will explain the methodology of the study—or rather, the methods and systems used to conduct the study—and clarify what the results mean. All of the steps are listed in detail in order to allow other researchers to conduct similar experiments.
One of the best ways to tell if you’re reading a research article is to look for phrases like “we found” or “I measured” or “we tested.” This indicated that the authors who are writing the article are the ones who also conducted the research.
Next, look at the formatting of the article. Research papers include sections that are listed in a particular order: abstract, introduction, methods, results, discussion, and references.
What is a review?
Review papers do not include original research conducted by the authors(s). Instead the author(s) give their thoughts on existing research papers for the purpose of identifying patterns or forming potential new conclusions based on a variety of research studies. For example, a researcher may look at a study performed in 1980 and compare it to a similar study from 2010 in order to provide an overview of the topic as a whole.
Reviews are particularly useful for people looking to get background information on a topic before diving into detailed or technical research papers. However, there is no formal process to dictate which articles must be included in a review, which gives authors the freedom to overlook existing research that may not fit their agenda. Thus, it can be difficult to determine if the author’s conclusions are biased.
What is a systematic review?
Systematic reviews were developed to eliminate that bias by requiring multiple authors to track down all available studies on a particular topic and execute high-level analysis of existing research in order to answer a clearly defined, clinical question. Systematic reviews can take months or years to complete, whereas standard reviews may only take a few weeks.
Systematic reviews contain a lot of data and to the untrained eye, can look a lot like original research. Systematic reviews are held in the same echelon as original research and are often presented to the public as if the research was new (like in the Keele University tweet.) This strikes me as potentially misleading, not because the research isn’t valid or useful, but because of the language used to promote the research.
For example, Farooq’s article concludes that based on his analysis, “Prevention of depression may be possible in patients who have high-risk conditions but the strategy requires complete risk and benefits analysis before it can be considered for clinical practice. However, not a single clinical study has been conducted to support or disprove that statement and the tweet says nothing about that and instead presents the research as if it were a new, exciting discovery.
What is meta-analysis?
Meta-analysis is a research process used to manage and interpret all the data for a systematic review. In layman’s terms, meta-analysis is how researchers make sense of the data in hundreds or thousands of individual papers. After extracting the data, analysts use a variety of methods to account for differences like sample size, variations in study approach that may affect the overall outcome of the systematic review, and overall findings.
Frankly, I don’t understand a lick of how meta-analysis works. But, I’ve learned that I don’t have to understand it as long as I understand what role it plays in research: meta-analysis pools the data sets from different studies into a single statistical set of data in order to analyze it and come to a single conclusion.
* * *
For or those of you who like visuals, check out this article by Concordia University that visually breaks down the structure of various journal articles so you can recognize what you’re reading.
More articles from the blog
see all articles
October 28, 2022
