A How-To On Reading Scientific Papers

“Be skeptical. But when you get proof, accept proof.” – Michael Specter

That quote is from Denialism: How Irrational Thinking Hinders Scientific Progress, Harms the Planet, and Threatens Our Lives, where New Yorker staff writer Michael Specter examined the distrust of science that’s turned discussion of scientific topics into a potential minefield. Some good examples of that minefield are climate change, and childhood vaccinations.

Anyone interested in scientific progress – full disclosure, I’m in that group – needs to understand the ideas being explored in scientific papers, the dispatches from the front lines of scientific thinking and discovery. To arrive at that understanding, you have to be able to understand what you’re reading, and I’ll be the first to admit that isn’t easy.

Scientific papers are written by scientists, for scientists, and follow a set of rules and formal structures that can feel like they’re designed to prevent any understanding by the average Joe/Jane “just plain human.” In this post, my goal is to help anyone interested in, but not formally trained in, science tackle reading – and understanding! – an article in any scientific journal.

10 steps to scientific (article) understanding

1. Check the source

   • What journal is publishing the article? Check Beall’s List, and if the journal appears there, you can stop reading – it’s a fake journal.
   • Who is the lead author, and what organization or institution is s/he affiliated with? If it’s an established university or research institute (University of Chicago or Scripps Institute, for example), keep reading.

2. Read the introduction first, not the abstract

   • The introduction will reveal the Big Question, the one that the research project worked to reveal the answer to. For instance, an article in the Christmas 2017 issue of The BMJ reports on research into the effects of pet ownership on human biomarkers of ageing; the introduction clearly lays out the Big Question as “ we examined the prospective link between pet ownership and a selected range of objective biomarkers of ageing proposed for use in large scale population based studies of older people.”

3. Write out your own summary of what the research was examining

   • This will give you a grasp of why the researchers wanted to ask the Big Question, and a framework for assessing what their answers to that question are.

4. Identify the null hypothesis

   • The null hypothesis could really be better termed the “nullifiable” hypothesis, since the purpose of the research project is to nullify the hypothesis that there are no differences in possible answers to the Big Question.
   • An example of a null hypothesis is “the world is flat,” which is what Copernicus worked to scientifically disprove a while back. He was successful, but there are some people who still reject his conclusions. (Warning: opening that link might be hazardous to your sanity.)

5. Look at the approach, and the methods, used in the research study or experiment(s)

   • What did the researchers do to answer the Big Question? What specific experiments did they run?
   • Sketch out diagrams of each experiment or data crunch.

6. Read the results section of the article

   • Look at the written results, as well as all charts and figures related to those results.
   • What are the sample sizes? Really small sample sizes are a red flag.
   • What results are listed as “significant,” and what as “non-significant”? If you want to totally geek out on this topic, this post will make your geeky day.

7. Do the results actually answer the Big Question?

   • Using your own judgment, do you think the study authors have answered the question asked in the introduction?
   • Do this before you read the paper’s conclusion.

8. Does the conclusion make sense, in light of everything you’ve read and evaluated while going through the paper?

   • Do you agree with the conclusion?
   • Can you identify an alternative explanation for the results in the article?
   • What are the next steps the authors see emerging from their research?

9. Read the abstract at the beginning of the paper

   • In light of the work you did in Steps 1 through 8, does the abstract line up with what the authors said their research purpose was?
   • Does it fit with your own interpretation of the paper?

10. What are other scientists saying about the paper?

    • Have other scientists written about this paper?
    • What other research is referenced in the paper?
    • Have the authors of that research weighed in on the paper you’re evaluating?

Reading, and understanding, scientific papers takes practice. It’s also fun, if you’re a science nerd, or just interested in new scientific discoveries. And it’s work worth doing, because the more you know, the more likely it is that you yourself might make a discovery that makes a difference.