The coronavirus pandemic requires many of us to change our behavior in fundamental ways. Scientists are working around the clock to learn as much as possible while the rest of us wait to hear their findings. The study of Covid-19 stretches across many branches of science: epidemiologists are examining how the disease spreads, immunologists are studying how the virus acts in the body, physicists are studying how to remove it from the air, and many other researchers are applying their expertise to the problem in their own way. Tens of thousands of scientific articles have been published on Covid-19 so far this year. Eventually, it is likely that hundreds of thousands will be published. While most of the scientific articles are free to read, nobody will ever read all of them.
Many of us rely on science-based journalism to summarize the latest findings. When different sources state conflicting information we need to read the researcher’s actual words. Unfortunately, even people with scientific backgrounds can find published research to be dry at best and impenetrable at worst. For the rest of us, it can be difficult to recognize the right bits of information to assemble the story. Scientists typically lack training in writing and have trouble putting themselves in the place of someone who is unfamiliar with their topic. It’s fortunate that all of the scientific disciplines adhere to published scientific literature as their standard of evidence.
Though these publications can be full of numbers and acronyms only interpretable by a scant few people in the world, we can still look to them to supply us with what matters – the truth. Looking at the original scientific literature cited by a journalist can help us make decisions guided by the latest findings by the most knowledgeable people. Below we describe how to look at scientific literature to easily understand it and whether or not we can trust its findings.
Science grows from branches
Before we read scientific literature, we need to understand its purpose. Science builds on itself, so all scientific literature published in the modern era is designed to build on literature that came before. Facts mentioned in an article that are not supported by the described experiment are referenced with previous research that supports those facts. Most scientific literature is subject to peer review, which is when a committee of scientists spend months studying every aspect of a paper and ensure that it meets certain quality standards to be included in the scientific canon. Sometimes the article is rejected completely, other times the reviewers will require some edits to make the findings less ambiguous.
When reading scientific literature, the first thing to look for is the real-world fact it investigates, which will build on previous scientific knowledge. All scientific literature is a description of one or many experiments that explore a factual statement, also known as a hypothesis.
The structure of a scientific paper
Though it may seem difficult to believe at first glance, all scientific papers are written to be easily understood. Each one contains the same basic elements so a reader can look at the paper and quickly understand what is important and what can be skipped over. Here is a summary of the basic elements included in almost all papers and how you can use each one:
- Title – The subject described in as few words as possible. Use the title to figure out if the paper contains the information you want to know.
- Abstract – A short paragraph (150-200 words) summarizing all of the sections below. The authors of the paper will state in simple terms whether the hypothesis was supported or not by the results of the experiment. Start with the abstract because sometimes this is all you will need to read to know if you can rely on the results or not.
- Introduction – This is the most information-rich section of any paper. This is where the authors will describe the history of the current science that informed the experiment. Here you can find references to other papers that explore more general hypotheses about the topic. If you want to learn about the research that came before and led up to the experiment, read the introduction thoroughly and look to the references section for papers exploring more general hypotheses.
- Methods & materials – This is the section of the paper where the authors describe how they went about the experiment. This section can usually be skipped since it will contain extensive technical specifics about the equipment used so it can be replicated by any other researcher, but can still be interesting to learn exactly what data the scientists are seeking to support the hypothesis.
- Results – Here is where the discovered facts are stated unambiguously. Numbers, tables, and graphs are often found in the results section. This section can also be skimmed since the interpretation of the results is usually best left to experts in the field, but look for the biggest and most obvious graphs as these will tell the main story of the article.
- Discussion – The discussion section is where the authors will describe in great detail how the results support or do not support the hypothesis and how the findings tie back to previous research. This section can be useful to understand all of the ins and outs of the results of the experiment and how best to think about them.
- Conclusion – This short section is where the authors firmly state if the hypothesis was supported or not, the shortcomings of the methodology, and what possible research in the future might help to further elucidate the topic.
- Acknowledgments – This is where the researchers will declare any conflicts of interest or other direct associations with the experiments.
- References – This section at the end of the paper is a list of the other research used to inform the experiment. It may mention other published papers, chapters in reference books, or other well-supported literature.
When Science Magazine asked several scientists in different stages of their careers how they read scientific literature, they all said they read articles out of order. In addition, most of them said they had a question in mind before reading the article to guide what they wanted. If you’re a non-expert like most of us, here’s a guide for what to look at.
- Abstract first, to understand what was being studied
- Skim the Results next to check the major graphs and figures to understand how well the hypothesis was supported or not supported.
- Introduction to get a background on how the results will matter.
- Conclusion to get a deeper understanding of the findings.
- Discussion for a clear and verbose interpretation of the findings.
- Detailed readings of Results and Methods & materials will usually come last, or not at all.
- Check References throughout, and look up any words or acronyms you do not understand.
What to watch out for
When journalists cite articles, they will usually use language to indicate how well the results can be relied upon. If the journalist writes “A study out of Harvard found…” then they are only indicating a single study discovered those results, and it might be worth reading the study over before relying on how the journalist describes it. If they don’t link to the study, use a search engine to find the study by searching for the author’s names and the institution. If nothing comes up, then it’s best to remain skeptical about the information.
It would make our lives very easy if we could just read a few lines of the abstract of any paper and trust its findings. Unfortunately, that is not the case and we have to be careful when using a paper to validate a hypothesis in the real world.
- Corroboration. This is the most simple check for a well-supported hypothesis. If more than one peer-reviewed paper supports it then it’s safe to assume the hypothesis is well-supported.
- Citations. If a paper is cited more than 10 times by other papers, it’s in the top 24% of papers worldwide and has clearly been widely accepted. The websites of most journals will list the number of times a paper has been cited.
- Publishing date. This is more dependent on the field of research, but the world changes, and later research is usually better research. For example, since industrialization has changed so much over the past few decades, a paper studying air pollution in the 1970s is not going to be as reliable fifty years later when pollution levels have changed. However, a paper studying the properties of aluminum will be reliable almost forever because aluminum will always be the same.
- Sample size. The more times an experiment is able to observe the hypothesis in action, the better the research. If the experiment did not look at a representative sample of a population, it probably is not reliable. There are very complex statistical methods to figure out the best sample size for any given experiment, but as a rule, sample sizes less than 100 should be interpreted with extreme caution. Sample sizes of 100 to 500 are more reliable, but keep an eye out for mitigating factors, which should be mentioned in the discussion or conclusion. Sample sizes over 500 are fairly safe to rely on, but each experiment is different, so be sure to read closely if you’re skeptical. If the sample size is over 5,000 then it’s almost reliable.
Different types of scientific literature
The most common and often-referenced type of scientific literature are articles published in scientific journals. These papers typically examine a single experiment and seek to add its findings to the scientific canon. There are thousands of science journals out there, but the best ones have a peer-review process and are cited by other journals frequently. The article itself most commonly takes the form of a description of original research and the author is typically the researcher that performed the experiment.
Some other forms of articles published in journals might be review papers, letters to the editor, case reports, or other commentaries. While articles describing novel experiments are useful, review papers can be the most valuable for laypeople. Review papers seek to summarize all of the research on a given topic and often perform a meta-analysis on the validity of the hypotheses that make up a body of knowledge about a subject. Regardless of the type of article, as long as they are published in a peer-reviewed journal, we can assume that an expert in the field has read the article very closely and vouched for its authenticity.
Published articles are not the only source of scientific knowledge. Here is a shortlist of other locations where hypotheses might be supported:
Patent applications. When applying for a patent extensive evidence of the product’s functionality must be supplied. As a result, most patents fit the same high standards of evidence required for scientific literature.
Textbooks. Books or individual chapters of books that are written by a researcher or team of researchers on a specific topic are often cited in scientific literature.
Presentations at conferences. Data presented by researchers at conferences are a way to validate a hypothesis.
Preprints. These are papers that appear to have merit but have yet to be peer reviewed. As a result, they may have misleading or unsupported claims but make for attractive headlines.
We are big fans of scientific literature here at Molekule and try to cite it in our blog as much as possible, from our article on the science of PECO and VOCs to our article on the dangers of PM2.5. So be sure to keep an eye on our blog for more articles that will point you to scientific literature that matters.
