10 things we wish we’d known earlier about research: Tips from Journalist's Resource

Here at Journalist’s Resource, we love research. Early in our careers, however, we as individual journalists didn’t always appreciate the value of research or interpret it correctly. We did not always use the best study to make a point or fact-check a claim. Learn from our mistakes. Here are some things we wish we knew years ago.

1. Academic research is one of the best reporting tools around.

• Reading studies early in the reporting process will give you a good general understanding of an issue. It also will help you ask better questions and understand the answers that sources give you.
• Use it to hold public officials accountable. Oftentimes, policymakers try new things because they assume a certain change will prompt a certain result. (For example, mandating uniforms in public schools to improve student achievement.) A review of the research often will help you gauge whether such a change will or could provide the result a policymaker wants. Research also will tell you what has and has not worked in other locations and under similar circumstances.
• Individual studies often offer ideas for other angles journalists might want to pursue.

2. General Google searches are not the best way to find good research.

• A better source is scholar.google.com, which lets you search for research published in peer-reviewed journals. Other good resources are PubMed, Microsoft Academic, PLOS and the National Bureau of Economic Research.
• Search academic journals for studies. But don’t limit yourself to journals that focus on your specific topic area. For example, if you are writing about criminal justice, you can find quality research in Criminology as well as in Social Problems, PNAS (Proceedings of the National Academy of Sciences) and The Quarterly Journal of Economics.
• At the end of each academic study, authors list the research that is cited within the study. These lists are great resources for additional research on a particular topic.

3. Researchers generally are accessible and like to talk about their work.

• We have found that researchers respond more quickly to email than phone calls. They also may share free copies of their work or tell you how to access them for free.
• If you are confused by a data analysis and don’t have a strong background in statistics or research methods, reach out to someone who does. Many scholars are eager to help journalists describe their research findings correctly.

4. When something is described as “significant,” that doesn’t necessarily mean it is important or noteworthy.

• Scholars often refer to their findings as being “significant” or “statistically significant” to indicate that a relationship they have discovered or a difference they have observed is not likely to be the result of chance or a sampling error. Determining whether something is “significant” or “statistically significant” is based on a mathematical analysis, not an opinion.

5. Correlation does not imply causation.

• Often, studies determine there is a relationship between two or more variables. Just because one variable changes in the presence of a second variable does not mean the second variable caused the change. Never make assumptions about what a study says or does not say. If in doubt, contact the author.

6. Don’t spend much time on the abstract.

• Many people think of the abstract as a summary of the most compelling findings. Oftentimes, this is not the case. The two best places to find information about key findings are 1) the “results” section, which typically is located in the middle of a research article and is where authors explain what they have learned and provide their statistical analyses and 2) the “discussion” or “conclusions” section, which is usually located at the end of the paper and offers a summary of findings as well as a discussion of the real-world implications of the author’s work.

7. Use caution when relying on research from think tanks, private consulting firms and special interest groups.

• The results of research from these organizations are not always independently reviewed prior to publication or distribution, whereas studies published in academic journals generally are.
• As a rule of thumb, journalists should avoid research funded or distributed by organizations with clear biases, including political affiliations.
• Sometimes, academic research does not exist on a certain topic, or there is little of it. Private consulting firms and other organizations often will try to fill that knowledge gap by doing their own research. While some of these organizations provide quality research, it’s important to give the information additional scrutiny.

8. The best research doesn’t always come from Harvard and Stanford.

• Scholars from prestigious universities such as Harvard and Stanford are considered among the best in their fields. But that doesn’t mean every research article and report they write is the best on a given topic. High-quality research comes from scholars in a variety of settings, including public universities and non-profit institutes.

9. The peer-review process does not guarantee quality research.

• Keep in mind that some peer-review processes are more rigorous than others and some academic journals are more selective than others. Publication in a top-tier journal is no guarantee that a piece of research is high quality. But it’s safe to assume research articles that are published in top-tier journals have been reviewed and given a stamp of approval by multiple top scholars.

10. Knowing statistics and research methods helps a lot.

• Many journalists shy away from math and science. But even a basic knowledge of statistics and research methods will help you understand the studies you’re reading and distinguish a good research study from a questionable one.