Release Date: October 9, 2017
Category: Scientific Writing
Author: Katherine S., Ph.D., E.L.S.
In Part I, we discussed how to define your aim for a systematic review. In Part II, we defined your inclusion and exclusion criteria. In Part III, we conducted your search. Now that you have a list of articles, you need to screen them to determine if they are relevant to your aim based on your inclusion criteria. First, I’d suggest finding a review buddy that you can include as a co-author on your review. The best systematic reviews have two people independently screening the articles and comparing notes afterward to ensure that there is no bias when deciding what studies meet the inclusion criteria. Using multiple independent reviewers also helps improve reproducibility.
How about Anjali, the other postdoc in my lab? She’s got some time between experiments.
That’s perfect. Now we can move onto screening.
Does this mean we must read the 383 papers we found during our search?
Definitely not. Screening is a three-phase process. During the first pass, just look at titles. Look for papers that are obviously not relevant to your review. For example, based on your aim, “to evaluate if treatment P improves disease-free survival and the ability to perform tasks R and S compared with placebo in patients with condition X,” you would automatically remove papers that conducted their studies in mice, didn’t study treatment P alone, or used treatment P as a preventative measure. You will also want to eliminate any duplicates. Because you searched multiple databases, the same article can show up multiple times. At this point you and Anjali will compare notes. Ideally, your lists should match. If they don’t, talk about why you want to keep or get rid of a study and decide together if you should include it in the next round.
Once you have a matching list of titles, move on to phase two, reading the abstracts. During this phase, you are determining if the study meets your inclusion criteria. Look at the experimental details and the outcomes reported. If the study matches your inclusion criteria or doesn’t give enough detail for you to determine conclusively whether it meets your criteria, keep the paper. If it doesn’t meet your inclusion criteria, eliminate it. Once you’ve made it through the abstracts, meet with Anjali again to compare lists and decide which abstracts to keep. At this point, you should have a more reasonable number of papers.
Phase three involves reading the full text. Double check the papers to ensure they meet ALL your inclusion criteria. This is also a convenient time to start extracting data from your papers. After that, meet with Anjali to determine the final list of papers to include.
At this point, some people go through the references of the included papers to see if they missed any relevant studies, but this step isn’t strictly required. No matter what, you need to keep track of how many articles you had at each phase, why you included or excluded them, and any information relevant to the selection process. This information is critical when making your PRISMA flowchart and can be helpful if a reviewer takes issue with excluding or including a specific paper.
What if I have too many or too few papers at the end of screening?
Then you will need to decide if you want to modify the inclusion criteria. If there are too few studies, you may want make the inclusion criteria less stringent. For example, you may want to include prospective case-control studies along with randomized controlled trials (RCTs). If you have too many papers, you may need stricter inclusion criteria, such as adding a requirement for double-blind RCTs to improve the strength of your evidence. If you change your inclusion criteria at this stage, disclose this in your review so people can reproduce your study.
This seems like a lot of information to coordinate and keep track of.
It is, which is why software is your friend. Many programs are available to help you coordinate and write your systematic review. They range from simple Excel templates to fancy software explicitly designed for systematic reviews and meta-analyses. A comprehensive list of available software with their pros and cons is available on the Health Librarian Wiki. Select the software that best meets your needs and fits within your budget.
Alright, I’ve got 15 papers in my final list. What now?
Now, you will want to make a table of data from your included papers. It is best to extract too much information rather than too little at this point. Obviously, you will want to record the outcomes of the study, but also pull all relevant experimental details. A table with lots of details allows you to quickly identify similarities and differences between studies. For example, you wouldn’t want to directly compare outcomes from studies that administered a daily dose of treatment P with those that administered a weekly dose. A detailed table will provide the strong foundation you need for later analysis. If you need a little help, some of the available software can build a customized table that allows you to input the data as you read through the full text and export it in an analysis-friendly format.
Now that you have selected your articles and extracted your data, it’s time to start screening and analyzing them. We’ll get into that in our next article, “Writing a Systematic Review Part V: Determining Bias.”
Our articles are based on the material from our scientific writing workshops, which cover these and many other topics more thoroughly, with more examples and discussion.
We offer on-site workshops for your event or organization, and also host workshops that individual participants can attend. Our on-site scientific writing workshops can range from 1-2 hours to several days in length. We can tailor the length to suit your needs, and we can deliver a writing workshop as a stand-alone activity or as part of scheduled meetings.
Our scientific writing workshops consistently receive high praise from participants including graduate students, post-docs, and faculty in diverse fields. Please see our scientific writing workshop page for details.
If you found this article helpful or if there is a topic you want us to address in a future article, please use our online comment submission form, or contact us directly. Your comments and suggestions are valuable! Click here to return to our scientific editing article library.