Release Date: December 4, 2017
Category: Scientific Writing
Author: Katherine S., Ph.D., E.L.S.
In Part I, we discussed how to define your aim for a systematic review. In Part II, we defined your inclusion and exclusion criteria. In Part III, we conducted your search. In Part IV, we screened your articles and extracted the data. In Part V, we determined the level of bias in your selected articles. The next step is to analyze your data, but how do you know which studies you can compare?
Ummm… Match up the ones that look the same?
Close, but not exactly. A lot of subtle things can make your outcomes vary, such as the location where the study was conducted or the time of day the drug was administered, but these things may not be reported. We call that heterogeneity. You can calculate the I2, which is a statistical measure of the heterogeneity between studies, using the chi-squared value and degrees of freedom. The bigger the I2, the more heterogeneity. If there are one or two big outliers, you may want to investigate them further or exclude them from analysis. If the studies are generally heterogeneous, note it in your review and mention it as a limitation in the Discussion. If you were hoping to do a meta-analysis with your data, I suggest talking to a statistician who can help you determine what analyses are appropriate.
Now can we compare the studies?
Yes, have at it.
How exactly do I do that?
That is entirely up to you, but the nature of the data influences your approach. If you have a lot of studies that are mostly identical experimentally and homogenous, you can briefly describe them all and present their results directly. If you have a mix and match of different studies, look for ways to classify them into subgroups. This is where the detailed data table comes in handy. You can scan the table for similar experimental details or outcome measures that would make good subgroups. Examples of experimental subgroups would be to group studies based on administration method or based on the type of tests used to measure the ability of patients to perform a task if different methods were used. An example of outcome subgroups would be to group studies based on which outcomes they measured if some of the studies didn’t measure all three outcomes. Pull out a few logical subgroups, keeping in mind that they need to relate to your study aim and that you will need to discuss why these subgroups are relevant or important in the Discussion section. You will still want to briefly describe the studies and results, but you will be grouping them under separate subheadings. Remember that one paper can be included in multiple subgroups if appropriate.
You will also want to make tables for publication. It is unlikely that you’ll be able to publish your detailed table as is. You will need to make a smaller table containing only the data you are presenting. If there is too much information for one table, you can split it into an experimental details table and an outcome table. Alternatively, you may want to make several subgroup-specific tables. Choose the table format that clearly conveys your data in an easily digestible manner.
For the Discussion, look at your data to see if you can draw any conclusions. If you have subgroups, compare studies within the subgroups. Are the results similar? If so, why is that? What implications does that have for patients with condition X? Use the literature to support your statements. If the results conflict, why do you think that is? Could differences between studies explain the contradicting results? Does one study have stronger or less-biased data than the other? Could someone do specific studies to resolve the conflict? Again, use the literature to support your statements. No matter what, make sure your conclusions are appropriate for the strength of your data. You will probably want to make one broad conclusion at the end of your review. Keep in mind that “there is not enough high-quality data available draw a conclusion” is a valid conclusion. Suggesting specific studies that could fill the knowledge gap is also helpful.
Anything else I should know?
The information presented here is a barebones outline of the systematic review process. If you decide to write a systematic review, I suggest digging deeper before starting. A wealth of resources is available for someone who wants to write a systematic review. Specifically, the Cochrane Collaboration Handbook for Systematic Reviews of Interventions, the Institute of Medicine Standards for Systematic Reviews, and the PRISMA website are excellent and comprehensive resources. Spending a little extra time with these resources will save you a lot of pain later.
In addition, remember that the best systematic reviews are a group effort. Individuals, such as research librarians and statisticians, may specialize in developing and analyzing systematic reviews and meta-analyses. If you find yourself in over your head, it is okay to ask for help. It is worth adding an author to your paper to preserve your sanity.
Overall, systematic reviews are a lot of work, but the answers they provide are invaluable to science. Hopefully this guide will help you break the process into manageable chunks. Good luck in your review writing endeavors.
Our articles are based on the material from our scientific writing workshops, which cover these and many other topics more thoroughly, with more examples and discussion.
We offer on-site workshops for your event or organization, and also host workshops that individual participants can attend. Our on-site scientific writing workshops can range from 1-2 hours to several days in length. We can tailor the length to suit your needs, and we can deliver a writing workshop as a stand-alone activity or as part of scheduled meetings.
Our scientific writing workshops consistently receive high praise from participants including graduate students, post-docs, and faculty in diverse fields. Please see our scientific writing workshop page for details.
If you found this article helpful or if there is a topic you want us to address in a future article, please use our online comment submission form, or contact us directly. Your comments and suggestions are valuable! Click here to return to our scientific editing article library.