BioScience Writers LLC

Follow us on Facebook Follow us on twitter Follow us on LinkedIn

  <     Client Comments    >   

  • "This is a great course for foreigners who do not use English as a mother tongue"

You have our 100% Satisfaction Guarantee!

Writing a Systematic Review Part V:
Determining Bias

Release Date: November 6, 2017
Category: Scientific Writing
Author: Katherine S., Ph.D., E.L.S.

In Part I, we discussed how to define your aim for a systematic review. In Part II, we defined your inclusion and exclusion criteria. In Part III, we conducted your search. In Part IV, we screened your articles and extracted the data.

I notice that there isn’t anything in the inclusion criteria about whether the study is good or not.

Excellent point, which is why you will need bias assessments. Bias assessments are important because they will help both you and your readers determine how reliable the included data are. Study bias is something that can occur in any study, including RCTs. The Cochrane Collaboration categorizes study bias into six categories (Chapter 8;

  1. Selection bias—Selection bias relates to differences between experimental groups at baseline. This can be counteracted with a good randomization scheme.
  2. Performance bias—Performance bias is a difference in care between study groups. For example, if you were looking at the effectiveness of a computer program, you would want your control group to spend the same amount of time on the computer as the study group.
  3. Detection bias—Detection bias is a difference in how you determine outcomes. This can be combated using blinded outcome assessors; however, this may not be possible depending on how the outcome is measured.
  4. Attrition bias—Attrition bias is a difference based on patient withdrawals, which can leave potentially biased holes in your outcome data. Some statistical methods are available to combat this, such as intention-to-treat analyses.
  5. Reporting bias—Reporting bias is a difference between reported and unreported outcomes. Reporting bias can be caused by many factors including:

    1. publication bias, where studies only report positive results,
    2. time-lag bias, where authors report results very quickly or slowly after completing the study,
    3. multiple or duplicate publication bias, where authors publish many papers on the same study data,
    4. location bias, when the paper is published in an easily accessible versus a less accessible journal,
    5. citation bias, when papers have different citation frequencies depending on the results,
    6. language bias, when a paper is published in a specific language depending on the results,
    7. outcome reporting bias, when papers only report positive outcomes. Most studies avoid outcome reporting bias by setting the primary and secondary outcomes before starting a study and reporting all outcomes, whether the differences were statistically significant or not.
    Reporting bias can usually be detected using a funnel plot. In a funnel plot, the intervention effect estimate, such as the odds ratio, is plotted against a measure of study size or precision in a scatter plot. In an ideal situation, the plot should look like an inverted funnel (Cochrane Collaboration Handbook Chapter 10.4.1) and be relatively symmetrical. If your funnel plot is asymmetrical, you can try a few other statistical analyses such as a Begg’s rank correlation test or an Egger’s linear regression. If you get to that point, it may be helpful to find a statistician who specializes in meta-analysis to help you out.
  6. Other bias—Other biases are any other source of between-group differences, usually relating to the study design.

The Cochrane Collaboration has developed a tool to determine the overall risk of bias. A study is judged as having “low risk of bias,” “high risk of bias,” or “unclear risk of bias” in each of the six categories. A description of judgement criteria and how to score them is provided in Table 8.5.d of the Cochrane Collaboration Handbook, and different methods for presenting bias data are provided in Section 8.6. Scoring bias is another co-author activity. You and your fellow postdoc Anjali will need to score the risk of bias separately and compare your scores to make sure they match. If they don’t, you should talk it out until you both agree on the scores.

Anjali and I talked it out and ran the numbers. Everything looks good here. Can we get to the part where we actually compare studies now?

Yes, you can finally compare your studies. We’ll get into that in our next article, “Writing a Systematic Review Part VI: Comparing Studies.”

Scientific Writing Workshops

Our articles are based on the material from our scientific writing workshops, which cover these and many other topics more thoroughly, with more examples and discussion.

We offer on-site workshops for your event or organization, and also host workshops that individual participants can attend. Our on-site scientific writing workshops can range from 1-2 hours to several days in length. We can tailor the length to suit your needs, and we can deliver a writing workshop as a stand-alone activity or as part of scheduled meetings.

Our scientific writing workshops consistently receive high praise from participants including graduate students, post-docs, and faculty in diverse fields. Please see our scientific writing workshop page for details.

If you found this article helpful or if there is a topic you want us to address in a future article, please use our online comment submission form, or contact us directly. Your comments and suggestions are valuable! Click here to return to our scientific editing article library.