Principles of systematic review

What are systematic reviews

Evidence pyramid

They systematically evaluate and summarise current knowledge.

A review of a clearly formulated question that uses systematic and identify select and critically appraise relevant research and to collect and analyse data from the studies that are included in the review. Statistical methods meta analyses may or may. It be used to summarise and analyse the results of the studies. 

Statistically pooling of study findings aka meta analysis 

Brings together all the high quality evidence on a research question. 

Highlight the gaps in the evidence, end confusion about the benefits and harms, dispel myths, yield new insights, reduce of any flaws or errors in a single study, tells us when enough research has been done to confidently reach a conclusion, can identify where publication bias is present.

Grey literature is academic term for unpublished data,literature, for army does a lot of work in research and they don’t publish it.

Conclusions can be misleading because you can’t control the data so if they messed up with data then your conclusions will be bad.

Most of the time the answer is that there is not enough good quality evidence..

Stages of systematic review:

Define your research question 

Write and agree a protocol (and publish it)

Conduct a systematic search of the literature 

Screen the literature for eligibility

Extract relevant data from the included studies

Assess the quality of included studies 

Synthesise the findings of the included studies 

Write up (and publish)

Systematic reviews journal and you can publish your protocol there.

Define your research question:

Topic ➡️ narrower ➡️ focus ➡️ research question


American civil war ➡️ battle of Gettysburg ➡️ residence ➡️ 

Search strategy:

Where to start

What databases will you use? Typically a minimum of 3. 

Will you look for grey literature and if so how? 

What search terms will you use and how will you combine them?

Will you use filters to exclude certain types of articles?

How will you manage your health results?

Tips for designing a search strategy:

Involve a librarian or someone with experience

Look at search strategies from other similar systematic reviews and borrow

Before committing to anything, test the search yourself from your proposed databases 

Maximise your search – triangulate:

Contact experts in the area

Reference lists of relevant articles

Google scholar citations of key articles – forward citation tracking aka click on cited by x number link on google.

A flow diagram should be in all systematic reviews

Inclusion/exclusion criteria

P.i.c.o model

Patient, population or problem 

Intervention, exposure, prognostic factor

Comparison or intervention (if appropriate)

Outcome you would like to measure or achieve

What type of question are you asking?

Type of study you want to find

Consider what types of study will you include,

Consider what types of participants 

What types of outcome measures will you include 

How many people will screen the titles and abstracts, and the full text 

Now you need to systematically collect all the relevant data:

General information e.g. Author, year, funder

Methods e.g. Study designs


This is typically a table

Next step:

Assess the quality of included studies 

Decide on a criteria for what a good quality study would look like

Rate the quality of included studies as low, medium,or high risk of bias -every study will have a limitation. Here you could do a sensitivity study.

Informs the confidence you state in the findings of the studies/systematic reviews 

In my apps is a Cochran review manager. Read the handbook


Data synthesis 

It has to be a result section taking into account the quality of the evidence.

This can be done in a number of ways:

Narrative synthesis 

Meta analysis qualitative synthesis 


Combines the results from 2 or more studies 

Estimates an average or common effect

Optional part of SR, not always possible 

Why to do one:

Quantity treatment effects and their uncertainty 

Increase power

Increase precision 

Explore differences between studies

Settle controversies from conflicting studies

Generate new hypotheses

Standard deviation is the variability 

In the forest plot diagram if anything crosses the line, there is no effect. I am 95% sure of…. there is no difference or there is difference.

The point estimate is minus…. so I am 95% confident that they successfully did what they wanted to do.

Managing the SR process:

Include someone with SR experience 

Tends to generate a lot of information and there is a lot to manage and record.

Need to think through carefully how this will be managed

Can do it all on paper but there is free software e.g. Review manager that you can use.

Consider how do we prioritise outcomes?

When do you reach the point of comparing apples and oranges? To what extent should data from studies be pooled together?

The Joanna Briggs institute healthcare and public health

University has a diploma with Cochran reviews.

Comments are closed.