LibGuides: Psychology: Doing a Systematic Review

Connect with us on social media

Contact the Library

For help with a variety of enquiries you can contact the Library by:

libraryhelp@tees.ac.uk

01642 342100 (Monday - Friday 9am to 5pm)

Find us and check our opening hours

Feedback on our services

Urgent help and support for mental health

Systematic Reviews

Overview of Systematic Reviews

What is a Systematic Review?

Definitions of systematic reviews vary but generally it is agreed that a systematic review is a structured literature review that involves identifying all relevant primary research using a rigorous search strategy to answer a focused research question. The next step is to assess the quality of the research and provide a summary and synthesis of all relevant available research on the topic.

Systematic reviews often refer to research undertaken by review groups with specialised skills such as the Cochrane collaboration.

This video from Cochrane explains why systematic reviews are important and how they are done.

Undertaking a systematic review as an individual researcher/ postgraduate student cannot be as thorough and in depth as a Cochrane review, however it does need to follow a structured process with identifiable stages. Cooper et al. (2018) examined the approaches to systematic reviews recommended in nine different documents including the Cochrane Handbook, Systematic reviews: the CRD’s guidance for undertaking reviews in health care, the Joanna Briggs Institute Reviewers’ Manual and the Campbell Handbook. These sources consistently described eight key stages, indicating there was a consensus on the key stages of literature searching that are required for a systematic review (Cooper et al., 2018, p.3).

The Cochrane Collaboration (cited in Kings College London, 2024) describe the 8 stages as follows:

1. Defining the review question and developing criteria for including studies
2. Searching for studies
3. Selecting studies and collecting data
4. Assessing risk of bias in included studies
5. Analysing data and undertaking meta-analyses
6. Addressing reporting biases
7. Presenting results and “summary of findings” tables
8. Interpreting results and drawing conclusions

This guide focuses on the first two stages, with some guidance on the third.

This video from The Evidence Synthesis Academy at Brown University talks through the basic steps of conducting a systematic review.

Who is this guide for?

It is important to consider whether you are undertaking a full systematic review or if you are carrying out a literature review, which may be more limited in scope. This guide is aimed at postgraduate researchers, who are carrying out a full systematic review. However, students on other levels of study, such as a taught postgraduate course who need to search systematically for a literature review, may find parts of the guide helpful too.

For help with structured searching, which is not part of a systematic review, please refer to your subject LibGuide.

A Systematic Review is not the same as a literature review (even though it may be carried out systematically). Although both aim to provide an accurate summary of the available literature on a topic, a Systematic Review is expected to be more rigorous, transparent and replicable.

This video from Research Shorts discusses the distinction between a systematic literature review and a literature review in the context of a major research paper.

The table below is taken from Bettany-Saltikov (2010, p.49) and explains the similarities and differences of systematic and literature reviews. There is also a useful comparison of literature reviews and systematic reviews available from Durham University (2023).

	Systematic Review	Literature Review
Question	Focused on a single question.	Not necessarily focused on a single question, but may describe an overview.
Protocol	A peer review protocol or plan is included.	No protocol is included.
Background	Both provide summaries of the available literature on a topic.
Objectives	Clear objectives are identified.	Objectives may or may not be identified.
Inclusion and exclusion criteria	Criteria stated before the review is conducted.	Criteria not specified.
Search strategy	Comprehensive search conducted in a systematic way.	Strategy not explicitly stated
Process of selecting articles	Usually clear and explicit.	Not described in a literature review.
Process of evaluating articles	Comprehensive evaluation of study quality.	Evaluation of study quality may or may not be included.
Process of extracting relevant information	Usually clear and specific.	The process of extracting relevant information is not explicit and clear.
Results and data synthesis	Clear summaries of studies based on high quality evidence.	Summary based on studies where the quality of articles may not be specified. May also be influenced by the reviewer’s theories, needs and beliefs.
Discussion	Written by an expert or group of experts with a detailed and well grounded knowledge of the issues.

Cochrane Systematic Reviews

The Cochrane Database of Systematic Reviews (CDSR) is one of the databases listed within the Cochrane Library and is the leading resource for systematic reviews in health care.

Campbell Collaboration

The Campbell Collaboration is an international social science research network that produces systematic reviews. Subject coverage includes: Business and management; Crime and justice; Disability; Education; International development; Knowledge translation and implementation; Methods; Nutrition; and Social welfare.

Joanna Briggs Institute (JBI)

The Joanna Briggs Institute (JBI) is an independent, international, not-for-profit research and development organisation based in the Faculty of Health and Medical Sciences at the University of Adelaide.

The JBI Reviewer's Manual provides guidance on different types of systematic reviews and scoping reviews.

JBI Reviewer's Manual

Prospero

PROSPERO is an international database of prospectively registered systematic reviews where there is a health related outcome. Key features from the review protocol are recorded and maintained as a permanent record.

PROSPERO aims to provide a comprehensive listing of systematic reviews which are registered. This is to help avoid duplication and reduce the opportunity for reporting bias, enabling comparison of the completed review with what was planned in the protocol.

Systematic reviews may examine quantitative or qualitative evidence. In the past systematic reviews were predominantly medical and often with a narrowly defined focus. Increasingly systematic reviews are attempting to deal with much broader topics, including topics allied to medicine but also topics outside of medicine. It is becoming more common in certain disciplines to see two or more types of evidence included and appraised and this is often called a mixed-method systematic review.

Quantitative

If the focus of your systematic review is to find quantitative research, you may be looking for studies:

that use experimental or observational methods
that involve the use of numbers in the collection of data
where there is no involvement between the researcher and the participant
which usually have a lot of participants and therefore generalised findings can be made and applied in wider contexts

Examples of quantitative research are:

Randomised Controlled Trials (RCTs)
Cohort studies
Case Control studies
Correlation studies
Cross-sectional studies (including surveys/questionnaires))

If you are unsure of the type of research you are looking at, some key features of quantitative research are: random sampling, descriptive or inferential statistics, p-values, confidence intervals, risk ratios and odd ratios.

Qualitative

If the focus of your systematic review is to find qualitative research, you may be looking for studies:

which explore meaning and phenomena
that include data which is not numerical
where results are descriptive rather than statistical

Examples of qualitative research approaches are:

Grounded Theory
Phenomenology
Action Research
Ethnography

If you are unsure of the type of research you are looking at, some key techniques which are used in qualitative research are: interviews, focus groups, observation, open-ended questionnaires, surveys, case studies, discourse analysis and textual analysis .

Mixed Methods

It is becoming increasingly common to see two or more types of evidence included and appraised in a systematic review. If several data types (both quantitative and qualitative) are combined in the review it is known as a mixed method systematic review. An example of this could be bringing together findings of effectiveness (quantitative evidence) and patient, family, staff or others' experiences (qualitative evidence) to enhance the usefulness of the recommendations to decision makers (taken from Briggs, 2020).

Hierarchy of Evidence

Not all evidence is the same and therefore cannot be treated as equal. Systematic reviews always consider the quality of evidence in each research study before summarising the information in a review. Evidence based medicine requires the use of the best available evidence, in order to be able to make effective recommendations.

(Greenhalgh, 2019, p.42-43) lists the traditional hierarchy of evidence in the following order:

1. Systematic reviews and meta‐analyses

2. RCTs with definitive results

3. RCTs with non‐definitive results

4. Cohort studies

5. Case-control studies

6. Cross‐sectional surveys

Cardiff University Library Services (no date) have also created an evidence pyramid, which illustrates the hierarchy of evidence based on quality. At the top of the pyramid is the most reliable and current evidence and at the bottom is the evidence which is regarded as the lowest quality, which includes background information or expert opinion. Click on the image to see more.

There are also some alternative resources which offer more complex versions of the hierarchy, for example:

Oxford University's Centre for Evidence Based Medicine
Cochrane sets out the levels of evidence for healthcare interventions.
York University Centre for Reviews and Dissemination set out the hierarchy levels on pg 10-11

Using material on this page

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.

Psychology