Skip to Main Content

Critical Appraisal for Health Students

Appraisal of a Qualitative paper : Top tips

undefined

Introduction

Critical appraisal of a qualitative paper

This guide aimed at health students, provides basic level support for appraising qualitative research papers. It's designed for students who have already attended lectures on critical appraisal. One framework  for appraising qualitative research (based on 4 aspects of trustworthiness) is  provided and there is an opportunity to practise the technique on a sample article.

Critical Appraisal of a qualitative paper: practical example

How to use this practical example 

Using the framework, you can have a go at appraising a qualitative paper - we are going to look at the following article: 

Schellekens, M.P.J. et al (2016) 'A qualitative study on mindfulness-based stress reduction for breast cancer patients: how women experience participating with fellow patients', Support Care Cancer, 24(4), pp. 1813-1820.

Step 1.  Take a quick look at the article

Step 2.  Click on the Credibility tab above - there are questions to help you appraise the trustworthiness of the article, read the questions and look for the answers in the article. 

Step 3.   Click on each question and our answers will appear.

Step 4.   Repeat with the other aspects of Trustworthiness: Transferability, Dependability and Confirmability .

Questioning the credibility:

Explanation of authors’ role on p.1814-15. 3 authors are unaccounted for in the article – what was their role? Potential bias as authors may be in favour of MBSR (having come from Centre for Mindfulness). Positive that they weren’t involved in running MBSR courses. 1 person doing the analysis not involved in the focus groups (positive) (MS) –know this from comparison of Focus groups/Data analysis sections. Authors know research area well.
There were 5 focus groups and 3 interviews (see Abstract and Methods). Interviews were conducted for palliative patient and two patients who participated in mixed cancer group. Used constant comparative method for data analysis – a recognised method but no justification as to why this was used (p.1815). No triangulation as each participant was only seen once either in a focus group or interview. The limitations of focus groups were acknowledged. Also mentioned that the results would be strengthened by using different methods. (p.1818-9)
See p.1814-15. Each participant only attended one focus group so data was only collected at one time point. Focus groups 100-120 mins and took place 1 week after the course. Interviews 30-60 mins – either in person or by phone. Stated that focus groups encouraged open, non directive discussion. They mention one question used (bottom of p.1814) which suggests that questions were only asked once.
Participants were shown a summary, not the full transcript (p.1815). Direct quotes were used but it wasn’t clear from which participants so it might not be a variety.
Mixed.

Questioning the transferability

Everyone who participated in the study was invited to focus groups. Large sample but missing the dropouts. Participants recruited from 2 hospitals and via website. Weighted towards curative (37) v palliative (3) (see table 1, p.1816). Positive that included detail about participants but would have been even better if had linked this to quotes.
Setting for focus groups and some interviews was the Centre for Mindfulness; potential for bias? Some interviews by phone – unclear how many. For sample, see p.1816 – table 1 where demographics of participants given. Clear inclusion criteria for study population.
No results from the 7 participants who dropped out of the training. 3 patients declined participation. Some negative comments included eg on feeling disconnected (p.1817). Data saturations mentioned (p.1815 under data analysis). Limitations (p.1818) recognise that focus groups don’t encourage people with different views.
Can only be transferred to those who have completed the whole course. Doesn’t include enough mixed cancer perspective (2 participants – see p.1819) or palliative perspective (3 participants)..

Questioning the dependability :

Yes, you can follow member checking and the coding. Content of focus groups and interviews with partners not clear – know themes but not questions asked. No field diary but clear limitations section.
No research plan was mentioned but a lot of peer scrutiny in data analysis (including 1 independent person to prevent bias). Transcripts coded independently and then reached consensus with another researcher.
Mixed – good on analysis, less good on what happened in focus groups/interviews.

Questioning the confirmability :

Constant comparative method to develop grounded theory. Lot of detail. Referenced. (Data analysis, p.1815)
Cross checking of themes. Reached consensus (See Data analysis, p.1815) Not sure of role of 3 of the authors of the article.
No mention of diary. Some reflection in limitations section but this doesn’t include researcher assumptions.
Fairly clear steps, lack of reflection.

Questioning the overall trustworthiness :

Mixed? Strong on some aspects of dependability and confirmability with clear steps and member checking. More trustworthy for curative patients than palliative.

Further Information

See Useful resources for links, books and LibGuides to help with Critical appraisal.