Medicine

Have you ever seen a news piece about a scientific breakthrough and wondered how accurate the reporting is? Or wondered about the research behind the headlines? This is the beginning of critical appraisal: thinking critically about what you see and hear, and asking questions to determine how much of a 'breakthrough' something really is.

The article "Is this study legit? 5 questions to ask when reading news stories of medical research" is a succinct introduction to the sorts of questions you should ask in these situations, but there's more than that when it comes to critical appraisal. Read on to learn more about this practical and crucial aspect of evidence-based practice.

What is Critical Appraisal?

Critical appraisal forms part of the process of evidence-based practice.
“Evidence-based practice across the health professions” outlines the fives steps of this process. Critical appraisal is step three:

1. Ask a question
2. Access the information
3. Appraise the articles found
4. Apply the information
5. Audit (1)

Critical appraisal is the examination of evidence to determine applicability to clinical practice. It considers (1) :

• Internal validity (trustworthiness)
  • Are the results of the study believable?
  • Was the study methodologically sound?
    
  • What is the clinical importance of the study’s results?
  • Are the findings sufficiently important? That is, are they practice-changing?
    
  • Are the results of the study applicable to your patient?
  • Is your patient comparable to the population in the study?

  Why Critically Appraise?

  If practitioners hope to ‘stand on the shoulders of giants’, practicing in a manner that is responsive to the discoveries of the research community, then it makes sense for the responsible, critically thinking practitioner to consider the reliability, influence, and relevance of the evidence presented to them.

  While critical thinking is valuable, it is also important to avoid treading too much into cynicism; in the words of Hoffman et al. (1):

  … keep in mind that no research is perfect and that it is important not to be overly critical of research articles. An article just needs to be good enough to assist you to make a clinical decision.

  How do I Critically Appraise?

  Evidence-based practice is intended to be practical. To enable this, critical appraisal checklists have been developed to guide practitioners through the process in an efficient yet comprehensive manner.

  Critical appraisal checklists guide the reader through the appraisal process by prompting the reader to ask certain questions of the paper they are appraising. There are many different critical appraisal checklists but the best apply certain questions based on what type of study the paper is describing. This allows for a more nuanced and appropriate appraisal. Wherever possible, choose the appraisal tool that best fits the study you are appraising.

  Like many things in life, repetition builds confidence and the more you apply critical appraisal tools (like checklists) to the literature the more the process will become second nature for you and the more effective you will be.

  How do I Identify Study Types?

  Identifying the study type described in the paper is sometimes a harder job than it should be. Helpful papers spell out the study type in the title or abstract, but not all papers are helpful in this way. As such, the critical appraiser may need to do a little work to identify what type of study they are about to critique. Again, experience builds confidence but having an understanding of the typical features of common study types certainly helps.

  To assist with this, the Library has produced a guide to study designs in health research.

  The following selected references will help also with understanding study types but there are also other resources in the Library’s collection and freely available online:

  • The “How to read a paper” article series from The BMJ is a well-known source for establishing an understanding of the features of different study types; this series was subsequently adapted into a book (“How to read a paper: the basics of evidence-based medicine”) which offers more depth and currency than that found in the articles. (2)
    
  • Chapter two of “Evidence-based practice across the health professions” briefly outlines some study types and their application; subsequent chapters go into more detail about different study types depending on what type of question they are exploring (intervention, diagnosis, prognosis, qualitative) along with systematic reviews.
    
  • “Clinical evidence made easy” contains several chapters on different study designs and also includes critical appraisal tools. (3)
    
  • “Translational research and clinical practice: basic tools for medical decision making and self-learning” unpacks the components of a paper, explaining their purpose along with key features of different study designs. (4)
    
  • The BMJ website contains the contents of the fourth edition of the book “Epidemiology for the uninitiated”. This eBook contains chapters exploring ecological studies, longitudinal studies, case-control and cross-sectional studies, and experimental studies.

  Reporting Guidelines

  In order to encourage consistency and quality, authors of reports on research should follow reporting guidelines when writing their papers. The EQUATOR Network is a good source of reporting guidelines for the main study types.

  While these guidelines aren't critical appraisal tools as such, they can assist by prompting you to consider whether the reporting of the research is missing important elements.

  Once you've identified the study type at hand, visit EQUATOR to find the associated reporting guidelines and ask yourself: does this paper meet the guideline for its study type?

  Which Checklist Should I Use?

  Determining which checklist to use ultimately comes down to finding an appraisal tool that:

  • Fits best with the study you are appraising
  • Is reliable, well-known or otherwise validated
  • You understand and are comfortable using

  Below are some sources of critical appraisal tools. These have been selected as they are known to be widely accepted, easily applicable, and relevant to appraisal of a typical journal article. You may find another tool that you prefer, which is acceptable as long as it is defensible:

  • CASP (Critical Appraisal Skills Programme)
  • JBI (Joanna Briggs Institute)
  • CEBM (Centre for Evidence-Based Medicine)
  • SIGN (Scottish Intercollegiate Guidelines Network)
  • STROBE (Strengthing the Reporting of Observational Studies in Epidemiology)
  • BMJ Best Practice

  The information on this page has been compiled by the Medical Librarian. Please contact the Library's Health Team (libhealth@deakin.edu.au) for further assistance.

  Reference list

  1. Hoffmann T, Bennett S, Del Mar C. Evidence-based practice across the health professions. 2nd ed. Chatswood, N.S.W., Australia: Elsevier Churchill Livingston; 2013.

  2. Greenhalgh T. How to read a paper : the basics of evidence-based medicine. 5th ed. Chichester, West Sussex: Wiley; 2014.

  3. Harris M, Jackson D, Taylor G. Clinical evidence made easy. Oxfordshire, England: Scion Publishing; 2014.

  4. Aronoff SC. Translational research and clinical practice: basic tools for medical decision making and self-learning. New York: Oxford University Press; 2011.