Levels of Evidence

The Levels of Evidence framework ranks study designs to guide clinical decision-making, with Level I representing high-quality randomized controlled trials (adequate randomization, blinding, power, and low attrition) or systematic reviews thereof—importantly, study design does not automatically determine evidence level, as a poorly conducted RCT may be downgraded to Level II or III. The hierarchy descends through Level II (lesser RCTs, prospective cohorts), Level III (case-control, retrospective cohorts), to Level IV-V (case series, expert opinion). The GRADE system introduces crucial nuance by separating evidence quality (confidence in effect estimate) from recommendation strength (should we act), acknowledging that strong recommendations can arise from lower-quality evidence when effects are large and harms minimal. Evidence can be downgraded by "RIIIP" factors (Risk of bias, Inconsistency, Indirectness, Imprecision, Publication bias) or upgraded by large effect sizes, dose-response relationships, and residual confounding favoring the null hypothesis.

Understanding Levels of Evidence

Levels of evidence provide a hierarchical framework for evaluating the quality of research studies. This system helps clinicians appraise the strength of evidence supporting clinical decisions.

Key Principles:

• Higher evidence levels indicate greater confidence in study findings
• Study design alone does not determine evidence level - quality matters
• Different question types have different evidence hierarchies
• Context determines appropriate evidence level for clinical decisions

Core Concepts in Evidence Appraisal

The Evidence Pyramid:

• Top: Systematic reviews and meta-analyses
• High: Randomized controlled trials (RCTs)
• Medium: Cohort and case-control studies
• Low: Case series, case reports, expert opinion

Why Study Design Matters:

• Randomization controls for known and unknown confounders
• Blinding prevents performance and detection bias
• Control groups allow comparison of intervention effects
• Prospective design avoids recall and selection bias

GRADE Framework:

• Separates evidence quality (confidence) from recommendation strength
• RCTs start as high quality, observational studies as low
• Quality can be upgraded or downgraded based on specific criteria

What is GRADE?

GRADE (Grading of Recommendations Assessment, Development and Evaluation) is the most widely used system for assessing evidence quality and recommendation strength.

Two Key Outputs:

1. Quality of Evidence: High / Moderate / Low / Very Low
2. Strength of Recommendation: Strong / Weak (for or against)

Assessing Evidence Quality

Start with Study Design, then apply modifiers:

Example: RCT with high risk of bias (-1) and wide confidence intervals (-1) = Moderate quality evidence.

Example: Cohort study with very large effect (+2) = Moderate quality evidence (upgraded from Low).

Understanding GRADE is essential for guideline development and evidence interpretation.

Australian Epidemiology and Practice

Evidence-Based Practice in Australian Orthopaedics:

• AOANJRR (Australian Orthopaedic Association National Joint Replacement Registry) provides world-leading Level II evidence on implant survival and outcomes
• Registry data is cited internationally as high-quality observational evidence for arthroplasty decisions
• The Whitehouse Report methodology underpins registry analysis and has influenced international registry standards

RACS Orthopaedic Training Relevance:

• Levels of evidence and GRADE methodology are core FRACS examination topics in research methodology and evidence-based practice
• Viva scenarios commonly test ability to critique study designs and assign evidence levels
• Key exam focus: differentiating study designs, identifying bias, applying GRADE downgrade factors (RIIIP)
• Examiners expect candidates to interpret evidence levels when discussing treatment recommendations

Australian Orthopaedic Research:

• Australian orthopaedic journals (ANZ Journal of Surgery, JBJS Open Access) require authors to assign evidence levels to studies
• NHMRC (National Health and Medical Research Council) evidence hierarchy aligns with Oxford CEBM levels
• Australian Clinical Practice Guidelines use GRADE methodology for recommendation development

Key Australian Databases and Resources:

• AOANJRR: Primary source for arthroplasty evidence in Australian practice
• Cochrane Musculoskeletal Group: Australian-based systematic review group contributing to global evidence synthesis
• NHMRC Guidelines Portal: Evidence-based guidelines for Australian clinical practice

Application to Clinical Practice:

• Therapeutic Goods Administration (TGA) requires evidence level assessment for device and drug approval
• Medicare Benefits Schedule (MBS) Review Taskforce considered evidence levels when evaluating orthopaedic procedures
• Private health fund prostheses list decisions incorporate evidence from AOANJRR and systematic reviews