TYPE I AND TYPE II ERRORS
False Positives | False Negatives | Error Rates
Error Types by Truth
Critical Must-Knows
- Type I Error (Alpha): Concluding there IS an effect when there is NOT (false positive). Set before study, usually 0.05.
- Type II Error (Beta): Concluding there is NO effect when there IS (false negative). Related to power: Power = 1 minus Beta.
- Trade-off: Reducing alpha (e.g., 0.01) reduces Type I error but increases Type II error risk unless sample size increases.
- Clinical Consequences: Type I leads to adopting ineffective treatments; Type II leads to discarding effective treatments.
- Multiple Comparisons: Testing many hypotheses inflates Type I error (family-wise error) - need correction (Bonferroni).
Examiner's Pearls
- "Alpha is set BEFORE study (usually 0.05), p-value is calculated AFTER from data
- "Underpowered studies have high Type II error risk - may miss real treatment effects
- "Type I error is considered worse in many contexts - adopting harmful treatment worse than missing beneficial one
- "Multiple testing without correction can inflate Type I error above 0.05
Clinical Imaging
Imaging Gallery
Critical Error Concepts
Type I Error (False Positive)
Definition: Rejecting null hypothesis when null is actually true. Example: Concluding new treatment is better when it actually is not. Alpha = 0.05 accepts 5% risk.
Type II Error (False Negative)
Definition: Failing to reject null when alternative is true. Example: Concluding treatments are equivalent when new treatment is actually better. Beta = 0.20 (power 80%) accepts 20% risk.
Alpha-Beta Trade-off
Relationship: Reducing alpha (stricter threshold) increases beta (Type II error risk) unless sample size increases. Cannot minimize both errors simultaneously with fixed sample.
Clinical Consequences
Type I Consequence: Adopt ineffective or harmful treatment. Type II Consequence: Discard effective treatment. Which is worse depends on context - severity of disease, treatment risks.
CRWAType I vs Type II Errors
Memory Hook:The Boy Who Cried Wolf - Type I is crying wolf falsely (false positive), Type II is missing the real wolf (false negative)!
PAWSError Consequences and Prevention
Memory Hook:Use your PAWS to prevent errors - proper planning prevents poor performance!
Overview/Introduction
What is Type I Error?
Definition: Rejecting the null hypothesis when the null hypothesis is actually true.
Common Name: False Positive
Example: Concluding a new surgical technique is superior when it actually has no benefit.
Consequences:
- Adopt ineffective or harmful treatment
- Waste resources implementing change
- Potential harm to patients
- False confidence in intervention
Alpha Level Selection
Alpha Thresholds and Implications
| Alpha | Type I Error Risk | When Used | Trade-off |
|---|---|---|---|
| 0.01 | 1% false positive rate | When Type I error is very costly (e.g., drug approval) | Requires larger sample or accepts higher Type II error |
| 0.05 | 5% false positive rate | Conventional in most research | Balance between Type I and Type II errors |
| 0.10 | 10% false positive rate | Exploratory or pilot studies | Easier to find significance but higher false positive risk |
Key Point: Alpha is set BEFORE the study. The p-value is calculated AFTER from the data. If p less than alpha, reject null.
Principles of Error Testing
Core Principles
The Error Trade-Off:
- Decreasing Type I error (lower alpha) increases Type II error risk
- Decreasing Type II error (higher power) increases sample size needed
- Cannot minimize both simultaneously without increasing sample size
Control Strategies:
- Type I (Alpha): Pre-specify alpha, use appropriate corrections for multiple testing
- Type II (Beta): Adequate sample size, appropriate effect size assumptions
Clinical Decision Framework: When is each error more serious?
- Type I more serious: Invasive treatment, irreversible decision, expensive intervention
- Type II more serious: Missing life-saving treatment, rare disease with few options
Understanding these principles guides appropriate study design.
Understanding Type II Error (Beta)
What is Type II Error?
Definition: Failing to reject the null hypothesis when the alternative hypothesis is actually true.
Common Name: False Negative
Example: Concluding two treatments are equivalent when one is actually superior.
Consequences:
- Discard effective treatment
- Delay progress in patient care
- Wasted research effort (failed trial)
- Miss therapeutic opportunity
Relationship to Power: Power = 1 minus Beta
Beta and Power
Beta and Power Relationship
| Beta | Power | Interpretation | Sample Size |
|---|---|---|---|
| 0.05 | 95% | Very high power - 95% chance detecting real effect | Very large sample needed |
| 0.10 | 90% | High power - 90% chance detecting real effect | Large sample needed |
| 0.20 | 80% | Adequate power - 80% chance detecting real effect | Moderate sample, conventional target |
| 0.50 | 50% | Underpowered - coin flip chance of detection | Small sample, high Type II error risk |
Understanding Type II error is critical for interpreting negative study results.
Error Matrix and Decision Framework
The 2x2 Truth Table
Statistical Decision vs Reality Matrix
Type I and Type II Errors
| Your Decision | Null is TRUE | Alternative is TRUE |
|---|---|---|
| Reject Null (p less than alpha) | TYPE I ERROR (False Positive) - Alpha = 0.05 | CORRECT DECISION (True Positive) - Power |
| Accept Null (p greater than alpha) | CORRECT DECISION (True Negative) - 1 minus Alpha | TYPE II ERROR (False Negative) - Beta = 0.20 |
Key Insight: We never know which column we are in (true state of nature is unknown). We set alpha and beta to control error rates.
Multiple Comparisons and Type I Error Inflation
The Multiple Testing Problem
Problem: Testing multiple hypotheses inflates overall Type I error rate.
Example: Testing 20 different outcomes at alpha = 0.05 each.
- Expected false positives: 20 × 0.05 = 1 false positive on average
- Family-wise error rate (FWER): Probability of at least one Type I error increases with each test
Formula for FWER: 1 minus (1 minus alpha)^n
- For 20 tests at alpha = 0.05: FWER = 1 minus 0.95^20 = 0.64 (64% chance of at least one false positive)
Bonferroni Correction
Method: Divide alpha by number of tests to maintain overall Type I error.
Formula: Adjusted alpha = 0.05 / n
Example: Testing 5 outcomes → Adjusted alpha = 0.05 / 5 = 0.01
- Use p less than 0.01 as threshold for each test to maintain overall Type I error at 0.05
Trade-off: Conservative - may increase Type II error (reduce power).
When to Correct for Multiple Comparisons
Correct: When testing multiple related hypotheses (e.g., multiple outcome measures in same trial).
May NOT need correction: Pre-specified primary outcome vs secondary/exploratory outcomes. Only primary outcome requires alpha = 0.05.
Understanding multiple comparisons prevents inflated Type I error rates.
Clinical Application
Which Error is Worse?
Context-Dependent: Type I (false positive) often considered worse - adopting harmful treatment. But Type II (false negative) can be worse if missing life-saving treatment. Balance depends on disease severity and treatment risk.
Screening Tests
Type I in Screening: False positive → unnecessary workup, anxiety. Type II: False negative → missed diagnosis, delayed treatment. Serious diseases (cancer) prioritize minimizing Type II (high sensitivity).
Underpowered Studies
High Beta Risk: Many orthopaedic trials underpowered (power under 80%, beta greater than 0.20). Negative results may be Type II errors. Always check power before accepting negative result.
Meta-Analysis Solution
Combining Studies: Meta-analysis increases power by pooling data from multiple studies. Reduces Type II error risk, provides more precise effect estimate.
Evidence Base
Type I Error and Multiple Comparisons
- Routine Bonferroni correction may be too conservative
- Should adjust for multiple comparisons when many unrelated tests performed
- Pre-specified primary outcome does not require correction
- Secondary outcomes should be interpreted with caution or corrected
- Recommendations: Define primary outcome, limit secondary outcomes, interpret cautiously
Power and Type II Error in Orthopaedic Trials
- 60% of orthopaedic RCTs did not report power calculation
- Of studies reporting power, 40% had power below 80% (beta greater than 0.20)
- Underpowered studies risk Type II error - may incorrectly conclude no difference
- Negative results from underpowered studies are inconclusive, not definitive
Balancing Type I and Type II Errors
- Traditional alpha = 0.05, beta = 0.20 is arbitrary convention
- Should consider consequences of each error type in context
- Serious diseases with safe treatments may justify higher alpha (0.10) to reduce beta
- Less serious diseases with risky treatments may justify lower alpha (0.01)
- Bayesian approaches allow explicit consideration of error consequences
Exam Viva Scenarios
Practice these scenarios to excel in your viva examination
Scenario 1: Error Type Identification
"A study concludes that a new fixation technique reduces nonunion rates compared to standard technique (p = 0.03). However, the new technique actually has the same nonunion rate as standard. What type of error has occurred?"
Scenario 2: Multiple Comparisons
"You are reviewing an RCT that tested 10 different outcome measures. One outcome showed p = 0.04. How do you interpret this result?"
MCQ Practice Points
Type I Error Definition
Q: What is a Type I error? A: Rejecting null hypothesis when null is actually true (false positive). Concluding there IS a difference when there is NOT. Probability is alpha (usually 0.05 or 5%).
Type II Error Definition
Q: What is a Type II error? A: Failing to reject null hypothesis when alternative is true (false negative). Concluding there is NO difference when there IS. Probability is beta (usually 0.20 or 20% for power = 80%).
Multiple Comparisons
Q: Why does testing multiple outcomes increase Type I error risk? A: Each test has 5% chance of false positive. Testing 20 outcomes means expecting 20 × 0.05 = 1 false positive on average. Family-wise error rate (probability of at least one false positive) increases with each additional test. Bonferroni correction divides alpha by number of tests to control overall Type I error.
Management Algorithm
TYPE I AND TYPE II ERRORS
High-Yield Exam Summary
Error Definitions
- •Type I = False Positive = Reject null when null is true = Alpha
- •Type II = False Negative = Accept null when alternative is true = Beta
- •Power = 1 minus Beta = Probability of correctly rejecting false null
- •Alpha set BEFORE study (usually 0.05), p-value calculated AFTER from data
- •If p less than alpha, reject null (risk Type I if null actually true)
Error Consequences
- •Type I consequence = Adopt ineffective or harmful treatment
- •Type II consequence = Discard effective treatment, miss opportunity
- •Type I often considered worse (false adoption) but context-dependent
- •Screening: Type II worse for serious diseases (miss cancer)
- •Treatment: Type I worse for risky interventions (adopt harmful therapy)
Error Control
- •Reduce Type I = Lower alpha (0.01 instead of 0.05) OR increase sample
- •Reduce Type II = Increase power (0.90 instead of 0.80) OR increase sample
- •Trade-off: Lowering alpha increases beta unless sample increases
- •Conventional: Alpha = 0.05 (5% Type I), Beta = 0.20 (20% Type II, 80% power)
- •Large sample reduces both errors
Multiple Comparisons
- •Testing n outcomes inflates Type I error (family-wise error rate)
- •FWER = 1 minus (1 minus alpha)^n
- •20 tests at alpha 0.05: FWER = 64% (not 5%)
- •Bonferroni correction: Adjusted alpha = 0.05 / n
- •Primary outcome: No correction. Secondary outcomes: Correct or interpret cautiously
Clinical Application
- •Underpowered studies have high Type II error risk (beta greater than 0.20)
- •Negative result from underpowered study = Inconclusive, NOT definitive
- •Pre-specify primary outcome to avoid multiple comparison issues
- •Meta-analysis reduces Type II error by pooling studies (increases power)
- •Always check power when interpreting negative results