Understanding the difference between trial results and real-world performance
These two terms are often confused but measure different things. Understanding the distinction is essential for interpreting vaccine data correctly.
Efficacy measures how well a vaccine works under ideal, controlled conditions — in a randomized clinical trial. Effectiveness measures how well it works in the real world, where conditions are messier and populations are more diverse.
Both numbers are valuable. Efficacy tells us the biological effect of the vaccine. Effectiveness tells us the practical public health impact.
Efficacy is measured in randomized controlled trials (RCTs) — the gold standard of clinical research. In a vaccine trial:
The formula:
Vaccine Efficacy = (Risk in unvaccinated - Risk in vaccinated) / Risk in unvaccinated
For example, if 1% of unvaccinated people get a disease and 0.1% of vaccinated people get it, the efficacy is (1% - 0.1%) / 1% = 90%. This means the vaccine reduces the risk of disease by 90%.
Effectiveness measures how well a vaccine works in routine, real-world conditions. Unlike trials, observational studies that look at health outcomes as they happen in the general population.
Trials often exclude elderly, immunocompromised, or those with comorbidities — the very people who may respond less strongly to vaccines.
Real-world vaccination may not follow the exact schedule, storage, or dosing as in trials.
Effectiveness is measured over longer time periods, during which protection may decline.
New variants may emerge that the vaccine wasn't designed to target.
People in trials may be more likely to seek care, maintain health behaviors, or use other interventions.
Real-world studies may measure different outcomes (hospitalization vs. any infection).
The distinction between efficacy and effectiveness became highly visible during the COVID-19 pandemic. Here's how it played out:
Phase 3 Trial Results (Efficacy): The original mRNA COVID-19 vaccines showed ~95% efficacy against symptomatic infection in trials conducted in 2020.
Real-World Results (Effectiveness): As the pandemic evolved, effectiveness dropped — first to ~80-90% with Delta variant, then to 40-60% with Omicron — not because the vaccines changed, but because:
However, critically, effectiveness against severe disease, hospitalization, and death remained much higher than effectiveness against mild infection — often 70-90% even when infection protection dropped significantly. This is because the immune system develops multiple layers of protection.
• CDC. "Vaccine Effectiveness Studies." Centers for Disease Control and Prevention.
• Halloran ME, Longini IM Jr, Struchiner CJ. Design and Analysis of Vaccine Studies. Springer; 2012.
• WHO. "Vaccine Efficacy, Effectiveness and Protection." World Health Organization, 2021.