Every news story about COVID-19 in the U.S. tends to include various numbers: How many people have been vaccinated (76,899,987 as of March 1), how many confirmed cases there have been (28,244,591 and counting), and what each COVID-19 vaccine’s effectiveness is — 95% for Pfizer, 94% for Moderna and 66% for Johnson & Johnson, which was authorized by the FDA for emergency use on Feb. 27.
But what do those efficacy rates actually mean? First of all, it’s important not to confuse efficacy rate with effectiveness, because they’re different.
“In short, efficacy is the performance of a treatment under ideal and controlled circumstances, and effectiveness is performance under real-world conditions,” said Zania Stamataki, a senior lecturer in viral immunology at the University of Birmingham in the U.K., writing for The Conversation.
Another way to think of it is that vaccine efficacy applies to a group of people who received the vaccination in a clinical trial, while effectiveness measures how well a vaccine works when administered to people in the community, outside of clinical trials.
Stamataki explained that “efficacy” is a term used during clinical trials to determine whether a vaccine is safe and if it works. To determine efficacy, researchers need to compare the vaccine to a “control” treatment — typically an irrelevant or known vaccine, or a similar preparation that shouldn’t work for the virus being tested.
According to Stamataki, the trials are often “double-blinded,” which means the participants don’t know which vaccine they received and the researchers don’t know which vaccine they administered until the study is complete.
So in a nutshell, COVID-19 vaccine efficacy represents a certain percentage of fewer cases of COVID-19. At this point, it’s important to understand what counts as a COVID-19 case, and to know that it can vary by vaccine. Both Pfizer and Moderna defined a case as having at least one symptom — even a mild one, like fever, cough, fatigue, headache or nausea — and a positive COVID-19 test. On the other hand, Johnson & Johnson defined a “case” as having a positive COVID-19 test plus at least one moderate symptom (for instance, shortness of breath, abnormal blood oxygen levels or abnormal respiratory rate) or at least two milder symptoms.
It’s difficult to draw direct comparisons between the three approved COVID-19 vaccines’ efficacy rates. The clinical trials took place in different geographic areas with different populations, and at slightly different times during the pandemic, when different variants of COVID-19 were circulating.
“There were more people who had the B117 [U.K. variant] or other types of variants during the time of the Johnson & Johnson trial than during the Moderna trial,” Brianne Barker, a virologist at Drew University in New Jersey, told Live Science.
Notably, none of the three vaccine trials looked for asymptomatic COVID-19, so their efficacy numbers represent protection from having symptoms, not protection from being infected.
For instance, the reported 95% efficacy of the Moderna vaccine means that the vaccine prevented COVID-19 symptoms for 95% of those who received the vaccine compared with the placebo. However, this doesn’t mean that 95% of people are protected from disease with the vaccine, or that 5% of vaccinated people get infected. This is a general misconception of vaccine protection, according to The Lancet.
In fact, the actual percentage of vaccinated people in the Pfizer (and Moderna) trials who got COVID-19 was only 0.04%.
“I think it’s important for people to understand that this is an extremely effective vaccine,” Barker said. “This is much more effective than you might think.”
As Stamataki pointed out, the monitoring of vaccines doesn’t stop after they’re approved for use, and researchers continue to gather data to study how well it works across all populations. And it’s still far too early to know how effective the COVID-19 vaccines are in preventing symptoms, severe illness or death, because it takes many years for reports to reach solid conclusions that take into account different ages and ethnic backgrounds of recipients, the duration of protection and the benefit/harm balance.
Plus, people who take part in vaccine clinical trials are usually closely monitored. They might complete daily symptoms monitoring diaries and report any side effects. The same steps aren’t taken in the community at large.
Remember, very few vaccines are 90% effective. The two-dose measles, mumps and rubella (MMR) vaccine is 97% effective against measles and 88% effective against mumps, according to the Centers for Disease Control and Prevention. On the other hand, the flu vaccine is around 40%-60% effective, depending on any particular year’s vaccine and flu strains, but it is still considered to be successful because it saves millions of lives every year.
Like the viruses that cause the flu, SARS-CoV-2 (the virus that leads to COVID-19) is prone to mutations, which can make vaccines less effective. So vaccine developers may update the COVID-19 shot every year to ensure the best possible match to the most prevalent strains.
If you can get your head around what COVID-19 vaccine effectiveness and efficacy rates mean, the only other thing you really need to know is that whether the vaccine you’re offered has 66% efficacy or 95% efficacy, those numbers are good. As Barker said, “we are incredibly lucky with how effective these vaccines have been.”