Be prepared to see a bunch of new estimates of effects in the days and weeks to come—and some of these could make the vaccines sound less useful at preventing Covid than did first reports. That’s not a sign that the first analyses are unreliable. Calculations of 95 percent efficacy were based on data from only those participants who took both of two vaccine injections, with some extra time allowed for the fullest possible immunity to kick in. That means a bunch of people were left out: There may be some who got Covid before that point was reached; some who dropped out of the study before getting their second injection; and some who stopped providing the researchers with follow-up, so it won’t be clear whether they ever did get sick. We’ll soon have analyses that take these people into account, and they will show somewhat different levels of efficacy. These may offer some insight into how well the vaccines will perform in practice, when rolled out in our communities.
That said, the new data may introduce some new problems. Watch out, in particular, for results of analyses that may not have been high-level outcomes planned out by the researchers ahead of time. These are known as “subgroup” or “exploratory” analyses, and they can be more like setups for questions to study in the future than solid answers on their own. We saw a case like this already with some of the Oxford-AstraZeneca results. The efficacy for that vaccine’s standard dosing was 62 percent, according to the press release. But a smaller group of people mistakenly got the wrong dosing, and when their data was separated from the others’, the efficacy was 90 percent. Initial media coverage ran with this finding as if it were a more meaningful, primary result. AstraZeneca has since acknowledged that they “need to do an additional trial” to test this new hypothesis.
Misleading Claim 3:
The vaccine was well tolerated, and side effects were mostly mild or moderate.
This is a classic example of what I call “placebo information”: It looks like the real thing, but it tells you nothing. Adverse reactions to vaccines that are designed for widespread distribution are always “mostly mild and moderate.” The vaccine may cause almost no severe effects, or it could cause a lot of them; but either way, mild and moderate reactions will be more numerous.
For months, we’ve been getting this type of placebo statement for all the Covid-19 vaccines. But that doesn’t mean the experience of taking them will be the same. For example, a press release about the BioNTech-Pfizer vaccine reported that the rates of most severe adverse events were below 2 percent, with only fatigue and headache showing up a bit more often. In contrast, Moderna’s press release reported rates rates higher than 2 percent for about half a dozen severe symptoms, including two—fatigue and myalgia (muscle pain and weakness)—that occurred in up to 9 or 10 percent of those who got the vaccine.
Even so, The New York Times reported only that participants in trials of both vaccines had no serious effects, but “reported sore arms, fatigue, fever, and joint and muscle aches that last for a day or two.” It’s true that neither study release included evidence of “serious” or “grade 4” side effects, which are defined by the FDA’s grading scale as being those that can be life-threatening, such as a fever over 104.0°F or anything that requires hospitalization, such as a stroke or an acute breathing problem. But the vague New York Times write-up, and many others like it, make no attempt to get across that “severe” or “grade 3” adverse events did occur, and that they were apparently more frequent for Moderna’s vaccine than for BNT-Pfizer’s. These adverse events aren’t life-threatening, but they matter. According to the FDA, a severe one prevents daily activities and requires medical help. It could leave you with a fever between 102.1 and 104.0°F, for example, in need of narcotic pain relievers or on an intravenous drip. In fact, among those who received the Moderna vaccine, severe adverse events could end up being described as “common,” according to the FDA’s threshold definition of that term. Watch out for specific data on these in the coming weeks, and also keep an eye out for what’s revealed about the “moderate” or “grade 2” adverse events, which can be hard-going, too. A full, transparent accounting of these details will only improve public trust in the vaccines.
More From WIRED on Covid-19