I read the Florida Department of Health's Flawed Vaccination Study so you don't have to
The analysis is clearly flawed, but don't expect a correction
Update: This is a terrible study, but I misread it. I’m leaving this post up as evidence of my error, but please see here for a correction:
The Florida Department of Health released “guidance” today indicating that an “analysis” found that young men shouldn’t receive mRNA COVID-19 vaccines. You will not be surprised to hear that the analysis is completely incorrect, but I think the way in which it’s wrong is fascinating: the authors set up their analysis with a subtle flaw that miscounts “lives saved by vaccines” as “people killed by vaccines”.
(By “flawed” I of course mean that the study doesn’t show what they claim it does and should be retracted with an apology. I have no idea whether the authors were aware of these problems.)
Some context: the study is trying to estimate the effects of COVID-19 vaccination on death from heart diseases like myocarditis. There is a strong scientific consensus that COVID-19 vaccines slightly increase the risks of myocarditis, and that COVID-19 infections vastly increase these risks.
In order to estimate this effect, the authors compare the risk of cardiac death in the 28 days following vaccination to the risk of cardiac death in the “control period” from 4 - 25 weeks following vaccination. If more people experience cardiac death just after getting the vaccine than months after getting the vaccine, the authors take this as evidence that the vaccine has, overall, increased the number of people dying of heart-related diseases.
This method is called the “self-controlled case series” method, and is widely used to examine vaccine side effects in other contexts. But there’s an important assumption you’re supposed to check in order to use this method:
Assumption: the vaccine does not affect the risk of cardiac death in the control period.
In many contexts this is a reasonable assumption: most vaccine side effects happen hours or days after receiving the vaccine, so it’s fair to assume most issues aren’t surfacing a full months later. But in the context of COVID vaccinations, we have to deal with the fact that cardiac death isn’t just a potential vaccine side effect: it’s also an extremely common symptom of serious COVID-19 infections, which the vaccine prevents throughout the control period.
More importantly, the COVID vaccines are better at preventing this kind of death during the control period than during first the month after vaccination. Here’s a plot from Lin et al. (2022) demonstrating the effectiveness of different vaccines over time. Notice that after two weeks the vaccine has only reached half of its full effectiveness, and the mRNA vaccines take nearly two months to peak!
So when the Florida analysis compares “the month following vaccination” to “the months after that”, it’s comparing people whose vaccinations aren’t fully protective yet to people whose vaccinations are. (The first month also includes a few people who may have caught COVID before their vaccination but died after.)
When they find a higher risk in the first month, the study’s authors want you to believe this is all due to vaccine side effects. But they have no way of distinguishing “vaccine side effects” from “people dying of cardiac arrest after getting COVID because the vaccine hasn’t fully taken effect yet,” so their study counts lives the vaccine saves when it takes full effect as lives lost due to the vaccine, which is essentially as wrong as you can be in a measurement like this.
If we try to (very roughly) put numbers to this, we discover that the entire “effect” measured in the Florida study is likely due to vaccines preventing COVID, and essentially none of it is due to vaccine side effects. To see this, note that COVID increases your chance of cardiac death by about fifty percent. The graphs above indicate that vaccines are about half as effective across the first month of vaccination compared to after, which I’ll round down to 30% because “full effectiveness” doesn’t mean “literally nobody ever gets COVID.” This would imply that in a world with no vaccine side effects we’d expect to see a 15% bump in cardiac deaths in the first month post-vaccination compared to the months after, which is the entire effect the Florida study uncovers.
Is there an effect of vaccines on myocarditis? Probably a small increase in the short term. That increase is almost certainly reversed in the long term, since vaccines reduce your chances of severe COVID, and COVID causes a much larger increase in your chances of myocarditis than vaccines do.
In short, the Florida DOH’s analysis is simply wrong, and you should put no trust in it. Florida should retract that analysis, apologize, and reverse the “guidance” they’re pretending is based on the study. COVID vaccines continue to be safe, even for young men. Get vaccinated, and please ask your state governments to stop killing their constituents for political gain.