Liveblogging the FDA hearing on the Moderna Covid 19 vaccine

So Far the efficacy data has been presented. As reported in the press earlier, the vaccine is roughly 95% effective, that is roughly 95% of people who got Covid 19 during the trial were participants who received the placebo.

Importantly, the null hypothesis that just one dose is just as good as two was not rejected. The test of this null had extremely low power as almost all participants received both doses, so basically this means cases less than 4 weeks after the first dose (so one week after the second dose). However, note the extreme rigidity of the FDA.

Before allowing vaccination, the FDA required proof of efficacy. Before allowing a modification from two doses 4 3 weeks apart to one dose, the FDA requires … I don’t know maybe if Jesus Christ returned and petitioned them for some flexibility, they would give Him a hearing, but I guess they would tell him he needed to propose (and fund) a new Phase III trial.

update: incorrect assertion of fact crossed out

It is also true that there is no evidence of benefit from the second dose of Pfizer’s vaccine. It is clear that people who have received one dose of either vaccine are among those least at risk of Covid 19.

See the raw data below from Polack et all 2020 . Can anyone see from the Kaplan Meier plot when the second dose was given ?

The vaccines are in very short supply. People are anxiously waiting for vaccination. Because the protocol had two doses, half of the vaccine will be reserved for the people who will benefit least.

Here there is a difference between careful science and optimal policy. In science it is crucial to write the protocol first then follow it mechanically. This is necessary so that the experimental interventions are exogenous and one can be sure they cause the observed outcomes and are not caused by observations.

However, it is not optimal policy to reduce the possible decisions to two, a priori with extremely limited data. This is what the FDA does. I think they should approve a single dose. Their rule is always to only act on extremely firm knowledge. It is, in this case, not going to be first do no harm. The second dose has side effects (mild but not zero). There is, I think, no weak evidence of benefits. (Again, the test has extremely low power (and I’m not sure protocol did not say the question would be addressed — if it didn’t then there is a problem — the rule decide what to do in advance applies to data analysis too — it is vital that the data not be dredged looking for a significant coefficient)). I think the point estimate is pretty much exactly zero benefit. of a benefit of the second dose much lower than of the first (and without proof of any benefit.

I think that people should be given a single dose. After everyone who wants one dose has been vaccinated, then it makes sense to give people a second dose. There is no reason to think spacing 4 3 weeks apart is optimal — the spacing was decided in advance (and it was 4 weeks for the Moderna vaccine hence my mistake).

Next speaker discussed safety. There is 0 evidence that vaccination increases the risk of anaphalactic shock. There were two cases one person who suffered anaphalaxis received placebo and one received the vaccine. The most common side effect was pain. There were no cases of severe side effects. People with a history of anaphalaxis were *not* excluded from the study.

Now a third speaker argues for unblinding the study and giving the vaccine to participants who were given the placebo. They can drop out and just get the vaccine when it is their turn. Losing the control group is not ideal but attrition will make it useless soon anyway (people will not settle for 50% chance they were vaccinated when the vaccine is approved — probably tomorrow). I agree, they have enough data and it is not ethical to leave people unvaccinated just as a control group.

Now they open for discussion with a few members of the public allowed to ask questions (the law requires this). I muted. Now they have taken a pause.

My question is why not give people just one dose until everyone who wants it has been vaccinated once ? I see no basis at all for allocating the scarce vaccine to a second dose. The scientific method does not say that optimal policy requires sticking to a protocol written before data were collected. The first do no harm principle (which I absolutely oppose in general) would imply giving one dose until there is statistically significant evidence of benefit of a second dose.

Consider the case of tests for Covid 19. The test kits sent out by the CDC contained powder in tubes. One tube was the positive control — it was supposed to contained DNA with sequences corresponding to the Sars Cov2 RNA genome sequences. The tubes which were supposed to contain one of 3 oligonucleotides to be used. was contaminated with traces of that DNA. The result was that the kit as shipped reported that distilled water was infected with Sars Cov2. The hospital labs which got the kits almost immediately figured out that they could test with valid results if they didn’t use the material in the contaminated tubes, and just used 2 oligonucleotides. They could determine who had Covid 19 using the kit. But that was a modified protocol which was not FDA approved, so the FDA did not allow them to do this. The FDA also did not approve dozens of tests which were developed by the private sector.

Here the FDA’s decision that they would rather be safe than sorry kept the US blind to Covid for … I think maybe a couple of weeks. Don’t look, because you haven’t proven that your glasses have exactly the right prescription is not good advice to someone on a highway. This was a very bad problem. I think the lesson learned is not that even the CDC lab sometimes makes mistakes. It was that rigidity and refusing permission is not the way to safety.

Since then, I have been very favorably impressed by the FDA’s efforts. But today I want more — I mean less — I mean approving less and allowing more flexibility. I see no case for insisting on giving people second doses with almost exactly zero limited evidence of efficacy (update: and a very small point estimate of benefit). I see no case for reserving vaccine for the people who are least at risk of Covid 19. Yet I see no chance that a single dosage will be allowed.

update ?: I may update when they reconvene.

Usual rant

In previous posts, I object to the confusion of the pure food and drug act with the scientific method. I note that it is simply a mistake to assert that the null hypothesis is to be treated as true until it is rejected by the data. The law says drugs are assumed ineffective until they are proved effective. That is US law not the scientific method. In general the decision of which of 2 hypotheses to treat as the null is arbitrary and should have no implications. I am not a scientist, but I am familiar with the Neyman Pearson framework and I consider my claims about the meaning of “null hypothesis” to be as solid as my assessment of 2+2. Both are simple math