Liveblogging the FDA hearing on the Moderna Covid 19 vaccine
So Far the efficacy data has been presented. As reported in the press earlier, the vaccine is roughly 95% effective, that is roughly 95% of people who got Covid 19 during the trial were participants who received the placebo.
Importantly, the null hypothesis that just one dose is just as good as two was not rejected. The test of this null had extremely low power as almost all participants received both doses, so basically this means cases less than 4 weeks after the first dose (so one week after the second dose). However, note the extreme rigidity of the FDA.
Before allowing vaccination, the FDA required proof of efficacy. Before allowing a modification from two doses
4 3 weeks apart to one dose, the FDA requires … I don’t know maybe if Jesus Christ returned and petitioned them for some flexibility, they would give Him a hearing, but I guess they would tell him he needed to propose (and fund) a new Phase III trial.
update: incorrect assertion of fact crossed out
It is also true that there is no evidence of benefit from the second dose of Pfizer’s vaccine. It is clear that people who have received one dose of either vaccine are among those least at risk of Covid 19.
See the raw data below from Polack et all 2020 . Can anyone see from the Kaplan Meier plot when the second dose was given ?
The vaccines are in very short supply. People are anxiously waiting for vaccination. Because the protocol had two doses, half of the vaccine will be reserved for the people who will benefit least.
Here there is a difference between careful science and optimal policy. In science it is crucial to write the protocol first then follow it mechanically. This is necessary so that the experimental interventions are exogenous and one can be sure they cause the observed outcomes and are not caused by observations.
However, it is not optimal policy to reduce the possible decisions to two, a priori with extremely limited data. This is what the FDA does. I think they should approve a single dose. Their rule is always to only act on extremely firm knowledge. It is, in this case, not going to be first do no harm. The second dose has side effects (mild but not zero). There is, I think,
no weak evidence of benefits. (Again, the test has extremely low power (and I’m not sure protocol did not say the question would be addressed — if it didn’t then there is a problem — the rule decide what to do in advance applies to data analysis too — it is vital that the data not be dredged looking for a significant coefficient)). I think the point estimate is pretty much exactly zero benefit. of a benefit of the second dose much lower than of the first (and without proof of any benefit.
I think that people should be given a single dose. After everyone who wants one dose has been vaccinated, then it makes sense to give people a second dose. There is no reason to think spacing
4 3 weeks apart is optimal — the spacing was decided in advance (and it was 4 weeks for the Moderna vaccine hence my mistake).
Next speaker discussed safety. There is 0 evidence that vaccination increases the risk of anaphalactic shock. There were two cases one person who suffered anaphalaxis received placebo and one received the vaccine. The most common side effect was pain. There were no cases of severe side effects. People with a history of anaphalaxis were *not* excluded from the study.
Now a third speaker argues for unblinding the study and giving the vaccine to participants who were given the placebo. They can drop out and just get the vaccine when it is their turn. Losing the control group is not ideal but attrition will make it useless soon anyway (people will not settle for 50% chance they were vaccinated when the vaccine is approved — probably tomorrow). I agree, they have enough data and it is not ethical to leave people unvaccinated just as a control group.
Now they open for discussion with a few members of the public allowed to ask questions (the law requires this). I muted. Now they have taken a pause.
My question is why not give people just one dose until everyone who wants it has been vaccinated once ? I see no basis at all for allocating the scarce vaccine to a second dose. The scientific method does not say that optimal policy requires sticking to a protocol written before data were collected. The first do no harm principle (which I absolutely oppose in general) would imply giving one dose until there is statistically significant evidence of benefit of a second dose.
Consider the case of tests for Covid 19. The test kits sent out by the CDC contained powder in tubes. One tube was the positive control — it was supposed to contained DNA with sequences corresponding to the Sars Cov2 RNA genome sequences. The tubes which were supposed to contain one of 3 oligonucleotides to be used. was contaminated with traces of that DNA. The result was that the kit as shipped reported that distilled water was infected with Sars Cov2. The hospital labs which got the kits almost immediately figured out that they could test with valid results if they didn’t use the material in the contaminated tubes, and just used 2 oligonucleotides. They could determine who had Covid 19 using the kit. But that was a modified protocol which was not FDA approved, so the FDA did not allow them to do this. The FDA also did not approve dozens of tests which were developed by the private sector.
Here the FDA’s decision that they would rather be safe than sorry kept the US blind to Covid for … I think maybe a couple of weeks. Don’t look, because you haven’t proven that your glasses have exactly the right prescription is not good advice to someone on a highway. This was a very bad problem. I think the lesson learned is not that even the CDC lab sometimes makes mistakes. It was that rigidity and refusing permission is not the way to safety.
Since then, I have been very favorably impressed by the FDA’s efforts. But today I want more — I mean less — I mean approving less and allowing more flexibility. I see no case for insisting on giving people second doses with almost
exactly zero limited evidence of efficacy (update: and a very small point estimate of benefit). I see no case for reserving vaccine for the people who are least at risk of Covid 19. Yet I see no chance that a single dosage will be allowed.
I may update when they reconvene.
In previous posts, I object to the confusion of the pure food and drug act with the scientific method. I note that it is simply a mistake to assert that the null hypothesis is to be treated as true until it is rejected by the data. The law says drugs are assumed ineffective until they are proved effective. That is US law not the scientific method. In general the decision of which of 2 hypotheses to treat as the null is arbitrary and should have no implications. I am not a scientist, but I am familiar with the Neyman Pearson framework and I consider my claims about the meaning of “null hypothesis” to be as solid as my assessment of 2+2. Both are simple math
I’m in the Moderna trial. It was silly to call this blinded. Anyone who looked at the phase 1/2 data knew they would have the typical reaction–bone and muscle ache, fever, headache–after the booster. I did. A month later, I got myself tested for spike antibody and had a robust positive.
They took a blood draw before the booster (8 vaccutainer tubes), so the study can determine what kind of immune reaction occurs with a single injection.
The US has purchase enough Pfizer vaccine for 50 million people and enough Moderna vaccine for 100 million. We have purchased 100 million units each of the AstraZenica and J&J vaccines when they get approved, which will probably be in early 2021. At that point, there will be enough to vaccinate 350 million Americans. Vaccine *will not* be limiting. The problem will be tens of millions of Americans who refuse the vaccine. Maybe instead of beating up on the FDA, you should put the blame where it really belongs–with the Trump administration and the right-wing wind machine that has been attacking science.
Did your evidence that you had the actual vaccine change your behavior?
When the third vaccine is approved, doses for 350 million will be approved and ordered. However they will not yet “be” as you assert. Your claim is incorrect. The doses will be made over months. The production started before approval. Approval if a third vaccine (especially if it is tge single dose JJ vaccine) will speed things up.
Actual experts predict that vaccination of the general public will start late March or April not early 2021 (as I use the vague phrase). The difference between March 31 and March 1 matters for the tens of millions who will die as a result of an additional month waiting for ghe vaccine (thus will include un-necessary deaths of people who will, in any case refuse to take thd vaccine because giving low effect second doses not high effect first fises will delay the achievement of herd immunity).
the current plan is to be cautious methodical and careful and accept hundreds of thousands of pointless deaths worldwide. the reason us tgst policy must ve obe of 2 options in a protocol written before almost all available data were collected, becsuse modifying strategy when new data becone available is inconsistent with the scientific method.
I trust my sarcasm was blatant enough. In fact science does not rule out using the data. it is a strange hybrid of sound rules for valid exleriments, the lure foid and drug act, and rigidity of regulators whose true useful role and self image ate based on resusting pressure.
But it is not science or goid policy and it will kill people.
Not really. While there’s good evidence that the vaccine will protect me from disease, there isn’t clear evidence whether it will prevent me from infecting others. Presently, I’m vaccinated and my wife isn’t.
so far I have heard that the virus has gone through at least 6 mutations. Perhaps one of the mutants will spread faster but not be as lethal, perhaps give the rest of us some immunity to all the mutations. Hope for the best! Hope that Congress will save some of the tax money for later when the going gets tougher!
The problem with flexibility is that it smears petroleum jelly on the lens. There is so much noise in the system as it is that allowing even minor changes in the protocol can vitiate the entire test. Every change in protocol means less statistical power, more confounding factors and a weaker signal. Look at the Astra Zeneca vaccine testing bollix and decide if you want two full doses, a half dose then a full dose, or perhaps a vaccine which was properly tested.
“The law says drugs are assumed ineffective until they are proved effective.”
That is also how the scientific method works. Just because you want a drug or vaccine to work doesn’t mean it works. If you want a rocket that can put a satellite into orbit, you don’t do a statistical test by launching a series of rockets and arguing that the hypothesis that the rocket can launch a satellite into orbit was not ruled out.
Word press seems to be eating my comments
If the efficacy is only 52 percent with one dose (as I find reported) and if recipients start relaxing social distancing (contrary of Joel), then giving people one dose could have some nasty unintended consequences.
@Arne Citation needed. If you found that, tell us where.
your assertion’ “in early 2021. At that point, there will be enough to vaccinate 350 million” is not correct. In fact there will not be enough to vaccinate 350 million people. The vaccine doses will have been ordered, but they will not yet actually exist in the physical world.
The amount purchased is not the amount available. The US has ordered doses and paid for them. The companies have promised to produce the doses. The companies have not yet produced the doses. They are producing at the rate of 40 million doses a month which means vaccinations for 20 or 40 million people a month. Not 350 million. The AstraZeneca and J&J vaccines will help (especially because the J&J vaccine is being tested without a second dose).
Actual experts predict that vaccine will be available to the general public in late March or early April. The difference between March 31 and, say, March 1 is roughly 100,000 deaths in the USA alone. Many will be people who would refuse to get the vaccine anyway, because a delay in vaccination is a delay in herd immunity.
The link https://talkingpointsmemo.com/muckraker/vaccine-officials-contradict-azar-on-when-it-will-be-available-to-wider-public
@Arne again. The number is in Polack et al 2020 and is in the figure I just added to my post. It reports effectiveness from the instant the first dose is injected. The graph shows a dramatic change in the rate of infection after 11 days with 35 cases in the first 11 days after the first dose and 4 in days 12 through 21.
Choosing the day 11 ex post is cheating. I would have chosen 10 before looking at the figure. So that is 34 cases in days 1-10 and 5 in days 11-21. That is a statistically significant difference
(p < 0.5^39 (1 +39 + (39*38)/2 + (39*38*37)/6 + (39*38*37*36)/24 +(39*38*37*36*35)/120) )< 0.0000012 ).
The point estimate for days 10-21 is way more than 80% effective (the slope is less than the slope of the line from day 1 to end of study which is reported to imply 80% efficacy).
I can’t use the figure to calculate the exact p value of a test of the null of no benefit of the second dose against the alternative of further benefit, because there is right censoring. The trial ended 119 days after the first vaccination but less than 119 days after the last participant got his or her first dose. The steps are 1/population at risk and you can see they get bigger.
All I can do with the published data is eyeball the curve. notably the slope doesn’t change visibly from day 10 on. Not at day 21 (second dose) and not at day 28 (7 days after second dose). New cases get rarer but there are fewer people at risk. Similarly there are fewer new cases per day among people who got the placebo, but the slope of the curve doesn’t change.
I googled “efficacy of vaccine with one dose”, to get the two significant figure number because I remembered seeing it before. I had no idea whether the referenced study meant what the headline said, so I think it is just as well for you to have found it and commented as you did.
You misread my comment. Here’s the part about vaccine availability that you critiqued. Read it again:
“The US has purchase enough Pfizer vaccine for 50 million people and enough Moderna vaccine for 100 million. We have purchased 100 million units each of the AstraZenica and J&J vaccines when they get approved, which will probably be in early 2021. At that point, there will be enough to vaccinate 350 million Americans. Vaccine *will not* be limiting.”
Nowhere do I say that everyone will be vaccinated in early 2021. I agree that vaccines won’t be available to the general public until spring at the earliest, and nothing I posted here contracts that assessment.
What I said is that availability of units will not be rate-limiting in getting everyone vaccinated. Supply chain and vaccine refusal will be the big challenges in reaching herd immunity.
What is more efficient with limited quantities, using those for very vulnerable folks or directing the resources right at the populations that seem most involved in propagation? If you think about “Great Barrington” the critique was not that it wouldn’t work, but that the time needed was long and lots would suffer along the way. Push it hard with relatively young and active because they provide much better leverage against the virus. Bet their young immune systems would do better than average on one dose, too. Taking the 88 year-old nursing home resident out of the contagion system is good for that person but maybe not overall best use for the early weeks.
With luck we will be compiling data during the vaccination process that will tell us who and when got shots and infection. Tests right before 2nd dose would be a handy indicator. Then after healthcare workers and the most vulnerable, particularly long term care residents, have been made safe there will be data to support giving the younger, stronger, and less proximate people single doses. The application logistics are more challenging the further from the healthcare system that we get. Herds of unprotected people lining up for shots sounds like a Trump plan since the vaccine likely does not protect from day one.
Problem not being taken into consideration much is the mental health degradation due to familial separation for healthcare workers and the institutionalized. Donald Trump was an endogenous disorder.
The head of Operation Warp Speed apologizes for shortfalls in vaccine deliveries to at least 14 states.
Gen. Gustave F. Perna, who heads Operation Warp Speed, the Trump administration’s multiagency effort to get coronavirus vaccines out to Americans, apologized repeatedly on Saturday morning for confusion over vaccine deliveries to states.
He attributed some of the problems to the federal government’s miscalculation of how many doses of Pfizer-BioNTech’s vaccine could be shipped. The discrepancies disrupted vaccination plans and stirred consternation in at least 14 states.
General Perna is in charge of the logistics for distributing the coronavirus vaccines to the states, and he took full and sole responsibility for the delays and confusion around the vaccine rollout, and for the discrepancies between the number of doses states were expecting and what they are receiving.
“It was my fault,” he said. “It was a planning error, and I am responsible.”
“I want to take personal responsibility for the miscommunication,” General Perna said at a news conference. He said the number of vaccines available to allocate ended up being lower than initial forecasts.
“I had to lower the allocations to meet the releasable doses that were presented to me,” General Perna said. “So to the governors,” he said, “please accept my personal apology if this was disruptive in your decision-making.” …
LONDON — Alarmed by a fast-spreading variant of the coronavirus, Prime Minister Boris Johnson abruptly reversed course on Saturday and imposed a wholesale lockdown on London and most of England’s southeast, banning Christmas-season gatherings beyond individual households. …
The variant is up to 70 percent more transmissible than earlier versions, officials said, though some scientists are skeptical. …
Viral mutations are not uncommon, and British officials said this variant had been detected in a handful of other countries, without naming them. But the government’s medical experts expressed alarm about its apparent infectiousness, noting that it now accounts for more than 60 percent of the new infections reported in London. …
@kaleberg I agree that in an experiment, the protocol must be followed so the experimenter’s actions are exogenous to the entity under study.
That does not mean that a policy should never be modified given new information. Think of the sort of policy I discuss more often — macroeconomic policy. The argument that there should not be a changed without a new experiment would imply never changing macro policy.
I think the argument for withholding vaccine to make sure all who are vaccinated once are then given a booster makes as little sense as sticking to a macro policy rule chosen 90 years ago (you know the rule).
Ion my post, I stressed that I agree that for a valid experiment, one must stick to the protocol. I did not object to the conduct of the trials. However, I was discussing the fact that, given the data collected in those trials, it is almost certain that current suboptimal policy will cause hundreds of thousands of deaths.
The 21st Century Cures Act allows the FDA to accelerate the approval of drugs also. Innovation in New Drug Approvals of 2019 Advances Patient Care Across a Broad Range of Diseases