I am going to write more on the topic of Covid 19 vaccines. I will argue that it would be better to give second (booster) shots only when vaccine supplies are plentiful. I think that so long as the vaccine is in short supply, people should be given one dose. I criticise the current policy of withholding vaccine to make sure that everyone who gets a first dose gets the scheduled booster either 3 weeks later (for the Pfizer vaccine) or 4 weeks later (for the Moderna vaccine). I argue that more lives will be saved with the one dose until supplies are plentiful strategy (even if many people don’t show up for the booster when supplies are plentiful). I think that the current policy will lead to tens of thousands of un-necessary deaths in the US alone with worse consequences for countries further back in line for vaccine supplies.
The post will have two sections. One will be an attempt to analyse the published data, which are mostly Kaplan and Meier plots with numbers read off by eyeballing. I will conclude that the best (poor) estimate of the reduction of infections from the first dose is more than 18 times the reduction due to the second dose. This calculation requires the key unproven assumption that the effect of one shot is weaker than but as durable as the effect of two shots.
This would mean that efficient deployment is one dose for as many as possible as soon as possible and second doses when giving them doesn’t interfere with this. I will go on to bore people with some p-values. I think that the null that it is as efficient to give second doses on schedule is rejected at at the 5% level (given published data). I note that benefits probably include the (unproven) prevention of transmission as well as benefits for the vaccinated people. I also note that giving two to some and zero to others is unfair especially given the difficulty of deciding who gets vaccinated first and the necessarily partly arbitrary decisions. The second section will come back to arguing about public policy and ethics as I just did.
So Far the efficacy data has been presented. As reported in the press earlier, the vaccine is roughly 95% effective, that is roughly 95% of people who got Covid 19 during the trial were participants who received the placebo.
Importantly, the null hypothesis that just one dose is just as good as two was not rejected. The test of this null had extremely low power as almost all participants received both doses, so basically this means cases less than 4 weeks after the first dose (so one week after the second dose). However, note the extreme rigidity of the FDA.
Before allowing vaccination, the FDA required proof of efficacy. Before allowing a modification from two doses 4 3 weeks apart to one dose, the FDA requires … I don’t know maybe if Jesus Christ returned and petitioned them for some flexibility, they would give Him a hearing, but I guess they would tell him he needed to propose (and fund) a new Phase III trial.
update: incorrect assertion of fact crossed out
It is also true that there is no evidence of benefit from the second dose of Pfizer’s vaccine. It is clear that people who have received one dose of either vaccine are among those least at risk of Covid 19.
See the raw data below from Polack et all 2020 . Can anyone see from the Kaplan Meier plot when the second dose was given ?
The vaccines are in very short supply. People are anxiously waiting for vaccination. Because the protocol had two doses, half of the vaccine will be reserved for the people who will benefit least.
Here there is a difference between careful science and optimal policy. In science it is crucial to write the protocol first then follow it mechanically. This is necessary so that the experimental interventions are exogenous and one can be sure they cause the observed outcomes and are not caused by observations.
However, it is not optimal policy to reduce the possible decisions to two, a priori with extremely limited data. This is what the FDA does. I think they should approve a single dose. Their rule is always to only act on extremely firm knowledge. It is, in this case, not going to be first do no harm. The second dose has side effects (mild but not zero). There is, I think, no weak evidence of benefits. (Again, the test has extremely low power (and I’m not sure protocol did not say the question would be addressed — if it didn’t then there is a problem — the rule decide what to do in advance applies to data analysis too — it is vital that the data not be dredged looking for a significant coefficient)). I think the point estimate is pretty much exactly zero benefit. of a benefit of the second dose much lower than of the first (and without proof of any benefit.
I think that people should be given a single dose. After everyone who wants one dose has been vaccinated, then it makes sense to give people a second dose. There is no reason to think spacing 4 3 weeks apart is optimal — the spacing was decided in advance (and it was 4 weeks for the Moderna vaccine hence my mistake).
Noah Smith wrote a very interesting post on how macroeconomists are behaving very differently in 2020 than in 2008-9 : The new macro: “Give people money” . He notes two differences : first there is little discussing of theory or of the models used in most peer reviewed articles, second there is (as far as he can tell and he would know more than me) a virtual consensus that we need stimulus and that austerity would be a mistake.
Or as a tweet
I think this is a vast improvement and demonstrates the success of a wonderful scientific counter revolution. I think it better to read his excellent brief post before reading this long wandering self indulgent post (and maybe best to read his post and not read this post).
Chronic Lymphocytic Leukemia is the most common kind of leukemia in adults. Current therapy is extremely unpleasant and often (usually) fails. There is a very large number of clinical trials of new therapies mostly immunotherapies. They are based on the fact that CLL cells are B-cells which express CD20.
I promised myself not to forecast. I promised you that I would call Pennsylvania when I could. I get Trump ahead by about 700,000 and 2,000,000 absentee ballots to be counted. Assuming independents split (generous to Trump) Dem-Rep on absentees is about 50% so … I call Pennsylvania for Biden.
(I may delete this post tomorrow).
But I promised.
Networks finally call Pennsylvania more than 3 days and 8 hours after I did
It is very likely that the candidate who wins Pennsylvania will become president. A problem is that the election officials are not allowed to look at absentee ballots before November 3rd. It is likely that Trump will lead the in person vote and try to declare victory.
It shows the amazing early voting turnout. One fault is that it has a list of states with detailed data and a map of states showing early votes cast so far as a fraction of total votes in 2016. This currently shows Pennsylvania with a relatively low ratio of early votes = 27.8 % of the 2016 vote.
The page for Pennsylvania shows that (so far) 69.7% of those early votes have been cast by registered Democrats and 20.8% by registered Republicans. This is an extraordinary imbalance. Assuming voters with no party affiliation split equally (which is generous to Trump given polls) this suggests a 49.7% of all early votes lead already booked by Biden. So about 13.3% of the 2016 turnout. Even with very high 2020 turnout that has to be over 10% of 2020 votes.
It will also be possible to do this election night.
I hope that this calculation will make it possible to call the election before the absentee votes are counted. I also hope TV networks do this (no reason not too — the call is always a forecast based on exit polls and models).
Last I heard (6:52 PM EST October 10 2020) Donald Trump wants to sign a huge new stimulus bill. I don’t try to read his mind, but I think he is sincere. It is the only way he can win re-election. On the other hand, Mtich McConnell does not want the Senate to pass a huge stimulus bill. I assume that he assumes that Trump will lose and has already switched to the worse it is the better it is. What is a poor President to do ?
1) Mitch McConnell is not President of the Senate. The President of the Senate is named Michael Pence. Michael Pence can actually preside. Then if a Senator (say Charles Schumer) introduces a Mnuchin/Pelosi compromise bill, it can be debated (Pence need show no more respect for regular order than McConnell ever has). Then there will be a filibuster. VP Pence can declare the debate over and call a vote. This would be a lie about Senate rules. McConnell would object and the Senate would vote on the objection. The Democrats, independents and 3 Republicans would make a tie so Pence’s decision would stand. This is how filibusters are nuked. McConnell doesn’t have to be involved. He can be outvoted if there are three Republican Senators who do not want to break with Trump, go against public opinion, and lose their seats. Then there would be a vote. The Democrats, independents, 3 Republican Senators and Pence could send the bill to Donald Trump to sign.
I think it all works fine. Pence can say no to Trump (it is a tradition that Vice Presidents are servants of Presidents not a provision of the Constitution (as Jefferson and Adams might explain). It is possible that no Republican Senators would see any gain in breaking with Trump. It is almost certain that Trump won’t do this.
I would like to propose a set of questions for the Amy Coney Barrett confirmation hearings.
I would ask about her interview with Donald Trump.
1) in the interview, did the president talk about himself at all ?
Both answers are costly. We all know he did (he always does) so to answer no is to blatantly like.
A yes answer leads to following questions (which I would ask in any case).
Barrett will refuse to answer, saying the conversation should be private.
2. I’m not asking if he said he had a headache, My concern is whether he said anything about the upcoming Presidential election aand whether it might be contested.
Again the answer no is an obvious lie. Also refusing to answer suggests that the answer is damaging to her (as the true answer certainly is).
She will refuse to answer
3. So you refuse to say that he didn’t suggest that he wants you confirmed so you can side with Trump in a possible upcoming Trump V Biden case ?
She has to refuse.
4. Did he ask you to assure him that you would vote in his favor if there were such a case ?
Here she has to answer no. It is very costly to refuse. If she answers yes (almost certainly the honest answer) then she can’t refuse to answer when asked how she replied.
5. Here under the extremely unlikely hypothesis that she answers yes, she would have to claim she told him she can’t make that promise about a purely speculative case for which the facts haven’t even occurred yet. That would be an obvious lie. If she had been asked and gave that answer, she would not have been nominated.
So she will answer no, but then argue that the conversation is confidential and she shouldn’t answer other questions about it. I think the questions lead her to contradicting herself.
She could stick to refusal to answer, but if she refuses to answer “did you promise to President Trump that you will favor him in an upcoming Trump V Biden case ?”
Then she has some trouble.
Then ask if she promises to recuse herself if there is a Trump V Biden case (she has to answer that one).
This doesn’t distract from the gross impropriety of blocking Garland then rushing Barrett.
The questions are not polite or normal, but the situation is clearly not normal either.
Polls vs Polls plus.
Rule number 1 of forecasting is do not quarrel with fivethirtyeight.com
Rule number 538 is not ever.
So here I go. I am going to start with the fivethirtyeight Senate forecast(s). (s) because there are three and an “pick a model” icon to toggle them. I like the “lite” just polls forecast. I like it because it estimates a 70% probability of a Democratic majority, while the “Classic” “polls, fundraising, past voting patterns and more” model gives them only a 67% chance and the “Deluxe” “we add experts’ ratings to the classic” gives them only 62%.
Which model is optimal ? The Deluxe model gives the best fit with past elections — this must be true because it nests the other two. It would be true also if the deluxe model gave worse forecasts. Generally, the problem is over-fitting if one estimates many parameters. The Deluxe model adds few new parameters (I guess only one but I won’t check). An argument against it has to be not the standard watch out for overfitting argument (It will also outperform using the Akaike Information Criterion).
So how can I argue against it (and then go on to argue against the classic model)) ? Basically, I will argue that things have changed, so past performance is not a reliable indicator of future performance. Some changes are obvious — many more polls are conducted than used to be. Everyone, even experts, knows about averaging polls and how much better it works than looking at them and trying to judge. There is extreme turmoil.
OK now something along the line of evidence. I am going to present data with states in alphabetical order (and below selected so the figure is almost legible).
Here are deluxe forecasts
Here are lite (polls only) forecasts
You can see (if you have excellent eyes — fewer but larger numbers after the jump) that the deluxe forecasts are systematically more favorable to Republicans than the lite forecasts. This would be very unlikely if all of polls, money raised, past voting, and experts’ ratings gave unbiased estimates. The logic of addiing more useful variables is that it increases the precision of the estimates not that it eliminates bias. If there has been a shift of support from Republicans to Democrats, then forecasts based on past voting will be biased in favor of Republicans. It still often makes sense to include data on past voting, because it reduces the variance due to random sampling of the forecast. There are two arguments against — one is that the error in polls has low (not zero) correlation from state to state (one part of it is the change in true public opinion from polling day to election day) so, while each state forecast’s mean squared error is reduced by adding past voting patters as an explanatory variable, the nationwide calculations are worseened.
Another argument is (see above) there are many more polls than their used to be. This makes polling averages better forecasts (not as much as it would if each poll had an genuine independent sampling error but still a lot). If one decided on weights optimizing using data on old elections, one would put too low a weigh on the polling average. I think this happened.
The big change comes when experts’ ratings are added. Here one thing is that experts’ ratings are given by category
Solid R, Likely R, leans R, tossup, Leans D, likely D, Solid D. Now lets pretend that the “experts” have learned that the best strategy is to average polls, do a “lite” calculation then classify based on estimated probabilities. Such “experts” would add no useful information and would remove information. Also they would outperform the other “experts” just as Nate Silver systematically outperformed the previously recognized experts.
The key word here is “learned”. I speculated about a change over time from experts trying to infer without relying totally on polls to experts presenting polling averages as judgment calls.
This is a kind of herding. There is a difference between the strategy which enables me to give my best forecast (lowest mean squared error) and that whic make my forecast the most useful contribution to an average. The best strategy for Robert Waldmann would be to just cut and past fivethirtyeight (see above). My effort to improve on their three forecasts by emphasizing one which they don’t headline above the one which they present as the default is an effort to add something useful. Just following them is probably the best strategy to avoid embarrassment. But challenging them might be useful.
My wild guess is that experts have learned to average polls, then use the average to assign races to categories (eliminating useful information) then change a few of the ratings so it isn’t obvious what they did (and so their ratings aren’t identical to those of another expert). If that’s true, then their ratings used to contain useful information and don’t anymore.