Chait vs Roy on Baicker not really vs Baicker

Jon Chait has a brilliantly ruthless takedown of the absurd defences of the AHCA (house obamacare replacement) and BCRA (Senate version) . His main point is that Republicans are lying claimiing the huge cuts to Medicaid aren’t cuts to Medicaid and that the huge increase in the number of uninsured will actually be zero.

He also criticizes Avik Roy. This quarrel has become very interesting. Roy praised the BRCA. He refused to answer when Chait asked if he had also written it. Roy will not say if he is commenting on his own work without noting the conflict of interest.

Jonathan Chait‏Verified account @jonathanchait
Update: @avik tells me his policy is to not disclose his role in crafting legislation.

Chait notes that Roy praises the bill for increasing deductibles and also notes that Republicans denounced the Obamacare deductibles — for being too high. This just shows hypocrisy ( ok psychopathic dishonesty) or GOP politiicians. I’m sure Roy believes dedcutibles should be high, and Republicans in Congress have revealed a preference for high deductibles.

Chait also objects to Roy’s claim that Medicaid doesn’t cause improved health. Interestingly I had the same debate with someone on twitter yesterday. In both debates, the case against Medicaid is based on a citation of Baicker et al (2013) the report on the Oregon Medicaid experiment.

Chait (and I) responded by citing Somers Gawande and Baicker (2017) who wrote

Insurance coverage increases access to care and improves a wide range of health outcomes. Arguing that health insurance coverage doesn’t improve health is simply
inconsistent with the evidence.

and

One head-to-head quasi-experimental study of Medicaid versus private insurance, based on Arkansas’s decision to use ACA dollars
to buy private coverage for low-income adults, found minimal differences.11

So is it Chait’s experts against Roy’s experts ?

Not at all. Roy bases his argument on Baicker et al and Chait on al et Baicker. Katherine Baicker PhD (who should know) does not think that Baicker et al (2013) showed that Medicaid doesn’t work.

Indeed she concedes much less than Chait does. He wrote “The study was unable to detect better physical health outcomes.” This is false. The study found better physical health outcomes in the treatment group than in the control group. What Chait should have written was “the study was unable to detect statistically signficicantly better physical health outcomes”.

Treating a statistically insignificant evidence improvement as evidence that there was no improvement is a gross error. It is also almost universal (I have doubts only about the “almost”). In fact Baicker et al found statistically significant effects on access to health care, diagnosis of diabetes, and treatment of diabetes. They did not find new proof that standard treatment of diabetes is better than no treatement. In every other context, this is not treated as an open question. The study did not find statistically signficant evidence that the benefit was smaller than predicted based on other studies either.

But the motto of the New England Journal (and all serious scientific journals) is first make no claims which go beyond the data.

Statistically insignificant is not an assertion. It doesn’t mean zero. It doesn’t mean small. So it is always favored. Then it is read as meaning small or zero.

Chait understands this. He argues that the Baicker et al (2013) study had low power so the fact that “The study was unable to detect better physical health outcomes.” [failed to reject the null of zero effect on physical health] doesn’t tell us much. But even in the context of a discussion of power, he refuses to distinguish zero from statistically insignificantly different from zero. I think there is some rule that people writing for general audiences must not use technical terms like “statistically insignificant”. The result is that they write falsehoods.

Tens of thousands of people a year may die partly because people just will not accept that the Neyman Pierson framework is what it is.

In any case, Roy is reduced to arguing that he understand Baicker et al (2013) and Baicker doesn’t. He is not in great shape totally aside from the question about unreported conflicts of interest.

Comments (3) | |