This post is not up to the standards of the New England Journal of Medicine
Compassionate Use of Remdesivir for Patients with Severe Covid-19
is an important article written and published with amazing speed. The (many) authors (including professional writers) assess the experience of 53 “patients who received remdesivir during the period from January 25, 2020, through March 7, 2020, and have clinical data for at least 1 subsequent day.”
I think I’m just going to fair use most of the abstract
… Patients were those
with confirmed SARS-CoV-2 infection who had an oxygen saturation of 94% or less while they were breathing ambient air or who were receiving oxygen support.Patients received a 10-day course of remdesivir, consisting of 200 mg administered intravenously on day 1, followed by 100 mg daily for the remaining 9 days of treatment. This report is based on data from patients who received remdesivir during the period from January 25, 2020, through March 7, 2020, and have clinical data for at least 1 subsequent day.
Of the 61 patients who received at least one dose of remdesivir, data from 8 could not be analyzed (including 7 patients with no post-treatment data and 1 with a dosing error). Of the 53 patients whose data were analyzed, 22 were in the United States, 22 in Europe or Canada, and 9 in Japan. At baseline, 30 patients (57%) were receiving mechanical ventilation and 4 (8%) were receiving extracorporeal membrane oxygenation. During a median follow-up of 18 days, 36 patients (68%) had an improvement in oxygen-support class, including 17 of 30 patients (57%) receiving mechanical ventilation who were extubated. A total of 25 patients (47%) were discharged, and 7 patients (13%) died; mortality was 18% (6 of 34) among patients receiving invasive ventilation and 5% (1 of 19) among those not receiving invasive ventilation.
I find the results *very* encouraging. As I have written from time to time, I don’t agree with current interpretation of the pure Food and Drug Act. I think Remdesivir should be approved with possible revocation of the approval if the results of the controlled trials are disappointing (that is, as always, I reject the current FDA approach). I know that won’t happen. I am going to try to add something interesting (while noting why the NEJM would not and should not publish it).
An impressive aspect of the medical literature is that they take mathematical statistics seriously. This is partly due to the FDA rules and very strictly applied to drug trials. It is required to describe the hypothesis test *before* collecting the data. This is the only way to rule out cherry picking.
The authors note that they report point estimates and standard errors and warn us to refrain from dividing, calling the quotient a t-statistic and claiming the null was rejected if there is a t-statistic greater than 2. Yes exactly. If one runs multiple tests and rejects if any reject, then the size must be adjusted (a safe way to do this is to bonforronize and assume that the overall size is the sum of the sizes of the individual tests — this is the worst case so it is safe).
They also do not test the statistical signficance of differences between the treated group and comparison groups from the published literature. Hypothesis testing is allowed only for genuine experiments with a control group and randomization. Economists often do such things and often obtain empirical results which are not valid at all.
I applaud this discipline. Also I can’t force myself to be so disciplined. I will do exactly what they refuse to do. They give the raw numbers; just typing some numbers, no insinuation of a hypothtest here, except it is humanly impossible to resist the temptation to guess what would happen if they (incorrectly) treated the two groups as two wings of a randomized trial.
My guess is that, if the comparison group were an actual control group, then the null that Remdesivir doesn’t help would be overwhelmingly rejected. For mechanically ventilated patients the death rate was 18%. The number one sees in the literature for the (non control) group is 67%. That’s a huge difference.
So the numbers I will analyse are death rates of patients who are ventilated or not and with and without Remdesivir. Those without are from published articles cited in the NEJM article.
Before going on, I have to explain one of the reasons this post is a sin against statistics and science. At the end of the sample period, 8 patients were still on ventilators and 2 on the even more extreme extracorporeal membrane oxygenation [ECMO]. Tragically their chances are not great. The 7 deaths so far are not all deaths due to Covid 19 of patients discussed in the manuscript. I am just going to assume that there is similar follow up of patients in the studies which I used as if they were control groups. There are many reasons that this isn’t up to NEJM standards, but I think the most important is that I am just assuming that the time from treatment until the end of counting deaths is the same in different studies.
From the abstract we have 6 of 34 ventilated patients who received Remdesivir died, 1 of 19 of non ventilated patients who received Remdesivir died.
Later for non ventilated non remdesivir
In a recent randomized, controlled trial of lopinavir–ritonavir in patients hospitalized for Covid-19, the 28-day mortalitywas 22%. It is important to note that only 1 of
199 patients in that trial were receiving invasive ventilation at baseline.
So that gives 43 deaths out of 198 patients who were not ventilated and did not receive Remdesivir.
among 201 patients hospitalized in Wuhan, China, mortality was 22% overall and 66% (44 of 67) among patients receiving invasive mechanical ventilation
so 1 of 19 vs 43 of 198 among non ventilated patients
6 of 34 vs 44 of 67 among ventilated patients
One way to test is to estimate the probability of dying with the frequency, calculated variances and note that, for independent samples the variance of the difference is the sum of the variances. This is a Wald type test.
for ventilated with Remdesivir
mean 1/19 = 0.05263
variance = (1/19)(18/19)/19 = 0,002624
mean 43/198 = 0.21717
variance = 0,0008586
the z score is
estimated reduction of the probability of dying is 16.45%
The z – score is -2,788
If this were a legitimate hypothesis test, the null that Remdesivir does not help patients who aren’t ventilated would be rejected at standard confidence levels
for Ventilated patients the estimated reduction in the probability of death is 48.03% with a z-score of -5,495
This rejected the null at all confidence levels that anyone ever uses.
A big problem is that I am acting as if the patients treated with Remdisivir who are still ventilated will survive as long as patients in the comparison groups survived. The Ritonavir study looked at death within 28 days. Not all surviving patients in the NEJM paper have lived that long. The Wuhan paper (as quoted) did not explain how long the patients were followed.
I think a conservative estimate of the effect of Remdesivir can be obtained by assuming that the patients who were still on ventilators when the study ended and the patients on ECMO will die of Covid 19.
This makes the estimate of deaths for the non ventilated patients 2 of 19 and the estimate for the ventilated patients 15 of 34.
The estimated benefit of Remdesivir is, of course, reduced.
For the non ventilated patients the reduction in probability of death is just 11.29 % with a z-score of -1.467 so the null is not rejected at standard confidence levels.
For the ventilated patients the estimated reduction is 21.55 % with a z-score of -2.092. This means that, even with the theextemely pessimistic guess about the fate of patients still on ventilators, the null of no benefit is rejected at the 5% level.
Personally, I think it is time to allow all MDs to prescribe Remdesivir which permission can be repealed if the genuine controlled trials are disappointing. I think my radical approach would save many lives.