IMET2000 Clinical Research Club

• Published: 15 April 2021

Abstract

Substantial COVID-19 research investment has been allocated to randomized clinical trials (RCTs) on hydroxychloroquine/chloroquine, which currently face recruitment challenges or early discontinuation. We aim to estimate the effects of hydroxychloroquine and chloroquine on survival in COVID-19 from all currently available RCT evidence, published and unpublished. We present a rapid meta-analysis of ongoing, completed, or discontinued RCTs on hydroxychloroquine or chloroquine treatment for any COVID-19 patients (protocol: https://osf.io/QESV4/). We systematically identified unpublished RCTs (ClinicalTrials.gov, WHO International Clinical Trials Registry Platform, Cochrane COVID-registry up to June 11, 2020), and published RCTs (PubMed, medRxiv and bioRxiv up to October 16, 2020). All-cause mortality has been extracted (publications/preprints) or requested from investigators and combined in random-effects meta-analyses, calculating odds ratios (ORs) with 95% confidence intervals (CIs), separately for hydroxychloroquine and chloroquine. Prespecified subgroup analyses include patient setting, diagnostic confirmation, control type, and publication status. Sixty-three trials were potentially eligible. We included 14 unpublished trials (1308 patients) and 14 publications/preprints (9011 patients). Results for hydroxychloroquine are dominated by RECOVERY and WHO SOLIDARITY, two highly pragmatic trials, which employed relatively high doses and included 4716 and 1853 patients, respectively (67% of the total sample size). The combined OR on all-cause mortality for hydroxychloroquine is 1.11 (95% CI: 1.02, 1.20; I² = 0%; 26 trials; 10,012 patients) and for chloroquine 1.77 (95%CI: 0.15, 21.13, I² = 0%; 4 trials; 307 patients). We identified no subgroup effects. We found that treatment with hydroxychloroquine is associated with increased mortality in COVID-19 patients, and there is no benefit of chloroquine. Findings have unclear generalizability to outpatients, children, pregnant women, and people with comorbidities.

Introduction

Coronavirus disease 2019 (COVID-19) caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has the potential of progression into respiratory failure and death¹. More than 1,500,000 persons with COVID-19 globally have died by December 2020², and treatment options are limited³. The COVID-19 pandemic has caused a hitherto unprecedented search for possible therapies, with almost 700 clinical trials initiated in the first quarter of 2020—and one in five of these trials target hydroxychloroquine (HCQ) or chloroquine (CQ)⁴. This remarkable attention was primarily due to in vitro data⁵, immunomodulatory capacities⁶, and the oral formulation and well-documented safety profiles.

In March 2020, the US Food and Drug Administration (FDA) issued an Emergency Use Authorization of HCQ⁷ and the number of prescriptions and usage outside clinical studies skyrocketed⁸. In many countries, HCQ or CQ were listed in treatment guidelines for COVID-19 (including, e.g., China, Ireland, and the United States)⁹. In a New York City cohort of 1376 COVID-19 inpatients during March–April 2020, 59% received HCQ¹⁰. However, the FDA revoked the Emergency Use Authorization on June 15, 2020¹¹. At that point, two large randomized clinical trials (RCTs), RECOVERY and the WHO Solidarity trial, had stopped enrollment to their HCQ treatment arms^12,13. An interim analysis of the RECOVERY trial showed no mortality benefit of HCQ¹³. Established as treatments of malaria and rheumatic disorders, HCQ and CQ may carry potentially severe adverse effects, especially related to cardiac arrhythmia⁶. Public uncertainty still remains, as illustrated by recent reports of planned use in pandemic epicenters in Central and South America¹⁴.

While many trials are ongoing, additional published evidence of potential benefits or harms may be several months away, if they even reach completion. Given the lack of favorable results in the large RECOVERY trial and the revoked Emergency Use Authorization, recruitment into HCQ and CQ trials has become increasingly difficult and many trials may run the risk of ending in futility. A rapid examination of data on all-cause mortality from as many trials as possible may offer the best evidence on potential survival benefits and to ensure that patients are not exposed to unnecessary risks if benefit is lacking. We used the infrastructure established with COVID-evidence¹⁵, a comprehensive database of COVID-19 trials funded by the Swiss National Science Foundation, to invite all investigators of HCQ or CQ trials to participate in an international collaborative meta-analysis. We aimed to identify and combine all RCTs investigating the effects of HCQ or CQ on all-cause mortality in patients with COVID-19 compared to any control arm similar to the experimental arm in all aspects except the administration of HCQ or CQ.

In this work, we find that treatment with HCQ is associated with increased mortality in COVID-19 patients, and there is no benefit of CQ. Findings have unclear generalizability to outpatients, children, pregnant women, and people with comorbidities.

Results

Our search identified 146 randomized trials investigating HCQ or CQ as treatment for COVID-19, of which 83 were deemed potentially eligible after scrutinizing the randomized comparisons. The investigators of these 83 trials were contacted and 57% (47 of 83) responded (Fig. 1). Of the responders, 19 trials were eligible and available (14 unpublished, one preprint, and four publications); 21 trials were ineligible according to information provided; five responding investigator teams were not ready to share their results yet; and two declined participation. For the 36 trials without response, six were confirmed eligible and available (four publications and two preprints); two were confirmed ineligible; and for the remaining 28, results were not available, nor could they be confirmed eligible.

We included 28 trials (14 unpublished trials, nine publications, and five preprints; of these, one publication and two preprints were identified for the first time in our search update)^{13,16,17,18,19,20,21,22,23,24,25,26,27,28}. Individual trial characteristics are presented in Table 1 (28 included trials) and Supplement Table S1 (34 potentially eligible but unavailable). Overall, trial characteristics were not different between included and unavailable trials (Table 2).Table 1 Group-level characteristics of randomized clinical trials evaluating hydroxychloroquine or chloroquine as treatment for COVID-19.Full size tableTable 2 Group-level characteristics of included and unavailable trials.Full size table

HCQ was evaluated in 26 trials (10,012 patients) and CQ was evaluated in four trials (307 patients). Two trials investigated both HCQ versus control and CQ versus control (63 patients). The median sample size was 95 (interquartile range (IQR) 28–282) for HCQ trials and 42 (IQR 29–95) for CQ trials. The two largest trials (RECOVERY and WHO SOLIDARITY) included 47% and 19% of all patients in the HCQ trials, respectively. Most trials investigated HCQ or CQ in hospitalized patients (22 trials; 79%), and only five trials (18%) had an outpatient setting. The average mortality was 10.3% (standard deviation 13.5%) in inpatient trials and 0.08% (standard deviation 0.18%) in outpatient trials. The comparator was in 11 trials placebo (39%) and in 17 (61%) no other treatment than standard of care. In most trials, patients and clinicians were aware of the treatment (15 trials; 54%), while in one trial (4%) the patients were blinded and in 11 trials (39%) patients and clinicians were blinded (Table 2). We identified no relevant risk of bias across all trials, with only one trial including seven patients having an overall high risk of bias (Table S2). We found no evidence of small-study effects (Figure S1).

Regarding HCQ, in the 26 included trials, 606 of 4316 (14.0%) patients treated with HCQ died and 960 of 5696 patients (16.9%) in the control groups died (within 19 trials with a 1:1 randomization ratio, 7.7% of patients in the HCQ arm died [181 of 2346] and 7.1% of patients in the control arm died [168 of 2352]). In the meta-analysis, the combined odds ratio (OR) was 1.11 (95% confidence interval (CI), 1.02–1.20, p = 0.02), with low heterogeneity (I² = 0%) (Fig. 2A). In 12 trials including a total of 1282 patients (representing 12.8% of the total sample size for HCQ), there were zero deaths in both arms.

Regarding CQ, in the four included trials, 18 of 160 (11%) patients treated with CQ died and 12 of 147 patients (8%) in the control groups died. The combined OR was 1.77 (95% CI 0.15–21.13, p = 0.21), with low heterogeneity (I² = 0%) (Fig. 2B). In two of four trials including a total of 217 patients, there were zero deaths in both arms.

The available evidence in this study is the result of publications, preprints, or unpublished trial results accrued from April 10, 2020 to October 16, 2020 as shown in the cumulative meta-analyses (Fig. 3A–C).

For HCQ, none of the exploratory subgroup analyses showed an effect modification (Supplement Table S3 and Figure S1A, B). When only including published information (publications and preprints, excluding unpublished trials), there was an increase in mortality among patients treated with HCQ (OR 1.12, 95% CI 1.08–1.16), while among the unpublished trials there was no such sign of increased mortality (OR 0.92, 95% CI 0.63–1.34, p for interaction = 0.23). We conducted no subgroup analyses for CQ, as there were only two trials with events. In the sensitivity analyses employing different meta-analytical approaches (Supplement Table S4 and Figures S2A–C), results were compatible with the main analysis.

Discussion

This collaborative meta-analysis of 28 published or unpublished RCTs, including 10,319 patients, shows that treatment with HCQ was associated with increased mortality in COVID-19 patients, and there was no benefit from treatment with CQ. No differences were seen across subgroup analyses on patient setting, diagnosis confirmation, control type, publication status, or dose and the between-study heterogeneity was low. For CQ, the number of studies was too small to draw clear conclusions.

This meta-analysis offers useful insights for a challenging health situation. Hundreds of thousands of patients have received HCQ and CQ outside of clinical trials without evidence of their beneficial effects. Public interest is unprecedented, with weak early evidence supporting HCQ’s merits being widely discussed in some media and social networks—despite the unfavorable results by a very large RCT. Numerous clinical studies have been investigating HCQ and CQ almost simultaneously. Although several systematic reviews and meta-analyses are already available, they only consider the small handful of RCTs being already published (which were all included here)^{29,30,31,32,33}. While data sharing has been rather limited to date in biomedical research, such openness can be transformative in generating knowledge. This pandemic has brought together a collaboration of clinical trialists agreeing to share their data, which allows this study to not only summarize the existing evidence but also illustrate the accumulation of evidence that would otherwise not be available.

For HCQ, evidence is dominated by the RECOVERY trial¹³, which indicated no mortality benefit for treated COVID-19 patients, together with longer hospitalization and higher risk of progression to invasive mechanical ventilation and/or death. Similarly, the WHO SOLIDARITY trial indicated no mortality benefit²⁶. The RECOVERY and WHO SOLIDARITY trials used HCQ in comparatively higher doses than all other trials except REMAP-CAP. There was no evidence for an effect modification by dose (p for interaction = 0.29), and the combined effect of all the trials with lower dose did not indicate a benefit of HCQ but tended to a null effect (OR 0.97; 0.73–1.30) with wide CIs, compatible with the main effect estimate.

This meta-analysis does not address prophylactic use nor other outcomes besides mortality. All but three trials excluded children and the majority excluded pregnant or breastfeeding women; generalizability remains unclear for those populations. Among the five studies on outpatients, there were three deaths, two occurring in the one trial of 491 relatively young patients with few comorbidities¹⁷ and one occurring in a small trial with 27 patients. For outpatients who are elderly or have comorbidities, evidence is sparse. Most of the 28 trials excluded persons with comorbid conditions carrying higher risk of adverse events from HCQ or CQ^16,17,27. No evidence is in the pipeline for these groups, which echoes clinical reasoning being reluctant to expose them to risk.

Although the published trials resulted in a conclusive treatment estimate, the unpublished trials tend towards a null effect. The tendency of published trials to report larger effect sizes than unpublished trials is well-documented and constitute one of the reporting biases that are discernable only when a body of studies are considered together³⁴. Null results are less expected to be rapidly disseminated, especially if the trial is small. Of note, RECOVERY results showing dexamethasone benefits have been published more rapidly³ than the unfavorable HCQ results¹³. We have found no evidence for small-study effects and we consider that the results are unlikely to have been materially affected by publication or reporting bias. This paper offers the most comprehensive summary on HCQ and mortality in COVID-19 to date.

Twenty-three percent of the potentially eligible trials were listed as discontinued, mostly because of fewer patients than expected. Among 28 included RCTs, only two had reached their target sample size at the time of censoring for this meta-analysis. As previously discussed⁴, most trials on HCQ and CQ in COVID-19 are small, reflecting both the strong motivation for individual efforts and underscoring the need for readily available research infrastructure to merge small-scale initiatives. Especially in the context of recruitment challenges, we encourage other researchers to form collaborations and combine trial results³⁵.

Our analysis has some limitations. First, although we adopted a comprehensive, systematic search strategy, our real-time initiative differs from traditional systematic reviews. We focused on collecting unpublished information, aiming to rapidly secure as much trial evidence as possible. We did not review individual trials, nor did we stratify results according to patient characteristics, and we have not collected information on other outcomes than mortality. Such analyses are planned in future publications using in-depth details disclosed in individual trial publications to come^36,37,38. The exploratory subgroup analyses did not support the hypothesis that blinding/use of placebo is associated with the observed effect (the test for an interaction gives p = 0.15 and the OR is 0.88 with wide CIs 0.55–1.41, compatible with the overall effect); moreover, attrition was negligible (median 0%, IQR 0–0%; range 0–19.5%). A meta-epidemiological study shows little evidence that mortality results would be affected by lack of blinding, or problems in randomization and allocation concealment, in contrast to less objective outcomes³⁹. Accordingly, we identified no relevant risk of bias across trials. Second, a majority of the potentially eligible trials were not available. Despite going far beyond the standard review of published evidence, we expect additional results from future trials on CQ to narrow the uncertainty of the treatment effect and possibly reveal benefits or harms not discernible based on the current evidence. We plan to perform an update when substantial additional evidence becomes available. Third, although this analysis intended to combine results from both inpatients and outpatients regardless of disease severity, trials enrolling patients with mild to moderate disease comprised a minority of the final sample size; many of which had zero or few events. Finally, although sensitivity analyses addressing model specifications were compatible with the main analysis, one combination (HKSJ model with SJ τ² estimator) yielded substantially wider CIs. This combination gave disproportionately low weight to RECOVERY (15%) and we consider the main model (HKSJ with PM τ² estimator) to be more valid in this situation.

Treatment with HCQ for COVID-19 was associated with increased mortality, and there was no benefit from CQ based on currently available randomized trial data. Medical professionals around the globe are encouraged to inform patients about this evidence.

Methods

This collaborative meta-analysis focused solely on all-cause mortality in order to provide rapid evidence on the most critical clinical outcome. Investigators of ongoing, discontinued, or completed trials were contacted via email to provide group-level (aggregated) mortality data per trial arm at any time point available. The protocol was published online before data collection⁴⁰. This review has been reported in accordance with the Preferred Reporting Items for Systematic Review and Meta-analysis⁴¹. The PRISMA checklist can be found in the supplement (Supplement 1).

Eligibility criteria

We considered all clinical trials that reported randomly allocating patients with confirmed or suspected SARS-CoV-2 infection to a treatment protocol containing HCQ or CQ (for any duration or dose) or the same treatment protocol not containing HCQ or CQ. In other words, the control group had to receive placebo or no treatment other than standard of care (we excluded comparisons of HCQ or CQ against an active treatment, e.g., HCQ versus azithromycin, since active controls were too heterogeneous to pool together and reveal the pure benefits and harms of HCQ or CQ). Eligible ongoing trials had to provide data on all-cause mortality and randomize the first patient before June 1, 2020 (time point selected arbitrarily as we did not expect trials launched later to recruit enough patients to provide relevant additional information). Trials that were published or posted as preprint were not restricted by date. Prevention trials were not included. We included trials regardless of follow-up time and whether mortality was a primary outcome or not; moreover, we put no restrictions on trial status, language, geographical region, or healthcare setting.

Search strategy

We searched for eligible trials registered at ClinicalTrials.gov and the WHO International Clinical Trials Registry Platform (ICTRP) by June 11, 2020 (COVID-evidence database; see Supplement 2)⁴². We additionally searched PubMed and the Cochrane COVID-19 trial registry (covering preprints, trial registries, and literature databases) by June 11, 2020, using terms related to HCQ and CQ combined with terms for COVID-19 and a standard RCT filter (Supplement 2)⁴³. We updated the literature search on October 16, 2020. Two authors (C.A. and A.M.S.) independently verified the eligibility criteria (Fig. 1) and solved any discrepancies by discussion.

Principal investigators of 83 potentially eligible trials were asked to confirm the eligibility criteria, as well as: “For each of your study arms: (a) What intervention did this group receive? (b) How many patients were randomized to this group? (c) Of these patients, how many have died? (d) Of these patients, for how many it is unknown if they are dead or alive?” (Supplement 3, email template). Investigators who were not responsive received two email reminders in English or Chinese, depending on trial origin.

Data extraction

The following information was extracted from all included RCTs by two reviewers (C.A. and A.M.S.) and verified by the trial investigators: experimental and control arms, number of randomized participants, treatment schedule, patient setting, eligibility criteria, study location, blinding, target sample size, and trial status. We also classified trials as published in a peer-reviewed journal, posted on a preprint server, or unpublished (the latter category not including preprints). For reasons of feasibility within this rapid assessment, we generally did not request descriptive information beyond items included in trial registrations.

Risk of bias assessment

Two reviewers (C.A. and A.M.S.) independently assessed the risk of bias of included RCTs using the Cochrane risk of bias tool 2.0⁴⁴. Disagreements were resolved through discussion. We used the information reported in preprints and journal publications, and for unpublished trials, we retrieved information from trial registrations, which was confirmed by trial investigators. We also assessed small-study effects with an inverted funnel plot and Egger’s test⁴³. The presence of small-study effects may be suggestive, but not definitive, of publication bias⁴⁵.

Data synthesis and analyses

The main analysis evaluated separately the effect on all-cause mortality of HCQ versus control and CQ versus control. We report absolute numbers and proportions, as well as the treatment effect estimate as an odds ratio (OR; odds of death in the HCQ or CQ intervention group divided by the odds of death in the control group) with 95% CIs. For multi-arm studies, we requested data for all arms and calculated treatment effect estimates for each eligible comparison. We combined mortality effects from all RCTs based on binary outcome data (2 × 2 contingency tables) in meta-analyses and describe the statistical heterogeneity using the I² statistic⁴⁶. In our protocol, we prespecified a random-effects model of the Hartung–Knapp–Sidik–Jonkman (HKSJ) approach⁴⁷, in order to provide more equality of weights between trials with moderate to large size (than, e.g., the DerSimonian–Laird approach). We did not prespecify the between-study variance estimator, τ², but chose the Paule and Mandel (PM) estimator based on provided guidance on choosing among 16 variants⁴⁸. Cases of zero events in one arm were corrected by adding the reciprocal of the size of the contrasting study arm⁴³. However, considering the range of sample sizes and the number of zero events across trials, we assessed the effects of alternative approaches with sensitivity analyses, as detailed below. To explore and illustrate evidence generation over time, we also performed a cumulative meta-analysis of all trials as well as stratified by dissemination status (publications/preprints versus unpublished), using the HKSJ approach with PM τ². We used the date of email response or publication/posting of preprint. The meta-analyses were completed using R version 3.5.1 and the “meta” package version 4.13-0.

Subgroup analyses

In exploratory subgroup analyses, we stratified trials by patient setting (as proxy to COVID-19 severity: outpatients, inpatients but not intensive care unit (ICU), and ICU), diagnostic confirmation (confirmed SARS-CoV-2 versus suspected cases), control type (placebo control versus other), and publications/preprints versus unpublished trials. We did not stratify for missing data since the amount was extremely low. A post hoc stratification by HCQ dose was added (trials with ≥1600 mg on day 1 and ≥800 mg from day 2 versus lower-dose trials) to isolate trials predicted to achieve blood levels of HCQ above the in vitro half-maximal inhibitory concentration value for SARS-CoV-2 (1.13 μM)⁴⁹.

Sensitivity analyses

We added exploratory sensitivity analyses to assess robustness across meta-analytic approaches: DerSimonian–Laird and Sidik–Jonkman τ² estimators, Mantel–Haenszel random-effects method, and Peto method. DerSimonian–Laird is a standard random-effects meta-analysis approach, but may underestimate uncertainty. The Sidik–Jonkman τ² estimator, on the other hand, may yield inflated estimates if heterogeneity is low⁴⁸. The Mantel–Haenszel method performs reasonably well with small and zero event counts, much like Peto and arcsine transformation. The Peto method is suboptimal in the presence of substantial imbalances in the allocation ratio of patients randomized in the compared arms (e.g., RECOVERY trial). We also modeled variants to handling zero events (arcsine difference, and excluding trials with zero events) as well as excluding trials with <50 participants.

Unpublished trial details

All unpublished trials were performed according to the principles of the Declaration of Helsinki and written informed consent was obtained from the study participants. The release of mortality outcome data was authorized by the respective data and safety monitoring boards and principal investigators. Ethical approval was granted by institutional review boards as follows: University of Pennsylvania, ref. #842838 (PATCH, NCT04329923); National Bioethics Committee (NBC) Pakistan, ref. 4–87/NBC-471-COVID-19-05/20/ (PROTECT, NCT04338698); Ethics Committee of the Capital Region of Denmark, ref. H-20025317 (CCAP-1, NCT04345289); Comité de Protection des Personnes du Sud-Ouest et Outre-Mer 4, ref. CPP2020-03-036/2020-001271-33/20.03.24.72431, and the Agence Nationale de Sécurité du Médicament et des produits de santé (ANSM), ref. MEDAECNAT-2020-03-00045 (HYCOVID, NCT04325893); Ethik-Kommission an der Medizinischen Fakultät der Eberhard-Karls-Universität und am Universitätsklinikum Tübingen, ref. 190/2020AMG1 and ref. 225/2020AMG1 (COV-HCQ, NCT04342221; and COMIHY, NCT04340544, respectively); London-Surrey Borders Research Ethics Committee in the UK, Medisch Ethische Toetsingscommissie Utrecht (METC Utrecht) in the Netherlands, Sydney Local District ethics Review Committee (Royal Prince Alfred Hospital) in Australia, Northern A Health and Disability Ethics Committee in New Zealand, St. Vincent’s Healthcare Group Ethics and Medical Research Committee in Ireland, King Abdullah International Medical Research Center Institutional Review Board in Saudi Arabia, University of Pittsburgh Institutional Review Board in the United States, Unity Health Research Ethics Board in Canada, National Ethics Committee for Clinical Research (CEIC) in Portugal, and the Romania Academy of Medical Sciences National Bioethics Committee for Medicines and Medical Devices (REMAP-CAP, NCT02735707); Comissão Nacional de Ética em Pesquisa (CONEP), ref. 3.961.681 (CloroCOVID19II A, NCT04323527, and CloroCOVID19II B, NCT04342650); a Single Ethics Committee from the Coordination of the National Institutes of Health and High Specialty Hospitals, ref. C13-20 (HYDRA, NCT04315896); the Ethics Committee of Beijing Youan Hospital, Capital Medical University, ref. JINYOUKELUN[2020]013 (ChiCTR2000031204); Partners Human Research Committee at the Brigham and Women’s Hospital, Boston, Ethisch Comite in Belgium, and Stichting Beoordeling Ethiek Biomedisch in the Netherlands (NCT04333654); The Queen’s Medical Center, ref. RA-2020-018 (OAHU-COVID19, NCT04345692); Duke University Medical Center Institutional Review Board, ref. Pro00105339, and UnityPoint Health Institutional Review Board (NCT04335552); and Medical Ethics Committee Utrecht (METC Utrecht), part of the Dutch Central Committee on Research Involving Human Subjects (ARCHAIC, NL8490).

Reporting summary

Further information on research design is available in the Nature Research Reporting Summary linked to this article.

Data availability

All trial-level data generated or analyzed during this study are included in this published article and its Supplementary information files. The data file is provided in the Open Science Framework [https://osf.io/qesv4/]⁴⁰. Source data are provided with this paper.

Code availability

The code file is provided in the Open Science Framework [https://osf.io/qesv4/]⁴⁰.

Change history

• 15 May 2021A Correction to this paper has been published: https://doi.org/10.1038/s41467-021-23559-1

Cite this article

Axfors, C., Schmitt, A.M., Janiaud, P. et al. Mortality outcomes with hydroxychloroquine and chloroquine in COVID-19 from an international collaborative meta-analysis of randomized trials. Nat Commun 12, 2349 (2021). https://doi.org/10.1038/s41467-021-22446-z

Download citation

• Received02 October 2020
• Accepted15 March 2021
• Published15 April 2021