Comparative efficacy and safety of COVID-19 vaccines in phase III trials: a network meta-analysis

There have been over 600 million confirmed cases of coronavirus disease (COVID-19) and over 6 million worldwide deaths by the end of 2022 since the onset of the COVID-19 pandemic [1]. The pandemic has significantly impacted healthcare and socio-economic development worldwide. The most prevalent clinical features of COVID-19 include fever, cough, and dyspnea [2]. While most cases are mild, the elderly and those with underlying diseases are at high risk of severe COVID-19. Moreover, some people also experience long-term effects after recovery. Novel oral antivirals such as molnupiravir, fluvoxamine, and paxlovid [3] are still under development, and heteropathy is believed to be the main clinical treatment. Therefore, vaccination is the first and most important step in stopping the spread of COVID-19 and reducing the social burden.

Vaccines can be divided into five categories according to their principles of antigen generation and production processes: inactivated virus vaccines, mRNA vaccines, DNA vaccines, viral vector vaccines, and protein subunit vaccines. Each type has certain advantages. Inactivated viral vaccines containing intact spike proteins and other proteins protect against viral variants by inducing a broader immune response [4]. mRNA and DNA vaccines are rapid and cost-effective platforms that can simulate natural infections by synthesizing endogenous proteins to induce a strong immune response [5]. Viral vector vaccines are characterized by robust immunogenicity, the absence of adjuvants, and long-term storage without freezing [6]. Protein subunits vaccines can produce robust and durable antibody responses and are expected to be safer because they do not utilize genetic materials [7].

Vaccine efficacy (VE) data are primarily obtained from phase III randomized controlled trials (RCTs). Previous studies have compared the efficacy and safety of vaccines using multiple post-hoc pairwise comparisons in meta-analyses [8‐10]. In June 2021, a meta-analysis was conducted for eight Phase III RCTs encompassing four vaccine types [8]. The study indicated that all vaccine types exhibited good preventive effects against COVID-19, accompanied by an elevated risk of overall adverse events in the vaccinated groups. However, these studies did not compare multiple vaccines administered under identical conditions [8]. A network meta-analysis (NMA) provides a methodological approach to simultaneously compare vaccines through a common comparator (placebo) since there are no head-to-head clinical studies directly comparing the relative efficacy and safety of COVID-19 vaccines. In April 2021, the first published NMA of four Phase III RCTs showed that the vaccine exhibited different efficacies to prevent COVID-19: BNT162b2 ≥ mRNA-1273 > Gam-COVID-Vac > AZD1222 [11]. Subsequently, Rotshild et al. reported no statistical differences among vaccines in the preventive effect against severe COVID-19 of the elderly [12]. The latest NMA evaluation of the efficacy of 16 vaccines (October 2022) revealed that BNT126b2 conferred the highest protection against symptomatic severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection [13].

This study aimed to integrate the latest published data from Phase III RCTs to compare the efficacy and safety of COVID-19 vaccines in adult populations. The efficacy of COVID‑19 vaccines was also conducted to prevent symptomatic disease among the elderly. This manuscript was written following the PRISMA-NMA checklist [14].

A total of 5606 records were identified by the search, with 24 published and one unpublished Phase III RCT [21‐45] involving 22 vaccines eventually included in the NMA (Fig. 1). Two of the search results included a small number of individuals under the age of 18 years [29, 31], and another study included Phase I/II/III RCTs of AZD1222 vaccines [34]. These three studies were included in the NMA to ensure a sufficient number of samples. None of the included studies directly compared two different vaccines. In total, 915,370 participants were included, and more than 50% were randomly assigned for vaccination. Study characteristics and raw data are summarized in Table 1 and S2. A comparison of basic features, including outcome definition and participant characteristics (age, sex, and race) is presented in Table S3 and Figure S1. There was no evidence of violation of the transitivity assumption. Among these articles, studies with some concerns accounted for 36%, but there were no serious risks of bias according to the RoB2 (Figure S2).

This study was based on 25 RCTs that included 915,370 patients randomly assigned to receive 22 vaccines or a placebo. This project updates and extends previous research and is the most comprehensive NMA to compare the efficacy of COVID-19 vaccines in preventing symptomatic disease and the incidence of SAEs in adults and the elderly.

In terms of safety, mRNA vaccines may increase SAEs versus the placebo, although this result was not statistically significant. Similar trends were described in an earlier meta-analysis of 11 trials [46]. Our results provided the following rankings according to RR in the indirect comparison: inactivated vaccines ≥ viral vector vaccines ≥ protein subunit vaccines > mRNA vaccines. This is unsurprising given the high safety of inactivated vaccines since no viral genetic material is involved. In addition to SAEs, inactivated vaccines have the lowest risk of local or systemic adverse events following immunization [47]. The ranking of individual vaccines was generally consistent with the vaccine type. BIV1-CovIran, an inactivated vaccine, had the lowest incidence of SAEs. Notably, most included studies did not specifically exclude patients with symptomatic COVID-19 from SAE, which may have affected the accuracy of the above ranking.

In terms of efficacy, all vaccine types versus placebo significantly prevented symptomatic SARS-CoV-2 infection, but the 95% CI for DNA vaccines indicated no effect. In the indirect comparison, our results provided the following ranking according to the RR: mRNA vaccines ≥ protein subunit vaccines ≥ viral vector vaccines ≥ inactivated vaccines ≥ DNA vaccines. The 95% CI for all vaccine types was compatible with no effect, although the RR values were significant. One possible explanation for the excellent efficacy of mRNA vaccines is the production of a fully functional protein through cellular translational machinery, which induces powerful and durable immunity against the coronavirus [48]. An earlier NMA compared nine vaccines to prevent symptomatic SARS-CoV-2 infection, based on the results of Phase III RCTs up to August 1, 2021 [12]. BNT162b2 had the highest efficacy, followed by mRNA‑1273, Gam‑COVID‑Vac, NVX‑CoV2373, CoronaVac, BBIBP-CorV, WIBP-CorV, and Ad26.COV2.S [12]. Similarly, one recent NMA reported that BNT126b2 conferred the highest protection, followed by mRNA-1273, Gam‑COVID‑Vac and NVX-CoV2373 [13]. In line with previous evidence, we ranked BNT162b2 with the highest efficacy, followed by mRNA-1273, Abdala, Gam-COVID-Vac, and NVX-CoV2373. We also found that BNT162b2 and mRNA-1273 mRNA vaccines performed best in preventing symptomatic COVID-19, while CVnCoV ranked lower. A possible explanation is that approximately 85% of COVID-19 cases in the CVnCoV trial were caused by variants that might alter VE owing to the increased transmissibility and evasion of neutralizing humoral immunity [49]. In addition, 12 µg mRNA contained in CVnCoV may be insufficient to elicit a protective immune response compared to 30 µg in BNT162b2 and 100 µg in mRNA-1273.

We found that BNT162b2 had the highest efficacy in terms of the efficacy in preventing symptomatic SARS-CoV-2 infection in the elderly population. This was consistent with the conclusion of an earlier study [47]. CVnCoV, CoVLP + AS03, and CoronaVac were interpreted as having no difference from the placebo, possibly owing to an insufficient absolute number of events in the short follow-up duration. In fact, the VE of CoronaVac in the real world has reached 66.6% in individuals aged > 60 [50]. In addition to the elderly, the impact of vaccines on children is gradually emphasized. Recently published Phase III clinical trials show that mRNA-1273 [51‐53], BNT162b2 [54], and BBIBP-CorV [55] are safe in populations younger than 18 years and trigger an immune response no less than that in young people. There is a lack of large-scale clinical trials to support the active use of COVID-19 vaccines for other populations, such as pregnant women, immunodeficient patients, and people that were previously exposed to SARS-CoV-2.

Our review has some limitations; the above results should be cautiously interpreted since inconsistencies were not assessed in the absence of trials that directly compared the two COVID-19 vaccines. The transitivity assumption underlying the NMA was evaluated by comparing key clinical features, including participant characteristics (age, sex, and race), and outcome assessment (definition and measurement). However, there are some differences in the research background and protocols, such as vaccine dose and different SARS-CoV-2 variants, which might lead to deviations in analytical results. Furthermore, vaccines face great challenges in terms of increasing the diversity of variants, and the ranking of VE can change. Booster vaccines are necessary to prevent SARS-CoV-2 variant infections and provide durable immunity. These data suggest that homologous and heterologous booster vaccines have an acceptable safety profile and heterologous boosting may be more immunogenic than homologous boosting [56]. Our conclusion aims to provide a primary reference for vaccine selection. However, other important factors such as the prevention of severe COVID-19, long-term side effects, and economic considerations should also be considered practical scenarios.

Our study is the most comprehensive NMA exploring the efficacy and safety of type-specific and individual COVID-19 vaccines based on the latest data. Our analysis showed that BIV1-CovIran inactivated vaccine had the lowest incidence of SAEs in adults, and BNT162b2 mRNA vaccine had the highest efficacy in preventing symptomatic SARS-CoV-2 infections in adults and the elderly population.