COVID vaccine surveillance efforts are a global priority, but safety monitoring for vaccines should not reflect a single global population. The largest international network study ever completed on the background rates of adverse events of special interest (AESI) being monitored in vaccine surveillance efforts identified that these rates vary substantially by age, sex, and database.

Led by researchers at Oxford University, Columbia University, Erasmus MC, UCLA, and Janssen, an international team of collaborators within the Observational Health Data Sciences and Informatics (OHDSI) network provided a timely reference of the background rates of AESIs in the recent study “Characterising the background incidence rates of adverse events of special interest for covid-19 vaccines in eight countries: multinational network cohort study” published June 14 by The BMJ (ADD LINK).

There were significant differences in the observed rates of AESIs based on the age groups and sex of more than 126 million people across four continents and 13 total databases in this observational study. Furthermore, differences were observed across people in distinct databases.

This analysis provides historical context for how often outcomes happen in the general population, and can facilitate comparisons with what is observed among those vaccinated. The findings, which suggest caution and adjusted analysis will be needed in vaccine safety analyses to avoid misleading conclusions, can support international efforts aimed at monitoring the safety of COVID vaccines.

“We knew regulators would be monitoring a long list of events for the surveillance of COVID vaccines safety,” said co-senior author Dani Prieto-Alhambra MD MSc PhD, Professor of Pharmacoepidemiology at the University of Oxford. “To do this, they need robust estimates of the background rates of these events in historical data. These results can be used as benchmark for the monitoring of these potential safety events and for any upcoming COVID-19 vaccines.”

There were 15 prespecified adverse events studied, matching those being monitored by the U.S. Food and Drug Administration and similar to those used by other top regulatory agencies, which include heart attack, stroke, and blood clotting. Incidence rates were classified by age groupings and gender across the 13 databases, though the outcomes of those specific groupings would even vary by database.

“We found significant heterogeneity in background rates between age and sex,” said co-lead author Xintong Li, DPhil student and Clarendon scholar at the University of Oxford. “If we compare these rates regardless of age or sex group, we may either find a false signal or neglect a real safety signal while monitoring vaccine surveillance. “The observed and expected rates comparison should also be conducted within the same health database whenever possible,” Li added. “While we understand that is not possible for all surveillance systems or vaccine safety studies, choosing a similar population and stratifying or standardizing by age and sex is highly recommended.”

Heart attack, for example, was observed as a very rare (<1/10,000) outcome for a 24-year-old female, but it was a common (<1/10 to ≥1/100) one for an 88-year-old male. The research team believes that the populations who are more likely to suffer these AESIs (like the older man in this example) should be analyzed separately from populations in much lower risk groups.

 “If a vaccinated population is older than an unvaccinated population, and we do not adjust for it, we may see a false increased risk of events following vaccination,” said co-lead author Anna Ostropolets MD, a PhD student in the Columbia University Department of Biomedical Informatics.The variability between different databases could reflect numerous factors, ranging from the process of data capture to population differences such as socioeconomic status and comorbidities. “The observed heterogeneity between databases was more than I expected,” Prieto-Alhambra said. “As a consequence, vaccine safety surveillance should be conducted using the same database for both post-vaccine and background rates.”

This multinational network cohort study used deidentified electronic health records and health claims data, all of which were mapped to the OMOP common data model, from eight countries. All rates of adverse events, as well as the protocol and study codes, are available in the publication.

The study was developed and executed by the OHDSI community, a global, multi-stakeholder network that has more than 2,100 interdisciplinary researchers from six continents, as well as standardized health records for nearly 600 million unique patients around the world. The community strives to promote better health decisions and care through globally standardized health data, continuously developing large-scale analytics and a spirit of collaboration though open science.

OHDSI research on COVID-19 has resulted in published studies on the safety profile of hydroxychloroquine, the characteristics and outcomes of more than 34,000 COVID-19 patients, the varied therapies used around the world for COVID treatment, as well as several others in both peer-reviewed journals or on preprint servers.

Funding:

This work was partially funded by the UK National Institute for Health Research (NIHR), European Medicines Agency, European Health Data and Evidence Network (EHDEN), US Food and Drug Administration CBER BEST initiative (75F40120D00039), and US National Library of Medicine (R01 LM006910). EHDEN has received funding from the Innovative Medicines Initiative 2 Joint Undertaking under grant agreement No 806968. The Innovative Medicines Initiative 2 Joint Undertaking receives support from the European Union’s Horizon 2020 research and innovation programme and EFPIA. The study funders had no role in the conceptualization, design, data collection, analysis, decision to publish, or preparation of the manuscript. This document expresses the opinion of the authors of the paper, and may not be understood or quoted as being made on behalf of or reflecting the position of the European Medicines Agency or one of its committees or working parties.

While fatality was fortunately rare, complications including hospitalization, hypoxemia and pneumonia were more frequent in children and adolescents either diagnosed or hospitalized with COVID-19 than with seasonal influenza. Furthermore, labored breathing, loss of smell and gastrointestinal issues were more prevalent symptoms for younger people inflicted with COVID-19 than with influenza.

These findings, along with that of significant treatment heterogeneity for children/adolescents hospitalized with COVID-19, were presented in the study “30-day outcomes of Children and Adolescents with COVID-19: An International Experience,” published May 28 by Pediatrics.

Early in the pandemic, opinions around the COVID-19 impact on children and adolescents ranged from it being no more than the common flu to fear of its potential impact on lesser-developed immune systems. This OHDSI global network study compared the real-world observational data of more than 242,000 children/adolescents diagnosed and nearly 10,000 hospitalized with COVID-19 to more than 2,000,000 diagnosed with influenza across five countries (France, Germany, South Korea, Spain, United States) to provide a clearer picture of its impact.

Asthma and obesity were the most common baseline comorbidities — a common finding in disease prevalence among a general pediatric population — in the studied databases, but there was a higher prevalence of rare conditions, including congenital malformation/s, neurodevelopmental disorders, and heart disease, among those hospitalized with COVID-19. Pediatric patients with COVID-19 also showed higher rates of labored breathing, loss of smell and gastrointestinal symptoms than those presenting with influenza, which could help improve early diagnosis of COVID-19 among this population.

Adjunctive therapies were the most common treatment options in children/adolescents, though there was global heterogeneity on which particular therapies were used (systemic corticosteroids and famotidine were most common). The most common 30-day complications for hospitalized children/adolescents with COVID-19 were hypoxemia and pneumonia, both of which occurred at a higher rate than hospitalized influenza patients.

There was limited knowledge of the COVID-19 impact on children/adolescents around the world during the first half of 2020, when the OHDSI community collaborated on the CHARYBDIS Project. Findings at the time ranged from a 5.7 % hospitalization rate to another that reported a 63% hospitalization rate. There was a need for reliable evidence on the demographics, comorbidities, symptoms, in-hospital treatments, and health outcomes among children/adolescents to inform clinical decision-making.

“This study addressed critical questions that were weighing down on both the healthcare community and the general population — how was COVID-19 impacting our youngest population,” said study lead Talita Duarte-Salles, PhD, an epidemiologist at IDIAP Jordi Gol. “While some claimed that COVID-19 was no different than the flu early last year, the real-world evidence we generated through open science showed something quite different. It was relieving to see that fatality was rare, but clearly both complications and symptoms showed the COVID-19 was no flu in children and adolescents.”

The study was developed and executed by the OHDSI (Observational Health Data Sciences and Informatics) community, a multi-stakeholder, interdisciplinary network that collaborates globally to bring out the value of health data through open science and large-scale analytics. This study resulted from the CHARYBDIS Project, which set out to:

1) Describe the baseline demographics, clinical characteristics, treatments, symptoms and outcomes of interest among individuals with COVID-19 overall and stratified by sex, age and specific comorbidities

2) Describe characteristics and outcomes of influenza patients between September 2017 and April 2018 compared to the COVID-19 population

This project has resulted in several published studies, including ones on general COVID-19 phenotyping, patients with autoimmune disease, and use of repurposed and adjunctive drug therapies. Several others are undergoing peer review and have been posted to a preprint server; OHDSI’s work on COVID-19 can be found here.

“Generating reliable evidence that can inform clinical decision-making for children and adolescents was so important, and it doesn’t happen without collaboration and the foundation of open-source tools and practices developed for years in this network,” Duarte-Salles said. “It was truly inspiring the way our OHDSI community rallied together globally in the face of this unprecedented pandemic and collaborated together.”

The IPCI database participated in a study on the Validation of the COVID-19 Vulnerability (C-19) Index across an international network of observational healthcare datasets, published today in JMIR Med Inform

Abstract

Background: SARS-CoV-2 is straining health care systems globally. The burden on hospitals during the pandemic could be reduced by implementing prediction models that can discriminate patients who require hospitalization from those who do not. The COVID-19 vulnerability (C-19) index, a model that predicts which patients will be admitted to hospital for treatment of pneumonia or pneumonia proxies, has been developed and proposed as a valuable tool for decision-making during the pandemic. However, the model is at high risk of bias according to the “prediction model risk of bias assessment” criteria, and it has not been externally validated.

Objective: The aim of this study was to externally validate the C-19 index across a range of health care settings to determine how well it broadly predicts hospitalization due to pneumonia in COVID-19 cases.

Methods: We followed the Observational Health Data Sciences and Informatics (OHDSI) framework for external validation to assess the reliability of the C-19 index. We evaluated the model on two different target populations, 41,381 patients who presented with SARS-CoV-2 at an outpatient or emergency department visit and 9,429,285 patients who presented with influenza or related symptoms during an outpatient or emergency department visit, to predict their risk of hospitalization with pneumonia during the following 0-30 days. In total, we validated the model across a network of 14 databases spanning the United States, Europe, Australia, and Asia.

Results: The internal validation performance of the C-19 index had a C statistic of 0.73, and the calibration was not reported by the authors. When we externally validated it by transporting it to SARS-CoV-2 data, the model obtained C statistics of 0.36, 0.53 (0.473-0.584) and 0.56 (0.488-0.636) on Spanish, US, and South Korean data sets, respectively. The calibration was poor, with the model underestimating risk. When validated on 12 data sets containing influenza patients across the OHDSI network, the C statistics ranged between 0.40 and 0.68.

Conclusions: Our results show that the discriminative performance of the C-19 index model is low for influenza cohorts and even worse among patients with COVID-19 in the United States, Spain, and South Korea. These results suggest that C-19 should not be used to aid decision-making during the COVID-19 pandemic. Our findings highlight the importance of performing external validation across a range of settings, especially when a prediction model is being extrapolated to a different population. In the field of prediction, extensive validation is required to create appropriate trust in a model.

JMIR Med Inform 2021;9(4):e21547

doi:10.2196/21547

The European Medicines Agency (EMA) has signed a contract with IQVIA in partnership with Erasmus University Medical Center, together with partners from 6 European countries to investigate the treatment patterns on the use of ranitidine and alternatives of ranitidine. For this study, all clinical data are standardised to the Observational Medical Outcomes Partnership (OMOP) -Common Data Model, which is maintained by the Observational Health Data Sciences and Informatics (OHDSI) community (www.ohdsi.org). The use of the OMOP-CDM enables standardised analytics, tools, and methodologies.

Ranitidine reduces the amount of acid in the stomach and is available both on prescription and as over the counter drug in pharmacies and supermarkets in many European countries. It is indicated for heartburn, stomach ulcers and acid reflux, but also for rare indications such as reduction of gastric secretion in specific tumours. Novel findings in July 2019 indicated that ranitidine can generate N-Nitrosodimethylamine (NDMA) a substance that is potentially carcinogenic. In view of this, the European Commission triggered a referral procedure in September 2019 to evaluate the relevance of these findings, the potential root causes and their impact on the benefit-risk balance of medicinal products containing ranitidine. Based on this evaluation, in April 2020 EMA’s Human Medicines Committee recommended the suspension of all ranitidine-containing medicines in the EU until further notice which resulted in a recall of ranitidine in the different member states.

Due to the unavailability of ranitidine, many patients will switch treatment to alternative medicines. The extent of switches to alternative medicines remains unknown as well as the rate of patients permanently discontinuing ranitidine treatment. By means of an international drug utilisation study using electronic health care data from 6 EU countries, we will explore drug utilisation and prescription patterns of ranitidine and describe switching from ranitidine to alternative therapies both in the pre-referral and post-referral phase. This study is a follow-up study from a previous EMA initiated rapid study exploring cumulative exposure to ranitidine and other H2 receptor antagonists led by Erasmus University Medical Center (EMA/2018/21/PE).

This multinational project is led by IQVIA by Hanne van Ballegooijen, senior epidemiologist and Deborah Layton, Professor of Pharmacoepidemiology at University of Keele, UK and Director of Drug Safety in close in collaboration with Peter Rijnbeek, Associate Professor of Health Data Science and Katia Verhamme, Associate Professor of Use and Analysis of Observational Data, both from the Erasmus University Medical Center, Rotterdam. The study will provide an impact assessment of the EMA referral procedure and switching to alternative medicines for a widely prescribed medicine. We are delighted to conduct such an important project and provide insight into the drug utilisation patterns. The results are expected in 2 years – spring 2023.

At the Annual Meeting of the international Society for Pharmacoepidemiology our reseachers participated in a symposium organised by the European Medicines Agency on the topic of rapid real world studies for regulatory purposes. It was great to be able to demonstrate the power of the OMOP Common Data Model and standardised analytics for the Ranitidine use case led by Erasmus MC that included the IPCI data. 

Regulatory decisions on medicinal products often require valid and generalisable real-world evidence on their utilisation and risks. In some conditions, data are needed wihin a few weeks or months, such as for identification of the population at risk of serious adverse effects or targeted for risk minimisation, rapid surveillance and prevention of potential safety concerns, or management of public health emergencies (e.g. pandemic influenza). In suchcases, multiple challenges need to be met with regard to study design, data access, collection and analysis and result integration, and some of these steps must be accelerated to support timely decisions without affecting internal and external validity. Given the increasing public expectation for fast regulatory decisions, solutions may need to be further developed and introduced in routine practice.

This symposium presented regulatory circumstances in the EU and the US where rapid real-world studies were needed and described the challenges to be addressed and some available solutions. It will contribute to define the infrastructure, research environment, methodological approaches and the analytical system necessary for rapid studies, and benefit regulators, researchers and industry who need to quickly answer regulatory questions.

Xavier Kurz (Data Analytics and Methods Task Force, EMA) introduced the topic and moderated the panel.

Sabine Straus (MEB, Chair of the EU Pharmacovigilance Risk Assessment Committee) presented examples of questions from the PRAC Pilot on rapid data analytics and how they were addressed.

Michel Kwa (Senior assessor, MEB) explained the need for timely data in the situation of the EU Referral procedure on H2 receptor antagonists assessing the potential risk associated with N-nitrosodimethylamine (a human carcinogen).

Peter Rijnbeek and Katia Verhamme (Department of Medical Informatics, Erasmus University Medical Center) presented the challenges met and solutions found for an EMA-funded multi-centre study from 5 EU countries on this topic which was completed within 4 months, and will discuss the opportunities of this approach to answer other types of regulatory questions.

Simone Pinheiro (Office of Surveillance and Epidemiology, FDA) and Judith Maro(Harvard Pilgrim Health Care Institute) discussed how the Sentinel system is used in situations where rapid analyses are needed, including sequential surveillance, cohort definition and acceleration of time segments for different study phases.

The session was ended with a panel discussion on the types of questions amenable (or not) to rapid studies and the research environment supporting them.