Immunizations are an important tool to reduce the burden of vaccine preventable diseases and improve population health.¹ High-quality immunization data is essential to inform clinical and public health interventions and respond to outbreaks of vaccine-preventable diseases. However, the de-centralized approach to public health service delivery, disparate state and local public health authorities, and multiple parties responsible for providing vaccination services, make high quality data collection challenging. Vaccine administration data is collected by systems at the point of care including electronic health records and pharmacy systems. For most vaccination providers, these data are then consolidated by jurisdiction public health authorities in Immunization Information Systems (IIS). IIS are confidential, population-based, computerized databases that record all immunization doses administered by participating providers to persons residing within a given geopolitical area.² IIS exist within Immunization Programs and play a critical role in providing data related to vaccine ordering and inventory management, clinical decision support (evaluation and forecasting), reminder/recall capabilities for patient follow-up, consolidated immunization histories, proof of immunity, and vaccination coverage estimates for specific geographic areas and population groups.³

IIS are governed by a complex array of state, territorial, and local laws and policies for patient consent and provider reporting; these specify which providers must report to an IIS, the types of data that can be collected, and what data may be shared for public health purposes⁴. IIS played a critical role in the COVID-19 pandemic response but by themselves were not sufficient to provide timely and complete information for public health decision making at the national level.

Although the Centers for Disease Control and Prevention (CDC) has historically provided funding for IIS via the 317 and Vaccines for Children Program Immunization Cooperative Agreement and has established the standards for IIS functionality and data exchange, it has not broadly collected data from IIS and did not have all the necessary infrastructure in place to do so. However, the COVID-19 pandemic required unprecedented access to vaccine ordering systems and vaccination data to inform rapid response, identify areas of need, track vaccine inventory, and enable efficient and timely vaccine distribution. To support the CDC COVID-19 Response, Vaccine Task Force, IIS functionality⁵ and data standards for interoperability⁶, standard vocabulary/vaccine code sets⁷, providers’ and partners’ systems, and CDC infrastructure were leveraged to develop a complex data and technology ecosystem (Table 1). This ecosystem enabled a streamlined process for vaccine ordering, distribution, administration, and vaccine data monitoring across the public health systems, including the ability to track provider enrollment, vaccine orders, inventory, distribution, wastage, and administration.

CDC established an integrated network of information systems to track COVID-19 vaccines and vaccinations. The network included entities that were participating in the COVID-19 federal vaccination program such as health care provider systems, health information exchange systems, IIS, pharmacy and dialysis systems, vaccine ordering systems, electronic health records (EHRs), and tools to support mass vaccination clinics. Data from all of these systems were reported either directly by the participating entity or indirectly via IIS to CDC’s COVID-19 data collection ecosystem, where it was processed, analyzed, and disseminated. In some cases, entities reported data directly to CDC (e.g., federal agencies) because they were unable to onboard to multiple IIS within the required timeframe, or they reported to both their IIS and to CDC to provide additional insight into distribution through different channels (e.g., pharmacy specific distribution). Establishing this complex ecosystem required both traditional and non-traditional partnerships, including vaccine manufacturers, data publishers, vendors (e.g., IIS, EHRs, and payers), providers (e.g., federal agencies, pharmacies, and dialysis centers), and data exchange partners (e.g., third-party entities that facilitate data exchange between partners such as between pharmacy chains and IIS, see Table 2).

As of May 11, 2023, which was the end of the COVID-19 public health emergency declaration, these systems have tracked:

This paper explores the systems, processes, and policies that enabled monitoring and reporting of COVID-19 vaccination efforts nationally and shares challenges and lessons learned for future public health emergency responses.

To address the scope and complexity of the pandemic response, public health technology infrastructure needed enhancement to support COVID-19 vaccination efforts across the country and ensure that data were available to drive public health decisions, such as equitable vaccine distribution. CDC assessed data requirements and conducted an analysis of existing capabilities in spring 2020. The analysis identified both CDC-owned systems and external systems that would have a role in implementing the COVID-19 Vaccine Program.

In anticipation of vaccine availability, CDC identified data requirements in areas including:

By comparing the data requirements and the analysis of existing capabilities (see Table 1), CDC developed a coordinated strategy to link and enhance existing systems and to develop new systems to fill gaps such as collecting, processing, and analyzing data at the national level and monitoring ordering, distribution, and inventory. The IT ecosystem was developed and deployed in December 2020 (Table 1). This strategy ensured coordination among multiple system types. The ecosystem accommodates numerous data flows and related processes, different system languages, differing approaches to implementing the same specification standards, multiple data transport mechanisms, integration of new partners to support their COVID-19 response, and varying resource challenges.

Tracking COVID-19 vaccines and administration was possible because decades of work resulted in a strong foundation for immunization data capture and exchange.⁸ This included jurisdiction IIS; CDC’s ordering and inventory management system, Vaccine Tracking System (VTrckS); FluFinder/VaccineFinder, a publicly available system for determining the availability of specific vaccines in communities; Vaccine Adverse Event Reporting System (VAERS); and data and technology standards. In addition, CDC developed multi-tiered emergency response policy requirements to support data collection for every dose of vaccine, new and expanded partnerships, and deployment of a multi-disciplinary team with the ability to design, build, and launch systems and services rapidly and iteratively.

Data collection at the national level was essential to ensure effective use of federally purchased vaccine, monitor vaccine uptake and inventory, make informed decisions about vaccine allocation, and inform public health interventions. Data collection was governed by clear reporting requirements for all providers involved in the response. The reporting requirements were specified in COVID-19 Vaccine Program provider agreements. The universal reporting requirement for all providers participating in the COVID-19 vaccine program enabled standardized, complete, and timely data collection for every dose of COVID-19 vaccine, which was not possible pre-response due to the constraints posed by disparate state and local laws. The implementation of the universal provider agreement facilitated the reporting of every dose of vaccine administered to the jurisdiction’s IIS or a system designated by CDC (e.g., COVID-19 Data Clearinghouse or the Immunization Data Lake). The provider agreement closed reporting gaps for jurisdictions with limited or no requirements for providers to report data to the jurisdictional IIS. Without this requirement, gaps may have existed in data from specific provider types or for specific age groups (e.g., children only) in some jurisdictions. The provider agreement required providers to report COVID-19 vaccinations to the IIS within 72 hours of administration. This requirement was critical to ensuring accurate, timely state-based assessment of COVID-19 vaccine uptake and timely information at the national level.

Data submission by jurisdictional IIS and other partners to the CDC, and how it was used, were facilitated through detailed legal agreements between the CDC and each participating entity. Jurisdictions signed data use agreements (DUA)⁹, and federal agency, pharmacy, and dialysis partners signed memoranda of agreement (MOA), that described data reporting requirements, data privacy and security, data usage, and aligned business processes with appropriate information systems, which formed the CDC COVID-19 technology ecosystem.¹⁰ Jurisdictions engaged with CDC to adjust DUA language when state laws required more limited data sharing. Changes to the standard DUA were identified as redlined agreements. In some cases, redlined agreements resulted in specific data elements being redacted or adjusted within line level records, for example, submitting month and year of birth instead of month, day and year of birth. In a few cases, these redlined agreements resulted in the submission of aggregate data (counts of vaccinations) for various demographic characteristics by the jurisdiction rather than de-identified record level data. Entities signing MOAs with the CDC in some cases also redlined the MOAs to limit the data submitted to reflect the sensitive nature of information related to personnel or locations. Data submitted to CDC by jurisdictions and other partners were a subset of the data received from participating providers. The data set excluded personally identifiable information and represented the minimum data set necessary to monitor efforts to intervene in a pandemic and ensure accountability for federally purchased vaccine.

CDC developed the COVID-19 Vaccine Reporting Specifications (CVRS) that defined data reporting requirements and the format for COVID-19 vaccination data submissions.¹⁰ The CVRS contained a limited subset of the data elements collected by providers during a vaccination event, including demographic and vaccination information. IIS and response partners submitted redacted (i.e., de-identified) vaccine administration data daily to CDC through a new system, the COVID-19 Data Clearinghouse (DCH) according to the CVRS. Providers were also required to place orders and report on-hand inventory using designated systems and specifications. Reporting specifications and supporting systems were updated as new vaccines became available throughout the response.

New and expanded partnerships were critical to mount an effective nationwide vaccine campaign, resulting in more complete and timely data, rapid development and publication of required vaccine codes, and improved data exchange for all vaccine administration data. CDC established a dedicated team to work directly with the Biomedical Advanced Research and Development Authority (BARDA) and the Food and Drug Administration (FDA) to assure rapid confirmation of new National Drug Codes (NDCs). Vaccine administration codes (CVX) and vaccine manufacturer codes (MVX) were documented and made available to all partners throughout the response effort to standardize data collection and reporting.

The CDC developed unprecedented partnerships with large chain and independent pharmacy organizations, dialysis centers, and federal agencies to expand the community-level reach and accessibility for COVID-19 vaccine administration and establish reporting of vaccination data to CDC for the first time. Early release and publication of vaccine code sets were critical in assuring that the data management systems used by electronic health record vendors, pharmacies, provider offices, jurisdictional IIS, and federal partners were prepared in advance of emergency use authorizations (EUA) to schedule appointments, administer COVID-19 vaccines, document the correct clinical information for each administration, and report the information to the IIS and CDC. The CDC worked closely with the American Medical Association (AMA) to assure the most current procedural terminology (CPT) was available upon authorization to complete billing workflows for vaccine administration. CDC also worked with the National Council for Prescription Drug Programs (NCPDP) to share information as quickly as possible and support NCPDP in releasing and updating guidance to pharmacies on COVID-19 vaccine billing and reimbursement claims for vaccine administrations. The rapid developments and updates to COVID-19 vaccines required constant engagement with these partners, and continuous communication to all partners to assure current, correct coding was used to document COVID-19 vaccines and vaccinations.

CDC’s COVID-19 Data Monitoring and Reporting team was highly engaged with the Advisory Committee on Immunization Practices (ACIP) COVID-19 workgroup. This partnership included providing data to support ACIP decision making for vaccine recommendations and translating ACIP recommendations into Clinical Decision Support for Immunization (CDSi) resources. The CDC publishes CDSi resources to support the evaluation and proper forecasting of vaccinations for individuals receiving COVID-19 vaccines. Partners use CDSi to update their vaccine forecasting algorithms, manage vaccination reminder and recall efforts, and ensure patients received the right vaccine at the right time, including completing the vaccination series and booster doses as appropriate.

In addition, CDC established a data management and governance structure that:

A multi-disciplinary team comprised of public health practitioners, informaticians, analysts, and epidemiologists worked collaboratively to design, build, and launch the COVID-19 vaccine data, technology ecosystem and business processes within a six-month period prior to vaccine availability. The ecosystem leveraged existing public health and health information technology systems and services whenever possible, but, in some cases, new tools were developed by the CDC and technology partners to address gaps. The highly integrated ecosystem includes systems and data flows for provider enrollment, vaccine ordering, distribution, administration, and vaccine and data disposition. These systems include:

Figure 1 shows the interactions between the COVID-19 vaccination reporting data systems and the different system users and Table 1 describes how each system was used to support the CDC COVID-19 response.

To operationalize the CDC COVID-19 Response, Vaccine Task Force monitoring effort, CDC dedicated teams to work with each partner in the response to assure the information systems and processes they used were technically ready to meet vaccination data reporting requirements. In addition, CDC assisted each partner with initiating accounts and establishing system access to complete the necessary business processes associated with the response. This work included verifying the identity of users for each partner, establishing two-factor authentication, processing account requests, and defining terms of agreement as needed for each system. Ongoing support for users was provided by a complement of help desk and technical assistance staff that system users could contact for assistance in troubleshooting system issues. For the 64 jurisdictional IIS, the first line of assistance was often a team of IIS subject matter experts who could direct users to the appropriate help desk, coordinate across teams supporting the response, and pursue information the jurisdiction needed to successfully partner in the response. Similarly, technical assistance was provided to federal agency, pharmacy, and dialysis partners by a dedicated team of subject matter experts.

Throughout the response, technical assistance was provided to participating entities to assure the highest quality vaccination data were reported and available for decision making. CDC developed data quality reports for federal agency, pharmacy, dialysis, and jurisdictional IIS partners. Each partner received weekly data quality reports that reflected missing, invalid, or illogical values submitted for the previous week’s data submissions and recommendations for approaches to address the identified issues. These reports were then used to determine the need to update previously submitted data. The DCH was enhanced to allow for both updates for incorrect or invalid values and deletion of duplicate records to improve data quality. The process for using updates and deletes to improve data quality included a pre-submission validation check by the organization, notification to response leaderships of the impact of the changes at the state level, and notification to impacted states of the pending change the week prior to the updates or deletes being processed in the production environment. Federal agency, pharmacy, dialysis, and jurisdictional IIS partners used this functionality beginning in July 2021 as a mechanism for data quality improvement. These efforts required close coordination between the jurisdiction, IIS subject matter experts, federal agency, pharmacy, dialysis centers, DCH, and IZDL teams for completion. Additional coordination was required with the Tiberius and CDT teams to manage updates to outward-facing systems, assure proper footnoting, and support responses to internal and external queries. The high-quality data resulting from this effort supported confidence for the use of vaccine administration data to enhance survey data collected through the COVID modules of the National Immunization Survey (NIS).

The unprecedented collection and visibility of vaccination data from multiple partners, complexities of data gathering, and data sharing guided by DUAs and MOAs resulted in many opportunities to identify and address concerns about discrepancies between data presented on state dashboards and data presented on CDT. The high visibility resulted in engagement with the media, state and local government officials, jurisdiction leadership, and other stakeholders. Outcomes of these inquiries included the development of resources to add clarity to the data displayed on CDT and the contribution of each group of participating providers. In addition, to support jurisdiction planning efforts, increased data visibility was added in the Tiberius platform that allowed jurisdictions to see federal agency data impacts at the county level, with detail that supported health equity and series completion goals. In-depth data reviews and follow-up were also provided to jurisdictions if specific concerns about data discrepancies arose. CDC used in-depth data reviews to work with jurisdictions to identify and resolve IIS related data discrepancies as well.

The policy framework to secure reporting from the diverse stakeholders participating in the COVID-19 vaccine response required several data sharing agreements and MOAs for each component of the COVID-19 vaccine response, including vaccine administration, distribution, and vaccination provider reporting to jurisdictions. The level of effort required to process multiple agreements, negotiate them with 64 jurisdictions and other participating partners, and complete the signatory process was time-consuming, confusing for participants, and required involvement of a significant number of staff within and across agencies. In addition, pursuit of the response related reporting agreements came amid requirements for agreements supporting data exchange between various partners of the response. This process identified a need for consolidating the data sharing and data exchange agreements. The result was the creation of the consolidated data use agreement between the CDC and each jurisdiction that included COVID-19 vaccines but also covered the submission of routine vaccination data, seasonal influenza vaccination data, and reporting requirements for outbreaks and pandemic events. In addition, CDC, in collaboration with partners, streamlined legal agreements required for participation in the IZ Gateway. The IZ Gateway consolidated data exchange agreement allows multi-jurisdiction vaccination providers and jurisdictional IIS to sign one time to allow data exchange to occur with all jurisdictions and multi-jurisdictional providers who also signed the agreement. The establishment of consolidated agreements and expansion of the IZ Gateway functionality has positioned the CDC to be proactive in allowing rapid implementation of vaccination data submission and reporting now and in the future.

After legal agreements were negotiated and data submission began, it was essential to have processes in place to reflect data quality back to jurisdictions, federal agencies, pharmacies, and dialysis partners, as well as to have a way to improve data quality. The use of the DCH update and delete process became a routine aspect of improving data quality for many partners. Overall, during the response, 55 entities utilized the update and delete functionality 263 times. About 260 million records were submitted through this process for update or deletion over the course of the response, leading to a more complete and accurate depiction of the data associated with the national COVID-19 response effort.

Existing standards for immunization data exchange¹² are broadly used by IIS but flexibility within the existing standards and variations in how the standards are implemented across systems inhibit the scope of data exchange needed to support a global pandemic response. CDC’s IZ Gateway provides the technical capability to exchange data between jurisdictions, from multi-state providers to jurisdictions, and from jurisdictions to consumers, but more work is needed to broaden its use and leverage the efficiencies it offers. CDC, federal partners, and jurisdictions have made significant strides in data exchange using the IZ Gateway as a result of COVID-19 investments. However, barriers persist. The allowable variances in the current data exchange standards add complexity to establishing connections between exchange partners. This complexity requires significant resources, time and sometimes development work by one or both exchange partners. There is significant variance in the technical capability and standardization of IIS. Not all systems meet the CDC’s IIS Functional Standards or operate using the national HL7 Implementation Guide.¹¹ There is not currently a certification process for IIS technology.⁴ Connecting to the IZ Gateway requires compliance with standards and the IZ Gateway onboarding process has accelerated improvements in IIS. However, this process has also highlighted additional efforts needed to achieve better alignment.

Some jurisdictions have legacy IIS that need replacement or modernization. Without critical modernization, not only will IIS be unable to fully leverage the efficiencies the IZ Gateway connection offers, but they will also not be able to leverage emerging standards and technology that accelerate interoperability. Federal providers have some unique challenges. Not all federal agencies use certified health IT, in part due to some of their unique use cases. As a result, some federal agency immunization systems lack the technical capability needed to exchange data using existing standards and/or to onboard the IZ Gateway and share data with jurisdiction IIS and other federal partners. The IZ Gateway has and continues to identify and implement process and technical improvements to gain efficiencies. However, to achieve efficient, seamless, and scalable interoperability, more consistency in the application of immunization exchange standards, standardization of IIS technology, and modernization of connecting systems is needed.

The collection and processing of daily COVID-19 vaccination data was only made possible by the expansion of IZDL. The IZDL is a cloud-based data processing repository capable of ingesting and processing an array of vaccine response-related data, including vaccine administration data from jurisdictions, pharmacies, dialysis centers, and federal agencies. Vaccine administration data processing included the transformation of information obtained from the CVRS data files into vaccination coverage and doses administered data for the externally facing CDT. The public and other government entities (including the White House) used CDT to monitor COVID-19 vaccination coverage. It was also integrated with other critical vaccine response data sources (e.g., cases, hospitalizations, deaths, and vaccine safety) to inform critical public health communications. In addition, IZDL fed data to response systems such as Tiberius, which was used by jurisdictions and federal agencies to monitor key metrics related to orders, distribution, inventory, and administration.

Vaccination data collection is de-centralized through provider and public health systems and historically very little routine vaccination data flowed to the national level. Data that is shared with CDC is de-identified, which poses challenges for patient matching, de-duplication, and linkage with other data sets to develop a more complete picture of population health. Due to policy restrictions at the state level, some jurisdictions were restricted to sharing aggregate counts of COVID-19 vaccination events rather than more detailed data on each administration. Submitting aggregate data was a time-consuming process for both the impacted jurisdictions and CDC staff who had to extract the data, use algorithms to process the data consistent with other data being processed by CDC, and integrate it with other data submissions, often using manual processes. The COVID-19 pandemic highlighted opportunities to improve access to and usefulness of vaccination data by implementing technology such as Privacy Preserving Record Linkage (PPRL) that results in higher quality linked data without identifying individuals^13,14. PPRL technology has broader applicability than immunization data and would enable linkage of deidentified data sets across data domains to inform public health decision making such as combining nationally notifiable disease surveillance data with immunization data to understand areas where there may be increased risk of a vaccine preventable disease outbreak and populations at greater risk.

The unprecedented tracking of the COVID-19 response was only possible because of the foundation laid by years of work by local, state, territorial public health agencies federal agencies, and their partners. Continued investment in the infrastructure that supported the COVID-19 response is critical for future public health emergencies and to support routine immunization activities to prevent vaccine preventable diseases. Many routine immunization activities can leverage the same technology resources and can benefit from the same data requirements and policies that were used for the COVID-19 response so that data can be collected in a consistent, timely manner and be used to inform public health decision making (Figure 2). This would ensure that knowledge and expertise are available to prepare for the next public health emergency. Investments in technology and immunization processes made prior to 2019 (e.g., IIS, VTrckS, IZ Gateway) were leveraged to support the response. Use of existing technology and processes allowed jurisdictions to focus on scaling systems and processes to meet response needs. Post-response systems established to fill gaps are being reviewed to determine if they are still needed, and if so, make the appropriate investments to incorporate them into infrastructure that supports routine immunizations across the lifespan and prepare for the next pandemic response. Ongoing investment in systems used to support business processes, data exchange, and data management needed to operationalize immunization programs and monitor vaccine distribution, uptake, and coverage are necessary to ensure public health’s ability to rapidly scale up for the future public health emergencies.