We used data from the Overcoming COVID-19 Network, a multistate surveillance network that conducts analyses of VE against COVID-19-related hospitalizations [20]. This analysis included hospitalized children and adolescents aged 5–18 years admitted to 35 participating hospitals and medical centers in 25 states from May 16, 2021 through October 29, 2023. Site personnel screened medical charts and SARS-CoV-2 specimen test results daily at participating sites to identify hospitalized patients eligible for enrollment. Patients admitted to the hospital for an acute illness and who tested positive for SARS-CoV-2 by nucleic acid amplification test (NAAT) or antigen test at the time of admission were included as COVID-19 case-patients. Control patients hospitalized for COVID-19–like illness were matched to case-patients (1:1) by site, age group (5–11 years, 12–15 years, 16–18 years inclusive) and date of hospital admission (+/− 3 weeks), but received a negative SARS-CoV-2 test result by NAAT during or ≤ 7 days before hospital admission, with no positive NAAT/antigen test result <3 days after hospitalization. Patients with multisystem inflammatory syndrome in children (MIS-C) were excluded. During hospitalization or after discharge, baseline demographic characteristics, clinical information, SARS-CoV-2 testing history, and COVID-19 vaccination history were obtained through parent or guardian interviews, and review of the patient’s electronic medical record.

Proof of vaccination was collected from several sources. Parent-reported vaccination data, including the number of doses received and vaccine manufacturer, were collected through structured in-person or telephone interviews. Study personnel also reviewed state vaccination registries, electronic medical records, or sources including documentation from primary care providers, schools, and pharmacy records. Documented vaccination was defined as vaccination status recorded on one or more of the following sources: COVID-19 vaccine card, hospital electronic medical record, state IIS (viewed at the time of parent interview), and documentation from primary care providers. If parents provided COVID-19 vaccination record cards, information from the card was recorded separately from parent-reported vaccination data during the interview.

Patients were included in the analysis of concordance between parent-reported and documented COVID-19 vaccination status if their parent or guardian completed an interview reporting the number of vaccine doses received, including unvaccinated status (i.e., zero doses) and vaccine documentation sources were checked. If no doses were identified in the documented sources, the participant was classified as unvaccinated (zero doses). We described characteristics of vaccinated and unvaccinated patients stratified by parent-report vs. documented vaccination status using counts and percentages for categorical variables, and medians and interquartile ranges for continuous variables. Characteristics of SARS-CoV-2 test-positive case-patients and test-negative control-patients were described similarly.

Parental report of child vaccination status was compared to documented vaccination records (considered the gold standard for this analysis) using several metrics. Percentage agreement was defined as the proportion of patients whose parents reported the same number of vaccine doses as was recorded in documented sources. Kappa (κ) statistics were estimated to measure the degree of nonrandom agreement between the number doses reported by parents and those recorded in documented sources, with κ = 1 representing perfect agreement and κ = 0 indicating no more agreement than would be expected by chance. Sensitivity of parent-reported child COVID-19 vaccination history was estimated as the proportion of patients with ≥1 COVID-19 vaccine documented (denominator) whose parent also reported child receipt of ≥1 COVID-19 vaccine (numerator). Specificity was estimated as the proportion of patients with no documented vaccine (denominator) whose parent also reported him/her as unvaccinated (numerator). Positive predictive value (PPV) was calculated as the proportion of patients reported by parents as having ≥1 COVID-19 vaccine dose (denominator) who also had documented receipt of ≥1 COVID-19 vaccine dose (numerator). Negative predictive value (NPV) was calculated as the proportion of patients reported by parents as being unvaccinated (denominator) who also had no documentation of vaccination (numerator). These concordance measures were stratified by case and control status to assess potential differences in accuracy by SARS-CoV-2 infection status.

We assessed factors associated with discordance between parental report and documented vaccination information by comparing the proportions of patients with discordant vaccine history by demographic characteristics (age, sex, race/ethnicity, region), clinical characteristics (presence of ≥1 underlying medical condition, SARS-CoV-2 test result), and time between hospital admission and parent interview using discordance prevalence ratios (PR). Adjusted PRs were estimated using multivariable log binomial regression including age group (5–11 years, 12–18 years), sex, race and ethnicity (non-Hispanic White, non- Hispanic Black, Hispanic or Latino, multiple or other races, or unknown), social vulnerability index (SVI, a continuous measure ranging from 0 to 1 calculated based on ZIP code of residence and categorized as low, medium, high) [21], Census region, presence of ≥1 underlying medical condition, days from admission to parent interview (≤7 days, 8–50 days, >50 days), and SARS-CoV-2 infection status at admission. We used the Wilcoxon rank sum test to compare distributions of days between hospital admission and parent interview for patients with discordant versus concordant vaccine status.

To assess whether VE estimates were similar across data sources, we compared VE estimates generated using (1) parent-reported data only and (2) documented sources only. The subset of patients included in the VE analysis differed from that of the concordance analysis (Fig. 1). For inclusion in the VE analysis using parent-reported data, we required participants’ parents to provide the approximate date of the last dose to confirm that it was administered at least 14 days before hospital admission. For both VE analyses, vaccination status was classified as (1) unvaccinated (no vaccine doses received before hospital admission), (2) vaccinated with 2 doses (second dose of a two-dose mRNA vaccine series received ≥14 days before hospital admission, 99 % original formula), or (3) vaccinated with ≥2 doses of an mRNA vaccine with the second dose received ≥14 days before hospital admission or the third dose received ≥7 days before hospital admission. Numbers were insufficient to estimate VE for ≥3 doses or for the bivalent COVID-19 vaccine; no patients had received an updated (2023-2024 formulation) monovalent dose.

VE was estimated by comparing the odds of COVID-19 vaccination among SARS-CoV-2 test-positive case-patients to the odds of vaccination among test-negative control-patients (VE = [1 – adjusted odds ratio] x 100 %). Multivariable logistic regression models included hospital site as a repeated effect using generalized estimating equations to account for clustering within hospital site, and adjusted for the presence of ≥1 underlying medical condition, age (in years), bimonthly interval of hospitalization, U.S. Census Bureau region of hospital, SVI, and race and ethnicity. We estimated VE for the entire study period and by period of SARS-CoV-2 variant predominance defined as follows: Delta (May 16, 2021–December 18, 2021); Omicron BA.1 (December 19, 2021–March 19, 2022); and Omicron BA.2 and subsequent subvariants (March 20, 2022–October 29, 2023).

SAS software (version 9.4; SAS Institute) was used to conduct VE analyses. R (version 4.4.0) was used for all other analyses. This activity was reviewed by CDC, deemed not research, and was conducted consistent with applicable federal law and CDC policy.²

From 16 May 2021 through 29 October 2023, 5496 inpatients aged 5–18 years without MIS-C were enrolled in the Overcoming COVID-19 surveillance network; children aged 5–11 years enrolled prior to November 2021, before FDA approved and CDC recommended COVID-19 vaccines for this age group, were excluded (n = 12) (Fig. 1). Test-positive case-patients and test-negative controls were evenly distributed over time (Fig. 2). The concordance analysis was restricted to 3262 (59 %) patients for whom both parent-reported and documented COVID-19 vaccination information was available (Fig. 1). Hospitalized patients included in the concordance analysis had a median age of 13 years (IQR: 9–16) and 49 % were female (Table 1). Most patients (76 %) had ≥1 underlying medical condition.

Of the 3262 children included in the concordance analysis, 1274 (39 %) were reported by parents to have received at least one dose of the COVID-19 vaccine. This proportion was higher compared to those confirmed through verified documentation (n = 1068, 33 %). However, demographic and clinical characteristics of patients classified as vaccinated (or unvaccinated) according to parent report were similar to those classified as vaccinated (or unvaccinated) according to documented sources. Similarly, demographic characteristics were similar between case-patients and test-negative control-patients (Table 2). However, case-patients were more likely to have an underlying medical condition (80 % vs 72 %, χ² p < 0.01) and to be unvaccinated according to documented sources (75 % vs 60 %, χ² p < 0.01) and parental report (68 % vs 54 %, χ² p < 0.01).

Among 3262 patients with both parent-reported and documented COVID-19 vaccination information, percent agreement between the number of parent-reported and documented vaccine doses was 88.0 % (95 % CI: 86.9–89.1 %), with a kappa of 0.77 (95 % CI: 0.75–0.79) (Table 3). Most discordant pairs (346/390) were because of parents reporting more doses than were found in documented sources (Table 4, Supplementary Table), which was reflected in the high sensitivity (98.2 %) and lower specificity (89.7 %) of parent report (Table 3). Parents reported an average of 0.8 doses (SD: 1.1) compared to a documented mean of 0.7 doses (SD: 1.0). Percent agreement was highest in the earlier periods at 89.6 % (95 % CI: 87.8–91.3) and 91.1 (95 % CI: (89.3–93.0) in the Delta (16 May 2021–18 Dec 2021) and Omicron BA.1 (19 Dec 2021–19 Mar 2022) predominant periods, respectively, but declined with time to 84.1 % (95 % CI: 82.0–86.1) when the Omicron BA.2 SARS-CoV-2 variant became predominant (20 Mar 2022–29 Oct 2023). A similar trend was observed with a kappa statistic of 0.77 (95 % CI: 0.74–0.81) and 0.84 (95 % CI: 0.80–0.87) in the Delta and Omicron BA.1 predominant periods, respectively that declined to 0.72 (95 % CI: 0.69–0.76) when Omicron BA.2 became predominant.

We explored whether the time between parent interview and hospital admission influenced the observed decline in agreement over time. The number of days between admission and parent interview increased from 18 days (IQR: 4–60) in the Delta period to 36 days (IQR: 5–137) by the Omicron BA.2 period. The median time between hospital admission and parent interview was greater for children with discordant vaccine status (58 days [IQR: 18–131]) than for those with concordant status (18 days [IQR: 3–63]) (p < 0.01). When stratified by time between hospital admission and parent interview, the trend of lower percent agreement between the Delta and Omicron BA.2+ periods was evident among participants interviewed within 7 days of hospital admission, with percent agreement declining from 97.9 % (95 % CI: 96.4–99.5) during the Delta period to 84.9 % (95 % CI: 81.0–88.8) during the Omicron BA.2+ period. A similar, though less pronounced, decline was observed among participants interviewed 8–50 days after admission, with percent agreement decreasing from 93.0 % (95 % CI: 90.3–95.8) in the Delta period to 86.7 % (95 % CI: 82.9–90.6) in the Omicron BA.2+ period. However, among participants interviewed more than 50 days after admission, no clear trend was observed; percent agreement fluctuated across periods: 78.0 % (95 % CI: 73.3–82.8) during the Delta period, 83.4 % (95 % CI: 78.3–88.6) during the Omicron BA.1 period, and 80.5 % (95 % CI: 76.9–84.0) during the Omicron BA.2+ period.

The proportion of children reported by their parents to have received ≥1 vaccine dose who also had documentation of receiving ≥1 vaccine, was lower for case-patients (PPV: 76.9 % [95 % CI: 73.0–80.3]) compared to control patients (PPV: 85.8 % [95 % CI: 83.1–88.1]), indicating more overreporting by parents of case-patients than by parents of control patients (Table 3). However, this difference between case-patients and control patients declined following the Delta period. The proportion of children reported by their parents as having received no vaccines who also had no documented vaccines, was high (>98 %) for both case-patients and control patients across all time periods, indicating a high level of accuracy in parental reporting when children were unvaccinated.

In addition to the time between hospital admission and parent interview, region was associated with discordance between parent report and documented vaccination (Table 5). Compared to patients in the South, the proportion of patients with discordant vaccination status was 1.9 (95 % CI: 1.4–2.4) times as high in the West and 1.9 (95 % CI: 1.4–2.5) times as high in the Northeast. Discordance did not differ by age, sex, race or ethnicity, SARS-CoV-2 infection status, or with the presence/absence of ≥1 select underlying medical conditions.

Of 5496 children and adolescents hospitalized from 16 May 2021 through 29 October 2023, 5215 (94.9 %) were included in the VE analyses. Among these 5215 patients, 3049 (58.5 %) were included in VE estimates using parent-reported vaccination status and 5143 (98.6 %) were included in the VE analysis using documented vaccination status (Fig. 1). Demographic and clinical characteristics of patients included the two VE analyses were similar (Table 6).

VE of two vaccine doses against COVID-19-related hospitalization was 57.8 % (95 % CI 49.5–64.7) for the entire study period when using vaccination status from documented sources and 59.5 % (95 % CI 50.3–67.0) when using vaccination status reported by parents (Table 7). During this period, the median time from the most recent vaccine dose to hospital admission was 167 days (IQR: 86–288) among patients with documented vaccination status (for whom date of vaccination was available). Over each successive variant predominant period, the time from most recent vaccine dose to hospital admission increased coinciding with a decline in VE. Among patients with documented receipt of ≥2 doses, the median time from most recent COVID-19 vaccine dose to hospitalization was 98 days (IQR: 59–139) in the Delta predominant period (16 May 2021–18 Dec 2021), 124 days (IQR: 46–193) in the Omicron BA.1 period (19 Dec 2021–19 Mar 2022), and 280 days (IQR: 158–408) when the SARS-CoV-2 Omicron BA.2 variant became predominant (20 Mar 2022–29 Oct 2023). Despite this decline in VE, estimates based on documented sources and parent report remained similar, with overlapping confidence intervals. Specifically, for vaccination status from documented sources, effectiveness of 2 doses dropped from 89.9 % (95 % CI: 85.6–92.9 %) during the Delta predominant period to 56.5 % (95 % CI: 39.3–68.9) during the Omicron BA.1 predominant period and further to 12.2 % (95 % CI: −6.3–27.4 %) by the Omicron BA.2 predominant period. Similarly, for vaccination status reported by parents, effectiveness dropped from 86.2 % (95 % CI: 80.6–90.2 %) to 60.1 % (95 % CI: 44.1–71.5 %) by the Omicron BA.1 period and to −7.6 % (95 % CI: −39.0–16.8 %) by the Omicron BA.2 predominant period.