The Advisory Committee on Immunization Practices (ACIP) recommends an immunization schedule for the United States where children receive 10 vaccines to protect against 14 diseases before the age of two [1]. Vaccines effectively protect against infectious diseases that are potentially fatal, and are widely recognized as one of the most successful public health interventions in modern history. However, vaccines may also be considered victims of their own success [2]. As vaccine-preventable diseases have become less prominent over time, some parents’ concerns have shifted from consequences of the disease to the safety of the vaccine [3]. Recent studies have shown that refusing or delaying vaccines is an increasing trend [2,4–7], and more than 1 in 10 parents are choosing alternative immunization schedules for their children [8]. Not only does this put young children at an increased risk for disease, but it also contributes to the spread of vaccine-preventable diseases in the community. While vaccine uptake on a national level remains high, pockets of low vaccine coverage have resulted in outbreaks of vaccine-preventable diseases [9–17].

In 2012, the Institute of Medicine (IOM) reviewed the safety of the recommended childhood immunization schedule, and concluded that although available evidence strongly supported the safety of the schedule as a whole, additional observational research was warranted to compare health outcomes between fully vaccinated children and those on a delayed or alternative schedule [18]. In addition, the IOM identified the Vaccine Safety Datalink (VSD) as an important resource for conducting this research. Guided by the IOM report, the Centers for Disease Control and Prevention (CDC) commissioned a white paper to assess how the VSD could be used to study the safety of the childhood schedule. All-cause mortality was identified as a priority outcome to study in the context of the immunization schedule because of both public health significance and public health concern [19].

There have been few studies evaluating mortality following vaccination [20–22]. One prior VSD study examined the risk of death in the 30 days following vaccination in older children and young adults and found no association [23]. Additionally, there have been multiple studies, primarily in developing countries, examining the incidence of mortality with respect to the order and timing of live and inactivated vaccinations [24–27]. In these studies, lower mortality rates were found for children who last received a live (e.g., measles-containing) vaccine compared to those who last received an inactivated vaccine. Although they may not be directly relevant to a high-income country such as the United States, these findings help illustrate the importance of studying mortality with regards to the immunization schedule.

We conducted a study that describes and compares mortality rates among young children in the VSD with respect to their vaccination patterns.

The VSD is a collaborative project between CDC and 8 integrated healthcare systems (sites). [28,29]. The project captures comprehensive medical and immunization data for over 10 million people annually, which represents approximately 3% of the U.S. population. This study included data from the following 6 VSD sites: Kaiser Permanente Washington, Kaiser Permanente Colorado, Kaiser Permanente Northwest, Kaiser Permanente Northern California, Southern California Kaiser Permanente, and Marshfield Clinic. The study was approved by the institutional review board at each participating VSD site and the CDC.

The VSD obtains data from electronic medical records and other administrative sources at each site on enrollees, including demographics, vaccinations, and medical outcomes, including deaths. Deaths are identified for members enrolled at the VSD sites at the time of death and continue to be captured during the 2 years or more following any stop in enrollment. VSD mortality files are updated annually and include data on the cause(s) and date of death. Immediate, underlying, and contributory causes of death are included in the files and coded using the International Classification of Disease 10th revision (1CD-10). The majority of the sites receive cause and date of death information from state death records; however, the National Death Index, Social Security Administration, electronic medical records, and administrative sources, such as health plan membership information, are also sources of mortality information.

We included all children born January 1, 2004 through December 31, 2009 who were continuously enrolled in the VSD from within 6 weeks of birth to 19 months of age. We required at least one outpatient medical visit before 19 months of age to ensure that children were receiving care at the VSD site. Children with potential contraindications to vaccination (e.g., human immuniodeficiency virus patients, hematopoietic stem cell transplant patients, and other immunodeficiencies including leukemia and lymphomas), were excluded from the cohort as they were unlikely be vaccinated according to the schedule. We also identified children with complex chronic conditions using the Pediatric Medical Complexity Algorithm (PMCA) [30], and excluded children with complex chronic diagnoses prior to 19 months of age as these conditions could affect the likelihood of vaccination according to the schedule as well as death. Follow-up began August 1, 2005, and we collected death information through December 31, 2013. In order to examine the early childhood recommended schedule as a whole, we identified deaths between 19 and 48 months of age using the VSD mortality files. Deaths due to injuries, congenital anomalies, or diseases with onset prior to 19 months of age were excluded from the study.

We identified vaccination patterns among children from 0 to 19 months of life for 8 recommended vaccines, including (1) hepatitis B (HepB), (2) rotavirus, (3) diphtheria, tetanus, and acellular pertussis (DTaP), (4) Haemophilus influenzae type b (Hib), (5) pneumococcal conjugate (Pneum), (6) polio, (7) measles, mumps and rubella (MMR), and (8) varicella [1,31]. Our primary analysis compared children vaccinated according to the ACIP recommended schedule to undervaccinated children. We also evaluated children with specific patterns of undervaccination, including undervaccinated but up to date by 19 months of age, received no vaccines, delayed starting vaccination until ≥4 months of age, consistent vaccine-limiting (≤2 vaccines per visit), and missing at least one vaccine dose or series at 19 months of age.

We implemented criteria set forth in the VSD white paper when determining undervaccinated status [19]. We allowed for a 30-day grace period following the recommended age for vaccination for all vaccine doses, apart from the recommended birth dose of hepatitis B, where the grace period began at 2 months of age. We also took into account national vaccine shortages, as well as changes in the ACIP recommendations during the study period. In this context, we defined an undervaccinated child as having received one or more vaccines ≥30 days after the recommended age of administration. Due to the rotavirus vaccine’s initial slow uptake, we did not require rotavirus vaccine administration to be considered up to date until after the point in time when the respective VSD site reached 80% coverage with rotavirus vaccine. Influenza vaccine was not included when identifying vaccination patterns, as the annual recommendation for influenza vaccination makes it distinct from the other childhood vaccines. However, receipt of influenza vaccine was included as a covariate in the statistical analyses. We also excluded hepatitis A vaccine because recommendations for universal immunization began during the study period, and coverage rates following the recommendation were low.

We used an algorithm originally developed by Luman et al. [32], and modified by Glanz et al. [4], to calculate the average number of days undervaccinated (ADU) for each child in the study cohort. ADU is a continuous metric that quantifies immunization status, and measures the difference between when the vaccine dose was administered and when the vaccine dose should have been administered according to the ACIP recommended schedule. Using this measure, a fully up to date child with no delays will have an ADU = 0, and an undervaccinated child will have an ADU ≥ 1.

We evaluated mortality rates for children vaccinated according to the recommended schedule and undervaccinated children. We calculated the IRR for children vaccinated according to the schedule compared to undervaccinated children using the Cox proportional hazards model, adjusting for outpatient utilization, influenza vaccination, sex, and VSD site. We also compared mortality rates with ADU as a continuous exposure with a 30-day unit of change, as well as with ADU in quartiles. The method of Schoenfeld residuals was used to test the proportional hazards assumption [33]. We conducted analyses including all causes of death, as well as only known causes of death. Using previously published VSD mortality rates and undervaccinated population distributions, we determined a priori that there would be 80% power to detect an incident rate ratio (IRR) of 2.5 when comparing mortality rates of undervaccinated children to rates of children on the ACIP schedule [4,20]. We repeated analyses with deaths due to injuries as a control outcome in order to evaluate potential biases associated with factors that are not captured in the VSD electronic data. Lastly, we used a scan statistic software program, SatscanTM [34], to identify any clusters of deaths between 19 and 48 months of age in our cohort. All other analyses were conducted using SAS version 9.4 (SAS Institute Inc., Cary, NC).

Among 341,297 children born January 1, 2004 through December 31, 2009, 312,388 children were included in the study (Fig. 1). We found 199,661 children (64%) were vaccinated according to the ACIP schedule, and 112,727 children (36%) were undervaccinated. Table 1 provides the population characteristics for the cohort. When comparing children on the ACIP schedule to those undervaccinated, there was a higher proportion of children receiving influenza vaccine and a higher number of outpatient visits among children who followed the ACIP schedule. Among the undervaccinated cohort, 48% became up to date on all doses by 19 months of age (Table 2). We identified additional mutually exclusive vaccination patterns within the undervaccinated cohort: children receiving no vaccines (3.3%), children with a delayed start to vaccination (2.0%), children with no more than 2 vaccines per visit (2.4%), and children missing a vaccine dose or series at 19 months of age (44.3%). The ADU for undervaccinated children ranged from 1 to 419 days, with a mean of 63 days and median of 22 days. Over half of the undervaccinated group (61%) had an ADU from 1 to 30 days.

We identified 91 deaths. We excluded 67 deaths due to either injuries or congenital anomalies, and 6 deaths in children who had diseases with onset prior to 19 months of age that were related to the death. As a result, 18 deaths were eligible for our primary analysis. Of those, 11 occurred in children following the recommended schedule (2.28 per 100,000 person-years), and 7 occurred in undervaccinated children (2.57 per 100,000 person-years). Table 2 shows the mortality rates by vaccination pattern. There were 3 deaths due to respiratory causes, 1 death due to diseases of the nervous system, 2 deaths due to malignancies, 3 deaths due to infectious diseases, and 9 deaths with an unknown cause of death. The deaths due to infectious diseases had causes that were not preventable by routinely recommended vaccines.

Mortality rates among children following the schedule were not significantly different from undervaccinated children when excluding deaths with unknown causes (IRR = 1.29, 95% CI = 0.33–4.99), as well as when including deaths with unknown causes (IRR = 0.84, 95% CI = 0.32–2.99). When examining mortality rates by ADU as a continuous measure with a 30-day unit change, there were no significant differences in mortality rates (IRR = 1.09, 95% CI = 0.96–1.25). We also found no significant differences in mortality rates implementing quartiles of ADU as a categorical exposure (data not shown). When evaluating deaths due to injuries as a control outcome, there were no significant differences in mortality rates among children following the ACIP schedule versus undervaccinated children (IRR = 0.99, 95% CI = 0.53–1.88). Lastly, no significant clusters of deaths were identified between ages 19 and 48 months.

In this study, we examined deaths among young children with respect to their vaccination patterns, and found mortality rates were not significantly different between the patterns identified. About a third of the study population was undervaccinated (36%). This is slightly lower than previous VSD studies with undervaccination rates of 45–49% [4,35,36], which is likely related to differences in our exclusion criteria (e.g., excluding children with complex chronic conditions and potential contraindications to vaccination). Overall, the number of deaths was low, which is expected as deaths among children 19 through 48 months are rare, and the most common causes of death in this age group are injuries and congenital anomalies, which were excluded from our analysis [37].

We did not find any difference in risk of mortality among children aged 19 through 48 months vaccinated according to the recommended schedule as compared to undervaccinated children. We also did not find a protective effect of the recommended schedule against all-cause mortality; however, we would not expect to be able to detect a protective effect of the schedule, as deaths from vaccine-preventable diseases in the U.S. are rare and none were observed in our study population [38]. Furthermore, those that do occur are more common among young infants [39], which was not the age group evaluated in our study. Also, while influenza-associated deaths do occur in the age group examined [40], we were unable to evaluate influenza vaccine directly in the context of the recommended schedule because children receive the vaccine at various ages.

The 2013 U.S. mortality rate among children 1–4 years of age was 25.5 deaths per 100,000 person-years [41]. For a more conservative power calculation a priori, we used the mortality rate among children 1–4 years of age in the VSD within 60 days of any vaccination (17.65 deaths per 100,000 person-years) [20]. However, the exclusion of children with chronic or immunocompromising conditions from the cohort, and excluding deaths due to external causes, such as injuries and congenital anomalies, lowered the mortality rate in this study, and subsequently lowered the power. When comparing mortality rates of children vaccinated according to the ACIP schedule with undervaccinated children, there was 80% power to detect IRR = 4.0 for all causes of death. We were underpowered to detect small difference in mortality rates, which is one of the challenges in examining mortality within a population with relatively few deaths. However, despite low power, mortality rates were similar among children following the ACIP schedule and undervaccinated children, which is reassuring.

Our study had some limitations. We relied on VSD electronic data, and may not have captured some vaccinations given outside of the VSD; however, by requiring children to be enrolled from within 6 weeks of birth to 19 months, and requiring at least one outpatient visit to ensure the child was receiving care at the site, there is less potential for exposure misclassification [35]. In addition, our analysis with deaths due to injuries as a control outcome demonstrated there is unlikely to be bias associated with factors that are not captured in the VSD electronic data. We were only able to capture deaths through 2013 due to the reliance on vital statistics data and the lag associated with obtaining those data from the states. Also, a small percentage of deaths may not have been captured, if for instance a health plan member ceased membership and died out of state. However, capturing deaths up to 2 years post enrollment should overcome this in part, as our ascertainment of death did not rely solely on state death records, but utilized sites’ administrative data as well. Only 1% of the cohort received live vaccine alone during their last vaccination visit prior to 19 months of age, and there were no deaths in this group, so we were unable to examine mortality with respect to the order and timing of vaccines in our cohort. The association between receiving live vaccine last and lower mortality rates has been reported from mostly low and middle income countries where children have a different immunization schedule and mortality rates are higher overall [24–27].

Half of the deaths eligible for analysis had unknown causes, either because the cause was not available in VSD data or because the ICD-10 code given was “death not otherwise specified”. We conducted separate analyses for all causes of death and known causes of death, as we could not determine whether these unknown causes were all accounted for by our exclusion categories. Regardless of cause of death, we found no association between immunization schedules and mortality. Among the undervaccinated group, most had an ADU between 1 and 30 days (61%). Future studies with more common outcomes may consider focusing on children on a more delayed undervaccination schedule (i.e., higher ADU).

The strengths of this study include the use of high quality vaccination and mortality data, and the ability to examine the recommended immunization schedule as a whole. In recent years, undervaccination has become an increasing trend partly due to vaccine safety concerns, and death represents the most serious outcome for a range of potential vaccine-related adverse events, such as severe allergic reactions [4,22]. Although there were relatively few deaths, our results do not indicate any increased risk of mortality among children following the currently recommended ACIP immunization schedule compared to undervaccinated children. Our findings support the safety of the recommended immunization schedule with regard to all-cause mortality.