U.S. states have a heterogeneous array of public health systems, defined as the “constellation of individuals and organizations in the public and private sectors that provide information and assets to promote population health, provide health care delivery, and prevent disease and injury.” ¹ But state-specific models are unique. Each has its own statutory framework, funding and staffing patterns, and performance requirements.^{2, 3} Legally, economically, and operationally, state public health systems function independently of each other and of federal controls ⁴ despite recent efforts to create national standards. ^{5, 6}

The World Health Organization recommends national-level pandemic influenza planning.⁷ But in the U.S., the state-based approach to planning produces substantial heterogeneities in emergency preparedness.⁸ For example, during the H1N1 pandemic of 2009–2010, there was wide variation among states in vaccination coverage.⁹ One study attributed these differences in part to factors arising from state-level policy and planning decisions, including time required of vaccinators to order allocated doses, number of vaccine shipments to vaccinator locations, and existence of school-based and public-access areas for vaccine distribution.¹⁰

Differences in emergency preparedness among states have been a matter of national policy concern for some time. The 2009 National Health Security Strategy lists “community interventions for disease control” (such as vaccine distribution) among the criteria for assessing performance capabilities but also acknowledges that relevant measures for this capability do not exist in the CDC’s guidance accompanying preparedness funding grants.^{11 p.86}

Motivated by the need to evaluate how effectively federal preparedness dollars were being used to improve state-level preparedness, the Centers for Disease Control and Prevention (CDC) has evaluated each state’s preparedness to distribute materiel from for its Strategic National Stockpile (SNS).¹² SNS is the national reserve of medical supplies, pharmaceuticals, and other resources to be deployed to states and distributed by them in response to epidemics and other disasters with public health consequences. CDC gave each state a SNS score based on numerous criteria including: coordination between health agency and community partners; receiving, staging, and distributing medical materiel; statutory support for the rapid dispensing of medications; and the type and frequency of training, exercising, and evaluation of response plans. Thus, the SNS score provides a metric for evaluating preparedness based on public health system characteristics with validity and relevance to the quality of state-specific responses to emergencies such as a novel influenza pandemic.

CDC did not use SNS scores explicitly for making inter-state comparisons or for evaluating national preparedness. However, the likelihood that under-preparedness in some states might affect outcomes in other states presents an important question for planning and policy-making. This study used SNS scores in an agent-based computational model as a proxy for state-specific preparedness to explore the potential inter-state and national effects of heterogeneity among states’ public health systems. The study simulated the emergency distribution of vaccines to state populations as a pandemic mitigation strategy, with timeliness scaled to each state’s standardized SNS score. The results offer insights for national and state-level preparedness plans and policy-making and support the use of modeling as a tool for policy-relevant research.

There is difficulty inherent in public health preparedness research. Public health systems are complex, ¹³ and public health emergencies like pandemics are infrequent and locally heterogeneous. Previously published studies based on expert panel methods and table-top exercises have contributed insights for policy makers^{14, 15} but have lacked empirical validation. Empirical evidence is sparse, since security concerns tend to inhibit the publication of emergency after-action reports. Case-controlled and randomized trials comparing disaster outcomes among localities are impractical and ethically problematic.

Large-scale computational modeling provides a novel research tool for public health systems and preparedness by facilitating inquiry into complex and dynamic problems that are not amenable to other methods. Agent-based models are, in effect, artificial societies in which every individual is represented as a distinct computational agent whose activities (e.g., going to work or school) and whose contacts with other agents are explicitly represented as events unfold both spatially and chronologically.¹⁶ Modeling approaches are already acknowledged as useful to preparedness researchers and policy makers ¹⁷ and have strong credentials in the study of infectious disease control.¹⁸ Such models have simulated the spread of disease through populations in considerable biomedical and demographic detail, for example, by representing such disease characteristics as a pathogen’s transmissibility and mortality and such population characteristics as age, residential density, commuting patterns, contact frequency, and immunologic status.¹⁹ Previously published simulations of influenza epidemics account for disease and population variations, model their stochastic (randomly occurring) interactions, and suggest how strategies such as mass vaccination or targeted vaccination strategies might affect a pandemic.^20–23 Previous simulation studies have experimented with the mitigating effects of antiviral distributions,²⁴ vaccination programs,^{20, 22, 25} and school closures.²⁶

Computational modeling requires that the relationships between sets of inputs and outputs—that is, how variations of one might affect the others—be defined in mathematical or quantitative terms. This study explored the effect of variations in states’ levels of preparedness by linking the SNS scores to calibrated delays in emergency vaccine distribution and by comparing how the delays would affect influenza infection rates within and among states. To date, there exists no experience-based evidence for a quantitative, scaled correlation between a state’s SNS score and its performance in a distribution of SNS resources. Nevertheless, as a proxy for state-specific preparedness, the SNS scoring system is logically compelling. Starting in 2002, each state’s allocation of federal preparedness funds carried an obligation to prepare for emergency distributions of SNS materiel. The CDC’s evaluation of state preparedness in 2007 included criteria for legal authorities, economic and human resources, and operational plans and exercises relevant to implementation of the SNS program.¹² The resulting score for each state represents a composite indicator of its preparedness that is comparable to that of other states.

This study uses a pandemic model of the U.S. population with the assumptions that the population has had no previous exposure and thus no immunity, there is an available vaccine, and each state’s SNS score has a scaled relationship to the timing of vaccine distribution. We explored three fictional pandemic scenarios: 1) no vaccine distribution in any state; 2) 35% vaccination coverage achieved in all 48 contiguous states within four weeks of the start of the U.S. pandemic; and 3) vaccine distributions in each state scaled to its SNS score. This study did not involve the use of human subjects or of any data collected from human subjects or patient records.

This modeling experiment is fictional, designed to explore the question of how state-specific preparedness might affect disease rates in the nation as a whole during an influenza pandemic. While the model’s specifications are grounded in data from actual population and influenza characteristics, this study is intended neither to represent any historical influenza outbreak nor to forecast the outcome of any possible future outbreak. Rather, it tests the hypothesis that total U.S. influenza attack rate (% of the population infected) in a pandemic is sensitive to variations among states in their preparedness to distribute vaccines.

Table 1 presents the overall results of simulations under each scenario and illustrates the model’s considerable sensitivity to the state-specific variations in timing of vaccine distribution. Nationally, the overall attack rate of influenza in the population fell from 44% with no vaccine distribution to 13% with optimal vaccine distribution; and SNS-modified vaccine distribution produced attack rate of 15% assuming only 1-day delay increments per 10-point decrement in SNS score and 28% assuming 5-day delay increments. Peak incidence was 5.5 million cases without vaccination, 1.3 million cases with optimal vaccination, and 1.59 million and 3.4 million cases, respectively, under the 1-day and 5-day delay increment SNS-modified scenarios. The peak day was only slightly different under each scenario, an effect attributable to the initiation of vaccine distribution no earlier than day 88 of the U.S. pandemic even in the optimal scenario.

Figure 1 shows the national results in terms of cumulative case incidence over the course of the pandemic for each of the four scenarios: no vaccine distribution resulted in 128 million cases; optimal vaccine distribution resulted in 38 million cases; SNS-modified vaccine distribution with 1-day delay increments resulted in 44 million cases; and SNS-modified vaccine distribution with 5-day delay increments resulted in 82 million cases.

Figure 2 shows how the SNS-modified vaccine distribution in low-scoring states affected not only intra-state rates of incident infection but also inter-state rates. Here the states and the District of Columbia are grouped by SNS scores in thirds: 17 top-scoring with scores at 90 or above; 17 middle-scoring with scores of 77 to 89; and the 17 low-scoring below 77. Each group’s average percent-difference in incident cases between optimal and SNS-modified scenarios is shown as a separate bar; and the two sets of bars show the 1-day and 5-day delay increment scenarios. Under the 1-day incremental delay SNS-modified scenario, incident cases increased over optimal by 1.67% in the high-scoring group, 2.09% in the middle-scoring group, and 4.29% in the low-scoring group. Under the 5-day incremental delay SNS-modified scenario, incident cases increased over optimal by 11.3% in the top-scoring group, by 17.5% in the middle-scoring group, and by 25.4% in the bottom-scoring group. Notably, the interstate effect is evident in the top-scoring group, in which all states had SNS scores of 90 or higher and therefore did not have any delay beyond optimal in vaccine distribution under the SNS-modified scenario. This group had a higher case incidence under both SNS-modified scenarios, indicating that infection was transmitted from the other states where vaccine distributions were delayed.

Using the SNS scores to calibrate a pandemic simulation, this large-scale agent-based model shows how delayed vaccine distributions nationally can result in higher peak case numbers and higher rates of incidence; and it shows how greater delays in some states result in higher peak cases and incidence rates even in states where no incremental delays occur. This approach does not require a literal or actual relationship between SNS score and vaccine-distribution time. Rather, the score provides a calibrated proxy for state-specific differences in preparedness. The model’s sensitivity to these state-specific differences underscores the importance of state-specific measures of public health system preparedness, and its demonstration of an inter-state effect suggests the need for national standards of preparedness.

State-specific measures of public health preparedness are important for state-level policy making and resource allocation, and the continued development and use of nationally standardized preparedness metrics like the SNS score are necessary to that purpose. However, as this modeling experiment shows, national standards such as for vaccine distribution policies and practices could improve outcomes for the nation as a whole. If influenza incidence rates among states are in fact interdependent, then the nation’s pandemic preparedness effort should strive to minimize inter-state disparities.

To date, federal officials have shown reluctance to impose uniform standards on states. The National Health Security Strategy of 2009 states that “Primary authority for health security lies with local, state, territorial, and tribal governments.” ^{11 p.19} This principle influences how CDC allocates preparedness funding among states, by using guaranteed state minimums and per-capita rates rather than need assessments or evaluation results.³⁴ Nevertheless, even conservative policy analysts argue in favor of shifting the balance of authority toward national uniformity of standards when deference to states’ sovereignty might diminish the overall effectiveness of response. ^{35 p. 80}

The CDC has now conducted and published results from two rounds of SNS-related evaluation and scoring, the first in 2008¹² and another in 2010.³⁶ These standardized evaluations can provide the base of evidence needed to inform performance benchmarks and guidance. The continued preparedness funding to states could provide a vehicle for setting performance benchmarks and for targeting extra funds to states where greatest improvements are needed.

There are three aspects of this study that limit the interpretation of its results. First, interstate travel and cross-border commuting patterns programmed into the model give rise to the observed interstate effect, in which disease transmission occurs between and among states. However, the state-level resolution of results lacks sufficient detail to show which cities or metropolitan areas might be serving as hubs for disease transmission between states. Further exploration of the inter-state effect would require county-level or metropolitan-level calibration of the vaccine distribution simulations for which standardized data are not currently published.

Second, this model simplifies reality—as do all models, omitting some possibly relevant details in order to explore how selected factors might affect results. Models of the kind used in this study do not purport to predict the course of a pandemic, dependent as that would be on innumerable excluded factors such as individual and social behavioral patterns.

Third, the SNS scores used here as proxy measures of states’ relative preparedness are based on evaluation of plans rather than actual capabilities; further, inter-rater reliability for CDC’s scoring process was not reported.^{12, App. 5,p156–157} Nevertheless, these issues are tangential to the modeling results, which show that even small delays in the distribution of vaccines can affect disease outcomes at both state and national levels.

The value of large-scale agent-based modeling is its utility for testing assumptions under varying conditions that cannot be observed or tested empirically. In this way, modeling offers a tool for those who must make resource-allocation policies and real-time emergency decisions in the absence of complete empirical evidence.

In this study, a modeling experiment isolates preparedness as measured by the SNS score from other factors to assess whether the wide variability of preparedness among states can affect the national outcome. The fact that states performed quite disparately in distributing H1N1 vaccines during the 2009–2010 pandemic substantiates this approach. The results suggest that investing a portion of federal preparedness funding to raise the capabilities of the lowest-scoring states would benefit the nation as a whole and would improve outcomes even among higher-scoring states.