The 1989 and 1990 distributions of daily collected female
Disease transmission forecasts can help minimize human and domestic animal health risks by indicating where disease control and prevention efforts should be focused. For disease systems in which weather-related variables affect pathogen proliferation, dispersal, or transmission, the potential for disease forecasting exists. We present a seasonal forecast of St. Louis encephalitis virus transmission in Indian River County, Florida. We derive an empirical relationship between modeled land surface wetness and levels of SLEV transmission in humans. We then use these data to forecast SLEV transmission with a seasonal lead. Forecast skill is demonstrated, and a real-time seasonal forecast of epidemic SLEV transmission is presented. This study demonstrates how weather and climate forecast skill verification analyses may be applied to test the predictability of an empirical disease forecast model.
St. Louis encephalitis virus (SLEV) is a mosquito-borne pathogen that is prevalent throughout much of North America. Florida is subject to periodic outbreaks of SLEV; five epidemics (>20 human clinical cases) have been recorded in south Florida since 1952 (
The annual dynamics of SLEV in south Florida can be divided into four phases: January–March maintenance; April–June amplification; July–September early transmission; October–December late transmission (
We analyzed historical sentinel chicken seroconversion, i.e., transmission incidence of SLEV, datasets from Indian River County from 1986 to 1991 (
A rationale for this empirical relationship was suggested by mosquito collection data, also from Indian River County from 1986 to 1991.
The 1989 and 1990 distributions of daily collected female
We expand on the approach presented in Shaman et al. (
Hydrologic modeling follows the methods set forth in Shaman et al. (
Changes in the annual timing and distribution of SLEV transmission to sentinel chickens have been strongly correlated with SLEV disease in humans (
Map of Indian River County. Asterisks indicate the locations of sentinel chicken flocks. “M” is the site of the Vero Beach 4W meteorologic station.
Generally, a 1.0 mL blood sample was drawn weekly from each bird during peak transmission periods (July through November), and twice a month during the rest of the year. Blood samples were assayed for HI antibodies to SLEV at the Florida Department of Health and Rehabilitative Services, Tampa Branch Laboratory. Individual chickens that tested positive for HI antibodies were replaced with fresh ones, and the entire flock was replaced each spring.
We define three categories of sentinel chicken seroconversion as transmission incidence, epidemic transmission, and transmission number. In a manner consistent with Shaman et al. (
High rates of sentinel chicken seroconversion are of principal interest to public health workers. These high levels of transmission connote the existence of large numbers of SLEV-infected mosquitoes and so identify times when humans are at greatest risk. We therefore define this category as epidemic transmission, which uses all the sentinel chickens in aggregate. It, too, is a categoric measure: one if ≥ 20% of posted chickens are seropositive in a given week; zero if ≤ 20% of the sentinel chickens are seropositive. These two categories represent epidemic level transmission and all other states, respectively (
Transmission number treats each chicken as a separate measure of SLEV transmission. Thus, for each chicken and week, the transmission number provides a categoric metric: one if the chicken is seropositive; zero if it is seronegative. For a given week, up to 48 such categoric measures are available. Unlike transmission incidence and epidemic transmission, which give a single weekly categoric value, the transmission number category provides multiple categoric measures at each point in time. These multiple measures are not necessarily independent. See
All three types of SLEV transmission (transmission incidence, transmission number, and epidemic transmission) were defined as categoric variables. Univariate and bivariate logistic regression were used to associate the probability of each of the types of SLEV transmission with single weekly lags of modeled WTD and combinations of two lags of WTD. We defined antecedent as the longer lag, and near coincident as the shorter lag. To account for the apparent dependence among chickens in the transmission number category, we performed these logistic regression using generalized estimating equations with a working correlation ranging from r = 0–0.6, following the methods of Liang and Zeger (
Logistic regression of an SLEV transmission category on modeled WTD derives the probability that this type of SLEV transmission will occur:
where
The quality of a forecast can be measured formally through an assessment of its skill. Skill refers to the accuracy of a forecast or set of forecasts relative to a standard control forecast. In weather and climate forecasts, often the control forecast is based on historic conditions. These historic conditions constitute a climatology, that is, a distribution of possible states. We defined our climatology as the documented frequency of epidemic SLEV transmission for each week of the year, and it is derived from the 1978 to 1997 sentinel chicken record. For each week of the 1978–1997 record, “epidemic transmission”, “no epidemic transmission”, or “no data” was recorded. The percentage of epidemic transmission that occurred in a given week, for instance, the 28th week of the years 1978–1997, provides the climatologic probability of epidemic SLEV transmission for that week. The climatology is fixed; hence, for each year of a forecast period, the same weekly climatologic values are predicted.
We evaluate the skill of our retrospective forecasts of epidemic SLEV transmission using the Brier score (
where
where
The skill score (SS) is computed directly from the Brier scores.
A skill score of 0 represents no improvement of forecast skill relative to climatology. A skill score greater than 0 demonstrates improvement of the forecast relative to climatology; a skill score of 1 is a perfect forecast.
Significance of the skill score value was assessed by using a Monte Carlo procedure. The null hypothesis is that the forecasts have no greater skill than modeled WTD climatology. No skill forecasts were simulated by randomly selecting weekly modeled WTD values from the 1949–1997 simulation record. One thousand such forecasts were made, and a mock SS was calculated for each. From this distribution of mock SS values, significance of the actual SS value was determined.
Time series of study data 1978–1997. The blue line is weekly modeled water table depth (WTD); the black bars are the weekly percentages of posted sentinel chickens in Indian River County testing positive for hemagglutination inhibition antibodies to St. Louis encephalitis virus.
The
| Predictand | 1986–1991 Transmission incidence | 1986–1991 Transmission no. | 1986–1991 Epidemic transmission | 1978–1997 Transmission incidence | 1978–1997 Transmission no. | 1978–1997 Epidemic transmission |
|---|---|---|---|---|---|---|
| Antecedent lag | 17 | 14 | 11 | 16 | 8 | 16 |
| Near coincident lag | 2 | 0 | 0 | 2 | 2 | - |
| Intercept | 19.03 (3.74) | 17.50 (1.79) | 20.98 (7.07) | 2.48 (0.39) | 6.33 (0.46) | 14.29 (3.50) |
| Antecedent slope | 18.06 (3.65) | 14.36 (1.45) | 19.56 (7.03) | 1.80 (0.36) | 2.59 (0.38) | 8.13 (2.50) |
| Significance | p < 0.0001 | p < 0.0001 | p < 0.01 | p < 0.0001 | p < 0.0001 | p < 0.005 |
| Near coincident slope | –6.21 (1.77) | –5.51 (0.79) | –8.26 (3.85) | –0.70 (0.34) | –0.53 (0.27) | - |
| Significance | p < 0.0001 | p < 0.0001 | p < 0.05 | p < 0.05 | p < 0.05 | NS |
| Whole model fit | p < 0.0001 | p < 0.0001 | p < 0.001 | p < 0.0001 | p < 0.0001 | p < 0.001 |
aEstimates of standard error are given in parentheses. For the transmission number category, the working correlation is r = 0.3. bWTD, water table depth; SLEV, St. Louis encephalitis virus; NS, not significant.
Probabilities predicted with the 1986–1991 logistic regression model equation for epidemic SLEV transmission range from 0 to nearly 1 when combined with realistic modeled WTD scenarios (
a) Best-fit bivariate logistic regression model of epidemic St. Louis encephalitis virus (SLEV) transmission based on the 1986–1991 record. Plotted for a continuous range of modeled water table depths (WTD) 11 weeks before transmission and fixed values of current modeled water table depths. b) Best-fit logistic regression model of epidemic SLEV transmission based on the 1978–1997 sentinel chicken record. Only antecedent drought conditions are statistically significant. Plotted for a continuous range of modeled water table depths 16 weeks before transmission.
The best-fit results from analysis of the longer 1978–1997 model show a slightly different picture. Again, for transmission incidence, antecedent drought and near coincident wetting contribute significantly to whole model goodness-of-fit (p < 0.0001). However, only antecedent drought (best fit 16 weeks) is significantly associated with epidemic SLEV transmission. Near coincident wetting is no longer a statistically significant explanatory variable. For transmission number, both antecedent drought and near coincident wetting contribute significantly over a range of antecedent drought lags (5–13 weeks), near coincident wetting lags (0–3 weeks), and working correlation values (r = 0–0.3). Near coincident wetting is not significant at higher working correlation values (r = 0.4–0.6). We have shown the best-fit model for a working correlation of r = 0.3.
Autocorrelation was nominal for all but the 1978–1997 transmission incidence model. For instance, weekly autocorrelation for the 1978–1997 epidemic SLEV transmission time series drops to r = 0.42 at lag one and doesn’t fall to zero until week 13. However, among the residuals of the 1978–1997 epidemic SLEV transmission regression model, the autocorrelation drops to r = 0.07 at lag one and remains still closer to zero for longer lags. These findings suggest that much of the autocorrelation of the 1986–1991 transmission incidence, 1986–1991 epidemic transmission, and 1978–1997 epidemic transmission is explained by modeled WTD. However, additional factors are needed to explain 1978–1997 transmission incidence.
We applied the empirical relationship established for epidemic SLEV transmission (1978–1997) to TBH model simulations of WTD for September 1997–March 2002 and produced weekly retrospective forecasts of epidemic SLEV transmission for January 1998 through June 2002. That is, we combined weekly, modeled WTD with the equation
where
Time series of weekly, retrospective epidemic St. Louis encephalitis virus transmission forecast probabilities, January 1998–June 2002, with 95% confidence intervals. Also shown are the weekly, epidemic SLEV transmission probabilities as would be predicted from climatology (1978–1997).
The Brier skill score was calculated for the weekly January 1998–June 2002 retrospective forecast of epidemic SLEV transmission. A high level of skill is found (SS = 0.461) and is shown to be statistically significant (p < 0.001).
Having found a high level of skill for our epidemic SLEV transmission forecast, we then developed a real-time forecast of epidemic SLEV transmission in Indian River County during 2002. (This forecast was in real time when this manuscript was prepared and initially submitted.) Model simulations from March to June 2002 were combined with equation 2 and are presented in
Real-time forecast of the probability of epidemic St. Louis encephalitis virus transmission in Indian River County, Florida, July–October 2002, with 95% confidence intervals. Also shown are the weekly climatologic probabilities of epidemic St. Louis encephalitis virus transmission.
In this study, we have defined three types of SLEV transmission as measured by sentinel chickens: transmission incidence, transmission number, and epidemic transmission. All three categories of SLEV transmission were found to be empirically associated with modeled WTD as simulated with the TBH model. For the shorter record, 1986–1991, antecedent drought and near coincident wetting were shown to be good predictors of all three transmission categories. These results, combined with the
The longer 1978–1997 record accounts for a wider variety of modeled wetness conditions and encompasses a longer, noisier record of sentinel chicken data, in which confounding factors, such as avian host susceptibility and host and vector mobility, may be strongly affecting transmission levels. Consequently, while not providing the same mechanistic insight into amplification dynamics as the shorter 1986–1991 record, empirical relationships derived from the 1978–1997 record provide a more realistic prediction of SLEV transmission based on modeled WTD.
For the longer 1978–1997 record, antecedent drought and near coincident wetting were significant predictors of transmission incidence. For transmission number, both antecedent drought and near coincident wetting were also significant predictors, but only for lower working correlation values (r = 0–0.3). Antecedent drought by itself, however, was a significant predictor of transmission number over the full range of working correlation values (data not shown). Lastly, logistic regression analysis of the longer record showed that only antecedent drought was significantly associated with epidemic transmission. Furthermore, for all three categories of SLEV transmission for 1978 to 1997, probabilities predicted using the logistic regression models and realistic simulated WTDs were considerably lower than those for the shorter 1986–1991 record. This finding corroborates the assertion that factors other than surface wetness conditions also control SLEV transmission rates. Consequently, drought-induced amplification may be necessary for high levels of SLEV transmission, but it alone is not a sufficient condition for such an event, nor must it occur locally.
The loss of significance for near-coincident wetting might have several causes. Epidemic SLEV transmission is a rare event. For the 1978 to 1997 Indian River County record, epidemic SLEV transmission only took place during the 1990 epidemic and on one occasion in 1997. These 20 years include many wet events, but these events were often not preceded by the drought needed for amplification of SLEV, and therefore were not associated with transmission. Other factors could also have been at play, such as avian immunity, mosquito migration, human activity, and land use changes, which might have countered the effect of wetting and reduced its association with epidemic SLEV transmission to levels below significance. However, recent analysis of human cases of SLE suggests that with a still larger record, wetting would again be significantly associated with epidemic SLEV transmission (
Epidemic SLEV transmission, associated with human SLE incidence, is of principal epidemiologic concern. We have presented a forecast of epidemic SLEV transmission, as measured with sentinel chickens, using the empirical association between 1978–1997 epidemic SLEV transmission and modeled WTD, and additional simulations of WTD with the topographically based hydrology model. Forecast skill has been demonstrated, and a real-time forecast presented. Because of wet conditions for the winter of 2001–2002 in Indian River County, springtime drought (lowering of WTD) was less severe than usual. As a result, probabilities of epidemic SLEV transmission were predicted to be lower than expected, based on climatology, for the July through October 2002 season. To our knowledge, these analyses are the first application of forecast verification methods to a predictive disease transmission model. This work demonstrates a means by which other empirical models of disease transmission can be tested for predictive skill.
We have shown that the TBH model can be used to predict SLEV transmission. Other hydrology models might also be developed and their simulations compared with those of the topographically based hydrology model. Such models would have to capture the spatial and temporal variability of near surface wetness conditions and be easily calibrated and computationally efficient. The TBH model was calibrated for 1983 to 2001 (
A forecast of SLEV transmission should incorporate additional information regarding the dynamics of the avian hosts, mosquito vector, and virus. For instance, monitoring of avian host susceptibility to the SLEV, in addition to modeling of local hydrology, is needed to determine whether conditions ideal for amplification exist. Remote sensing data should also be incorporated to delineate the effects of changes in land use, urbanization, and habitat fragmentation. Future investigations might also characterize the direct effects of large-scale climate phenomena, such as the El Niño-Southern Oscillation or North Atlantic Oscillation, on SLEV transmission. Such information would help further constrain epidemic SLEV transmission forecasts and permit more accurate identification and prediction of local amplification “hot spots” throughout south-central Florida. Such forecasts could be run operationally at the state and county level in conjunction with water management and public health agencies. Forecasts of epidemic SLEV transmission in excess of climatology would then warrant response and the targeting of mosquitoes during the amplification phase.
Whether other, past south Florida SLEV epidemics conformed to similar amplification dynamics, epicenters for these epidemics must be identified, and the local hydrology modeled to determine whether a similarly timed drought and wetting pattern preceded SLEV transmission. The findings of such studies will no doubt modify the empirical relationship between modeled local hydrologic conditions and epidemic SLEV transmission (equation 2). Research is also under way to determine whether West Nile virus transmission is similarly affected by hydrologic variability. By accounting for the interaction of the physical (climate) and biologic (vector, pathogen, and host) systems, a more robust means of monitoring and forecasting disease should be attained.
The Topographically-Based Hydrology (TBH) Model
Dependence in the Transmission Number Category Data
Empirical Methods
Suggested citation for this article: Shaman J, Day JF, Stieglitz M, Zebiak S, Cane M. Seasonal forecast of St. Louis encephalitis virus transmission, Florida. Emerg Infect Dis [serial on the Internet]. 2004 May [date cited]. Available from:
We thank D. Lohmann and B. Cosgrove for provision of the Global Energy and Water Cycle Experiment (GEWEX) Continental Scale International Project (GCIP) Land Data Assimilation System forcing datasets; Lillian Stark and her staff at the Florida Department of Health Tampa Branch Laboratory for their analysis of sentinel chicken and wild bird serum samples; and P. Shaman for many helpful statistical discussions.
This research was supported by the National Aeronautics and Space Administration Headquarters under the Earth System Science Fellowship, Grant NGT5-50323, the National Oceanic and Atmospheric Administration Postdoctoral Program in Climate and Global Change, administered by the University Corporation for Atmospheric Research, and the National Aeronautics and Space Administration Seasonal-to-Interannual Prediction Project at Goddard Space Flight Center.
Dr. Shaman is a National Oceanic and Atmospheric Administration postdoctoral fellow in climate and global change at Harvard University. His research interests include climatology, hydrology, vector-borne diseases, and mosquito ecology.