Public reports of dead birds were obtained from the DBSP. Through press releases and various types of media campaigns at state and local levels, citizens were encouraged to use the hotline (1-877-WNV-BIRD) and website (www.westnile.ca.gov) to report dead birds (7). Information regarding location, date found, and species was collected for each dead bird reported to the hotline; multiple dead birds included in a single report were treated as multiple reports. Hotline staff screened and entered these data into an Access database (Microsoft Corporation, Redlands, WA, USA); data were subsequently geocoded by using ArcMap version 9.1 and associated 2005 StreetMap USA Plus AltNames street dataset (Environmental Systems Research Institute, Inc., Redlands, CA, USA). WNV became a reportable disease in California in 2005, and human data were collected by local health departments by standardized case history forms. Data were subsequently stripped of personal identifiers, and addresses were geocoded by using a CDPH batch geocoding service that used multiple reference databases (www.ehib.org). Use of human data was approved by the institutional review board at the California Health and Human Services Agency (project no. 05-06-51).

The DYCAST procedure was implemented by using GIS software, Smallworld 3.2.1, and Magik programming language (General Electric Company, Fairfield, CT, USA). Regions comprising 32,517 km² among 16 participating agencies in 17 counties were superimposed by grids consisting of ≈0.44 km² (≈0.17 mi²) cells (Figure 1). Clustering of dead bird reports was quantified by using a Knox test (16,17) implemented from the center of individual cells; spatial and temporal parameters were defined by using biologically relevant values (Figure 2). The 2.4-km (1.5-mi) radius of the spatial domain represents 2× the daily feeding distance (14) of Culex spp. mosquitoes in California (18). The effective flight range of these mosquitoes is also 2.4 km (19), which corresponds to the maximum distance from breeding sites over which a sufficient number of vectors are able to disperse a mosquito-borne disease (20). The temporal domain of 21 days was based on a 7-day extrinsic incubation period of WNV, which ranges from 5 to 8 days for Culex spp. mosquitoes at 28ºC in California (21), plus 2 avian infection cycles of 7 days each (approximate maximum time from infection to death 5,14,22; ). Candidate values for defining proximity of dead birds in space (0.40, 0.56, and 0.64 km) and time (3, 4, and 5 days) were based on the limited mobility (caused by lethargy, ataxia, and reluctance to fly) and lifespan of infected amplification hosts (5,14,23). During model calibration, locations of human cases were compared with DYCAST risk maps generated by using various combinations of candidate values; 0.40 km (0.25 mi) and 3 days were selected as the optimal combination for the final model. For this calibration, the daily DYCAST procedure was run retrospectively (once during May 2005) by using dead bird and human case data from May 1 through September 30, 2004 within Los Angeles, Orange, Riverside, and San Bernardino Counties, which contained 664 (85.2%) of 779 statewide cases in 2004.

The DYCAST procedure was run at the center of every cell for which a minimum of 15 birds (the analysis threshold) was reported within the spatiotemporal domain, to minimize statistical instability that otherwise occurs at lower numbers of birds (14). Clustering was evaluated by comparing the observed number of pairs of dead birds that were close in both space and time (based on aforementioned values of proximity), with the expected number of pairs given a random spatiotemporal distribution of these reports (15). The resulting p values were assigned to individual cells, which were considered to indicate high risk for WNV transmission to humans at p<0.1 (15).

Model evaluation was conducted by analyzing the relationship between the location of human cases and the ability of DYCAST to predict their occurrence in both space and time. Prediction was defined as the identification of a cell as high risk before or on the date of illness onset (15) of the earliest case located within a cell. Sensitivity was calculated as the number of high-risk cells classified as predicted (true positives) divided by the total number of cells in which a human case occurred. Specificity was calculated as the proportion of low-risk cells without cases (true negatives) to the total number of cells without cases. Because agreement between model predictions and cases can occur by chance, a spatiotemporal implementation of the κ statistic was used to provide a measure of chance-adjusted agreement (15,24).

An initial pilot phase and subsequent prospective implementation occurred through a cooperative agreement with the Center for Advanced Research of Spatial Information at Hunter College, City University of New York. The Center for Vectorborne Diseases (CVEC) at the University of California Davis provided server infrastructure (Microsoft SQL Server, Microsoft Corporation; ArcIMS, Environmental Systems Research Institute, Inc.) for data exchange and implementation of interactive online risk maps, in collaboration with CDPH and the Mosquito and Vector Control Association of California. The Center for Advanced Research of Spatial Information calibrated and ran the DYCAST procedure and exported the data to the CVEC map server. During the pilot phase, animations of daily risk from June 1 through June 23, 2005, were retrospectively generated for 3 study areas that were selected based on high numbers of dead bird reports: the south Sacramento Valley region (Sacramento, Placer, and Yolo Counties), the central San Joaquin Valley region (Fresno, Kings, and Tulare Counties), and the greater Los Angeles area.

Prospective modeling began on June 17, and on July 1 the system was fully implemented and integrated into the CDPH WNV Surveillance Program. This implementation involved running the DYCAST procedure for analysis regions every weekday through November 1, 2005; daily risk maps (Figure 3) were generated and made available in real time to mosquito control agencies via the CVEC password-protected website, the California Vectorborne Disease Surveillance Gateway (www.calsurv.org). These interactive maps were overlaid with county boundaries, streets, and locations of reported and WNV-positive dead birds.

Beginning in 2006, DYCAST was implemented for the entire state of California and adopted as a formal component of the California Mosquito-Borne Virus Surveillance and Response Plan (8). Addresses of where dead birds were found were automatically geocoded in real time by using the Yahoo Maps application programming interface (Yahoo! Inc., Sunnyvale, CA, USA), which allowed hotline staff to validate location data while callers remained on the line; birds not automatically geocoded were omitted from DYCAST analysis. Interactive DYCAST risk maps were made available online to local mosquito control agencies and integrated with dead bird, mosquito, and sentinel chicken surveillance data from May 1 through October 31, 2006, May 1 through August 31, 2007 and 2008, and March 1 through August 31, 2009. Statewide reports of DYCAST activity, including maps and animations of high-risk areas over time, were sent to local agencies on a routine basis. A real-time alert system was also introduced in 2006 to provide custom DYCAST reports and interpretations for counties experiencing rapidly increasing or elevated levels of high-risk areas (8).

In December of 2006 and 2007, links to web-based surveys regarding the DBSP were provided by email to 64 local mosquito control agencies in 47 counties, in part to assess which agencies used DYCAST to assist mosquito larviciding or adulticiding activities each year. For agencies that participated in the 2005 DYCAST program, the 2006 survey also asked if DYCAST results were used “to assist public education or to promote dead bird reporting” in 2005 (control activities were not surveyed for this year). Rate ratios (RRs) were used to compare annual DYCAST prediction rates of reported human WNV cases between agencies that did and did not use DYCAST to assist each mosquito control activity (25).

Results

During 2005, a total of 124,876 calls were placed to the DBSP hotline, >3 million hits were made to the website, and 109,358 dead birds were reported in California (Table 1) (26). DYCAST identified high risk in 9.7% of the analysis regions (7,160/73,767 cells; 3,139/32,517 km²), with cells identified as high risk for a mean total of 39.0 days (range 1–117, median 39, SD 22.8 days). Relative risk of a WNV case in high-risk cells compared with low-risk cells was 39.10 (95% confidence interval [CI] 29.80–51.30; p<0.0001). Sensitivity and specificity of the DYCAST system were 80.8% (269/333 cells) and 90.6% (66,543/73,434 cells), respectively (Table 2). Prevalence of cells containing cases was 0.45% (333/73,767 cells), which resulted in low positive predictive value (3.8%; 269/7,160 cells) and high negative predictive value (99.9%; 66,543/66,607 cells). κ values maintained a moderate strength of chance-adjusted agreement (0.40<κ<0.60; 27) for >4 weeks before onset of illness (Figure 4). Overall, 289/354 (81.6%) of cases were predicted (Table 3), with cells identified as high risk before onset of illness by a mean of 37.2 days (range 0–126, median 34, SD 20.9 days). A total of 252/354 (71.2%) of cases were predicted 15 days before onset of illness, and >50% of cases (179/354, 50.6%) were predicted 30 days before onset of illness.

According to the 2006 survey, 10/14 (71.4%) of responding local mosquito control agencies within the analysis regions used DYCAST results to assist public education or to promote dead bird reporting in 2005. DYCAST risk maps were also used to direct WNV surveillance (and ultimately control) efforts as early as the pilot phase, when 2 emerging clusters of high-risk cells were identified around the Arden-Arcade and Citrus Heights regions of Sacramento County on June 24, 2005 (Figure 3, June 23, 2005). These results were immediately shared with the Sacramento-Yolo Mosquito & Vector Control District (SYMVCD), which used the risk maps to deploy mosquito traps within the 2 high-risk clusters on June 28 (D. Brown, SYMVCD, pers. comm.). On June 29, SYMVCD collected and detected 4 WNV-positive Cx. pipiens mosquito pools from these traps in both areas, which represented the first positive mosquito pools in Sacramento County that year (29). Additionally, both human cases from Sacramento County with onset of illness before June 23 were located within cells identified by DYCAST as high risk, 12 and 2 days before onset of illness (Figure 3, June 23, 2005). Within Sacramento County, DYCAST predicted 142/152 (93.4%) of cases (11/163 cases were missing spatial or temporal data); 122/152 (80.3%) and 84/152 (55.3%) of cases were predicted 15 and 30 days, respectively, before onset of illness (Figure 4).

After 2005, the number of reported dead birds generally decreased (Table 1); the percentage of successfully geocoded dead bird reports ranged from 98.8% to 100% each year. The statewide DYCAST prediction rates for reported human cases during 2006–2009 were 26.3% (67/255), 33.0% (110/333), 16.3% (64/392), and 3.2% (3/93), respectively (Table 3). Responses to the 2006 and 2007 surveys were received from 47 agencies in 36 counties and 18 agencies in 19 counties, respectively. Results indicated that most of the agencies that responded each year used DYCAST to assist larviciding or adulticiding activities (Table 4). DYCAST prediction rates were significantly higher for agencies that answered “yes” to questions regarding larviciding (RR 10.06, 95% CI 2.45–41.32) and adulticiding (RR 10.91, 95% CI 2.65–44.88) in 2006 and larviciding (RR 10.16, 95% CI 1.41–73.00) in 2007 (Table 4). Conversely, the prediction rate was significantly lower for agencies answering “yes” to the question regarding adulticiding (RR 0.37, 95% CI 0.15–0.91) in 2007. However, excluding the most extreme outlier with respect to total number of cases, an agency in Kern County whose jurisdiction included 43 and 124 cases in 2006 and 2007, respectively, RRs were not significant in 2007 and were incalculable for 2006 (Table 4).

Discussion

Results from prospective implementation of the DYCAST system in California indicate that the risk model provided accurate and early identification of areas at high risk for WNV transmission to humans during a statewide epidemic in 2005, and was used by local agencies to assist public education campaigns, surveillance, and mosquito control programs. Our findings indicate that DYCAST yielded high levels of sensitivity and specificity for predicting human cases during the 2005 epidemic and that relative risk for a WNV case was >39× higher in high-risk cells than in low-risk cells (this value should be considered somewhat inflated, however, because not all low-risk cells contained populated areas). Given the low prevalence of cells containing cases (0.45%), the dynamic nature of DYCAST, and the (>1 cell) spatial scale of WNV transmission and mosquito control (8), positive predictive value is considered inferior to other metrics such as κ for evaluating model predictions. κ values >0.50 indicate that DYCAST correctly identified >50% of cells expected to be misidentified by chance alone, which is considered high because WNV causes symptoms in only ≈20% of infections (28). Values maintained a moderate strength of chance-adjusted agreement for >4 weeks before onset of illness, which indicates temporal robustness of model predictions.

Cells containing predicted cases were identified as high risk before onset of illness by a mean of 37.2 days; given the 2–14 day range of the human WNV incubation period (28), this identification preceded transmission to humans and provided sufficient time to respond and potentially reduce the number of infections (Figure 4, Figure 5). Indeed, 252/354 (71.2%) of cases were predicted 15 days before onset of illness, before the maximum range of the incubation period. Additionally, because the DYCAST procedure only analyzes dead bird reports, it provided for more timely results than did active systems relying on the collection and testing of bird carcasses.

Results from Sacramento County in 2005 demonstrate the practical application of DYCAST for conserving and directing public health resources, such as targeting surveillance efforts that detected the county’s first positive mosquito pools that year. During subsequent months, Sacramento County was the location of the largest WNV epidemic in the United States, with 163 reported human cases (30) and an incidence rate of 14.5 infections per 100,000 population (31). DYCAST results played a key role in SYMVCD’s decisions for implementing and targeting emergency aerial mosquito control in the county (D. Brown, pers. comm.; 31), which ultimately reduced human illness and potential death from WNV infection (32).

Notably, prediction rates during 2006–2009 were substantially lower than in 2005, which has implications for the robustness of the model in nonepidemic years or regions. The fairly prevalent use of DYCAST results to assist mosquito control activities in 2006 and 2007 may have played a role in reducing the model’s prediction rates in circumstances in which WNV transmission was successfully interrupted before human infection occurred (31,32). However, while DYCAST could have helped to reduce the absolute number of cases, relatively higher prediction rates were generally observed for agencies that used DYCAST results compared with agencies that did not (Table 4). One explanation is that these areas may have had higher rates of WNV transmission initially, which in turn may have increased agencies’ likelihood of utilizing DYCAST for directing control activities or of simply conducting control activities in general. Furthermore, higher rates of WNV transmission may yield greater numbers of subsequent cases within high-risk cells or clusters, thereby increasing the prediction rate. This phenomenon could have also contributed to the higher prediction rates observed during the more severe epidemic in 2005, as could have the self-selecting nature of agencies that participated in the DYCAST program that year, which may have included areas with higher rates of WNV transmission compared with the rest of the state or to subsequent years.

Efficacy and sustainability of the DYCAST system may be compromised by declines in dead bird reporting, which could be caused by public fatigue or apathy, reductions in reporting infrastructure, or declines in bird deaths caused by herd immunity (33). Potential approaches for ameliorating these effects could include recalibration of DYCAST parameters (e.g., lowering the analysis threshold), strategic timing and targeting of press releases and media campaigns, and technologic solutions such as mobile phone application software and text messaging to disseminate information and facilitate the reporting of dead birds. Furthermore, it is uncertain how DYCAST results are affected by spatial and temporal heterogeneities of WNV transmission, including inter- and intraspecies variability in the competence (21,34), pathology (6), and distribution of vector and host populations (35,36). Other confounding factors may include demographic and socioeconomic composition of human populations (37) as well as environmental (38) and meteorologic variation.

Regardless, DYCAST proved to be a timely and effective early warning system during a severe WNV epidemic. The use of such prospective measures enable the conservation and focus of valuable human and financial resources, which in some cases could be the difference in making an otherwise chaotic epidemic situation tractable. More responsive and efficient surveillance and control can prevent additional human disease, decrease reliance on more substantial control activities later in the season, and reduce indirect costs from medical expenses and productivity loss. The total cost of the 2005 WNV epidemic in Sacramento County alone has been estimated at ≈$3 million (39). Furthermore, dynamic monitoring of risk throughout the season may inform decisions for redirecting and triaging resources and may also provide a means for evaluating efficacy of mosquito control efforts. Ultimately, the DYCAST system illustrates the utility of establishing a biologically relevant, spatiotemporal framework for disease surveillance, and adaptation of the DYCAST method may be useful for detecting other infectious diseases and clustering phenomena.

This study also highlights the benefits of interdisciplinary and interagency collaboration; synergies between 2 academic institutions and a governmental public health agency shortened the time from research to implementation, and engagement with local mosquito control agencies enabled the practical application of results in real time. Furthermore, our findings demonstrate the potential of harnessing the public’s ability to provide timely and useful surveillance data through telephone and internet communications. The leveraging of similar sociotechnologic infrastructure, from mobile phones to internet search queries and social networks, may play a major role in the success, scalability, and cost-effectiveness of predicting and preventing emerging diseases in the future.