Statistical algorithms applied to disease surveillance data aim to identify which cases most likely represent outbreaks, before local public health authorities would otherwise become aware of them. Early detection of outbreaks may increase the ability of local authorities to prevent additional outbreak-related cases. The algorithms work by applying statistical techniques to reported cases of disease, laboratory data, or pharmacy data to identify unusual deviations from expected values; some techniques use historic data to detect deviations from temporal trends and others examine spatial or spatio-temporal differences in disease concentrations [1,2]. Non-statistical methods may also be applied to detect changes in the spatio-temporal distribution of disease burden.

Tuberculosis (TB) incidence has been declining in the United States for almost two decades, resulting in 10,528 reported cases and a rate of 3.4 per 100,000 in 2011 (5.8% and 6.4% decrease, respectively, from 2010) [3]. Molecular characterization of Mycobacterium tuberculosis isolates (TB genotyping) provides a mechanism to detect TB transmission events that might otherwise go unnoticed, based on the principle that epidemiologically linked cases share a similar genotype. In the United States, routine genotyping of M. tuberculosis isolates from culture-positive TB cases started in 2004 through the Centers for Disease Control and Prevention’s (CDC) National Tuberculosis Genotyping Service [4]. In 2010, CDC launched the TB Genotyping Information Management System (TB GIMS), a secure web-based database available to all 50 states, the District of Columbia, Puerto Rico, and the U.S.-affiliated Pacific Islands [5]. In 2008 and 2009, 80% and 87% of U.S. culture-positive cases, respectively, had complete genotype and surveillance records available in TB GIMS.

We retrospectively assessed the effectiveness of three statistical methods for detecting infectious disease outbreaks, using surveillance and genotyping data associated with TB outbreaks investigated by CDC during 2008–2009. We selected cumulative sums (CUSUM) as one of our outbreak detection methods since they are an industrial standard and used by CDC in the Early Aberration Reporting System [2]; however, CUSUM is more challenging to implement with rare diseases such as TB. SaTScan and county-based log-likelihood ratio (LLR) both use the likelihood ratio method, with SaTScan adding a geographic search function [6]. We included county-based LLR as one of our methods for comparison because it is simple and consistent with how TB control is structured in the U.S., and thus may be easily applied in our setting.

Our results demonstrate that the three statistical methods, when applied retrospectively to routinely collected TB data, can successfully detect known TB outbreaks, potentially months before local public health authorities became aware of the problem. Early detection of outbreaks allows more prompt intervention potentially averting additional cases.

The three statistical methods we compared (county-based LLR, CUSUM, and SaTScan) showed similar results and no single method was clearly superior. The methods were all, to some degree, based on geospatial concentration, and predictably did not identify outbreak “I” where the TB diagnoses occurred in three different states, even though M. tuberculosis transmission had occurred in a single workplace [15]. County-based LLR and SaTScan did not identify outbreak “D,” which was associated with the most common genotype in the United States, accounting for 1,077 (4.1%) of the 25,973 TB cases with a genotype result during 2008–2009. While county-based LLR and SaTScan have successfully identified other outbreaks associated with common genotypes (unpublished data), we do not yet have enough experience with CUSUM to know if it consistently performs better than county-based LLR and SaTScan with common genotypes. Recent improvements in our genotyping methods may also improve our ability to detect outbreaks involving common genotypes. In 2009, CDC expanded the routine panel for Mycobacterial Interspersed Repetitive Units genotyping from 12 to 24 loci. This additional discriminatory power should increase the specificity of our detection methods, particularly for common TB strains such as the one associated with outbreak “D.”

For other diseases with shorter incubation periods, studies that have evaluated methods to detect outbreaks or aberrant clusters of disease have generally been temporally focused, often using syndromic surveillance data [1,16-18]. Some researchers have applied quality control schemes, such as CUSUM charts or variations on the CUSUM method, to detect shifts in reported counts of health events [2,19-23]. Others have used time series methods such as the autoregressive integrated moving average (ARIMA) model when cases of a disease are cyclical in nature and have seasonal trends [2,24-29]. Both methodologies use baseline data to establish expected counts in a given time interval, where the baseline period is typically defined using historical data that are representative of usual patterns in disease counts and do not reflect outbreaks or unusual events. Given the low incidence of TB in the United States and even lower counts of cases with a specific genotype, in addition to the often years-long incubation period for TB disease, our application of CUSUM involved sparse data. The lack of genotyping data before 2005 limited our ability to use an historical period to define the baseline, which led to our use of multiple 2-year time windows to establish the CUSUM background rate; we plan on revisiting our determination of baselines when a longer period of genotyping data is available.

Our examination of geospatial statistical methods was based on the assumption that TB transmission is mostly local [30-32]. Both the county-based LLR and SaTScan look for an elevated concentration of cases with a genotype in a geographic area compared with the concentration of that genotype outside the area. SaTScan addresses the bias introduced by the county-based LLR when cases are aggregated using a predefined geographic area [33]. Sensitivity analyses using different values of the SaTScan maximum search radius suggested that a scanning radius of 50 km was sufficient to detect the geographically concentrated outbreaks in this study.

Our analysis has several important limitations. First, the success of aberration detection methods fundamentally relies on the completeness, quality, and timeliness of the underlying data. Regarding the issue of missing data, several investigations in the United States have concluded that TB case detection and reporting are excellent [34-36]. All three methods use genotype data and genotyping is only possible on culture-positive TB cases, or approximately three quarters of TB cases reported to CDC [3]; 89% of culture-positive TB cases in 2009 had isolates submitted for genotyping. The omission of TB patients with culture-positive TB who do not have a genotyped isolate as well as culture negative cases may affect the performance of our methods in terms of detecting outbreaks and timeliness of the alerts. Of note, we captured 78% of the cases confirmed to be part of the nine outbreaks (data not shown).

With respect to timeliness our retrospective analysis assumed no delays in reporting; however, delays in data availability are inevitable, and real-life performance of our outbreak detection methods will be less robust. The median time between specimen collection and availability of both genotyping and surveillance data for a patient was 142 days when TB GIMS debuted in 2010 [37]. Factoring in a delay of 142 days (a delay anticipated to decrease as TB GIMS use increases), our statistical methods would have issued an alert before local public health workers noticed the problem for four of the nine outbreaks.

Second, as demonstrated by outbreak “I,” our methods were only designed to detect outbreaks in geographically limited areas. CUSUM or a purely temporal scan statistic could potentially detect geographically dispersed outbreaks if applied on a national or regional level. Third, we examined outbreaks that were investigated by CDC, and our results may not be generalizable to all U.S. TB outbreaks.

Fourth, our analysis used known outbreaks to examine the sensitivity and timeliness, but not specificity, of the statistical methods studied. During the 2-year period 2009–2010 only 282 county-based clusters met the alert threshold (LLR≥5) used in this paper and 590 SaTScan clusters met the threshold of p<.05, but the proportion of these clusters that represent false-positive alerts is unknown. Our focus in this study was the sensitivity of our methods in retrospectively detecting outbreaks and is an important first step in the assessment of the value of statistical outbreak detection; the next step is an examination of specificity. In a recent study local health authorities investigated 24 TB clusters of 6 cases or more to determine which ones were outbreaks; an algorithm using the SaTscan method had a specificity of 83.3% (5 of 6 clusters determined not to be outbreaks were not alerted using the algorithm) [38].

Finally, the number of cases that occurred after the alerts and before detection by local authorities that could have been prevented is indeterminate.

In conclusion, the best future application of these statistical methods to outbreak detection might be to use all three approaches, while gaining experience about the relative uncertainty associated with each method’s performance and applying this experience to improve model accuracy. Determining how to interpret discrepant results and exploring how to optimally set model input parameters will be important steps as well. We are now prospectively examining the performance of these methods in detecting TB outbreaks.

CDC: Centers for Disease Control and Prevention; CUSUM: Cumulative sums; LLR: Log-likelihood ratio; TB: Tuberculosis; TB GIMS: Tuberculosis Genotyping Information Management System.

The authors declare that they have no competing interests.

JG, TRN, and JSK provided the conception and design for the study. JSK, SA, MBH extracted and collected the data. NS, SA, JSK, MBH conducted the analyses and interpretation of results. JSK wrote the manuscript. JG and TRN provided overall study supervision. All authors participated in the review and approval of the manuscript.