IntroductionGroup A streptococcal (GAS) pharyngitis is the most common cause of bacterial pharyngitis affecting over a half-billion people annually worldwide.1 GAS pharyngitis is both the antecedent for invasive streptococcal infections such as necrotizing fasciitis and the post-infectious immunologic complication of rheumatic fever/rheumatic heart disease, a leading cause of cardiovascular morbidity and mortality in many developing parts of the world. Physical examination of the posterior oropharynx is an inaccurate method to distinguish GAS from other causes of acute pharyngitis2, so the Centers for Disease Control and Prevention (CDC) and the American College of Physicians-American Society of Internal Medicine (ACP-ASIM), endorse applying the four point Centor clinical scoring scale to classify risk of GAS and guide management of acute pharyngitis in adults (Table 1.)3, 4 Developed three decades ago based on evaluation of 286 adults at a single emergency department, the Centor score helps clinicians distinguish GAS from viral pharyngitis, and thereby appropriately prescribe antibiotics to alleviate symptoms and decrease the rates of acute rheumatic fever, suppurative complications, missed school and work days, and disease transmission.5 The McIsaac score, derived from 521 patients from a University-affiliated family practice in Toronto and validated on 621 patients from 49 Ontario communities, adjusts the Centor score based on the patient’s age.6, 7 Since younger patients are more likely to have GAS than older patients, the McIsaac score is calculated by adding one point to the Centor score for patients ages 3-14 years, and subtracting one point for those age 45 years and above. Because clinical prediction models may perform poorly when applied to new settings, it is important to validate them on different populations and over time.8, 9 Further, despite endorsement from CDC and ACP-ASIM, the clinical scores have gained poor traction in clinical practice,10 perhaps in part due to the perception that the scores were derived from a relatively small sample. Here we analyzed a geographically diverse population of patients who presented with sore throat to MinuteClinic, a large retail health chain, to perform the largest validation studies of the Centor and McIsaac scores.
MethodsStudy DesignWe analyzed retrospective data collected from patients tested for GAS pharyngitis when they presented with a painful throat from September 1, 2006 to December 1, 2008 to MinuteClinic, a large, national retail health chain with over 500 sites in 26 states.11-14 From the retail clinic’s 581 sites, the dataset included 238,656 patient encounters across 25 states. In this setting, physician assistants or nurse practitioners collect standardized historical and physical exam information based on algorithm-driven care. The clinicians enter these codified data in real-time, and the information is stored in a common database across all clinic locations. MinuteClinic providers have demonstrated greater than 99% adherence to an established acute pharyngitis protocol, the “Strep Pharyngitis Algorithm” from the Institute for Clinical Systems Improvement.15, 16 According to this algorithm, medical providers collect structured information about patients’ relevant signs and symptoms, obtain rapid antigen testing on all patients with pharyngitis (with confirmatory testing used for patients whose rapid test is negative), and treat only those patients with a positive test for GAS. The dataset included only patient visits where there was complete information about age, all signs and symptoms included in the Centor and McIsaac scores, and test results.
We included patient-visits if a patient presented with a chief complaint of painful throat and was tested for GAS pharyngitis, or if a patient had symptoms of pharyngitis and was tested for GAS pharyngitis. Patient-visits were excluded if the patient reported having been treated for GAS within the one month prior to the visit. Patients under three years old were excluded since neither the Centor nor McIsaac rule is intended for use in those patients. For patients with multiple visits during the study period, we included the first visit only. Patients were not excluded if they were pregnant or had co-morbid conditions. MinuteClinic practice is to not care for patients with septic appearance but to refer them to emergency department care.
Test methodsAll MinuteClinic locations used the Clinical Laboratory Improvement Amendments-waived Quick Vue In-Line Strep A test (Quidel Corp., San Diego, California). The confirmatory test was a streptococcal DNA probe (74%) or throat culture (26%). Patients were categorized as GAS positive if either test (rapid or confirmatory) was positive.
Statistical AnalysisPredictor variables and covariates were developed for age, sex, history of fever in previous 24 hours, history of exposure to someone with GAS pharyngitis, presence of cough, duration of pharyngitis symptoms (days), presence of erythematous tonsils, presence of tonsillar exudates, presence of swollen tonsils, presence of swollen anterior cervical lymph nodes, presence of swollen posterior cervical lymph nodes, and presence of rhinorrhea. Streptococcal test results were extracted for each patient.
All patients fulfilling the inclusion and exclusion criteria were used to validate the McIsaac score and all patients age 15 years and older were used to validate the Centor score. The Centor score was calculated by summing the following clinical factors: history of fever, presence of tonsillar exudates, presence of swollen anterior cervical lymph nodes, and absence of cough. The McIsaac score was calculated for all patients aged three years and older by adding one point to the Centor score for those under age 15 years and subtracting one point from the Centor score for those aged 45 years and older.17 McIsaac scores of −1 and 5 were normalized to 0 and 4.7
Two approaches were taken to validate the scores. First, we compared the likelihood of GAS pharyngitis by clinical score in the MinuteClinic patients to the likelihood of GAS pharyngitis by clinical score in the published data. Second, we applied logistic regression to the MinuteClinic data to derive new prediction models, maintaining the same parameter that they be limited to no more than four clinical variables. The four chosen variables derived from the cohort of patients aged 15 years and older were then compared with the four variables that comprise the Centor score.
Calculation of GAS probabilitiesThe percent of patients aged 15 years and above in the retail health data who tested GAS positive by Centor score (0-4) was calculated and compared with the original Centor report4 and the Wigton validation study18. The percent of patients aged 3 years and older in the retail health data who tested GAS positive by McIsaac (0-4) was calculated and compared with the McIsaac studies.6, 7 95% confidence intervals were calculated for the proportion of patients testing positive at each score. The 95% confidence intervals around the proportion testing positive by score in the retail health data were compared with the 95% confidence intervals in the Centor and McIsaac studies.
Selection of variablesVariables included in the Centor and McIsaac scores as well as variables not included in the scores were examined to determine the best predictors of GAS pharyngitis among the MinuteClinic patients. Univariate and multivariate analyses were performed to identify predictors of GAS pharyngitis. Significance of association of categorical variables with GAS pharyngitis was tested by Chi-square. In the multivariate analysis, candidate predictors were entered into a stepwise logistic regression to identify independent predictors of patients with GAS pharyngitis. P value cutoffs for entry and departure for the multivariate regression models were 0.25 and 0.10, respectively. For the purpose of simplicity and usability and to facilitate comparison with the prior studies, the final model was limited to four predictor variables and assessed by area under the receiver operator characteristic curve (AUC).
Statistical analyses were performed using JMP Pro software, version 9.0.2 (SAS Institute, Cary, North Carolina).
ResultsOf 238,656 patient-visits, 5653 were excluded due to treatment for GAS within the prior month and an additional 1399 were excluded because the patient age was under three years old, leaving 231,604 patient-visits. For patients with multiple visits only the first visit was included, leaving 206,870 patients to validate the McIsaac score. Of these, 64,789 (31%) visits occurred in patients under age 15 years, leaving 142,081 visits for the validation of the Centor score (Figure 1).
Validation of the Centor scoreAmong the 142,081 retail health visits for patients age 15 years and older, 23% (95% confidence interval 22-23%) tested positive for GAS, compared to 17% (14-23%) in the original Centor paper and 26% (24-32%) in the validation study of the Centor score. Two-thirds of the patients in the retail health dataset were female, and the average age was 34 years. Table 2 displays the characteristics of the study population for age, sex and clinical signs and symptoms of pharyngitis by GAS result for those aged 3 years and older and for those aged 15 years and older. In both groups, patients who tested positive for GAS pharyngitis were more likely to present with tonsillar exudates, swollen anterior cervical lymph nodes, tonsillar swelling, history of fever in the previous 24 hours, absence of cough, lack of rhinorrhea, swollen posterior cervical lymph nodes, exposure to GAS, and temperature above 101 degrees Fahrenheit at the time of presentation.
Table 3 displays the percent of patients testing positive for GAS by clinical score in the retail health data, compared to the published literature by Centor et al4, Wigton et al18, and McIsaac et al.6, 7 Patients in the retail health population had GAS positivity rates in an intermediate range between the Centor and Wigton reports, and were more likely to have GAS pharyngitis than patients in the McIsaac study. The 95% confidence intervals around the percent of patients testing positive in the retail health cohort overlapped with the 95% confidence intervals around the percent testing positive by score in the Centor and Wigton studies. Table 4 shows the risk of GAS pharyngitis according to the number of predictors present, stratified by the ages used in the McIsaac classification. In the multivariate logistic regression model, the same four candidate predictors were selected from the retail health data, as in the original Centor report. Presence of tonsillar exudates conferred the highest odds of having strep infection (3.1, 95% confidence interval 3.0-3.2) followed by swollen anterior cervical lymph nodes (2.2, 2.1-2.3), history of fever (1.7, 1.7-1.8) and absence of cough (1.6, 1.5-1.6).
The overall performance of the model as applied to the retail health data was evaluated by comparing the areas under the receiver-operator characteristic curves. For patients aged 15 years and older, applying the Centor score to the retail health data yielded an AUC of 0.72. For patients aged 3 years and older, applying the McIsaac score to the retail health data achieved an AUC of 0.71.
DiscussionWe evaluated two commonly used prediction models to classify risk of GAS among patients presenting with a painful throat. The purpose of a clinical prediction model is to provide clinicians with a practical and applicable tool to improve medical decision-making, the health of individual patients and the public health. The Centor score is one model that is particularly robust; it has withstood 30 years of changes in diagnostic testing, information technology, and population dynamics.19 Our study validated the Centor score in a clinical setting (retail health chain) with a less acutely ill population than the emergency department setting in which it was derived. While the Centor score was derived from a relatively small number of patients (n=286) seen in one setting during a single two month period, we analyzed data from multiple locations spanning more than one calendar year, mitigating the potential impact of seasonality as the data are collected throughout the normal peaks and ebbs of GAS incidence. Logistic regression selected, from among the candidate predictors shown in Table 3, the same four predictors that were chosen in Centor’s landmark paper.
With data from over 140,000 patients, our analyses provide precise interpretations of risk for each Centor score category, that still lie within the 95% confidence interval of Centor’s original study based on fewer than 300 patients. As we have shown previously, the recent, local incidence of GAS pharyngitis further improves the accuracy of estimating an individual patient’s risk of GAS.20 Ebbs and peaks of GAS occur naturally throughout the year, so the retail health data in our analyses collected over more than one year average over those variations and should provide more reliable characterization of the score than the original study by Centor conducted over two months.
Area under the receiver operator characteristic curve (AUC) is one metric widely used to reflect the overall accuracy of a diagnostic test or overall performance of a clinical prediction model. The AUC of the Centor score in the retail health population (0.72) was lower than in Centor’s 1981 study (AUC=0.78) but the same as in Wigton’s validation study (AUC=0.72), arguing for the discriminating validity of this score. Clinical prediction models tend to perform less well in validation studies, but our data are consistent with the model’s performance in other validation studies.21 While McIsaac did not report an AUC on his original data, his score performed similarly to the others on this large data set.
The observed proportion of Minute Clinic patients testing positive according to clinical scores fell within the 95% confidence intervals of the Wigton and McIsaac validation studies (except for McIsaac score 0), supporting the calibration validity of the Centor and McIsaac scores.
Leveraging codified data from retail health clinics where uniform, algorithm-driven care is provided and data captured in a single electronic medical record, our study demonstrates the strengths of the Centor and McIsaac scores as useful tools in clinical decision-making. Though many clinicians in the primary care or emergency medicine setting do not routinely test adult patients who are either very likely or very unlikely to have GAS pharyngitis (i.e. those with Centor scores 0, 1 and 4), because MinuteClinic protocol mandates testing for all patients presenting with a painful throat, a further unique strength of our large validation study is ascertainment of GAS status on all subjects.
LimitationsThough all clinical and laboratory data were collected prospectively, the analyses were conducted retrospectively. There may be some variability in clinical interpretation of the Centor criteria by the nurse practitioners in the MinuteClinic setting; whether anterior cervical nodes are enlarged, for example, might be more subjective than other criteria such as temperature above 101.22 Further, data are not available for calculating inter- or intra-observer reliability.
Though very useful for diagnosing GAS, retail health data would be unlikely to detect most other bacterial causes of pharyngitis, including group C streptococcus or Fusobacterium necrophorum, the latter of which may cause severe disease especially in adolescents and young adults.23
All patients in the data set were symptomatic with sore throat, so our analyses do not address the important issue of the asymptomatic streptococcal carrier state. Serologic testing was not performed, so symptomatic patients with a positive test, were assumed to be true positives, not carriers.
Because these data were collected recently, we could not quantify potential changes in antibiotics attributable to the 2002 Infectious Diseases Society of America guideline24 and by the 2001 American College of Physicians guideline.25