Conceived and designed the experiments: RC EB AW FT MC. Performed the experiments: RC AW FT. Analyzed the data: JFC CL RC MC. Contributed reagents/materials/analysis tools: PB EB. Wrote the paper: JFC MC CL EB RC. Approved the final version of the manuscript: JFC MC CL PB FT AW EB RC.
The stability of the accuracy of a diagnostic test is critical to whether clinicians can rely on its result. We aimed to assess whether the performance of a rapid antigen detection test (RADT) for group A streptococcus (GAS) is affected by the clinical spectrum and/or bacterial inoculum size.
Throat swabs were collected from 785 children with pharyngitis in an office-based, prospective, multicenter study (2009–2010). We analysed the effect of clinical spectrum (i.e., the McIsaac score and its components) and inoculum size (light or heavy GAS growth) on the accuracy (sensitivity, specificity, likelihood ratios and predictive values) of a RADT, with laboratory throat culture as the reference test. We also evaluated the accuracy of a McIsaac-score–based decision rule.
GAS prevalence was 36% (95CI: 33%–40%). The inoculum was heavy for 85% of cases (81%–89%). We found a significant spectrum effect on sensitivity, specificity, likelihood ratios and positive predictive value (p<0.05) but not negative predictive value, which was stable at about 92%. RADT sensitivity was greater for children with heavy than light inoculum (95% vs. 40%, p<0.001). After stratification by inoculum size, the spectrum effect on RADT sensitivity was significant only in patients with light inoculum, on univariate and multivariate analysis. The McIsaac-score–based decision rule had 99% (97%–100%) sensitivity and 52% (48%–57%) specificity.
Variations in RADT sensitivity only occur in patients with light inocula. Because the spectrum effect does not affect the negative predictive value of the test, clinicians who want to rule out GAS can rely on negative RADT results regardless of clinical features if they accept that about 10% of children with negative RADT results will have a positive throat culture. However, such a policy is more acceptable in populations with very low incidence of complications of GAS infection.
Group A streptococcus (GAS) is found in 20% to 40% of cases of childhood pharyngitis; the remaining cases are considered viral
Because of the need to reduce antibiotics consumption, clinical decision rules were developed to help clinicians determine which patients should undergo testing and/or treatment with antibiotics. A decision rule based on the McIsaac score was recently proposed for adults and children
We aimed to determine whether the diagnostic accuracy of a RADT for GAS pharyngitis is affected by the clinical spectrum effect and/or bacterial inoculum size and to validate a McIsaac-score–based decision rule.
This is a secondary analysis of data from an office-based, multicenter, prospective study that took place in France between October 2009 and June 2011 (unpublished data). The aims of the princeps study were to evaluate the frequency of GAS carriage in healthy children and to compare the performance of a RADT between children with pharyngitis and healthy children, with throat culture in a microbiology laboratory as the reference test (intermediate results presented at the 28th Annual Meeting of The European Society for Paediatric Infectious Diseases, Nice, France, 2010; abstract A-229-0021-00920). This ancillary analysis was restricted to patients with pharyngitis from the first year of inclusion. The STARD statement was followed for reporting the results of the study
Eligible patients were 3 to 15 years old who were evaluated by their paediatrician between October 2009 and May 2010, who had a diagnosis of pharyngitis and did not receive antibiotics for 7 days before inclusion. Pharyngitis was defined by inflammation of the pharynx and/or tonsils. In total, 17 French paediatricians who are part of a research and teaching network (ACTIV) participated in the study
Throat samples were obtained by use of a double–swab collection–transportation system (Venturi Transystem Amies agar, COPAN Diagnostics, Corona, CA, USA). The RADT (StreptAtest, Dectrapharm, France), a GAS-specific immuno-chromatographic strip assay, was performed immediately with swab #1 collected in the paediatrician's office. Swab #2 was held at ambient temperature and sent within 72 hr to the Robert Debré Hospital laboratory by an express messenger service. On receipt, the swab was rolled over one-quarter of a trypticase soy agar plate with 5% sheep blood, and the inoculum was further distributed on the plate by streaking with a sterile wire loop. The plates were incubated anaerobically at 37°C and read after 18 to 24 hr. Negative cultures were reexamined after an additional 24 hr of incubation (48 hr total). ß-haemolytic colonies were further investigated by latex agglutination (Prolex, Pro-Lab Diagnostics, Richmond Hill, ON, Canada). With GAS positivity, inoculum size was estimated as follows: 1+, <10 colonies; 2+, 11–50 colonies; and 3+, >50 colonies per plate. Data for patients with 1+ and 2+ cultures were combined to produce a dichotomized variable (light or heavy GAS growth). Microbiologists were blinded to individual clinical data and RADT results.
We collected clinical data needed to calculate the McIsaac score for each patient. The McIsaac score predicts GAS pharyngitis and ranges from 0 to 5
In the original study, sample size was estimated so that the 95% confidence interval (95CI) for RADT sensitivity would be a +/−5% estimation. Assuming sensitivity to vary between 75% and 95% and a GAS prevalence of 35%, a sample of 714 children with pharyngitis was needed. No other sample size calculation was performed for this secondary analysis.
The study protocol was approved by the Institutional Review Board Comité de Protection des Personnes Ile-de-France XI (n°09016) and the French administrative authorities (CNIL, n°1354254; Afssaps, n°2009-A00086-51). Parent and patient approval for participation was obtained before inclusion. Data were double entered into 4D software version 6.4 (4D) and the database was fully anonymized.
Hospital laboratory throat culture was considered the reference test. First, we described patient demographic features and overall prevalence of groups A, C and G β-haemolytic streptococci. Patients with group C or G β-haemolytic streptococci were considered GAS-negative in the analysis. Then, GAS prevalence by McIsaac score and each McIsaac criterion were described. Second, we used chi-square tests to investigate the clinical spectrum effect on RADT diagnostic accuracy (sensitivity, specificity, likelihood ratios and predictive values) by McIsaac score and each McIsaac criterion. Third, chi-square tests were used to compare the distribution of GAS inoculum size by office-to-laboratory delay (≤48 vs. >48 hr) and to study the effect of inoculum size (light or heavy GAS growth) on RADT sensitivity. Fourth, the distribution of heavy inocula by McIsaac score was explored by a chi-square test for trend, and the association of clinical signs and inoculum size was evaluated by comparing the frequency of heavy inoculum according to each clinical criterion by chi-square tests. Fifth, a binomial model with an identity link was used to evaluate the combination of clinical spectrum effect and inoculum size on RADT sensitivity
In total, 807 children met the inclusion criteria; 1 patient with unreadable RADT results, 2 with lost throat swabs and 19 with missing data for McIsaac score calculation were secondarily excluded. Therefore, analysis involved 785 children (351 girls [44.7%]), with mean (SD) age 6.1 (2.5) years. Overall, GAS prevalence was 36.3% (95CI, 32.9%–39.8%). In all, 4 (<1%) Group C and 7 (<1%) Group G β-haemolytic streptococci were identified on BAP. The mean age of secondarily excluded children was 5.9 (3.0) years and GAS was found in throat cultures for 4/20 (20.0% [7.9%–39.2%]).
GAS prevalence increased with increasing McIsaac score (p<0.001) (
| n (%) | ||||
| McIsaacscore | Total | RADT-positive | GAS-positive | Heavy inoculum |
| 1 | 12 (1) | 2/12 (17) | 3/12 (25) | 2/3 (67) |
| 2 | 103 (13) | 21/103 (20) | 25/103 (24) | 20/25 (80) |
| 3 | 262 (33) | 82/262 (31) | 89/262 (34) | 73/89 (82) |
| 4 | 286 (36) | 115/286 (40) | 114/286 (40) | 100/114 (88) |
| 5 | 122 (16) | 54/122 (44) | 54/122 (44) | 48/54 (89) |
All scores were ≥1 because all included patients were 3 to 15 years old.
Laboratory throat culture positive for GAS.
p = 0.09, χ2 test for trend for the frequency of heavy inoculum by McIsaac score.
| n (%) | GAS (%) | Sensitivity (95CI) | Specificity (95CI) | PLR (95CI) | NLR (95CI) | PPV (95CI) | NPV (95CI) | |||||||||||||||
| McIsaac score | ||||||||||||||||||||||
| ≤2 | 115 (14) | 24.3 | <0.001c | 75.0 (55.1–89.3) | 0.02c | 97.7 (91.9–99.7) | 0.04c | 32.6 (8.2–130.5) | 0.23 | 0.26 (0.13–0.49) | 0.09 | 91.3 (72.0–98.9) | 0.61c | 92.4 (84.9–96.9) | 0.83c | |||||||
| 3 | 262 (33) | 34.0 | 84.3 (75.0–91.1) | 96.0 (91.8–98.4) | 20.8 (10.0–43.3) | 0.16 (0.10–0.27) | 91.5 (83.2–96.5) | 92.2 (87.3–95.7) | ||||||||||||||
| ≥4 | 408 (51) | 41.2 | 89.9 (84.3–94.0) | 92.5 (88.4–95.5) | 12.0 (7.7–18.7) | 0.11 (0.07–0.17) | 89.3 (83.7–93.6) | 92.9 (88.9–95.8) | ||||||||||||||
| Age (years) | ||||||||||||||||||||||
| 3–8 | 663 (84) | 38.5 | <0.01 | 89.0 (84.5–92.6) | <0.001 | 94.1 (91.4–96.2) | 0.45 | 15.1 (10.2–22.4) | 0.62 | 0.12 (0.08–0.17) | <0.001 | 90.4 (86.1–93.8) | 0.48 | 93.2 (90.3–95.4) | 0.26 | |||||||
| 9–14 | 122 (16) | 24.6 | 66.7 (47.2–82.7) | 96.7 (90.8–99.3) | 20.4 (6.5–64.0) | 0.34 (0.21–0.57) | 87.0 (66.4–97.2) | 89.9 (82.2–95.0) | ||||||||||||||
| Lack of cough | ||||||||||||||||||||||
| Yes | 428 (55) | 44.4 | <0.001 | 90.0 (84.8–93.9) | 0.02 | 94.1 (90.3–96.7) | 0.65 | 15.3 (9.2–25.5) | 0.89 | 0.11 (0.07–0.16) | 0.02 | 92.4 (87.6–95.8) | 0.07 | 92.2 (88.1–95.2) | 0.76 | |||||||
| No | 357 (45) | 26.6 | 80.0 (70.5–87.5) | 95.0 (91.7–97.3) | 16.1 (9.4–27.7) | 0.21 (0.14–0.31) | 85.4 (76.3–92.0) | 92.9 (89.2–95.7) | ||||||||||||||
| Anterior cervical adenopathy | ||||||||||||||||||||||
| No | 407 (52) | 34.6 | 0.32 | 83.0 (75.7–88.8) | 0.07 | 97.7 (95.2–99.2) | 0.001 | 36.8 (16.6–81.4) | 0.003 | 0.17 (0.12–0.25) | 0.12 | 95.1 (89.7–98.2) | 0.01 | 91.5 (87.7–94.5) | 0.33 | |||||||
| Yes | 378 (48) | 38.1 | 90.3 (84.2–94.6) | 91.0 (86.6–94.4) | 10.1 (6.7–15.2) | 0.11 (0.06–0.18) | 86.1 (79.5–91.2) | 93.8 (89.9–96.6) | ||||||||||||||
| Tonsillar swelling or exudates | ||||||||||||||||||||||
| No | 157 (20) | 42.0 | 0.10 | 83.3 (72.1–91.4) | 0.36 | 95.6 (89.1–98.8) | 0.80 | 19.0 (7.2–49.7) | 0.71 | 0.17 (0.10–0.30) | 0.38 | 93.2 (83.5–98.1) | 0.47 | 88.8 (80.8–94.3) | 0.11 | |||||||
| Yes | 628 (80) | 34.9 | 87.7 (82.6–91.7) | 94.4 (91.7–96.4) | 15.6 (10.5–23.3) | 0.13 (0.09–0.19) | 89.3 (84.4–93.1) | 93.5 (90.6–95.6) | ||||||||||||||
| Fever | ||||||||||||||||||||||
| No | 246 (31) | 31.3 | 0.049 | 88.3 (79.0–94.5) | 0.62 | 93.5 (88.7–96.7) | 0.43 | 13.6 (7.6–24.2) | 0.47 | 0.13 (0.07–0.23) | 0.66 | 86.1 (76.5–92.8) | 0.15 | 94.6 (90.0–97.5) | 0.22 | |||||||
| Yes | 539 (69) | 38.6 | 86.1 (80.6–90.5) | 95.2 (92.3–97.2) | 17.8 (11.0–28.8) | 0.15 (0.10–0.21) | 91.8 (87.0–95.2) | 91.6 (88.1–94.3) | ||||||||||||||
| Total | 785 | 36.3 | 86.7 (82.2–90.4) | 94.6 (92.2–96.4) | 16.1 (11.1–23.2) | 0.14 (0.10–0.19) | 90.1 (86.0–93.4) | 92.6 (89.9–94.7) | ||||||||||||||
Abbreviations: GAS, group A streptococcus; PLR, positive likelihood ratio; NLR, negative likelihood ratio; PPV, positive predictive value; NPV, negative predictive value.
χ2 test; b Mantel-Haenszel's χ2 test; c χ2 test for trend.
Overall, the sensitivity and specificity of the RADT was 86.7% (82.2%–90.4%) and 94.6% (92.2%–96.4%), respectively; positive and negative likelihood ratios 16.1 (11.1–23.2) and 0.14 (0.10–0.19), respectively; and positive and negative predictive values 90.1% (86.0%–93.4%) and 92.6% (89.9%–94.7%), respectively.
We found significant variations in RADT sensitivity and specificity by McIsaac score (
The RADT performance varied significantly by McIsaac criteria. For example, sensitivity was significantly lower in children ≥9 years old than in younger children (66.7% [47.2%–82.7%] vs. 89.0% [84.5%–92.6%], p<0.01). The negative predictive value varied, but not significantly, by McIsaac score or clinical criteria, ranging from 88.8% (80.8%–94.3%) for children without tonsillar swelling or exudate to 94.6% (90.0%–97.5%) for children without fever.
Among 285 GAS-positive throat cultures, 243 (85% [80.6%–89.2%]) showed heavy inoculum. The distribution of inoculum size did not differ by office-to-laboratory delay (≤48 vs. >48 hr, p = 0.97). RADT sensitivity varied widely and significantly between patients with light and heavy inoculum (40.5% [25.6%–56.7%] vs. 94.7% [91.0%–97.1%], p<0.001). Light inoculum was found in 65.8% (50.0%–81.6%) of false-negative RADT cases versus 6.9% (3.7%–10.1%) of true-positive cases (p<0.001).
Frequency of heavy inocula increased with increasing McIsaac score, although not significantly (p = 0.09,
The results of univariate binomial modeling for RADT sensitivity were almost identical to those of stratified analysis (
| Univariate analysis | Multivariate analysis | ||||||||||
| Variable | All inocula | Light inoculum | Heavy inoculum | Light inoculum | Heavy inoculum | ||||||
| (n = 285) | (n = 42) | (n = 243) | (n = 42) | (n = 243) | |||||||
| Age (years) | |||||||||||
| 3–8 | 0.01 | <0.01 | 0.49 | 0.03 | <0.001 | 0.36 | |||||
| 9–14 | −22.4 (−39.7; −5.1) | −37.4 (−64.1; −10.7) | −4.6 (−17.4; 8.3) | −40.9 (−57.9; −24.0) | −5.7 (−17.9; 6.4) | ||||||
| Lack of cough | |||||||||||
| Yes | 0.03 | 0.09 | 0.14 | 0.19 | <0.001 | 0.08 | |||||
| No | −10.0 (−19.1; −9.0) | −25.0 (−53.6; 3.6) | −5.2 (−12.1; 1.7) | −23.6 (−23.7; −23.5) | −6.0 (−12.5; 0.6) | ||||||
| Anterior cervical adenopathy | |||||||||||
| No | 0.07 | 0.01 | 0.88 | 0.01 | 0.01 | 0.37 | |||||
| Yes | 7.3 (−0.6; 15.2) | 36.1 (7.7; 64.5) | −0.4 (−6.1; 5.2) | 40.5 (16.7; 64.3) | 2.0 (−2.3; 6.2) | ||||||
| Tonsillar swelling or exudate | |||||||||||
| No | 0.40 | 0.12 | 0.21 | 0.07 | n/a | 0.29 | |||||
| Yes | 4.3 (−5.6; 14.3) | −25.0 (−56.2; 6.2) | 5.4 (−3.1; 13.9) | 4.0 (−3.4; 11.4) | |||||||
| Fever | |||||||||||
| No | 0.61 | 0.27 | 0.77 | 0.26 | n/a | 0.41 | |||||
| Yes | −2.3 (−10.8; 6.3) | −19.1 (−53.0; 14.8) | 1.0 (−5.6; 7.6) | 2.3 (−3.3; 8.0) | |||||||
Data RADT sensitivity difference (95% confidence interval) unless indicated.
Abbreviations: ref, reference; n/a, not applicable.
Binomial modelling.
Interaction test between clinical variables and inoculum size (Wald test).
Model adjusted on age, lack of cough, anterior cervical adenopathy.
Model adjusted on age, lack of cough, anterior cervical adenopathy, tonsillar swelling or exudate and fever.
Not included in the model because of non-convergence.
Because of these multiple interactions and because of no convergence in the multivariate model after introducing interaction terms, 2 multivariate models of RADT sensitivity were fit by stratifying on inoculum size (
Among 785 children, 522 (66%) would have received antibiotics according to the decision rule. The rule had 98.9% (97.0%–99.8%) sensitivity and 52.0% (47.5%–56.5%) specificity.
Accuracy of a diagnostic test can vary across patient subgroups within a population, a phenomenon referred to as spectrum effect (or spectrum bias)
This study confirms the important effect of inoculum size on RADT sensitivity, already described in other studies, and reports for the first time an association of clinical spectrum and inoculum size. Our data suggest that having more streptococci in the throat (greater inoculum size) might be associated with more intense symptoms (higher McIsaac score) (p = 0.09), although we might not have had enough statistical power to validate this hypothesis. However, because we were not able to differentiate GAS carriers from truly infected patients (i.e., by assessment of streptococcal antibody response)
We chose to investigate the McIsaac score in 3 categories rather than compare the accuracy of the RADT above and below a defined breakpoint
One of the strengths of this study is its prospective, multicenter design, which limits the selection bias potentially present in other studies
Another limitation to our study is that some throat swabs were plated >48 hr after collection. This delay had no significant effect on the distribution of GAS inocula, but prolonged or inadequate shipping conditions could have resulted in the loss of viability of GAS if the original swab had only small numbers of bacteria. Therefore, some swabs with light inocula could have been falsely read as negative on throat culture. Because RADTs are more likely to give negative results with light inocula, these swabs might have led to a systematic decrease in number of RADT false-negative results and systematic increase in number of RADT true-negative results, with over-estimation of the negative predictive value of the RADT as a result. We can assume that this bias occurred at random because shipping and all bacteriologic investigations were blinded to clinical data and RADT results – non-differential measurement bias. Such a bias usually leads to loss of power, and our results regarding the stability of the negative predictive value of the RADT should be considered with caution until they are confirmed with a larger sample of patients.
The American Academy of Pediatrics advises that “
We thank A. Liboz (Department of Microbiology, Robert Debré Hospital), M. Boucherat, MD (database design), F. de La Rocque, MD (study conception), I. Ramay, D. Menguy, S. Tortorelli and M. de Pereira (ACTIV), Pr A. Morabia, MD, PhD (spectrum effect analysis methodological support) and all the physician investigators who participated in the RADT GAS study: M. Benani, MD, F. Corrard, MD, P. Deberdt, MD, A. Elbez, MD, M. Goldrey, MD, J. Gosselin, MD, M. Koskas, MD, P. Martin, MD, A.S. Michot, MD, N. Panis, MD, D. Qutob, MD, C. Romain, MD, O. Romain, MD, C. Schlemmer, MD.