Virus designations with stock concentrations are listed in Figure1. Frozen aliquots of stocks quantified by chicken embryo infectious virus titer (EID₅₀/ml) or MDCK tissue culture (TCID₅₀/ml) received from the Influenza Division, WHO Collaborating Center for Surveillance, Epidemiology and Control of Influenza, Centers for Disease Control and Prevention, Atlanta, GA, USA (CDC), were used for all determinations.

NP concentration (μg/ml) was measured by isotope dilution mass spectrometry as described for hemagglutinin and neuraminidase proteins.13,14 Virus stocks were enzymatically digested with trypsin and spiked with ¹³C- and ¹⁵N-labeled analogs of the NP target peptides (LIQNSITIER, LIQNSITIEK, and LIQNSLTIER for IAV and ALVDQVIGSR, VVLPISIYAK, and SGATGVAIK for IBV). Reverse-phase separation of peptides and analysis by mass spectrometry were as described.13,14 Publication with complete details of this method and applicability to a broader range of viruses is in process. Mass spectroscopy analysis was performed at the Division of Laboratory Science, National Center for Environmental Health, CDC.

One virus stock was used each day with the real-time RT-PCR and all RIDTs described in Table1. Each morning, a single virus stock was thawed and diluted in 0·9% saline (Sigma-Aldrich Company, St. Louis, MO), the only liquid medium compatible with all RIDTs used in this study. Virus stocks were thawed on ice and diluted in serial half-log-dilutions from 10⁻¹ to 10⁻⁴. Each virus dilution was transferred into 200 μl aliquots and stored on ice or in the 4°C refrigerator until used that day.

The CDC Influenza Virus rRT-PCR Diagnostic (Flu A&B) Panel (Influenza Reagent Resource, Manassas, VA, USA) was performed on each dilution as previously described.15 XY scatter plots of log₁₀ dilution versus the corresponding C_t value were generated (Microsoft Excel 2010, Microsoft Corp., Redmond, WA, USA). All 24 viral dilution series had a linear regression r² value above 0·95 (generally >0·99), assuring consistent dilution series for each virus. The C_t values for the 10⁻¹ dilution are used in analyses (see Figure1), as dilution curves tended to deviate from linearity when C_t values from the undiluted virus stock were included in the regression (data not shown).

Testing with RIDTs and RT-PCR was performed between October and December 2012 at the Medical College of Wisconsin. Rapid influenza diagnostic tests are listed in Figure2. Complete detailing of these RIDTs is available at http://www.cdc.gov/flu/professionals/diagnosis/clinician_guidance_ridt.htm#Table 2. Five of the RIDTs are CLIA-waived, categorized as simple laboratory examinations that have an insignificant risk of an erroneous result.

Positive and negative controls provided with each RIDT kit were tested upon receipt for each lot in every shipment. All aspects of the evaluation including diluent, swabs, and virus input were standardized. The manufacturer's instructions for testing a swab specimen directly (without placing the swab in transport medium) were always followed (nasopharyngeal swab instructions were used for most RIDTs; throat swab instructions for the BD Directigen EZ Influenza A+B).

Following virus stock dilution, 50 μl of each dilution was placed into three 1·5-ml microcentrifuge tubes and held on ice. A sterile foam swab (Catalog # 25-1506-1PF, Puritan Medical Products Co. LLC, Guilford, ME, USA) was used to absorb each of the 50 μl samples in the microcentrifuge tubes and used as the input. Adjustments to this procedure were used when RIDT instructions required input with liquid suspensions of swab samples. For the 3M™ Rapid Detection Flu A+B test and BD Veritor for Liquid Samples, after absorbing the sample the swab was placed into a tube containing 1 ml of UTM (Quidel Corporation, San Diego, CA, USA) and mixed prior to using the manufacturers' specified volume input for these two RIDTs. BinaxNOW requires placing the swab in an elution solution (either purchased or substituted with 500 μl saline, used in this testing). Both FluAlert (SA Scientific, San Antonio, TX, USA) RIDTs are only indicated for nasal wash and aspirate samples. As the required sample input for this RIDT is 250 μl, we combined the 50 μl dilution sample with 200 μl of 0·9% saline. Even though this is a CLIA-waived test, the moderate complexity protocol was used due to multiple invalid results during quality control testing with the waived protocol. Kit-provided flocked swabs were used with the Status® Flu A+B test (Princeton Biomedical, Monmouth Junction, NJ, USA), as instructions do not allow for foam swabs used with other RIDTs.

After the study started, Quidel issued a recall of previously used Sofia FIA lots. At that point, 1 false-positive influenza B result was recorded with a negative control during the use of over 450 Quidel Sofia tests (Table1). The manufacturer replaced remaining kits and no further influenza B false positives occurred with replacement kits. With the Status RIDT, four false positives for influenza B were observed: two in negative control replicates, one in a 10^−1·5 dilution of an A/Minnesota/03/2011 replicate, and one in a 10⁻¹ dilution of an A/Victoria/361/2011 replicate.

For analyses, the highest dilution reactive (HDR) was determined as the one in which two of the three replicates were positive for any one virus. Spearman's rank correlation analyses between stock ID₅₀ titers, NP concentration, and 10⁻¹ dilution C_t for each virus were used to assess the associations between these measures. A Spearman's rank correlation was also performed comparing the mean HDR for all RIDTs to the stock ID₅₀ titers, NP concentration, and 10⁻¹ dilution C_t for each virus. TCID₅₀ quantitations (three viruses) were omitted only from correlation analyses with ID₅₀ due to unverified comparability of TCID₅₀ and EID₅₀ methods. Any virus and RIDT combination with no reactivity in the 10⁻¹ dilution was not included in the correlation calculations; however, for subsequent analyses, the nominal value of the 10^−0·5 dilution was used. To compare virus groups for each RIDT, a log transformation was applied to normalize the variances of the reactivity measures before performing one-way anovas. P-values between significantly different IAV groups for individual RIDTs were determined by Tukey's honest significant difference test. All analyses were performed in Microsoft Excel 2010.

Figure1 lists the ID₅₀, C_t values, and NP concentration for the virus stocks. Correlation coefficients for the association between ID₅₀, NP concentration, and C_t values across viruses were weak for ID₅₀ and C_t versus NP (0·094 and −0·35) and moderate for ID₅₀ versus C_t values (−0·75). When the viruses with TCID₅₀ quantitation were included, the correlation was stronger for ID₅₀ versus NP and was weaker for ID₅₀ versus C_t values, but had no impact on interpretation of the results.

Figure1 shows the number of tests that were positive in at least two of three replicates at each dilution for each of the 24 viruses. All 13 RIDTs were reactive with seven of the 18 IAVs and all six of the IBVs. Only four RIDTs (Sofia, both Veritors, Directigen) were reactive with all IAVs in the initial dilution (10⁻¹) tested (Figure2 and Table1). The remaining RIDTs were not reactive with at least one IAV at any dilution tested. Notably, nine RIDTs detected all pH1N1 viruses, six RIDTs detected all H3N2 viruses, and eight RIDTs detected all H3N2v viruses in the 10⁻¹ dilution. One RIDT (SAS FluAlert Influenza A test and Influenza B test, which are separate test units, but boxed together in one kit) had reactivity in only seven IAVs (none in the H3N2v group, three in the pH1N1, and four in the H3N2 seasonal group). The reactivity of other RIDTs ranged from detection of 11 (Status) to 17 (X/Pect, OSOM, Alere) IAVs in the 10⁻¹ dilution (Table1).

For IAVs, patterns of reactivity are variable across all virus groups for both individual viruses (Figure1) and for individual RIDTs (Figure2). While we chose to score a dilution as reactive if 2 of 3 replicates at that dilution were positive, the majority of RIDTs yielded 3 of 3 positives at the highest dilution scored as reactive, and 0 of 3 positive results at all higher dilutions. There were seven occurrences for which an RIDT was scored reactive with 2 of 3 replicates positive. A total of 20 occurrences had only 1 of 3 positive replicates for any one RIDT in the next dilution beyond the HDR. The majority of these (15 of 20) were with RIDTs interpreted by automated readers from both fluorescent (Sofia, 3M) and reflectance (Veritor) signals. These readers may discriminate subtle differences in reaction intensity that are not apparent in visual reads.

Differences in mean stock NP concentrations between IAV that were reactive in all RIDTs and those that were not reactive in more than one RIDT suggest a link between NP concentration and test reactivity (Table S1). Those IAVs not reactive in more than one RIDT had a mean stock NP concentration of 1·9 μg/ml (range: 0·7–3·2), whereas the mean for all IAVs was 4·3 NP μg/ml (range: 0·7–13·4 μg/ml). ID₅₀ titer ranges were similar and overlapped considerably across each of the virus groups as did the C_t value ranges at the 10⁻¹ dilution for the IAVs. The range of stock NP concentrations was narrower for IBVs, yet HDRs varied widely across RIDTs.

Figure3 shows the mean log HDR for each virus plotted versus stock log NP, stock log ID₅₀, and 10⁻¹ dilution C_t values. For IAV, the correlation is strong (−0·86) between stock NP concentration and the mean HDR for all test kits. On the other hand, correlation between stock ID₅₀ and HDR is practically zero (−0·015) and weak between C_t values and HDR (0·24).

As NP concentration versus mean HDR had a strong association across all RIDTs for IAV, the mean NP (for all viruses in a virus group) was plotted against each RIDT (Figure4). anovas showed no significant difference (P-value >0·05) between any of the IAV groups for nine individual RIDTs. These nine RIDTs include those reactive with all IAVs (n = 4). While individual RIDTs showed some variation between IAVs, no individual IAV subtype was significantly less reactive across all RIDTs, when compared with NP concentrations by anova as shown in Figure4 with four exceptions. The FluAlert RIDT was apparently less reactive with pH1N1 and H3N2v (only 3 and 0, respectively) than with H3N2 viruses (four reactive). This particular RIDT, however, was less reactive for H3N2 than other RIDTs when compared to NP levels detected as shown in Figure4. Additionally, this RIDT failed to react with the pH1N1 virus with the highest stock NP measure (13·4 μg/ml for A/California/07/2009). Status was also less reactive with pH1N1 and H3N2v than with H3N2 viruses (4 and 1 reactive versus all 6, respectively) yet was similar to other RIDTs in the calculated NP reactivity for H3N2 viruses. QuickVue was less reactive with H3N2v than with either the pH1N1 or H3N2 group (only 2 of 6 H3N2v viruses were reactive, while all other IAVs were reactive). Note: Reduced reactivity with H3N2v viruses for QuickVue and FluAlert was also observed in a previous study.1 TRUFLU, on the other hand, was more reactive with the H3N2v group than with the pH1N1 group (6 reactive with H3N2v versus 5 with pH1N1) yet no statistical difference was found between pH1N1 and H3N2 virus groups for mean reactive NP levels. Figure4 notes IAV groups that were significantly more or less reactive (P-value <0·05) for any one RIDT.

Figure4 also shows the mean NP levels at HDR for each of the RIDTs with IBV. Trends or correlation for IBV were not evident as the virus NP stock concentrations (and also ID₅₀ titers and C_t values) for this small group of viruses were notably uniform, yet HDRs varied widely across individual RIDTs.