Before the COVID-19 pandemic, infection with influenza virus resulted in 9–41 million cases of illness, 140,000–710,000 hospitalizations, and 12,000–52,000 deaths annually in the US [1]. Clinical trials for an influenza vaccine began in the 1930s [2], but the Advisory Committee on Immunization Practices (ACIP) did not recommend that everyone ≥6 months be vaccinated until the 2010–11 season [3]. The influenza vaccine formulation and effectiveness changes yearly [2], and observational studies conducted seasonally to determine vaccine effectiveness (VE) directly inform recommendations; for example, the ACIP did not recommend a live-attenuated influenza vaccine for the 2016–17 season [4] in the U.S. as a result of earlier VE monitoring showing low effectiveness [4,5].

A common observational method to study VE is the test-negative design (TND), where biospecimens are collected from patients with acute respiratory illness (ARI) presenting to a clinic with either utilization of clinically ordered test results or use of a standardized case definition to prompt prospective specimen collection. The TND was originally designed to reduce confounding from healthcare-seeking behavior, as individuals present for care with a pre-specified set of potential symptoms [6]. Those testing positive for influenza virus are considered cases and those testing negative controls. Logistic regression models are commonly used to compute odds ratios for the association between vaccination and positive test result, which can be converted to a VE estimate [7,8].

A methodological concern in using the TND is the choice of control group, as there are some instances where the control exchangeability assumption may be violated potentially introducing bias. The introduction of COVID-19 and respiratory syncytial virus (RSV) vaccines have raised potential issues of control selection when multiple diseases are vaccine preventable. Already, the TND has been used to estimate COVID-19 VE [9]. Testing positive for these vaccine-preventable viral illnesses could indicate lower probability of any vaccination, and could also indicate changes in the probability of contracting influenza [10]. Additionally, confounding is likely present between receipt of multiple ARI vaccinations [10].

Recent simulation studies have shown that including other vaccine-preventable ARI’s, particularly COVID-19, in the control group of influenza VE calculations may underestimate VE [10]. The objective of this paper was to examine the impact and potential biases associated with inclusion or exclusion of those with RSV from the control group when computing VE in an ambulatory population. Given the very recent introduction of the RSV vaccine, we simulate the potential bias introduced to influenza VE estimates assuming RSV was a vaccine preventable infection. We then use the 2017–18 and 2018–19 seasons within the Michigan Henry Ford Influenza Vaccine Effectiveness (MFIVE) TND cohort to estimate effects prior to RSV vaccination introduction, purposefully removing the impact of SARS-CoV-2.

Participants for non-simulation analysis were enrolled in MFIVE in the 2017–18 and 2018–19 influenza seasons. Inclusion criteria, viral testing, and data collection procedures have been previously described [7,11]. Briefly, patients aged ≥6 months presenting to an ambulatory care center for ARI who had not received any influenza antiviral treatment for their current illness were eligible for inclusion and collection of a study nasal-throat swab [7,11]. For this analysis, those who received an influenza vaccine <14 days prior to infection, were missing self-reported health data or vaccination status, had an RSV/influenza co-infection, or had their specimen collected >7 days after illness onset were excluded. Laboratory confirmation of infection was determined by RT-PCR for influenza type and subtype using primers and probes provided by the U.S. Centers for Disease Control and Prevention. RT-PCR for RSV infection used a previously reported assay [12]. This study was approved by the Institutional Review Board at our institution (HUM00119183).

In this simulation study and single-center analysis of influenza VE during the 2017–18 and 2018–19 influenza seasons, we found little evidence that including RSV, another significant ARI, in the control group biased estimates of influenza VE. Simulation analysis showed that when RSV is also a vaccine-preventable illness, influenza VE estimates could be moderately biased with RSV prevalence >10 % of controls, particularly with significant differences in VE between the two vaccinations. Together, these findings suggest that general inclusion of RSV+ in the control group of TND for influenza VE, particularly in populations where RSV is not vaccine-preventable, currently should not significantly bias VE estimates.

Our real-world analysis showed 8–25 % of the control population was RSV+, with this proportion being widely variable by population and season. Controlling for all other viral-positive infections in a TND could yield problems in sample size. A 2016–21 surveillance study of children with acute respiratory illness found that 63 % had some virus detected; notably, 7 % for flu, 17 % for RSV, and 7 % for SARS-CoV-2 [16]. In a study of adult ARI cases from 2015 to 19, 12 % had influenza virus detected and 5 % RSV [17]. Removal of large proportions of the control group may reduce sample size, which could widen confidence intervals, particularly within subgroup analyses, or bias estimates. Additionally, removal of viral positive cases could lead to a sample biased towards chronic diseases with ARI-like symptoms like COPD with a standard case definition [18], which would not be representative of the underlying population from which influenza cases arose.

We found little difference in influenza VE estimates when those RSV+ were included in the control group, compared to VE estimates without RSV+ individuals in the control group. These results are comparable to those by Segaloff et al., who similarly found little difference in VE estimates with varying control populations (anyone negative for influenza, only those positive for other respiratory viruses, and those negative for all common respiratory viruses) in the 2014–15 and 2015–16 influenza seasons [18].

The recent introduction of RSV vaccines for adults ≥60 years [19], along with existing programs for COVID-19 and influenza vaccination, creates the potential for up to three vaccine-preventable viral illnesses with similar clinical presentations during ascertainment within the TND. Simulation work by Doll et al. showed the potential for moderate to severe bias in influenza and COVID-19 VE estimates as the proportion of those in the control group with another vaccine-preventable ARI rose, similar to our simulation findings with influenza and RSV [10]. Notably, the simulation models do not account for adjustment of additional confounders such as comorbidities likely to drive the decision to receive one or multiple vaccinations. While our results and that of Doll et al [10] suggest that the currently lower uptake of COVID-19 [8] and RSV [20] vaccines should not independently impact influenza VE estimates, the potential impact of additional RSV-vaccine eligible populations and increasing vaccination coverage towards CDC targets should be considered in analyses of VE estimates.

Strengths of our study include a well-defined population, robust laboratory testing, and replication of methods used in other studies [10]. Given the single-center nature and small sample size, our VE estimates had wide confidence intervals. Additionally, the current study only considered pre-pandemic influenza and RSV due to data availability; future studies should document the additional impact of SARS-CoV-2 and RSV vaccinations to VE estimation methods in real-world models.

More real-world research is needed to determine the adjusted impact of multiple vaccine-preventable ARI’s within the control group of TNDs. This is particularly important where vaccination is not approved for all ages within a cohort, such as with RSV. These findings suggest that general inclusion of RSV in the control group of TND for influenza VE, particularly in populations where RSV remains not vaccine preventable, does not currently pose a major concern for bias in VE estimates against medically-attended influenza illness.

Appendix A. Supplementary data

Supplementary data to this article can be found online at https://doi.org/10.1016/j.vaccine.2024.126493.