The Emory University Institutional Review Board (IRB) approved the study as exempt (#0004491). The Washington State IRB approved the study as non-human subject research (#E-072110-H). Informed consent was obtained via courier delivery of a Frequently Asked Questions (FAQ) document included with the survey which addressed the purpose, risks and benefits, confidentiality, incentives, and voluntary nature of the survey.

We drew a stratified random sample of 800 vaccine providers from 2,523 eligible practices who ordered H1N1 vaccine from the Washington State Department of Health and Human Services during the 2009–2010 H1N1 influenza pandemic. The sample size of 800 was based on a minimum anticipated response rate of 50%, and we sought survey estimates accurate within ±5% for all measures. All women's health providers (n = 107, Table 1) and correctional facilities (n = 31, Table 1) were selected, and the remaining providers were selected by stratified random sample. Women's health providers and correctional facilities were oversampled for pooled analysis with surveys in other states. The remaining six categories of provider types were proportionally represented in the sample: non-traditional vaccinators (e.g., alternative medicine, rehabilitation, occupational health, specialists), under-25-year-old priority group practices (e.g., pediatrics, college health services), pharmacies, government providers (e.g., Indian Health Service, local health jurisdictions, Veterans Affairs), hospitals and acute care, and traditional family practices. After eliminating 34 duplicate addresses and 1 Oregon address, 765 questionnaires were delivered.

Identical printed and online survey instruments were used to collect data from study participants. The printed survey instrument was a five-page, single-sided questionnaire. The paper survey was single-sided in order to facilitate the option of returning the survey by fax. The questionnaire consisted of 39 questions divided into 5 sections: practice demographics (5 items), communication with public health and the public (6 items), 2009 H1N1 vaccination administration (15 items), staff participation in public health preparedness activities (4 items), and use of IIS (8 items). We collected information on practice demographics including questions about provider type, participation in Vaccines for Children (VFC, a federal vaccine program), role of the contact (i.e., the onsite vaccine coordinator to whom the survey was targeted) in the practice, and size of the practice. The communications section of the survey addressed sources of public health information, effective communication methods from public health agencies, and effectiveness of previous local and state public health department communications. Questions addressing vaccine administration covered topics of priority group guidelines, staff vaccine coverage, and challenges of vaccine administration. We asked questions covering preparedness activities including questions about past participation in training or preparedness drills and past involvement in actual emergency responses. Finally, a section on the use of IIS included questions about the use and ease of use of Washington's IIS, Child Profile.

On September 15, 2010, sampled providers received a fax that informed them about the upcoming survey and outlined the survey goals. Two weeks later, we sent the survey by FedEx to study participants as a “survey kit”. Each was addressed to the person identified by the Washington State Department of Health and Human Services as the primary contact for ordering H1N1 vaccine at the practice. We used FedEx for delivery with the goals of increasing response rates and tracking signed receipt of the survey and gift card. Also included in the survey kit were a hard copy of the survey instrument, a cover letter, an informed consent framed as a FAQ page, a postage-paid addressed return envelope, a pen, and a $25 gift card to Target to thank the contact for their time. Target is the second largest discount retailer in the United States [15]. Target stores sell household items, apparel, electronics, and health and grocery products. We chose this retailer for the incentive because it offers a wide selection of merchandise, good geographic coverage, gift cards that are redeemable online, and the ability to delay gift card activation to protect our investment. The cover letter described the contents of the survey kit and the objectives of the survey, provided contact information of the investigators, and indicated ways that respondents could complete the survey (mail, fax, or online). In addition to addressing general concerns about confidentiality, the voluntary nature of the survey and the risks and benefits of the survey, the FAQ addressed gift card use, survey funding, and the multiple ways to return the survey. The website address of the online survey tool was chosen to be simple and was printed on all survey materials. Gift cards could be used in-store at any Target location or could be used online at Target.com. The online survey tool was administered using Feedback Server version 2008.1 (Geneva, Switzerland).

Non-respondents received a fax reminder two weeks after the first mailing, including the full survey instrument, cover letter, and survey FAQ document. Three weeks after the first mailing, we contacted non-respondents by telephone a maximum of three times over a period of 9 weeks. We left voicemail messages with the provider contact if direct contact was not possible after the first follow-up. Missing, incomplete, or outdated information was updated during telephone follow-up with the vaccine provider. Nine weeks after the initial mailing, remaining non-respondents received a personalized fax reminder. The reminder included the full survey instrument, a history of follow-up with that individual to date, and a reminder that his/her response was valuable for obtaining a representative sample.

We assessed non-response bias by comparing survey respondents with non-respondents, and by comparing early respondents to late respondents by demographic and practice-related variables: road distance in miles or in minutes to the nearest Target store, type of practice, geographic region of Washington, degree of urbanicity (as defined by the U.S. Census Bureau), whether the practice regularly administers vaccines, and size of practice. Each of these predictors has either been shown previously to affect response rates or was of particular interest in this study.

We used GoogleMaps to calculate road distance in miles and driving time needed to reach the nearest Target store from each provider by the shortest possible route. Late response was defined as responding after the first fax follow-up two weeks after delivery, and early response was defined as responding before the first follow-up. Demographic and practice-level data about providers was obtained from the Washington State Department of Health and Human Services, including physical address, local health jurisdiction, and whether the practice had registered to provide vaccines other than H1N1. We categorized geographic regions in Washington using region categories defined by the Washington State Department of Health and Human Services [16]. Respondents self-reported provider category (type of practice) and size of practice. Degree of urbanicity was determined using Rural-Urban Commuting Area (RUCA) codes obtained by ZIP code approximation through the Rural Health and Research Center [17]. We combined small town and rural designations to permit analysis due to small strata sizes.

The three survey estimates of interest were ease of adherence to priority group guidelines, perceived capability to respond to future public health emergencies, and participation in training drills or emergency preparedness exercises. We dichotomized ease of adherence to priority group guidelines into those responding “Easy - The guidelines made it easy for our practice to make decisions on who should or should not receive the vaccine.” compared to those responding “Moderate -The guidelines gave us general guidance, but we still had to make some case-by-case decisions that we were not sure were covered by the guidelines.” or “Hard - In most cases, the guidelines were not specific enough to help our practice make decisions on who should receive vaccine.” We categorized perceived capability to respond to future public health emergencies into those practices responding “Strongly Agree” or “Agree”, compared to all other responses on a five-point Likert Scale from “Strongly Agree” to “Strongly Disagree” to the statement “The H1N1 vaccination campaign illustrated that our practice or pharmacy branch is capable of responding to large scale public health events.” We assessed participation in training drills or preparedness exercises by comparing practices responding “Yes” compared to those responding “No” or “Not sure”.

We used Fisher exact test to evaluate the bivariate association between each predictor and the outcome. We used logistic regression modeling to assess the effect of each predictor on response status and response timing. Models of best fit were determined based upon whether adding additional variables confounded the relationship between the primary variables of interest and the dependent variable by more than ten percent and whether they contributed significantly to R-squared. Only models containing miles to the nearest Target location and practice type were eligible for consideration as these were the primary variables of interest.

We used Fisher exact test to evaluate the association between late response (a proxy for non-response) and survey estimates to determine if late respondents answered differently than early respondents. We used late response as a proxy because we were interested in the impact of non-response bias on survey estimates, but we do not know how the non-responders would have answered. Late responders more closely resemble non-responders than early responders since, without follow-up, late responders would have likely been non-responders. For those factors that affected response status, we used logistic regression to assess the effects of each predictor on survey estimated effect measures of interest: ease of adherence to priority group vaccine guidelines, perceived capability to respond to future public health emergencies, and participation in training drills or preparedness exercises. If, for example, rural and metropolitan clinics adhere to guidelines similarly, then the finding that rural clinics are more likely to respond would not affect the quality of survey estimates. However, if rural clinics are more or less able to adhere to guidelines, we would want to determine the magnitude and direction of non-response bias on survey estimates.

All statistical analyses were performed using SAS v. 9.2 (Cary, NC). Results were considered statistically significant at an alpha level of 0.05 for all tests.

Completed questionnaires were returned by 619 out of 765 (80.9%) vaccine providers sampled. Of these, 25 did not provide identifying information and could not be matched to the sample list. Since these providers could not be matched to demographic and geographic variables, these responses were included as non-respondents in this analysis. Of the 594 responses with identifying information and a valid time stamp, 404 (69.2%) were returned before any reminders were received. Number of responses and cumulative response rate by timing of follow-up are shown in Figure 1. Descriptive statistics are presented in Table 1.

Bivariate results stratified by response status and response timing are presented in Table 1. There was no significant difference between respondents and non-respondents by distance to Target store in miles (p = 0.730) or in minutes (p = 0.955). There was a significant difference among respondents and non-respondents by whether the practice regularly administers vaccines and by type of practice (Table 1). Those practices that regularly administer vaccines were significantly more likely to respond to the survey than those practices that provided H1N1 influenza vaccine only (82.4% compared to 74.6%, p = 0.013). By type of practice, response rates were lowest for pharmacies (69.4%) and highest for pediatric practices (91.4%) (p = 0.002).

Bivariate results stratified by early versus late response are also presented in Table 1. There was no significant difference between early respondents and late respondents by distance to the nearest Target in miles (p = 0.215) or in minutes (p = 0.196). There was a significant difference between early respondents and late respondents by whether the practice regularly administers vaccines (Table 1). Those practices that provided H1N1 influenza vaccine only were significantly more likely to respond late than those practices that regularly administer vaccines (34.9% compared to 25.1%, p = 0.014). The proportion of pharmacies that responded late was significantly higher than the proportion of traditional family practices that responded late (44.0% compared to 27.7%, p = 0.047).

Table 2 presents the logistic regression odds ratios (OR), 95% confidence intervals (95% CI) and p-values for the relationship between practice characteristics and survey response. Adjusting for type of practice, vaccinating for only H1N1, region of Washington, and degree of urbanicity, practices that were further from their nearest Target store were less likely to respond to the survey. The odds of receiving a response from a practice ten miles further from the nearest Target than another practice were 0.73 (95% CI = 0.60−0.89) times the odds of the nearer practice. Pediatric providers were significantly more likely than traditional family practice providers to respond to the survey (OR = 4.03, 95% CI = 1.36−11.96; Table 2). Non-traditional providers were significantly more likely to respond to the survey than traditional family practice providers (OR = 2.08, 95% CI = 1.06−4.08; Table 2). Providers that only provided H1N1 influenza vaccine were significantly less likely to respond than providers that regularly administer vaccines (OR = 0.56, 95% CI = 0.32−0.97; Table 2). Small town or rural providers were significantly more likely to respond to the survey than metropolitan providers (OR = 7.68, 95% CI = 1.44−40.88; Table 2).

The logistic regression model predicting response status based on the set of predictors (distance to nearest Target store, type of practice, vaccinator for only H1N1, region of Washington, and degree of urbanicity) was significant (p<0.05), but the model R-squared was low (0.082). The self-reported predictors of practice size and VFC enrollment were not significantly associated with response status, and were not included in the final logistic regression model. Regression coefficients were calculated using both distance in miles and minutes to the nearest Target; using one or the other strategy gave similar regression coefficients and identical conclusions. The final model used distance in miles because its R-squared was slightly higher than the model using distance in minutes.

The logistic regression model comparing early respondents versus late respondents is presented in Table 2. There was no significant association between the set of predictors (distance to the nearest Target store, type of practice, vaccinator for only H1N1, region of Washington, degree of urbanicity) and late response (p  = 0.064, Table 2). Self-reported predictors practice size and VFC enrollment were not significantly associated with response status, and were not included in the final logistic regression model.

Next, we assessed the consequences of non-response bias in terms of demographic variables on the survey variables of interest: easy adherence to guidelines on priority groups, capability of the practice to respond to future public health emergencies, and whether the practice participated in disaster training or preparedness exercises.

There was no significant association between response timing (early/late) and key survey responses (Table 3). Table 4 presents logistic regression results predicting three survey estimates of interest based on the set of predictors shown to affect response (distance to nearest Target store in miles, type of practice, administering vaccinations for only H1N1, region of Washington, and degree of urbanicity). The model for ease of adherence to guidelines was not significant (p = 0.061). None of the models had a high R-squared, although R-squared was greater for the model predicting training or preparedness activities (R-squared = 0.254) compared to the models for ease of adherence to guidelines (R-squared = 0.067) and perceived practice capability to respond to public health emergencies (R-squared = 0.104). Each of the models for the survey estimates were influenced by different sets of independent variables. None of the independent variables were significant in more than one of the models predicting survey estimates.

There was no significant observed effect of the number of miles to the nearest Target store on self-reported ease of adherence to guidelines (OR = 0.87, 95% CI = 0.72−1.05), perceived capability to respond to future public health emergencies (OR = 0.87, 95% CI = 0.65−1.16), or participation in disaster training or preparedness exercises (OR = 1.15, 95% CI = 0.92−1.43). Small town or rural location of the practice was positively associated with perceived capability to respond to future public health emergencies (OR = 20.83, 95% CI = 1.02−425.51). Practices located in small town or rural locations comprised 5.4% of sampled practices (41 of 765, Table 1) which accounts for the wide confidence intervals. The other variables shown to affect response (vaccination for only H1N1, and pediatric and non-traditional vaccinator provider types) were not significant in all models.

Our estimate of overall perceived capability of respond to public health emergencies is likely biased up and away from the null, because small town or rural location of the practice was positively associated with response and was also associated with increased perceived capability to respond to future public health emergencies. The magnitude of the bias is small. No other estimates associated with response were also associated with survey estimates; there is no evidence that survey estimates of ease of adherence to guidelines or participation in training exercises or preparedness drills were biased due to non-response.