Participation as a provider in HVF is completely voluntary and open to any provider who administers vaccines. To participate in HVF, interested providers must first sign up for an account. Providers submit an organization name and contact information in a request for an account. The system automatically checks the contact e-mail address against the existing e-mail addresses in the database to prevent duplicates. A member of the HVF team then approves or rejects the account.

Once the account is approved, the provider may access the provider admin tool, where he or she will upload basic information about the provider’s vaccination services. The information includes store number, site or clinic name, start and end dates for influenza vaccinations, street address, city, state, zip code, phone number, days and hours of operation, insurance accepted (yes–no), walk-in patients accepted (yes–no), Web address, provider type, vaccines offered, age restrictions, vaccine cost, prescription requirements, and other notes.

The information can be uploaded in 1 of 2 ways. If submitting an individual location, the provider can complete and submit a Web form with the fields described. If providers own or manage multiple locations, they can download and complete a spreadsheet template and then upload their data as a comma-separated values file.

Although HealthMap received contact information for providers who had participated in the program under Google.org, the team has since made efforts to expand the number and types of providers in the database. To recruit new providers into the system, HealthMap has worked with partners such as the US Department of Health and Human Services (http://www.flu.gov and http://www.vaccines.gov), the National Association of City and County Health Officials, the American Pharmacists Association, the Immunization Action Coalition, and other professional and government organizations. Outreach efforts include publishing articles in newsletters and via mailing lists, attending and presenting at relevant conferences, and communicating via social media channels.

To better understand the participating providers, we present the number of provider accounts, number of locations uploaded to those accounts, and a breakdown of the types of participating providers (pharmacy, health departments). All data were queried from the HVF database and analyzed using R (version 3.1.1, Revolution Analytics, Palo Alto, CA).¹⁵

The HVF data are stored in a relational MySQL database in the AWS Cloud on a Linux operating system. The Web interfaces for providers to insert and update data are maintained in PHP, as is the end-user interface, which also features a map using the Google Maps API. There is also a RESTful Web service available for authorized users that allows for querying provider data that are returned in JSON format.

When a user goes to the publicly available site (http://vaccinefinder.org), the geographic location is automatically detected via the IP address or mobile device. The system queries the database to identify all providers within a 10-mile radius of the search location that stock the selected vaccines. The default search on landing on the site during the study period was for all influenza vaccines. Users could limit the search to specific influenza vaccines by clicking checkboxes from a list of available vaccines. All providers returned by a search are displayed in order of distance from the search address and appear in list form and as markers on a map.

The user can click on any vaccine name in the search box to view the name of the vaccine, the disease or pathogen it protects against, recommended or approved groups for vaccination, and a link to the Vaccine Information Statement found on the Centers for Disease Control and Prevention Web site.¹⁶

Similarly, a user can click on any location marker on the map to open an infobox with more information about the provider. The infobox is populated with the information previously collected from the providers. The provider infobox also contains a link to driving directions via Google Maps.

Web traffic is tracked using Google Analytics. In addition to pageviews, unique users, and other common Web metrics, this tool logs the number of searches for specific vaccines, specific locations entered into the search bar, and clicks on provider infoboxes. However, autodetection of the user’s location and subsequent display of a map centered on that location allows many visitors to find an influenza vaccine provider without otherwise interacting with the site. Although this is certainly a positive experience for users, it does create certain limits to our understanding of interaction with the Web site. Our reported numbers of specific vaccine searches are drawn from a smaller pool of visits that included at least 1 click.

In 2012, we launched the vaccine recommendation tool, an interactive component that gives users a personalized list of recommended vaccines. To access the tool, the user clicks on the “What vaccines do I need” button that links to the quizzes. The user then chooses between a quiz for influenza vaccines or for other adult vaccines. Answering a series of questions on demographics and medical conditions results in a list of potentially recommended vaccines. The user can search for providers stocking the listed vaccines. He or she can also print the list to bring to the provider visit.

The logic underlying the tool was adapted from the Recommended Adult Immunization Schedule and Prevention and Control of Seasonal Influenza with Vaccines, both prepared by the Advisory Committee on Immunization Practices.^17,18 The logic is updated as the advisory committee updates the recommendations and contraindications for relevant vaccines.

Although the recommendation tool was launched in late 2012, user data storage capabilities did not come online until June 18, 2013; thus, recommendation tool usage data for the study period are not available. However, it is a useful tool and an important component of HVF’s overall utility.

In January 2013, the HVF team began to receive e-mails and social media messages from users unable to obtain an influenza vaccine at listed providers. There was no true shortage because vaccine was still available nationally, but some areas experienced stockouts as vaccine was not available locally. To improve the reliability and accuracy of the displayed information, a crowdsourced reporting tool was created specifically for influenza vaccines.

If a user is unable to receive an influenza vaccine at a specific provider, he or she can report this shortage to HVF using the Shortage Reporter tool. The user chooses from the existing provider list either by finding the provider on the map or by searching within a zip code. The user can then report the specific type of vaccine that was unavailable; during the 2012–2013 influenza season, 4 types of influenza vaccine were on the market and listed on the public HVF map. Users can then provide comments on their experience, and enter an e-mail address for future follow-up, if needed. A warning symbol will appear on the indicated provider’s information, showing the date the shortage was reported. This feature allows users to know about potential supply issues before they travel to their desired provider, potentially saving time, effort, and frustration.

The user-submitted report also generates an automatic e-mail to the provider in question. The e-mail includes a link to the specific location in the admin tool and explains possible next steps. The provider can update the supply information to show that the vaccine is no longer available at that location, or the provider can reject the shortage report if it is incorrect. Once the vaccine has been restocked, the provider can then remove the shortage report, by extension removing the warning from the map.

By the end of the 2012–2013 influenza season, HVF had 1256 active provider accounts, representing 46 381 locations. Of the 1256 provider accounts, 869 represented single locations. Of the remaining 387, the number of locations ranged from 2 to 8419. The median number of locations was 4.

Pharmacies were the most highly represented provider type, with 38 813 locations (84%). In the database, 2562 locations (5%) identified as clinics, and 700 locations (2%) identified as health departments. Community immunizers and health care providers together accounted for a small percentage of total locations (1%). Approximately 8% of providers did not specify a provider type. HVF includes information on vaccine providers in all 50 states, the District of Columbia, and Puerto Rico (Figure 1).

From September 1, 2012, to April 30, 2013, HVF received 625 124 visits (Figure 2). It experienced its highest traffic the week of January 6–12, 2013, with 205 254 visits. The median number of visits per day was 739 (range = 96–75 490). A maximum of 75 490 visits was recorded on 1 day (January 11). Of the total traffic, 548 240 (87.7%) visits were from new users, and 76 884 (12.3%) visits were from returning users.

Web visitors found HVF through a variety of sources. Of the total visits, 396 054 (63%) arrived via referral from other Web sites. Referral traffic was largely driven by flu.gov and the Centers for Disease Control and Prevention Web site. Approximately one fourth of visits (155 642) came via direct traffic, and 11% of visits (72 388) were search traffic from Google, Bing, and other search engines.

During the course of the influenza vaccination season, HVF drew traffic from all 50 states. The most visits, controlling for population, came from the District of Columbia, with 184.78 visits per 10 000 people. The 10 states with the highest and lowest number of visits, controlling for population, are represented in Table 1, with population taken from July 1, 2012, estimates by the US Census Bureau.¹⁹

During the 2012–2013 influenza vaccination season, the Web site had 220 450 visits that included at least 1 click or interaction with the Web site, or 35.3% of total Web visits to HVF. The total clicks numbered 483 623, for an average of 2.19 clicks per visit.

Of the total interactions, there were 201 894 searches (Table 2). Again, searches are only tracked if the user clicks on 1 of the vaccines to search; users who leave the default search of “all flu vaccines” selected are not included. The most searched-for vaccine was the high-dose flu shot, followed by intradermal flu shot, flu nasal spray, and standard-dose flu shot.

Current limitations of HVF include missing providers and low or misrepresentative usage by the public. Thanks in part to the tool’s focus on pharmacies when administrated by Google. org, the database contained 63% of the nation’s pharmacies by the end of the 2012–2013 influenza season. Coverage of other provider types is lower, and provider recruitment remains ongoing.

Regarding public usage, although we believe HVF experienced high usage in its inaugural influenza season, it is difficult to know whether the user audience is representative of the broader population. In promoting the site to the public, the strategy has thus far largely relied on a partnership with the US Department of Health and Human Services, including the flu.gov Web site, the Centers for Disease Control and Prevention, and social media campaigns. Members of the public who do not frequent government Web sites, use social media, or turn to Internet search engines are unlikely to find or use HVF. Conversely, if the target population is limited to those who look to the Internet for health information, HVF’s presence across different channels (government Web sites, social media, search engine results) means it is more likely users will find the tool. Future promotions will add to the existing efforts and attempt to engage new audiences.

An additional limitation is that we currently only have data on Web visits and clicks with no indication of the number of people who went on to receive a vaccine. A future study is needed to validate Web behavior as an indicator of real-life vaccination behavior. The research team hopes to partner with local health departments and independent pharmacies to attempt to answer this question.

HVF is a unique tool that uses the Internet to connect the public to vaccine providers in their communities. The site relies on data from participating providers and translates the data into an easy-to-use, publicly available map through which users can search the provider database, find recommended vaccines, and report any supply issues. In its first year of operations, HVF recruited a variety of providers into the database, as evidenced by the different types and number of locations reported. The tool experienced high public usage, in part driven by attention from national media and government.

In addition to serving as a resource supporting a core public health goal, HVF data have a number of uses in understanding trends in public interest in vaccines. Future research questions include analysis of influenza vaccine preference, the relationship between geographical provider density and vaccine uptake, and vaccine searches as an indicator of disease activity.