A web-based HIV behavioral surveillance system (WHBS) has potential to collect behavioral data from men who have sex with men (MSM) not reached through traditional sampling methods. Six U.S. cities conducted a WHBS pilot in 2005-2007 to determine the feasibility to conduct a behavioral surveillance project entirely on the internet.
Three sampling methods of adult MSM on the internet were explored: direct marketing (DM) using banner advertisements; respondent-driven sampling (RDS) using peer recruitment; and venue-based sampling (VBS) using internet venues.
A total of 8,434 complete MSM surveys were obtained: 8,109 through DM, 130 through RDS, and 195 through VBS. By methods, enrollment rates ranged from 70-90%; completion rates ranged from 67-95%. DM obtained the largest proportions of racial/ethnic minority MSM (36%) and MSM 18-20 years (19%).
Only the DM method achieved a substantial number of complete MSM surveys. Successful implementation of an internet-based systematic sampling method may be problematic, but a convenience sample of MSM using banner advertisements is feasible and may produce useful and timely behavioral information from a large number of MSM.
The Centers for Disease Control and Prevention (CDC) released the HIV Prevention Strategic Plan in 2000 with a goal to reduce new HIV infections in the US. Objectives to reach this goal included strengthening national capacity to monitor the HIV epidemic to better direct and evaluate prevention efforts [
The sampling method for NHBS among MSM uses physical venues, such as bars, clubs or other locations, where MSM are approached and consented to an in-person interview [
A web-based HIV behavioral surveillance system (WHBS) pilot project was initiated in 2005 to examine whether the internet may be used to collect behavioral data from MSM populations that may not be sampled through NHBS. The specific objectives of the WHBS pilot project were to 1) develop and field-test methods for conducting WHBS to determine the feasibility of conducting this type of project entirely on the internet; 2) evaluate the potential of recruiting MSM not otherwise easily reached through sampling at physical venues; and 3) compare HIV-related behavioral risks between MSM reached over the internet with MSM reached through NHBS to determine if these populations have similar or different HIV infection risks.
Internet-based behavioral research is an emerging field, but several researchers have successfully implemented online behavioral research projects using a variety of convenience sampling methods, some of which have focused on HIV and MSM [
The WHBS pilot consisted of a period of formative research activities in 2005 (data not presented) followed by two rounds of recruitment and online surveys that occurred in 2006 and 2007. WHBS was implemented in 6 MSAs: Boston, MA; Baltimore, MD; Dallas, TX; Los Angeles, CA; New York City, NY; and San Francisco, CA.
Three cross-sectional sampling methods were utilized for this round: 1) direct marketing using banner advertisements on websites; 2) respondent-driven sampling using peer recruitment; and 3) venue-based sampling using internet “venues”. Each MSA conducted at least one method from among these three. Data collection for WHBS Round 1 occurred during an 8-week period in 2006; each WHBS project area had a sample size goal of 500 MSM with complete surveys. In order to be eligible, men must have been at least 18 years of age and a current resident of one of the 6 MSAs. Additionally, for the respondent-driven sampling eligibility, men must have had sex with a man in the past year and reported using the internet to either “get information on topics of personal interest” or “meet sexual partners or friends” in the past year. Participant reimbursements/incentives were not used. Sampling methods are described below; recruitment statistics are presented in Table
Direct marketing (DM) is a convenience sampling method and consisted of banner advertisements (banner ads) displayed on a variety of websites. This method has been used for internet-based recruitment of MSM by several other researchers [
Respondent-driven sampling (RDS) is a systematic chain-referral sampling method that has been used in other HIV studies and may be particularly well suited to recruit individuals from hard-to-reach populations [
To recruit seeds, WHBS staff utilized their own expertise and formative research feedback to develop a list of internet chat room or personal profile websites that local MSM visited. Staff used instant messages and emails to interact with potential participants. Persons who agreed to take the survey and met eligibility criteria received a unique study ID and a link to the secure WHBS web site where eligibility screening was repeated. After a seed completed the survey, he was instructed in how to recruit internet-using MSM peers for the study and received 3 unique study ID numbers and a link to the WHBS web site that he was asked to give to his peers. Peers used the study ID and link to enter the WHBS website. Peer referral was allowed to continue through to the end of data collection. Round 1 RDS was conducted in 3 MSAs: Boston, Dallas, and New York City.
Venue-based sampling (VBS) is a systematic method which recruits MSM at public locations such as bars or clubs and is the methodology used for the MSM cycle of NHBS [
WHBS venues were not physical locations as in NHBS, but internet-based “venues” such as web sites or chat rooms. Internet venues that were thought to be popular among local MSM were identified during the formative research and through internet searches. Examples of internet venues included online chat rooms where men log in and use instant messages to have live conversations with other men, or websites where individuals post personal profiles in an attempt to meet other men. Identification of venues was not limited to those where MSM seek companionship or sex. Similar to other VBS methods, WHBS staff maximized recruitment for each identified venue by identifying 4-hour blocks of time called venue-day-time periods (VDTs) during which at least 7 MSM were expected to be enrolled. Once a list of internet venues and their respective VDTs had been assembled, staff randomly selected the VDTs to develop a weekly sampling schedule noting the days and times during which study recruitment would occur.
Recruitment methods varied depending on the type of venue where recruitment occurred: chat rooms or profile websites. Recruitment within internet chat rooms allowed live conversation
Recruitment within profile web sites did not always allow for instant live conversation with men, but, rather, used email links provided within each user’s profile. To recruit persons from these sites, staff systematically emailed every
Upon the completion of Round 1 data collection, all local WHBS principal investigators and CDC reviewed experiences with the different methods (see Table
For Round 2, all WHBS MSAs used an identical set of banner ads and fewer websites for the DM sampling. Otherwise, DM sampling methods did not differ from Round 1.
Staff did not recruit RDS seeds in Round 2. All participants who were recruited through DM and completed the survey were considered seeds. Upon completion of the survey, DM participants were given the option of recruiting up to 3 of their peers by providing their peers’ email addresses. Email addresses were not saved after the WHBS message containing the survey link was sent. The recruiter was given an option to include a personal note to their recruits in the email. Emails contained a brief introduction to the study and a link to the WHBS website. As in Round 1, once an RDS recruit was determined to be eligible and completed the survey they too were given the option to recruit up to 3 peers.
The secure WHBS web site could only be accessed by a unique study ID, clicking a banner ad, or using a hyperlink from an RDS email. Study IDs and hyperlinks expired if unused for 30 days. After entering the website, brief study information was provided and screening questions were administered to determine eligibility to participate. If eligible, participants were given more detailed information about the study and were invited to participate and provide consent. If they consented, the survey began immediately.
The WHBS survey consisted of seven main sections: 1) demographics; 2) MSM venue attendance and internet-usage behaviors; 3) sexual behaviors; 4) drug-use behaviors, 5) HIV and STD testing experiences; 6) accessing internet-based prevention services or information; and 7) accessing other HIV prevention services. The survey was designed to take approximately 10 to 15 minutes to complete. A participant who exited the survey early for any reason and later re-entered the WHBS site was required to restart the survey from the beginning. His previous survey responses could not be viewed by him and would appear as an incomplete survey record in the database. Once started, a participant had 24 hours to complete a survey before the link or study ID became inactive. The WHBS website did not collect any identifying information on the participants, including IP addresses, email addresses, or web logs.
Recruitment statistics were analyzed for Round 1 (2006) and Round 2 (2007) separately for each sampling method. The count of recruitment attempts varied by method and included the number of persons that started screener questions (DM and 2006 RDS [except seeds]), the number persons that staff attempted to actively recruit (2006 RDS seeds and VBS), or the number of email recruitments sent (2007 RDS). The count of persons screened also varied by method and included the number of persons that started screener questions on the WHBS website (DM and RDS [except 2006 seeds]) or the number of persons that staff screened during active recruitment (2006 RDS seeds and VBS). The count of persons eligible only used data from the WHBS website screener questions. It was not possible to consistently count eligible persons from staff-collected VBS and RDS seed recruitment information. Some who were initially screened and were eligible during staff-led recruitment may not have accessed the WHBS website to complete the process (and would not be counted as eligible). Counts of reasons for ineligibility (not 18 years or older, not male, not an MSA resident, not RDS eligible) are also provided for the 2007 DM; specific reasons for ineligibility were not consistently tabulated in 2006. The count of persons enrolled was the same for all sampling methods and was defined as those starting the WHBS survey. The completion count was calculated based upon those completing all 3 of the final questions in the WHBS survey. Surveys with duplicate study IDs that were incomplete were excluded from the calculations for the completion rate. Since the WHBS sampling goal was based on complete surveys among MSM, the counts of men reporting sex with at least one man in the past 12 months (MSM in the past 12 months) were obtained from answers to the sexual behavior questions within the WBHS survey. For the 2007 DM method, a count of the number of banner ad impressions was available from most (but not all) websites. An impression is a measure of each time a banner ad appears on an accessed webpage and can be thought of as a passive recruitment attempt.
Demographic characteristics of participants were analyzed for 2006 and 2007 separately for each sampling method. Race/ethnicity, age category, college education, and sexual identity were selected for this report. Self-reported HIV serostatus was defined using the results of the most recent HIV test and categorized as positive, negative, or untested/unknown status. The demographics and HIV serostatus analysis was restricted to only complete surveys from those men who reported sex with a man in the past 12 months. These analysis criteria were employed to be consistent with the stated sampling goal for the pilot project.
The purpose of this research project was not to determine whether there were statistically significant differences between sampling methods. Rather, the goal was to develop an understanding of the feasibility of conducting different sampling methods and each method’s yield of completed surveys for the population of interest, MSM. Therefore no tests of significance were performed; only descriptive data are presented in this report.
The DM sampling method in both rounds produced the largest numbers of complete MSM surveys and resulted in acquisition of the target sample size of 500 per MSA in 2006 and 2007 (Table
For the DM sampling method in 2007, over 21.5 million impressions resulted in 44,801 persons being screened in a 16-week period (Table
Overall, the MSM participants in WHBS were mostly non-Hispanic whites, between the ages of 21 and 40 years, college educated, self-identifying as homosexual and HIV-negative (Tables
The use of banner ads through the DM method was highly successful during both rounds of WHBS data collection. The systematic sampling methods used in the WHBS pilot were less successful but produced important insights regarding the feasibility of these methods. We found that a VBS method designed to approximate the methods used for real-world venues by NHBS was theoretically feasible using internet “venues”, but was labor-intensive, had difficulty meeting adequate enrollment proportions, and did not capitalize on some of the benefits of an internet-based sampling method (e.g., enrollment required a staff recruiter and was not automated). VBS is particularly complicated due to rapid changes in internet technology and the fluctuation of sites most commonly used by MSM for networking and socializing. RDS is another potentially viable systematic sampling option for the internet, but in both rounds of WHBS data collection, we were unable to sustain the necessary peer recruitment. RDS may not have been successful for WHBS because of the lack of incentives or the potentially weak ties between individuals of internet-based social networks; dual incentives and a strongly connected peer network have been recognized as potential preconditions for RDS success [
DM resulted in a large sample of eligible MSM enrolled in a very brief period of time. These results are consistent with the experiences of several other research studies of MSM on the internet that used banner ad recruitment [
Several limitations for the WHBS pilot study should be noted. We were unable to provide participant incentives in either round of data collection; participant incentives may be an integral part of the success of some systematic sampling methods (RDS in particular) and may have adversely impacted the success of these methods. We are unable to determine sampling bias or external validity for any of the WHBS samples. Because of the variability in the way that recruitment information was collected through different sampling methods, some of the differences in screening, eligibility, and enrollment rates may be due to the way information was collected and not to true differences between sampling methods. Because of the anonymous nature of the survey, we are also unable to determine whether an individual participated in the survey multiple times which may have been more feasible with the DM method, but we believe this would be minimized given the lack of incentives for survey participation. We also cannot determine the veracity of responses to survey questions, though social desirability bias is likely minimized by the anonymous nature of the survey. Finally, the sample sizes for VBS and RDS were inadequate for us to conduct statistical comparisons between the samples of MSM with an appropriate degree of scientific rigor, but this was also not the intent of this analysis, which was to determine the feasibility of the methods to reach target sample sizes.
The way in which MSM interact and seek sexual partnerships has expanded to include the internet, and as such, an internet-based sampling method for MSM has a role in monitoring HIV-related risk behavior. The WHBS pilot project has shown that an internet-based systematic sampling method of MSM is complex and may not be feasible, but obtaining a convenience sample of MSM using banner ads is feasible and may produce useful and timely behavioral information from a large number of MSM. Another paper published in this journal issue examines whether MSM surveyed through this sampling method differ from NHBS and explores whether internet-based sampling methods may have a role in providing behavioral surveillance information that is comparable or complementary to NHBS [
The WHBS pilot project was research conducted in compliance with federal regulations governing protection of human subjects and was reviewed and approved by all relevant human subjects review boards.
Declared none.
Colin Flynn and Frangiscos Sifakis, Baltimore, MD; Eric Rubinstein and Gabriella Garcia, Boston, MA; Richard Yeager and Douglas Shehan, Dallas, TX; Trista Bingham and Alexander Carruth, Los Angeles, CA; Christopher Murrill and Sabina Hirshfield, New York City, NY; H. Fisher Raymond and Greg Rebchook, San Francisco, CA; Matthew Amsden and Tom Coggia, Reveal Communications; Paul Young and Benjamin Neal, NOVA Research Company.
The authors have no institutional or commercial affiliations that might pose a conflict of interest regarding the publication of this manuscript.
Recruitment Rates of the Web-Based HIV Behavioral Surveillance System, 6 U.S. Metropolitan Statistical Areas, 2006-2007
| 2006 | 2007 | ||||
|---|---|---|---|---|---|
| DM (MSAs=4) | RDS (MSAs=3) | VBS (MSAs=3) | DM (MSAs=6) | RDS (MSAs=6) | |
| No. (%) | No. (%) | No. (%) | No. (%) | No. (%) | |
| Recruitment attempts | 34,399 | 462 | 6,634 | 76,294 | 708 |
| Screened | 17,774 (52) | 172 (37) | 2,097 (32) | 44,801 (59) | 131 (19) |
| Eligible | 4,782 (27) | 84 (49) | 293 (14) | 13,785 (31) | 105 (80) |
| Enrolled | 3,338 (70) | 75 (89) | 265 (90) | 10,920 (79) | 93 (89) |
| Completed | 2,528 (76) | 71 (95) | 239 (90) | 7,296 (67) | 72 (77) |
| MSM, past 12 months | 2,094 (83) | 67 (94) | 195 (82) | 6,015 (82) | 63 (88) |
MSM = men who have sex with men; DM = direct marketing; RDS = respondent-driven sampling; VBS = venue-based sampling; MSA = metropolitan statistical area
Screened proportion is among total recruitment attempts.
Eligible proportion is among total screened.
Enrolled proportion is among total eligible.
Completed proportion is among total enrolled.
Proportion is among total completed. Defined as a man who had sex with at least one man in the past 12 months.
Recruitment Rates by Website Used for the Web-Based HIV Behavioral Surveillance System, 2007
| Impressions | Screened | Eligible | Completed | MSM in Past 12 Months | |
|---|---|---|---|---|---|
| Total No. | No. (%) | No. (%) | No. (%) | No. (%) | |
| 2007 Total | 21,862,924 | 44,801 (0.20) | 13,785 (31) | 7,296 (53) | 6,015 (82) |
| MySpace | 5,513,672 | 10,317 (0.19) | 4,610 (45) | 2,517 (55) | 2,053 (82) |
| Manhunt.net | 794,633 | 7,302 (0.92) | 3,783 (52) | 2,148 (57) | 1,809 (84) |
| Dlist | 3,222,819 | 9,677 (0.30) | 2,182 (23) | 1,101 (50) | 900 (82) |
| BlackGayChat | 1,210,630 | 7,998 (0.66) | 958 (12) | 381 (40) | 309 (81) |
| BigMuscle | 8,753,839 | 2,746 (0.03) | 479 (17) | 206 (43) | 166 (81) |
| 365Gay | - | 1,972 - | 405 (21) | 202 (50) | 168 (83) |
| Friendster | 1,317,188 | 303 (0.02) | 250 (83) | 143 (57) | 123 (86) |
| Logoonline | - | 508 - | 78 (15) | 38 (49) | 30 (79) |
| Gay.com | 1,050,143 | 88 (0.01) | 42 (48) | 27 (64) | 25 (93) |
| AfterElton | - | 156 - | 15 (10) | 15 (100) | 10 (67) |
MSM = men who have sex with men
Totals are from all direct marketing websites, but only the top 10 websites (by total completed surveys) are included in the recruitment data by individual website.
Demographic Characteristics of MSM Participants in the Web-Based HIV Behavioral Surveillance System, 2006
| Overall (n=2,356) | DM (n=2,094) | RDS (n=67) | VBS (n=195) | |
|---|---|---|---|---|
| No. (%) | No. (%) | No. (%) | No. (%) | |
|
| ||||
| White, not Hispanic | 1,778 (75) | 1,606 (77) | 45 (67) | 127 (65) |
| Black, not Hispanic | 95 (4) | 65 (3) | 2 (3) | 28 (14) |
| Hispanic | 275 (12) | 241 (12) | 14 (21) | 20 (10) |
| Asian or Pacific Islander | 107 (5) | 99 (5) | 1 (1) | 7 (4) |
| Multiracial or other race/ethnicity | 90 (4) | 74 (4) | 5 (7) | 11 (6) |
| Missing | 11 (<1) | 9 (<1) | 0 (0) | 2 (1) |
|
| ||||
| 18-20 | 125 (5) | 116 (6) | 2 (3) | 7 (4) |
| 21-30 | 886 (38) | 810 (39) | 18 (27) | 58 (30) |
| 31-40 | 648 (28) | 570 (27) | 30 (45) | 48 (25) |
| > 40 | 697 (30) | 598 (29) | 17 (25) | 82 (42) |
|
| ||||
| No | 170 (7) | 155 (7) | 5 (7) | 10 (5) |
| Yes | 2,186 (93) | 1,939 (93) | 62 (93) | 185 (95) |
|
| ||||
| Homosexual | 2,056 (87) | 1,830 (87) | 60 (90) | 166 (85) |
| Bisexual | 243 (10) | 215 (10) | 7 (10) | 21 (11) |
| Heterosexual | 13 (1) | 11 (1) | 0 (0) | 2 (1) |
| Missing | 44 (2) | 38 (2) | 0 (0) | 6 (3) |
|
| ||||
| Positive | 212 (9) | 183 (9) | 3 (4) | 26 (13) |
| Negative | 1,810 (77) | 1,596 (76) | 54 (81) | 160 (82) |
| Untested/Unknown | 334 (14) | 315 (15) | 10 (15) | 9 (5) |
MSM = men who have sex with men; DM = direct marketing; RDS = respondent-driven sampling; VBS = venue-based sampling
Self-reported results of last HIV test.
Demographic Characteristics of MSM Participants in the Web-Based HIV Behavioral Surveillance System, 2007
| Overall (n=6,078) | DM (n=6,015) | RDS (n=63) | |
|---|---|---|---|
| No. (%) | No. (%) | No. (%) | |
|
| |||
| White, not Hispanic | 3,916 (64) | 3,868 (64) | 48 (76) |
| Black, not Hispanic | 458 (8) | 456 (8) | 2 (3) |
| Hispanic | 1,096 (18) | 1,088 (18) | 8 (13) |
| Asian or Pacific Islander | 205 (3) | 203 (3) | 2 (3) |
| Multiracial or other race/ethnicity | 356 (6) | 353 (6) | 3 (5) |
| Missing | 47 (1) | 47 (1) | 0 (0) |
|
| |||
| 18-20 | 1,164 (19) | 1,159 (19) | 5 (8) |
| 21-30 | 2,737 (45) | 2,711 (45) | 26 (41) |
| 31-40 | 1,228 (20) | 1,211 (20) | 17 (27) |
| > 40 | 949 (16) | 934 (16) | 15 (24) |
|
| |||
| No | 1,130 (19) | 1,124 (19) | 6 (10) |
| Yes | 4,948 (81) | 4,891 (81) | 57 (90) |
|
| |||
| Homosexual | 5,140 (85) | 5,082 (84) | 58 (92) |
| Bisexual | 777 (13) | 772 (13) | 5 (8) |
| Heterosexual | 36 (1) | 36 (1) | 0 (0) |
| Missing | 125 (2) | 125 (2) | 0 (0) |
|
| |||
| Positive | 458 (8) | 449 (7) | 9 (14) |
| Negative | 4,467 (73) | 4,414 (73) | 53 (84) |
| Untested/Unknown | 1,153 (19) | 1,152 (19) | 1 (2) |
MSM = men who have sex with men; DM = direct marketing; RDS = respondent-driven sampling; VBS = venue-based sampling
Self-reported results of last HIV test.