Homosexuality has long been stigmatized, with serious social consequences for sexual minority people (Hatzenbuehler, Phelan, & Link, 2013; Meyer, 2003). As a result, sexual minorities have historically been a “hard-to-reach” population (Ellard-Gray, Jeffrey, Choubak, & Crann, 2015). In part because of this challenge, early sexual minority population studies relied almost exclusively on samples derived from clinic- and prison-based populations. These studies suffered severe selection effects, that not surprisingly, yielded findings that portrayed sexual minority people as sick and morally flawed compared to heterosexuals (Hammack, Mayers, & Windell, 2013; Meyer & Wilson, 2009; Morin, 1977). Evelyn Hooker’s approach to studying homosexual men in the 1950s stood in stark contrast to the earlier clinic-based studies (Minton, 2002). Hooker called for the use of samples of sexual minority people derived from the larger, general community of homosexual men; her findings showed that homosexual men recruited through community samples were generally well-adjusted and healthy (Hooker, 1957). This community-based sampling approach was a revolution for sexual minority research and represented a marked shift in the approach to studying sexual minority people that continues to this day. Importantly, the research derived from community-based samples helped to establish that sexual minorities did not systematically differ from heterosexuals, except for their experiences related to adversity and stigma. Such research helped to alter larger societal views of homosexuality (e.g., leading to the removal of homosexuality from the DSM list of mental disorders in 1973) (Bayer, 1987), and set the stage for contemporary understandings of sexual minority lives and health (Meyer, 2003).

Due in part to the difficulty associated with collecting probability-based samples of sexual minority people, community-derived samples remain the bedrock for sexual minority population research. The sophistication by which such samples are collected has improved over time, including the use of targeted recruitment across distinct venues to capture a diverse range of sexual minority people, random digit dialing, computer-assisted and internet-based surveys, or respondent-driven methods that apply weights to accommodate non-random sample collection (Heckathorn, 1997; Meyer, Schwartz, & Frost, 2008; Muhib et al., 2001; Rothblum, 2007; Sadler, Lee, Lim, & Fullerton, 2010; Watters & Biernacki, 1989). Important lessons have been gleaned from community-based samples, which have allowed researchers to advocate for the inclusion of measures of sexual minority status in larger population-based studies (Sell & Holliday, 2014). The inclusion of measures in large-scale surveys has been a significant advance for sexual minority researchers and advocates, and results from such studies have contributed to arguments for improvements in social, legal, and political equality for sexual minority people (e.g., the repeal of “Don’t Ask, Don’t Tell” and the legalization of same-sex marriage). Given the growing availability of both types of samples, it is now possible to assess the efficacy of community samples to reach populations that are achieved in probability samples.

Relatively little research has attempted to compare community-based and probability samples of sexual minority populations. In the most extensive assessment of this question to date, Salway et al. (2019) performed a systematic review of 21 studies that assessed selection bias among non-probability samples of sexual minority people, using a variety of methods (e.g., by comparing non-probability samples to general population-based surveys or random digit dial surveys). They revealed consistent evidence of selection bias; in the majority of cases, single, lesbian/gay-identified, and higher-SES sexual minority people were over-represented in studies using non-probability samples.

Although studies derived from both sample types generally show disparities between sexual minority and heterosexual populations (e.g., in mental or behavioral health), the magnitude of difference may vary by sampling strategy (Salway et al., 2019). For instance, studies derived from community-based samples of men who have sex with men may overestimate HIV and sexual risk behaviors, as compared to probability-based samples (Dodds et al., 2006). In another study, Drabble et al. (2018) compared substance use outcomes between heterosexual and sexual minority women in a national probability sample, as well as to sexual minority women from a community-based sample from Chicago. They obtained mixed results; in comparison to heterosexual women, sexual minority women from the community sample reported greater drug use disparities, but fewer tobacco use disparities, as compared to sexual minority women from the probability sample.

A key consideration when making comparisons across distinct samples is to understand how similar (or distinct) the intended populations under study were. Sexual minorities are diverse and include distinct subpopulations (e.g., monosexual [e.g., gay, lesbian] vs. plurisexual [e.g., bisexual, pansexual] people; men vs. women; diverse race and ethnic groups) often with varying social, economic, or health statuses (Balsam, Beauchaine, Mickey, & Rothblum, 2005; Koh & Ross, 2006; Marshal et al., 2011; Saewyc et al., 2008). For instance, bisexual women have lower incomes, report less social support, and are less likely to finish college compared to lesbian women (Krueger & Upchurch, 2019). Further, the emergence (or reclaiming) of new sexual minority identities such as “queer” or “pansexual” may introduce new diversity into samples of LGB people (Goldberg, Rothblum, Russell, & Meyer, 2019). Whereas several population-based studies now include measures that assess LGB identities, these newer identities (e.g., queer, pansexual) are typically excluded (Horner, 2007; Morandini, Blaszczynski, & Dar-Nimrod, 2017; Watson, Wheldon, & Puhl, 2019). The exclusion of new sexual identity labels may bias who selects into a sample of sexual minority people. Thus, comparing distinct sampling approaches also requires attention to intended sample frames, but also other recruitment or inclusion criteria for a study such as sex and gender, age, race and ethnicity, or location.

We compare the sample characteristics of participants from two companion studies, each designed to capture the same demographic profile of U.S. sexual minority adults. One sample was recruited for the first national probability survey of U.S. sexual minority adults. The other sample came from the screener survey that was collected to identify a community sample from four U.S. regions. Recruitment for the screener survey used a venue-based targeted nonprobability approach (Frost et al., 2018). Given known differences among sexual minorities in sociodemographic characteristics, for each sample we examine whether sociodemographic characteristics varied based on sexual identity.

This study uses data from the Generations Study (www.generations-study.com), designed to examine identity, stress, health outcomes, and health services utilization in three generations of sexual minority adults who came of age during different historical contexts. Two samples were collected, referred to as the national probability and the community screener samples. Eligibility criteria for both the probability and community samples included (a) identification with a sexual minority identity label (e.g., gay, lesbian, bisexual, queer); (b) reported ages 18–25, 34–41, or 52–59 at the time of recruitment¹; (c) residence in the United States between ages 6–13 (to most closely capture those who “came of age” during the historical contexts defining each cohort); and (d) completion of at least a 6^th grade education. The study was reviewed and approved by the UCLA Office of the Human Research Protection Program. The combined final analytic sample included 2,637 participants: 1,507 from the probability sample and 1,130 from the community sample.

For the current study, parallel measures were developed for each sample to allow analysis in a combined, merged datafile.

Missing demographic data in the probability survey was imputed using Predictive Mean Matching by the Generations study team (Krueger et al., 2020). For all demographic variables, less than 2% of respondents had missing responses. With the exception of zip code, demographic information was required in the community sample screener, so no imputation was required.

Demographic characteristics were presented for each sample, and differences were assessed across the two samples as follows: frequencies obtained for each characteristic in the probability sample were used to estimate expected frequencies in the community sample. Expected frequencies were then compared to the obtained frequencies from the community sample, and differences were assessed using chi-square goodness of fit tests. Effect sizes of each comparison were estimated with Cohen’s W statistic. The cutoffs used for interpreting the effect sizes were <0.3 (small), 0.3–0.5 (medium), and >0.5 (large) (Cohen, 1988). Next, demographic and recruitment differences were assessed across the three sexual identity categories (lesbian/gay, bisexual, and “newer” [i.e., queer, pansexual, other]), separately for each sample (See Table S1 for comparisons across all five sexual identity groups, separately). For tests in which an overall difference was found, post hoc pairwise comparisons assessed differences between each of the subgroups. To account for the use of survey weights, design-adjusted F-tests assessed differences in the probability sample. Pearson chi-square tests assessed differences in the community sample.

In prior studies that have compared community to general population-based samples of sexual minorities, those from community samples have tended to be older, employed, have higher incomes, and be more highly educated, as compared to respondents in population-based samples (Dodds et al., 2006; Drabble et al., 2018; Salway et al., 2019). Our results generally support some of these prior findings, with significant differences across most demographic characteristics. However, our results also provide additional insights. Like in the probability sample, birth sex, race/ethnicity, and age cohort were included in the sampling design of the community sample and despite statistically significant differences between the samples across these characteristics, the effect sizes were relatively small.

There were, however, marked differences between the samples across other characteristics, highlighting areas to consider when recruiting community samples of sexual minority adults. Notable differences were found across sexual identities. Roughly 35% more bisexuals were recruited in the probability sample, compared to the community sample (40.6% vs. 26.5%). Since the probability sample was weighted to the U.S. adult population of sexual minorities, this suggests that bisexuals were underrepresented in our community sample. This finding is consistent with a recent systematic review, which showed that bisexual-identified respondents tend to be underrepresented in community-based samples compared to general population surveys (Salway et al., 2019). It is possible that this difference was related to a selection bias in our community sampling strategy; prior research has shown that bisexual people feel less connection than gay and lesbian people to the LGB community, on average (Frost & Meyer, 2012), and so fewer members of this population may have been recruited from the sexual minority-specific spaces targeted for recruitment (e.g., LGBTQ book shops, Pride events). To capture a diverse array of lived experiences, future community-based recruitment efforts should consider targeted recruitment of bisexual, queer, and pansexual respondents. Successful recruitment from these populations may be challenging in more traditional, brick-and-mortar spaces (e.g., LGBTQ bars), and more luck may be found online. Indeed, over half of bisexual respondents (54.5%) and respondents using newer identity labels (62.8%) in our community sample reported that they were recruited on websites and social media – significantly more than gay/lesbian respondents (48.3%).

There were small racial/ethnic differences between the samples, with a greater proportion of White respondents in the probability sample, and a slightly greater proportion of Latinx participants in the community sample. Yet for both samples there were no significant racial/ethnic differences by sexual identity. There were also marked differences between the samples in terms of education, consistent with prior research (Drabble et al., 2018). Nearly twice as many respondents in the community sample as in the probably sample had a college degree or more. This highlights the education bias common among community-based samples, especially those conducted around universities. Results also showed lower reported levels of education for bisexuals relative to gay/lesbian people across both samples. Future community-based studies may consider making explicit efforts to seek participant diversity in education levels to offset the education bias – and the potential overrepresentation of lesbian/gay-identified respondents.

Perhaps not surprisingly, our probability sample was able to capture more urban/rural diversity than the community sample. This is largely due to the recruitment design of the community sample, for which data were collected within an 80-mile catchment of four cities. Given an established urban bias in much population research, relatively little is known about rural sexual minority populations (Leedy & Connolly, 2008), and future community-based recruitment efforts could specifically target rural sexual minority populations. While there were no sexual identity differences in urban/rural residence in the probability sample, there were significant identity-based differences found in the community sample, with bisexual respondents being more likely to live in non-urban settings than other groups. Together, this finding may suggest that in order to increase the proportion of bisexual respondents in community-based surveys, researchers should be more intentional about recruiting in non-urban spaces.

This study also offers important lessons for future research using probability samples. Large-scale, population-based studies that include measures of sexual identity often rely on measures that capture historically “traditional” sexual minority identities (e.g., lesbian, gay, or bisexual identities) (e.g., National Epidemiologic Survey on Alcohol and Related Conditions). However, research has shown that sexual minority identities span a great diversity of labels not captured by these standard measures (Horner, 2007; Morandini et al., 2017; Savin-Williams, 2005), and so measures typically used in probability-based sample surveys may fail to capture the full sexual minority population (Igartua, Thombs, Burgos, & Montoro, 2009; Vrangalova & Savin-Williams, 2012). Notably, 12.5% of respondents from the probability sample identified with a newer identity label, compared to 17.4% in the community sample. It is possible that the proportion in the probability sample is smaller than in the community sample because a subset of the queer and pansexual populations did not respond affirmatively to the initial Gallup interview question (which asked about LGBT identity). Further, appreciable proportions of both samples selected the “write-in” sexual identity option. While these response options present challenges for data cleaning and management, including such an option on population-based surveys may allow researchers to capture a greater diversity – and thus may elicit participation by a sample more representative – of sexual minorities, while also accounting for changes in the use of sexual identity labels over time.

Respondents in the probability sample were invited to participate by interviewers at Gallup, while those recruited into the community sample were recruited across multiple venues, including by different recruiters. The sample composition and findings here cannot be distinguished from factors specific to the study recruitment. Further, there was a small amount of missingness for several demographic variables from the probability sample (less than 2% for all variables).

There are limitations with the measures of sociodemographic characteristics. Sexual orientation encompasses multiple domains (i.e., identity, attraction, and behaviors), yet, respondents were recruited on the basis of sexual identity for both surveys. Thus, people who do not identify as sexual minorities (e.g., those who are heterosexual but same-sex attracted) were omitted, and therefore our study only pertains to sexual minority-identified adults. We also acknowledge that many people, especially non-monosexual people, use multiple labels to describe their sexual identities (Galupo, Mitchell, & Davis, 2015). Therefore, the discrete categorization of bisexual people and people who use newer labels may not capture the lived experiences of people who use bisexual, pansexual, and queer interchangeably or in different contexts. In addition, the “other” sexual identity category included a heterogeneous mix of responses (e.g., asexual, fluid). Low frequencies of endorsement precluded us from examining these “other” identities separately. Thus, we continue to know little about these groups of sexual minorities. Further, by design, our samples did not include transgender-identified respondents, but did include genderqueer/non-binary respondents. We acknowledge that genderqueer/non-binary is sometimes included under the larger transgender umbrella, and as such it is possible that our samples include some who might have identified as genderqueer/non-binary plus transgender, given the opportunity to select multiple options.

Finally, there are several important limitations with the reliance on three racial/ethnic categories. On one hand, this design choice was to allow development of a probability sample of LGB adults for which robust race/ethnic group comparisons could be made. Yet this measure by definition excludes groups (e.g., Asian American and Pacific Islander Americans; Native Americans) whose numbers are too few to allow statistical comparison, or they are hidden within one of the 3 primary groups which include some multi-racial respondents. (Indeed, 17.3% in the community sample and 8.5% in the probability sample reported multiple or “other” race/ethnicities beyond the single-race categories used in this study; however, in sensitivity analyses, results did not vary between models that included vs. excluded multirace and “other” race participants). Such an approach to racial/ethnic categorization is problematic for individual study participants who must decide “which box to check” (Eisenhower, Suyemoto, & Lucchese, 2014), and ultimately, the approach obscures complexity and diversity in racial and ethnic identities among sexual minorities. Such complexity and diversity deserves further attention in studies designed to explicitly explore the intersections of sexual, racial, and ethnic identities.