We reviewed methodology reports, codebooks, and other documentation files for each survey to extract information on available rural-urban measures, accessibility of rural data, years of data available, mechanisms for accessing data for analysis, survey modality, sampling method, and available cancer-related variables.

To examine the representation of “rural” in population-based surveys and to determine the challenges of using such data to study rural cancer disparities, our team performed a review of published articles that used these surveys as the key data source. We performed a search on February 27, 2019 for each survey in PubMed using the following search strategy: “non-metropolitan” OR “rural” OR “Appalachia” OR “Delta Region” OR “Deep South” AND [survey name] AND “cancer”. We included “Appalachia”, “Delta Region”, and “Deep South” in our search because these are primarily rural regions of the country that experience notable cancer disparities (Zahnd et al., 2017; Blackley et al., 2012; Coughlin et al., 2002). This initial search yielded a total of 95 articles: 47 articles for BRFSS, 18 articles for MEPS, 17 articles for NHIS, and 13 articles for HINTS. We excluded articles that used questions from the respective surveys in primary data collection efforts but did not use the national survey itself. We also excluded articles for which “cancer” or “rural” was part of an author affiliation or were noted in the abstract but were not specifically examined in the study. In total, we examined 32 BRFSS, 9 MEPS, 15 NHIS, and 12 HINTS studies (68 articles total: See Appendix 1 for list of reviewed articles). From each article, we extracted information on the metric used to define “rural,” the way that metric was used to categorize “rural”, sample size, the percent of the sample that was “rural”, year(s) of data used, study outcome(s) of interest, whether the author affiliations were federal or non-federal (e.g., academic, non-profit), limitations noted by study authors, and additional limitations or relevant information identified by the investigative team.

The BRFSS is a phone-based CDC survey that has been conducted annually since 1984, includes more than 400,000 participants each year, and is administered at the state level (Table 1) (Centers for Disease Control and Prevention, 2014). Although states are required to ask a core set of questions, there are optional modules on a variety of topics that states may include in their annual survey. Thus, nationwide BRFSS data are publicly available from the CDC, but may also be obtainable from each state’s public health department. The survey covers a range of health behaviors, chronic diseases, and utilization of preventive health services. Most relevant to the study of cancer, in even numbered years, the BRFSS currently includes questions on screening for colorectal, cervical, breast, and prostate cancer. These questions are also offered as an optional module in some odd numbered years. Optional modules offered in recent years include modules focused on cancer survivorship, human papillomavirus (HPV) vaccination, and lung cancer screening. CDC has provided access to geographically specific data through its Selected Metropolitan/Micropolitan Area Risk Trends since 2013 (Centers for Disease Control and Prevention, 2018b). However, these datasets only include areas with at least 500 survey participants, which would likely not include rural areas. In 2008, the BRFSS began piloting a cell phone survey, and beginning in 2011, both landline and cell phone participants were included in the publicly available dataset. Landlines are sampled using a disproportionate stratified sample and cell phones using a random sample. In 2011, the BRFSS began to employ a new raking weighting methodology that considers more than age, race/ethnicity, and gender in weighting, but also considers educational status, marital status, property ownership, and telephone ownership. This approach ensures that weights are appropriate for each state based upon key demographics; reducing biases, and improving representativeness (CDC, 2012). Metropolitan statistical area (MSA) status of a survey respondent is an available geographic metric in publicly available BRFSS data. MSAs are urbanized areas with 50,000+ residents (United States Census Bureau, 2018b). However, in the most recently available 2017 BRFSS data, the codebook indicates that only those who participated by landline had a known MSA status. Data from BRFSS codebooks show that unknown MSA status (i.e., data were previously indicated as “Blank” in the codebook) in the publicly available data has consistently increased over time since the inclusion of cell phones; from 15.3% unknown in 2011 to 57.4% unknown in 2017 (BRFSS, 2013; CDC, 2018a, 2018b). In 2010, the year prior to the inclusion of cell phone participants in the publicly available BRFSS data, the proportion of survey participants unknown on MSA status was only 1.4% (BRFSS, 2011). Further, BRFSS questionnaires ask respondents to provide their ZIP code and county, but neither these data, ZIP code level nor county-level rural-urban measures, are publicly available (Centers for Disease Control and Prevention, 2019c).

Seventeen of the 32 BRFSS articles (Table 2 and Appendix 1) that we reviewed were nationally focused; seven articles were focused on a single state; and seven were regionally focused (e.g., Appalachia, Delta Region). One article used data from all states that included an optional module (i.e., HPV vaccination) (Monnat et al., 2016). USDA-based Rural-Urban Continuum Codes (RUCC), Urban Influence Codes (UIC), or Rural-Urban Commuting Area (RUCA) codes were the most commonly used metrics, but these studies were performed by federal researchers who may have had more access to county information to enable linkage or were performed on single state data that may be more readily accessible from the respective state (Table 3) (Berkowitz et al., 2019; Coughlin and Thompson, 2004; Henry et al., 2014). RUCCs and UICs are both county-based measures of rural-urban status developed by the USDA to categorize counties based on population size and adjacency to metropolitan areas (USDA, 2019b, 2019c). RUCA codes are USDA-developed codes that categorize census tracts based on population density, urbanization, and commuting patterns (USDA, 2019d). Most studies did not note the proportion of the study sample that lived in a rural area, but studies that did report the proportion indicated between 12.7% and 49.4% of the study sample lived in rural areas (Henry et al., 2014; Nuno et al., 2012). Overall sample sizes ranged from 1437 in a single state to 316,763 in a study that used national data (Moss et al., 2012; Bennett et al., 2012). Most studies used BRFSS data at the individual level, but a few state or regionally focused studies that employed an ecological design used county-level estimates of health risk factors or screening alongside rural-urban or regional designations (Christian et al., 2011; Sadowski et al., 2016). Studies primarily focused on breast, cervical, and colorectal screening while a few studies focused on cancer-relevant health behaviors and outcomes (e.g., smoking, alcohol use, obesity). Despite the considerable amount of missingness on MSA status, most studies did not note this as a limitation. One study by Bennett noted that, beginning in 2006, the BRFSS stopped publicly releasing data for counties under 10,000 residents, which led to an underrepresentation of rural respondents in BRFSS data, but this was only noted in three subsequent studies (Bennett et al., 2012; Bennett et al., 2011; Bennett, 2013). One study did note that no data from Alaska were released, a largely rural state (Doescher and Jackson, 2009).

NHIS is a CDC-sponsored nationally representative survey conducted since 1957 that covers a range of health topics (CDC, n.d.). Specific to cancer, NHIS includes questions on cancer screening, cancer-relevant health behaviors, genetic testing, family history, cancer risk, and cancer survivorship. This survey is administered in person through a computer-assisted personal interview approach (Table 1). More than 87,000 persons are sampled from 35,000 households using an area probability sampling approach that does not currently oversample racial/ethnic groups. Rural-urban variables are not available in the publicly accessible dataset but can be accessed through a research data center (RDC). RDCs are centers, such as the CDC’s National Center for Health Statistics (NCHS), RDC located in Hyattsville, MD or 29 Federal Statistical RDCs across the country, that provide researchers with access to restricted-use data while protecting the confidentiality of survey participants (Centers for Disease Control and Prevention, 2019; US Census Bureau, 2019 ). These data are only accessible onsite at one of these RDCs. In order to use data from RDCs, researchers must submit a proposal to the respective agency or center and gain approval prior to using the data. The cost of data access is typically estimated at $3000; this does not include any costs associated with travel or accommodations while performing analysis at an RDC (Centers for Disease Control and Prevention, 2019c).

Of the 15 studies we reviewed which used NHIS data (Tables 2–3 and Appendix 1), six used an MSA/non-MSA measure to assess rural-urban status (Calle et al., 1993; Duelberg, 1992; Fischer et al., 1998; James et al., 2006; Kleinman and Kopstein, 1981; Leach and Schoenberg, 2007; Makuc et al., 2007), four studies used RUCC (Palmer et al., 2013; Singh et al., 2017; Weaver et al., 2013a, 2013b), two used a census tract measure based on population size (Carlson et al., 2018; Whitfield et al., 2018), and the other studies did not report how rural-urban was categorized. Among the 10 studies that reported adequate details on the samples, sample populations in these studies ranged from 18.8% to 43.2% rural (Fischer et al., 1998; Whitfield et al., 2018), though one study that simultaneously explored rural-urban and racial disparities indicated the rural samples ranged from 5%–65% depending on the racial/ethnic group (Singh et al., 2017). The most commonly studied outcomes were cancer screening and cancer survivorship-related concerns.

MEPS is a nationally representative survey conducted by the Agency of Health Research and Quality (AHRQ) since 1996. The “Experiences with Cancer” supplement was included in 2011 and 2016 (Table 1) (NCI Healthcare Delivery Research Program, n.d.). MEPS includes survey questions on cancer-related behaviors, screening, and cost of care. The “Experiences with Cancer” supplement is administered to individuals reporting a previous or current cancer diagnosis as an adult and includes questions on financial burden related to cancer, access to care, employment, and use of health care services and prescription drugs. The MEPS draws its sample from households that participated in the previous year’s NHIS. As a panel survey, participants are interviewed five times over a two-and-a-half-year period. In the most recently available MEPS iteration (2016), more than 33,000 individuals were surveyed from among 13,587 families. MSA and non-MSA indicators were available publicly until 2013, but are currently only available at either AHRQ’s RDC in Rockville, MD or a federal statistical RDCs (Agency for Healthcare Research and Quality, 2019a; US Census Bureau, n.d.). Like the NHIS protocol, such analyses must be performed at an RDC for an associated cost. At RDCs, researchers can link MEPS data to state and county-level variables such as the Area Health Resource File (AHRQ, 2019a, 2019b, 2019c).

Six of the nine articles that we reviewed (Tables 2–3 and Appendix 1) used an MSA/non-MSA designation to define rural (Han et al., 2015; Horner-Johnson et al., 2014, 2015; Litzelman et al., 2017, 2018; Whitney et al., 2016). The remaining articles used a USDA definition (i.e., RUCC, UIC, RUCA) (Caldwell et al., 2016; Dobalian et al., 2003; Larson and Correa-de-Araujo, 2006). Rural populations ranged from 14.9% to 26% depending on the outcome of interest, although two studies did not provide information on the proportion of the population living in rural areas (Dobalian et al., 2003; Han et al., 2015). Studies utilizing MEPS primarily focused on financial or caregiver challenges surrounding a cancer diagnosis or cancer screening. Of these studies, only two of them used the Experiences with Cancer Supplement.

HINTS is another population-based survey that is administered by the NCI and focuses primarily on cancer communications, caregiving, screening, risk perception, and cancer-related health behaviors (Table 1) (National Cancer Institute, n.d). The publicly available data includes the RUCC for participants’ county of residence. Five iterations of HINTS have been administered since 2003 with data from 2011 to 2018 (the most recent data available) collected in cycles over multiple years. In the early years of data collection, HINTS utilized both random digit dialing and/or an address sampling frame (i.e., mailed survey) but since 2011 have only used the mailed survey approach. As part of its sampling approach, HINTS has identified and created high- and low-minority strata (i.e., areas with high or low proportions of minority populations) and oversampled high-minority strata to facilitate better estimates for minority populations. RUCA codes are also available upon request from the NCI. Additional geographic coverage categorizations include Census Region and Division along with the recent inclusion of Appalachia-specific categories (NCI, 2018). State-level data may be available upon request, but sample sizes may be too small to present stable findings without sufficient aggregations across survey years (National Cancer Institute, n.d).

Nine of the twelve HINTS articles (Table 2 and Appendix 1) that we reviewed examined rural-urban differences at the national level (Befort et al., 2013; Goldner et al., 2013; Greenberg et al., 2018; Hong and Cho, 2017; Jiang et al., 2017; Mohammed et al., 2018; Robertson et al., 2018; Zahnd et al., 2010), while the remaining two examined Appalachian/non-Appalachian disparities and rural-urban disparities within Appalachia (Rice et al., 2018; Vanderpool et al., 2007; Vanderpool and Huang, 2010). Most of the nationally focused studies used a dichotomous rural-urban measure with the rural populations comprising 5.6% to 22.1% of the study population (Jiang et al., 2017; Zahnd et al., 2010). Overall sample sizes ranged from 542 (cancer survivors only from one year of data) to 33,749 (all survey participants across multiple years: 2003 to 2014). Study outcomes of interest included: behavioral determinants of obesity, physical activity level, HPV vaccination knowledge, use of technology for cancer self-management, use of health information technology and other cancer information seeking behavior, fatalistic beliefs about cancer, and sunscreen use.

Our examination of survey documentation and review of articles that used BRFSS, NHIS, MEPS, and HINTS to examine rural cancer disparities identified three key problems: 1) limited accessibility of rural-urban variables; 2) variability in defining rural-urban across surveys; and 3) inadequate sample sizes of rural residents.

First, there are difficulties in accessing rural-relevant variables in BRFSS, NHIS, and MEPS. The BRFSS survey has become increasingly challenging to use for rural-urban analysis. BRFSS is phone-based, and the only publicly available variable that indicates any kind of geography other than state is MSA. Since 2011, when the BRFSS began to include both landlines and cell phones, such geographic indicators have become increasingly missing in the publicly available datasets, making rural-urban comparisons impossible to achieve without significantly biased results. For example, a recent BRFSS study that examined lung cancer screening uptake among eligible individuals identified the inability to examine rural-urban differences as a study limitation (Zahnd and Eberth, 2019). The lack of readily accessible rural-urban variables is particularly problematic for the examination of lung cancer screening. Rural populations are disproportionately burdened by lung cancer, and due to the relative recency of this screening recommendation, it is important to take advantage of population-based survey data to monitor its uptake, particularly among disparate rural populations (Odahowski et al., 2019b; Rai et al., 2019). ZIP code and county-level BRFSS data are collected from survey respondents and appear to be more readily available to federal researchers, as researchers employed by the CDC were able to use small area estimation approaches to generate county-level estimates of mammography utilization (Berkowitz et al., 2019; CDC, 2019c). For the most current NHIS or MEPS data, in order to perform rural-urban analyses, researchers must complete and submit a research proposal to CDC or AHRQ, respectively, pay data set-up fees, and subsequently travel to a RDC to complete the analyses. This required process may be cost and time prohibitive for many researchers, especially early stage investigators. HINTS, the final survey we examined, has at least one rural-urban metric (e.g., RUCC) in each year the survey was fielded allowing for rural-urban comparisons. Further, the RUCC variable includes all nine levels, allowing researchers to categorize rural in the most appropriate way for their research question and the data.

Second, limited availability of rural-relevant variables across the four survey types leads to inconsistency in defining rurality. The studies we examined used seven types of rural-urban and regional measures: Office of Management and Budget metrics (i.e., MSA/non-MSA), USDA metrics, Census metrics, NCHS metrics, % urban by state, presence/absence of health care services, and federally designated regional status. However, each survey varied in what variables or geographic identifiers were readily available. While the BRFSS and MEPS have historically included a binary MSA/non-MSA measure, such dichotomization leads to “underbounding” of rural where geographically large counties with rural areas are classified as “urban” (Hart et al., 2005). NHIS data can be available for analysis at the county level from RDCs, which enables researchers to use county level rural-urban measures of their choosing. HINTS includes RUCC codes in their publicly available dataset. Each of these rural-urban metrics has its own set of strengths and weaknesses (Hart et al., 2005; Zahnd et al., 2019b). However, the lack of readily available geocoded data in which researchers can link these measures prevent them from examining those strengths and weaknesses in the context of research questions and the population distribution of their sample of interest. For example, studies examining access to cancer care relative to cancer screening may wish to use a metric like RUCAs that consider commuting patterns or RUCCs or UICs. Other studies may prefer a county-based measure that may provide a more intuitive context for policies or interventions (e.g., local health departments are often county based).

The third challenge of these datasets is inadequate sample sizes for rural research, particularly in the study of subpopulations (e.g., minorities or cancer survivors). Two studies that utilized the MEPS Experiences with Cancer Survivorship Supplement had very wide confidence intervals due in part to small rural sample sizes (Litzelman et al., 2017; Whitney et al., 2016). As another example, in order to examine cancer screening rates using BRFSS data, which had the largest annual sample size of the surveys that we examined, Cole and colleagues combined data from 1998 to 2005 to have sufficient sample size to examine colorectal cancer screening rates in rural minority populations (Cole et al., 2012). In turn, this aggregation of data makes findings difficult to interpret as rates change drastically over time among subpopulations. Additionally, a study using HINTS data noted that there were challenges in assessing rural-urban differences in skin cancer prevention behaviors among cancer survivors or those with a family history of skin cancer because of small sample sizes (Zahnd et al., 2010).

In order to mitigate these methodological and analytic challenges, we propose two key solutions. First, we suggest that federal agencies make geocoded data more readily available to outside researchers either publicly through virtual mechanisms (e.g., remote portal) or through clear data request procedures. This will allow researchers to perform analyses at their home institutions with security precautions in place. Additionally, having geocoded data would encourage researchers to use the rural-urban metric most appropriate for their study and not the one prescribed by the dataset. HINTS is a good example of providing multiple rural-urban measures options (i.e., RUCCs and RUCAs as continuous). Cancer registry data (e.g., SEER) and some federal administrative datasets (e.g., datasets maintained by the Healthcare Cost and Utilization Project [HCUP]) both have accessible geocoded data (e.g., county or zip code level) (Agency for Healthcare Research and Quality, 2019b; National Cancer Institute, n.d). These data sources provide an established precedent and set of procedures to ensure that privacy and confidentiality of data are maintained, and that, when necessary, results are required to be suppressed or presented based upon the statistical stability of the estimates. In order to obtain and use SEER data – either through SEER*Stat software or through the receipt of data on hard media – researchers are required to sign a data use agreement (DUA) stating that they will not share data in such a way that individuals can be identified or that individual level data cannot be linked to other datasets (National Cancer Institute, n.d., 2019). Researchers who use HCUP datasets, such as the National Inpatient Sample, are required to complete a tutorial as part of the DUA process that emphasizes data protection and ensures that researchers understand their responsibility to be good data stewards both in the analysis and presentation of data (Agency for Healthcare Research and Quality, 2019c). Such approaches could be leveraged and utilized by the federal health agencies that maintain these data to enhance the ability of researchers to explore rural cancer disparities.

Our second recommendation is for these surveys to alter their sampling approaches and subsequent weighting methodologies to oversample rural populations. For example, HINTS currently over-samples for racial and ethnic minorities and subsequently weights the data accordingly. HINTS also has obtained a specific sample of the 13state Appalachian population. This provides a precedent for future iterations of HINTS or other surveys meant to be representative of the nation. The BRFSS, although under the umbrella of the CDC, is conducted at the state level, which may preclude standardized approaches to sampling and survey dissemination. Most states disproportionately sample from strata within sub-state regions to help improve sample sizes in geographically defined regions, but this approach is not nationally standardized (CDC, 2017).

Our review of population-based surveys was not without limitations. While we reviewed surveys that have been commonly used to examine rural cancer disparities, we did not examine all CDC, AHRQ, or NCI-sponsored surveys that may be used for disparities research. For example, although not population-based, NCI has linked several surveys to SEER data that may have been used to examine rural-urban cancer disparities such as the Medicare Consumer Assessment of Healthcare Providers and Systems patient surveys and SEER-Medicare Health Outcomes Survey (Mollica et al., 2018; National Cancer Institute, 2019a, 2019b). Additionally, although we performed a comprehensive review of articles using these surveys, we did not perform a systematic review. There may be articles that used these surveys that we did not identify during our search process. However, our review of articles using these surveys was complemented by an evaluation of survey codebooks, methods reports, and other documentation to assess the utility of these surveys for rural cancer disparities research and subsequently make recommendations to improve the accessibility of these surveys for researchers. Our objective was not to identify the universe of all published studies using these surveys, but to identify and define noted challenges.