The U.S. Centers for Disease Control and Prevention’s (CDC’s) Behavioral Risk Factor Surveillance System (BRFSS) collects a wide range of health information, including whether respondents have a personal doctor. BRFSS data are used by state health departments to plan interventions, allocate scarce resources, and provide information to the general public. While the BRFSS focuses on state-level estimation, there is demand for county-level estimation of health indicators using BRFSS data. The ability to make the most productive use of the BRFSS data and to improve county-level planning based on BRFSS data is essential.

The overall large sample size of BRFSS allows for county-level survey-based estimation of health indicators. However, survey-based estimates can only be produced for counties with sample survey data available and are subject to high uncertainty for counties with small sample sizes. The sparsity of the survey data in some counties and the availability of data from auxiliary sources motivate researchers to explore estimation methods that extend beyond the traditional survey-based estimation. For example, model-based small area estimation (SAE) provides a principled way to combine survey and auxiliary data, while exploring the relationship between the outcome of interest and the auxiliary information, and accounting for the sources of uncertainty in the model components.

There are two main classes of SAE models: unit-level models, introduced in Battese, Harter, and Fuller (1988) ¹, and area-level models, introduced in Fay and Herriot (1979) ². The first class of SAE models take as input unit-level survey and auxiliary data, while the second class of SAE models take as input area-level survey estimates and auxiliary data. The area-level SAE models are often preferred because of their practical applicability, while overcoming challenges related to the availability of auxiliary data, linking between the survey unit and the auxiliary source unit, inclusion of survey design effects, and others (for example, definition of unit and computational resources). These models are fit to the set of areas with sample data, but estimates can be constructed for all the areas of interest.

In this paper, an area-level SAE model is developed to combine county-level BRFSS survey data with county-level data available from auxiliary sources, for the purpose of estimating the proportions of individuals who do not have a personal care provider or doctor. The areas of interest comprise of the set of U.S. counties (3,142 counties), but the model is fit only to the set of counties for which BRFSS data on having or not a personal care provider or doctor are available (3,115 counties). For evaluation purposes, the model is also fit to the set of counties with BRFSS sample size of at least 500, also known as the Selected Metropolitan/Micropolitan Area Risk Trends (SMART) counties (213 counties). Model-based county-level estimates are produced for all the U.S. counties.

SAE models have been considered in the past for estimating BRFSS quantities. We first review some of the unit-level models studies. For county-level estimation of prevalence of self-reported diagnosed diabetes, Cadwell et al. (2010)³ developed a census region-specific model taking as input BRFSS survey data at a level defined by the cross-tabulation of age, sex, race/ethnicity, and county; this model fits under the unit-level modeling framework. For county-level estimation of chronic obstructive pulmonary disease prevalence, Zhang et al. (2014) ⁴ developed a model taking as input BRFSS survey data at a level defined by the cross-tabulation of age, sex, race/ethnicity, county, and state, with extensions to finer geographies including census blocks and tracts; this model fits under the unit-level modeling framework. The model in Zhang et al. (2014) ⁴ was then applied to BRFSS data on health status and access indicators, after removing sex and state from the definition of the domain level (see Pierannunzi et al., 2016⁵), to BRFSS data on colorectal cancer screening prevalence (see Berkowitz et al., 2018 ⁶), and to BRFSS data on mammography screening rates, after removing sex from the definition of the domain level (see Berkowitz et al., 2019 ⁷). Holt et al. (2019) ⁸ developed a similar model to the one in Zhang et al. (2014) ⁴ for estimating the number of at-risk community-dwelling adults with a chronic condition (chronic obstructive pulmonary disease), while combining BRFSS and National Hurricane Center (NHC) data. As documented in these studies, the current methods are not capturing the BRFSS complex survey design effects, hence are lacking at accounting for a key source of uncertainty in the survey data. Moreover, the pool of auxiliary data is constrained to BRFSS demographic variables for most of these studies. Unlike these studies, we explore a large pool of auxiliary data and adopt an area-level modeling approach that starts with BRFSS survey estimates and accounts for the sampling variability in the survey data.

Raghunathan et al. (2007) ⁹ developed a joint county-level model for BRFSS and the National Health Interview Survey (NHIS) current-smoking and mammography screening rates; this model fits under the area-level modeling framework. For estimating county-level smoking and screening for female breast cancer, cervical cancer, and colorectal cancer rates, Liu et al. (2019) ¹⁰ extended the model in Raghunathan et al. (2007) 9 that com bines BRFSS and NHIS data. Like in Raghunathan et al. (2007) ⁹, we also model the arcsine-square-root-transformed survey-based county-level estimates, in order to mitigate unstable survey variances and help support a normality assumption for the survey-based county-level esti mates. As a result, the sampling variances for the transformed survey estimates are functions of the effective sample sizes. For counties with small sample sizes, the authors imputed the design effect and set a lower bound for the effective sample sizes. To mitigate such scenarios, as well as extreme survey-based proportions, we approximate the effective sample sizes using Kish’s approximation (see Kish, 1965 ¹¹).

Multi-fold SAE models have been studied in the literature to further account for the nested data structure of the survey data and allow for reliable estimation at various levels of aggregation. Area-level multi-fold SAE models were initially introduced in an unpublished manuscript by Fuller and Goyeneche (1998)¹², as two-fold models, and then considered in Torabi and Rao (2014)¹³. Both of these studies employed frequentist estimation of model parameters. Erciulescu, Cruze, and Nandram (2020)¹⁴ specified a two-fold SAE model as a hierarchical Bayes model and derived closed-form expressions for model predictions for counties with survey data. Krenzke et al. (2020) ¹⁵ applied three-fold SAE models specified as hierarchical Bayes models to predict U.S. adult competency at the county-level. We consider Bayesian inference to be a straightforward inferential approach for multi-fold SAE models, while allowing for the construction of full posterior distributions for quantities of interest. In addition, we model county-level BRFSS and auxiliary data, while accounting for the nested structure of counties within states and states within census divisions and allowing for reliable estimation at these three levels: county, state, and census division. In this aspect, the hierarchical Bayes SAE model developed in this paper is similar in structure to the ones considered in Krenzke et al. (2020) ¹⁵.

The rest of the paper is organized as follows. In Section 2, we describe the data available for the application study and the initial survey estimation and variable selection steps necessary to prepare the model input data. The hierarchical Bayes SAE model is presented in Section 3, along with a framework for model fit, internal validation, estimation and prediction, and comparison. Final selected results are provided in Section 4 and include external validation checks. A general discussion is given in Section 5.

The survey data for the application consist of micro-level BRFSS data for the reference year 2018, subject to unit-level adjustments described next. Using the BRFSS information on whether or not an individual has a personal care provider or doctor, an indicator variable is constructed having these two categories. The item nonresponse rate for this indicator variable is 0.64%. Missing values are imputed using a hot-deck imputation approach. The survey weights are adjusted using a raking procedure to align the county-level population totals estimated from the survey to corresponding fixed controls available from the 2014–2018 American Community Survey (ACS) Summary File (also known as Detailed Tables). The resulting micro-level data are used to produce county-level survey estimates for proportions of individuals who do not have a personal care provider or doctor. Specifically, Hájek-type point estimators and associated Taylor series variance estimators are constructed for all the counties with available survey data. Let these be denoted by county-level pairs (p^ijk,V^ijk) where i=1,…,m indexes the census divisions, j=1,…,mi indexes the states in census division i, and k=1,…,mij indexes the counties in state j and census division i. Let the county-level population totals from the 2014–2018 ACS Summary File be denoted by tijk.

County-level survey estimates and associated variances on the arcsine-square-root scale serve as inputs into the models described in the next section. Let these be denoted by county-level pairs (yijk,σijk2) where yijk:=sin−1p^ijk, and σijk2≔14n~e,ijk.

The county-level effective sample sizes are approximated as n~e,ijk≈(∑a∈kwijkaf)2∑a∈k(wijkaf)2 where wijkaf is the adjusted survey weight associated with the individual a in county k.

The pool of variables related to the transformed proportion of individuals who do not have a personal care provider or doctor consists of demographic characteristics (i.e., race/ethnicity, age, sex, marital status), socioeconomic status (i.e., poverty, income, employment status, occupation), education (i.e., education, English-speaking ability), location (i.e., urbanicity), immigration status (i.e., length of stay for foreign-born people, migration), health (i.e., insurance, health professions, facilities), and other (i.e., journey to work, housing unit tenure, tax, birth rate, fertility rate, infant mortality rate, crime rate, Federal aid, energy consumption). A list of data sources and auxiliary variables is provided in the Appendix Table 1.

As the initial step in the variable selection process, we identify the most complete and less prone to error county-level auxiliary information. Next, a correlation analysis is conducted to reduce the pool of auxiliary variables to those identified to be highly correlated with the transformed proportion of individuals who do not have a personal care provider or doctor and minimally correlated with other auxiliary variables in the pool. Finally, a small set of auxiliary variables is selected using the least absolute shrinkage and selection operator (LASSO), with 20-fold cross-validation. Two choices are considered for the LASSO shrinkage/penalty parameters, both close to the optimal value that minimizes the mean cross-validated error but resulting in different numbers of selected variables. The two corresponding models constructed with these two sets of covariates will be referred to as the full model and reduced model, respectively. The full model contains a larger number of covariates than the reduced model does. Irrespective of the model, let the matrix of covariates be denoted by xijk; its rows correspond to counties with survey sample and its columns correspond to an intercept, followed by the selected variables. The exact number of counties with survey sample and the exact number of selected variables used to fit the models are discussed in the next section.

The overall structure of the proposed small area model follows closely the structure of the small area model for average adult proficiency presented in Krenzke et al. (2020) ¹⁵. First, a sampling level is specified for the survey estimates on the transformed scale: Samplinglevel:yijk|(θijk,σijk2)~N(θijk,σijk2), where θijk are the county-level quantities of interest on the transformed scale, i.e., the transformed proportions of individuals who do not have a personal care provider or doctor. Then, we borrow strength from auxiliary data and across small areas, while accounting for the nested structure of the data, via a linking level: Linkinglevel:θijk|(β,cijk,sij,di)=xijkβ+cijk+sij+di, cijk|σc2~N0,σc2, sij|(σs2)~N(0,σs2), di|σd2~N0,σd2, where β is a vector of unknown regression coefficients, and cijk,sij, and di are the county, state, and census division latent effects, respectively, assumed to follow normal distributions with zero means and unknown variances σc2,σs2, and σd2, respectively.

To fully specify the model as a hierarchical Bayes model, we adopt independent weakly informative priors for the unknown model parameters β,σc,σs, and σd:

Priors:β~N(0,104),component−wise

σc,σs,σd~Cauchy0,5,component−wise.

These choices of prior distributions ensure that little information about the values of these parameters is provided to the model, the data (likelihood) having the major role in shaping the posterior distributions. See Krenzke et al. (2020)¹⁵, Browne and Draper (2006)¹⁶, Gelman (2006) ¹⁷, and Polson and Scott (2012)¹⁸, for some recent discussions about the choice of prior distribution for the scale parameter in a univariate hierarchical Bayes model.

Variable selection and cross-validation results are presented in Tables 1 and 2, for the models fitted to all the counties with sample data and for the models fitted to the SMART counties, respectively. The auxiliary variables selected in the models fitted to all the counties with sample data are available at the county-level from ACS, Small Area Health Insurance Estimates (SAHIE), or Statistics of Income (SOI). The first six variables listed in Table 2 are available at the county level, but the rest are only available at the state level. Compared to the auxiliary data selected for the models fitted to all the counties with sample data, data from three additional government agencies were selected for the models fitted to the SMART counties: the National Center for Health Statistics (NCHS), the National Highway Traffic Safety Administration (NHTSA), and the Energy Information Administration (EIA). Recall Table 1 in the Appendix with the information on the data sources. The full models have better predictive power than the reduced models, having smaller sum of squared differences between the model predictions and the survey estimates. The results from the model comparison using WAIC are also in favor of the full models, as shown in Table 3.

For the counties with available survey estimates, we compare the distributions of the survey estimates and the model predictions produced using the four models under investigation. The results are illustrated in Figure 1. There are minor differences between the full and reduced models for each of the two pairs considered. Based on these results and the results from the model comparison and cross-validation, we conclude that the reduced models are no longer of interest.

As illustrated in Figure 1, the point estimates are comparable across the different sources considered in this comparison, with exception being the main mode of the distributions: the model predictions fall in a narrower range than the range of the survey estimates, as a result of smoothing, with the model predictions based on the models fitted to the SMART counties being smoothed towards slightly larger values than the other model predictions. The uncertainty in the model predictions based on the models fitted to the SMART counties is noticeably smaller than both the uncertainty in the survey estimates and the uncertainty in the model predictions based on the models fitted to all the counties with sample data. This result is a consequence of constructing a much larger number of not-in-sample model predictions under the approach where only the SMART counties are used in the model fit, while ignoring the survey data available for the non-SMART counties.

To further evaluate the model fits using different sets of counties, we compare the estimates for large geographies: census divisions and nation. Pairwise comparisons of the survey and model predictions for the high levels of aggregation are presented in Table 4. Note that for most of the census divisions, there is closer agreement between the survey estimates and the model predictions based on the model fitted to all the counties with sample data than the model predictions based on the model fitted to the SMART counties. In addition, for the nation and most of the census divisions, the standard errors of the model predictions based on the model fitted to the SMART counties are larger than the survey standard errors. This result is a consequence of constructing a large number of positively highly correlated not-in-sample model predictions under the model fitted to the SMART counties only, which impacts the variance estimates at aggregated levels. Hence, we identify the model fitted to all the counties with sample data to be the preferred or final model. A map of all the county-level model predictions constructed based on the final model is illustrated in Figure 2.

Finally, we note that the covariates selected do not present a very strong linear relationship to our outcome of interest (the proportion of individuals who do not have a personal care provider or doctor); the R squared for a multiple linear regression fitted to the county-level survey estimates and using the set of covariates selected for the model fitted to all the counties with sample data is approximately 0.08. Hence, it is important to use all the available county-level survey estimates in fitting the model, so that the number of not-in-sample predictions is as small as possible.

In summary, we developed model-based small area estimation methodology for the proportion of individuals with no personal care provider or doctor in the 2018 population. The models combine survey data from the BRFSS with data from other sources, while accounting for the error in the survey data and the nested structure of the data. Model predictions were constructed for all the counties in the nation, irrespective of the availability of survey data. For counties where survey estimates are available, the model predictions are more precise than the survey estimates. For evaluation purposes, the models were developed using either all the counties with survey data or only the counties with sample sizes at least 500. The former was preferred because it resulted in more accurate point estimates than the latter.

As part of the model development, we investigated other models and compared their performance to the performance of the model presented in this manuscript. Among these alternative model specifications, we investigated area-level models for the BRFSS survey estimates on the original scale or transformed using other transformations: square root and logit. Similarly to the transformation adopted for the proposed model, the square root and the logit transformations help relax a normality assumption for the survey estimates on the original scale. However, the logit transformation is only applicable to survey estimates greater than zero and smaller than one. Moreover, for the original scale of these two alternative transformations, the estimated variances associated with the survey estimates would not be defined for all the counties, either due to sample sizes or due to the transformation. Finally, unlike the transformation adopted for the proposed model, none of these three alternative model specifications would stabilize the sampling variances, so a smoothing step would be necessary, too.

Lacking unit-level auxiliary data for the entire population, we did not investigate any unit-level models. However, we did consider a hybrid-level model, where the sampling level is specified at the unit level and the linking level is specified at the area (county) level. As an initial step, we ignored the survey design effect and the imputation. That is, we specified the sampling level for the BRFSS unit-level raw data. The computational challenges led us to quickly set aside a nation-wide model, and instead fit a model to all the data available for one state. As part of the internal validation process, we noticed residuals with heavy-tailed distribution, so the model specification indicated there was room for improvement. Also, the state-level (aggregated) model prediction was significantly smaller than the state-level survey estimate, again indicating the model specification can be improved.

Future investigations may include closer attention to the hybrid-modeling and extensions to the model presented in this manuscript: (1) multivariate specifications, for example for modeling the BRFSS proportion of individuals who do not have a personal care provider or doctor and the proportion of individuals without health insurance jointly; and (2) measurement error specifications, to account for the uncertainty in the county-level covariates.