Bayesian hierarchical regression (BHR) is often used in small area estimation (SAE). BHR conditions on the samples. Therefore, when data are from a complex sample survey, neither survey sampling design nor survey weights are used. This can introduce bias and/or cause large variance. Further, if non-informative priors are used, BHR often requires the combination of multiple years of data to produce sample sizes that yield adequate precision; this can result in poor timeliness and can obscure trends. To address bias and variance, we propose a design assisted model-based approach for SAE by integrating adjusted sample weights. To address timeliness, we use historical data to define informative priors (power prior); this allows estimates to be derived from a single year of data. Using American Community Survey data for validation, we applied the proposed method to Behavioral Risk Factor Surveillance System data. We estimated the prevalence of disability for all U.S. counties. We show that our method can produce estimates that are both more timely than those arising from widely-used alternatives and are closer to ACS’ direct estimates, particularly for low-data counties. Our method can be generalized to estimate the county-level prevalence of other health related measurements.

National and state-level surveys are crucial to public health surveillance in the United States. These are typically designed to provide direct estimates. However, direct estimates are, due to small sample sizes, often impractical at the county level (a unit of local government in the United States). Some U.S. states use other names for these units of government, such as ‘parish’. Since they function like counties, we will call them all counties.

Small area estimation (SAE), through modeling, provides estimates for counties that do not have large enough samples for direct estimation. In brief, by borrowing “strength” from the entire domain (i.e., whole counties across state-wide or/and nation-wide) as well as auxiliary variables from other survey studies (

Frequentist model-based methods, such as those of

Additionally, BHR requires specification of a prior. Specifying an informative prior often requires more information than is readily available. Further, informative priors are inherently subjective. Non-informative priors avoid this difficulty. However, for small sample size counties, the posterior distributions are heavily influenced by the subjective priors.

The power prior (_{0} ∈ (0,1) and combining it with the uninformative ‘flat’ prior. This results in a proper posterior distribution. Many studies (

Here, we propose an approach, which we call the power prior log-weights estimates (PLOW). The PLOW incorporates adjusted design-based sampling weights and uses a power prior. As an example, we apply the PLOW to obtain estimates of county level prevalence of impaired vision from 2015 Behavioral Risk Factor Surveillance System (BRFSS) data. We provide estimates for all 3142 counties in the United States.We validate the estimates using both simulations and data from the American Community Survey (ACS), a survey large enough to provide direct estimates for many counties.

Behavioral Risk Factor Surveillance System (BRFSS) is an annual state-level telephone surveillance system conducted by the Centers for Disease Control Prevention (CDC). It collects data on risk behaviors, preventive health practices and health-related conditions in the non-institutionalized adult household population with ages 18 years and older. Details on BRFSS have been previously published (

In 2013, five survey questions concerning disability were used in common by both BRFSS and ACS.

Auxiliary variables are variables other than the variable of interest used to construct models for SAE. We use the county-level covariates age, sex and race/ethnicity as auxiliary variables. Similar to

Let _{ijk} be the binary disability outcome of survey participant _{ij}, _{ij} is the sample size of county _{ij} as the total population size in county _{ijk} to denote the sampling raking weight attached to _{ijk} and _{ijk} were derived from 2015 BRFSS while _{ij}

The most common design-based inference for the SAE is the Horvitz-Thompson (HT) estimator (

In particular,

In general, the bias

When historical data are available, it is possible to ‘borrow’ strength from it. Let _{0} ∈(0,1) be the power parameter (defined later) which controls how much the historical data impacts the prior. Let _{0}(_{ij}_{ij}_{0} and the likelihood function of _{0} is L(p∣_{0}). The power prior for _{ij}

From Bayes’ theorem, the posterior distribution of _{0} can be re-written as:

For notational convenience, we combine

We employ the BHR model for county-level disability prevalence estimates. Let _{ij}

Let _{ij}^{T}_{ij}_{i}_{s(i)} are independent. Borrowing “strength” (_{ij}

We apply the power prior, as described earlier. For example, to estimate the prevalence of disability in 2015, we use 2015 BRFSS data (most current available) and 2013 and 2014 BRFSS data (historical). We integrate them with current data through use of the power prior _{i}

The ACS releases single-year disability data for the 835 large counties (population>65000). For those counties, validation is done via direct comparison of HT (hereafter, direct estimates), PLOW estimates, and the unweighted, non-informative prior estimates of

For counties for which single year data are not released, direct comparison is not possible. For those counties, we validate through a simulation. We create fictional small sample size ‘pseudo-counties’ by sampling from actual counties for which ACS data are available. That is, we:

Randomly select 200 counties from 835 counties covered by ACS 1-year data

Randomly select 1%~5% survey samples from each selected county to form pseudo-counties; some pseudo-counties may have no data

Apply PLOW to estimate the prevalence of disability of each pseudo-county

Repeat steps 1 to 3 for 500 times and average the estimates

Validate simulation results with the real ACS 1-year results for each county

Weightings heavily influence the estimates, particularly in small population counties where the weights can be large. The weights tend to have a highly right-skewed distribution. To limit this effect, we rescale weights by a logarithmic transformation.

Validation, expressed as scatter plots, appears in

Next, we plot PLOW results against the ACS 5-year county-level data. ACS 5-year county-level data were collected in the range of five years, e.g., survey data from 2011 to 2015 were aggregated as the 2015 reports for all 3142 counties. Based on the Census Bureau guideline, we classify 3142 counties into three groups: large size with population>65000, (“large”, hereafter), medium size with population>20000 and <65000 (“medium”, hereafter) and small size with population<20000 (“small”, hereafter). Using Cognitive Difficulty for the example,

Additionally, we compared the results of the three methods at the state-level. We aggregated the county-level estimations into state-level and compared them with ACS state-level results.

To compare the power prior with flat prior implemented in the Bayesian hierarchical model, we applied PLOW to estimate prevalence of Vision Difficulty with these two types of prior, respectively, using BRFSS 2015 data. The difference is that we use BRFSS 2013 and BRFSS 2014 data as “historical” data in power prior modeling, whereas we chose the inverse Gamma distribution as the non-informative prior in the flat prior modeling.

Previous work has been done on SAE for county-level prevalence estimates using BRFSS survey data.

We have introduced PLOW, a design assisted model-based approach, and used it to estimate county-level disabilities prevalence. This approach has several advantages over model based estimates that do not use weights and design based estimates that are directly derived from weights. We demonstrate that PLOW can produce estimates with less bias and variance than other approaches.

First, PLOW calculates “effective” disability case counts using the adjusted sampling weights before applying the Bayesian hierarchical model. Smaller skewness for the weights reduces bias and variance in the downstream model-based estimates (

Perhaps most importantly, PLOW makes possible annual county-level prevalence estimates for all counties, using single-year data. Other methods are typically limited to counties with larger samples sizes or combine multiple years of data. Combining multiple years of data obscures secular trends and makes timely detection of rapid changes difficult or impossible. For example, Cadwell estimates, used by CDC for estimating county level prevalence of diabetes, obesity, and regular physical activity (

In short, PLOW, when applied to survey data for which historical data are available, can provide prevalence estimates that are both more useful and more timely. While we have provided estimates for disability prevalence, the PLOW method does not depend on any characteristic that is unique to disability. It can be used to provide county-level estimates of any measure with historical data.

Optimal “Shrinkage” Factors

The weights “shrinkage” index

MSE

Scatter plots showing validation of the disability (Vision, Cognitive, Ambulatory, Self-care and Independent living) prevalence estimates through HT direct estimate, Cadwell estimate and PLOW estimate using ACS 1-year results for “large” counties

Scatter plots of prevalence estimates of Cognitive Difficulty using PLOW against ACS 5-year county-level results by county size (“large”, “medium” and “small”)

Validation plot for comparison of results of ACS 1-year data with results of HT direct estimator (black circle), Cadwell estimates (black square) and PLOW (red triangle) for the small sizes counties using “pseudo-county” data. The diagonal reference line (dash line) represents the estimates results are same to ACS 1-year county-level results

Box-plot comparison of Bayesian hierarchical model implemented with power prior and flat prior to estimate the Vision Difficulty prevalence

The descriptions of five disabilities listed in ACS and BRFSS

Disability | ACS Disability Definitions | BRFSS Disability Questions |
---|---|---|

Vision | Blind or having serious difficulty seeing, even when wearing glasses | Are you blind or do you have serious difficulty seeing, even when wearing glasses? |

Cognitive | Because of a physical, mental, or emotional problem, having difficulty remembering, concentrating, or making decisions | Because of a physical, mental, or emotional condition, do you have serious difficulty concentrating, remembering, or making decisions? |

Ambulatory | Having serious difficulty walking or climbing stairs | Do you have serious difficulty walking or climbing stairs? |

Self-care | Having difficulty bathing or dressing | Do you have difficulty dressing or bathing? |

Independent | Difficulty because of a physical, mental, or emotional problem, having difficulty doing errands alone such as visiting a doctor’s office or shopping | Because of a physical, mental, or emotional condition, do you have difficulty doing errands alone such as visiting a doctor’s office or shopping? |

The distribution of BRFSS sampling weights and the adjusted sampling weights (N = 426218). GF= max _{ijk}_{ijk}

Weighs | min | max | median | skewness | GF |
---|---|---|---|---|---|

Raw | 1.18 | 36700 | 231.28 | 5.74 | 31102 |

Adjusted | 1.24 | 1612 | 68.16 | 2.49 | 1300 |

Mean squared error (MSE) of HT direct estimates, unweighted model estimates and new approach (weighted model estimates) for estimation of disabilities (Vision, Cognitive, Ambulatory, Self-care and Independent living) prevalence

Method | Mean Square Error (MSE)×10^{4}
| ||||
---|---|---|---|---|---|

Vision | Cognitive | Ambulatory | Self-care | Independent | |

HT direct estimates | 7.12 | 34.65 | 43.53 | 3.36 | 6.32 |

unweighted model estimates (Cadwell estimates) | 3.52 | 26.17 | 34.80 | 1.52 | 3.25 |

weighted model estimates (PLOW estimates) | 1.02 | 2.61 | 4.05 | 0.88 | 2.07 |