pmc988062736116J Stat Comput SimulJ Stat Comput SimulJournal of statistical computation and simulation0094-96551563-5163388839681117758210.1080/00949655.2023.2293124HHSPA1967526ArticleMultiple imputation of missing data with skip-pattern covariates: a comparison of alternative strategiesZhangGuangyuHeYuleiCaiBillMoriarityChrisShinHee-ChoonParsonsVanIrimataKatherine E.National Center for Health Statistics, Hyattsville, MD, USCONTACT Guangyu Zhang, VHA1@CDC.GOV, National Center for Health Statistics, 3311 Toledo Road, Hyattsville, MD 20782, US1320242023146202494715431570https://www.tandfonline.com/action/journalInformation?journalCode=gscs20Full Terms & Conditions of access and use can be found at https://www.tandfonline.com/action/journalInformation?journalCode=gscs20

Multiple imputation (MI) is a widely used approach to address missing data issues in surveys. Variables included in MI can have various distributional forms with different degrees of missingness. However, when variables with missing data contain skip patterns (i.e. questions not applicable to some survey participants are thus skipped), implementation of MI may not be straightforward. In this research, we compare two approaches for MI when skip-pattern covariates with missing values exist. One approach imputes missing values in the skip-pattern variables only among applicable subjects (denoted as imputation among applicable cases (IAAC)). The second approach imputes skip-pattern covariates among all subjects while using different recoding methods on the skip-pattern variables (denoted as imputation with recoded non-applicable cases (IWRNC)). A simulation study is conducted to compare these methods. Both approaches are applied to the 2015 and 2016 Research and Development Survey data from the National Center for Health Statistics.

Multiple imputationmissing skip-pattern variablesRANDS surveyIntroduction

Multiple imputation (MI) has been widely used to address missing data issues in many fields [1–3]. To construct an imputation model, an appropriate missing data mechanism (e.g. missing at random) is first assumed. Then a statistical imputation model is formulated to relate the missing variable(s) to the observed variable(s) and other available information. In MI, missing values are replaced (imputed) by draws from their posterior predictive distributions based on the imputation model. Such a procedure is independently repeated multiple (say M) times, resulting in M sets of imputed values. Each of the M completed datasets is analyzed separately, and the M sets of results are then combined to yield a single set of statistical inference using Rubin’s combining rules [1,2].

In practice, a complicated data structure often imposes challenges for implementing MI. In this paper, we evaluate and compare two approaches of MI with missing skip-pattern covariates. Skip-pattern questions are regularly used in surveys, where survey respondents are either skipped or directed to certain question(s) based on their responses to a prior question(s) in the survey [4–7]. While skip-pattern questions help to reduce respondents’ burden and shorten survey completion time, incorporating skip-pattern questions in surveys remains a challenge for general statistical analysis including MI. To our limited knowledge, there is scarce previous literature on the evaluation of the performance of different approaches in the application of MI with missing skip-pattern variables.

This research is motivated by a missing income data problem in the Research and Development Survey (RANDS) (https://www.cdc.gov/nchs/rands/), a series of surveys conducted by the National Center for Health Statistics (NCHS) [8–10]. RANDS was designed to explore the feasibility of using recruited web panels to collect information on national health outcomes and to augment NCHS’ question-response evaluation and statistical research. RANDS 1, conducted in 2015, and RANDS 2, conducted in 2016, are probability-sampled web-based surveys which collected information on health-based topics although both surveys contained missing values for income (overall 21% missingness). Missing data on income-related questions occur very often in surveys, and MI has been a popular approach to address missing income data issues [11–16]. Following this literature, we use MI to impute missing household income data in RANDS. However, several skip-pattern covariates with missing values exist and impose challenges.

Specifically, around sixty covariates, including several skip-pattern variables on smoking, health insurance and working status, are included in the imputation model (Table A1 in Appendix C and Figure 1). For example, for smoking related questions, the first question is ‘have you smoked at least 100 cigarettes in your entire life?’ (an ever smoked question), with responses ‘yes’ or ‘no’. If a respondent’s answer is ‘yes’, then the follow-up question is ‘how often do you now smoke cigarettes?’ (a current smoking question). If the answer to the ever smoked question is ‘no’, then the current smoking question would not be applied to the individual, i.e. skipped. The skip-pattern variables, like other survey questions, can contain missing values themselves. Thus, missing data in the skip-pattern covariates need to be imputed if they are included in the imputation model. Missing data in the skip-pattern questions occur when the prior question is missing or when a respondent provides an answer to the prior question but not to the skip-pattern question. For example, if an individual does not answer the ever smoked question, then the individual would have missing data on both the ever smoked and the current smoking questions. If the individual answers ‘yes’ to the ever smoked question but does not answer the current smoking question, then the individual would have missing data on the current smoking question.

In the literature, two approaches have been proposed to address missing data in the skip-pattern variables in MI practice. One approach is to impute the skip pattern questions sequentially [14,17,18], i.e. first impute missing values in the prior question among all subjects (e.g. impute missing values for the ever smoked question among all subjects) and then impute the missing values on the skip-pattern question only among those who are expected to have an answer for that question (e.g. impute missing values for the current smoking question only among those who have a ‘yes’ for the ever smoked question). Another approach that has been used in practice is to first recode skip-pattern questions as missing data or as some observed values for non-applicable subjects and then impute missing data in the skip-pattern questions among all subjects; after imputation, recode the imputed/recoded values for skipped items back to ‘not applicable’ in the imputed data sets to preserve skip patterns [19,20]. In this paper, we denote the first approach as imputation among applicable cases (IAAC) and the second approach as imputation with recoded non-applicable cases (IWRNC).

The first approach, IAAC, appears to be logical. However, to the best of our knowledge, only one software package, IVEware [18], has a built-in function for skip-pattern variables. Other software packages, such as SAS [21] and R [22], do not distinguish skip-pattern variables from non-skip-pattern variables. Thus, it would require additional programming to conduct imputation (i.e. impute the prior question(s) in one procedure and impute the skip-pattern variable(s) in a separate procedure), which could increase the complexity of multiple imputation programming, especially for hierarchical high dimensional data with multiple skip-pattern variables. The second approach, IWRNC, is easy to implement and does not require separate imputation procedures. However, imputing meaningful values for subjects who have been skipped seems to be counterintuitive. In addition, there are no common rules on how to recode the skip-pattern variables before imputation or when this approach can be applied. Finally, there are limited theoretical or empirical evaluations behind this approach.

In this paper, we evaluate the approaches mentioned above via simulation studies. The goal is to provide general guidance on multiple imputation of missing data when skip-pattern variables exist and contain missing values. The paper is organized as follows. Section 2 provides some background on sequential regression multiple imputation (SRMI) for complex survey data. Sections 3 and 4 describe two simulation studies for multiple imputation with skip-pattern covariates: Simulation 1 (Section 3) includes a categorical skip-pattern covariate, and Simulation 2 (Section 4) includes a continuous skip-pattern covariate. Section 5 summarizes the results of the simulation studies. Section 6 describes a multiple imputation procedure for imputing missing income in RANDS 1 and 2, where several skip-pattern covariates (smoking, health insurance, working status) with missing data were included in the imputation model. Section 7 contains concluding remarks.

Sequential regression multiple imputation for missing data problems in complex surveys: some background

In our motivating example and other practical settings, missing data problems with skip patterns are often in the context of large surveys that involve complex survey designs and many variables with missing values simultaneously. To achieve approximately valid inference, the imputation model and the subsequent analytical model (i.e. the model fit on the imputed data) must be congenial [23,24]. In the context of complex survey data, design information (sampling weights, stratification, clustering, etc.) is critical for valid design-based inference, and thus it is important to include features of the survey design in the imputation model [1,14,23–27].

To describe the idea briefly, let Y=Ymiss,Yobs, where Y is a vector of variables included in MI, Ymiss denotes a vector of variables with missing values, and Yobs denotes a vector of variables without missing values; let Z be a vector of variables containing survey design information without missing values; and M be a vector of missing data indicators. Then an imputation model based on the likelihood approach for the complex survey data can be formularized as f(Y,M,Z∣θ,ψ)∝f(Ymiss,M∣Yobs,Z,θ,ψ)=f(Ymiss∣Yobs,Z,θ)f(M∣Ymiss,Yobs,Z,ψ), where f() denotes the distribution function; θ, ψ are vectors of unknown parameters for the prediction model, fYmiss∣Yobs,Z,θ, and the missing-data-mechanism model, f(M∣Ymiss,Yobs,Z,ψ, respectively. When missing data are missing at random (MAR) and the parameters θ and ψ are distinct, the missing data mechanism can be ignored for the purpose of estimating θ, and this is the so-called ‘ignorable missingness’. Under ignorable missingness, an imputation model can be constructed solely based on fYmiss∣Yobs,Z,θ. When the missingness is not ignorable the missing data mechanism needs to be included in the imputation procedure. In practice, MAR (or ignorable missingness) is a popular assumption which has been used in many applications [28,29]. To make the MAR assumption plausible, Little and Rubin [2] recommended to include as many variables as possible related to the missingness and the response(s), as long as it is computationally feasible, in the imputation model. In the context of surveys, variables included in the imputation model should contain both the typical survey questions (i.e. Y) and survey design variables (i.e. Z). For the latter, we also include sampling weights as a proxy of the survey design information.

To incorporate a large number of variables with different distributional forms and various amounts of missing values, in general, sequential regression multiple imputation (SRMI) or MICE, which stands for multiple imputation via chained equations, is currently a preferred MI strategy to handle missing data problems in large-scale complex surveys [17,30]. In SRMI, each missing variable is imputed based on a model predicting the variable using other variables in the dataset as covariates. This algorithm is run through multiple iterations, and in each iteration the missing values of each variable are updated by sequentially imputing the values across all the variables. To describe the idea briefly, let Ymiss contain two variables Y1 and Y2. Using SRMI, Y1 and Y2 are imputed iteratively from two predictive distributions fY1∣Y2,Yobs,Z and fY2∣Y1,Yobs,Z. These predictive distributions (models) can be freely specified by the imputer, but they typically follow established customs in statistical modelling. For instance, if Y1 is continuous, then a natural choice for fY1∣Y2,Yobs,Z is a linear regression model using Y2, Yobs, and Z to predict Y1. If Y2 is binary, then a simple option for fY2∣Y1,Yobs,Z is a logistic regression model using Y1, Yobs, and Z to predict Y2. In addition, the specified models are expected to fit the real data well. For instance, transformations or adding interactions might be necessary. Multiple modelling options are provided in existing SRMI software packages.

However, missing data in skip-pattern variables might draw further attention to the use of SRMI, i.e. additional imputation procedures may be needed to impute skip-pattern variables. Using the aforementioned example, suppose that Y2 is only applicable when Y1 has certain values (e.g. Y1 is the ever smoked question with ‘yes’ and ‘no’ as responses; and Y2 is the current smoking question which is only applicable when Y1=‘yes’). The two approaches mentioned earlier, IAAC and IWRNC, correspond to imputing Y2 from either fY2∣Y1=‘yes’,Yobs,Z) (i.e. imputing Y2 on a subset of subjects who are applicable for Y2 after imputing Y1 in a separate imputation procedure) or fY2∣Y1,Yobs,Z (i.e. imputing Y2 on data from all subjects without utilizing a separate imputation procedure). Though both approaches have been applied in practice, the performance of these two approaches deserves further evaluation. In Sections 3 and 4, we compare these two approaches via simulation studies.

Simulation 1- MI with a categorical skip-pattern covariateSetup of simulation 1

To mimic the motivational example, the RANDS 1 and 2 data, we generate a sampling frame with 500,000 observations as a finite target population with eight variables (the outcome variable Y1, four covariates X1-X4, and three survey design variables Z1-Z3 which form 50 sampling strata). The variables are generated as follows:

X1∼N(0,1);

X2 is a binary variable, with PX2=1=0.2 when X1≥0; and PX2=1=0.5 when X1<0;

X3 is a skip-pattern variable, with X3∼Binomial⁡(0.4) when X2=1; and X3 is skipped when X2=0.

X4∼Multinomial⁡(0.2, 0.6, 0.2).

Z1∼ Binomial (0.5),

Z2∼ Multinomial 0.2, 0.2, 0.2, 0.2, 0.2,

Z3∼Multinomial⁡(0.15, 0.2, 0.15, 0.25, 0.25).

Y1 is an ordinal outcome variable ranking from 1 to 10 since the income-category variable in RANDS is an ordinal variable (Section 5). To generate Y1, we first generate a latent variable Y1_A from a normal distribution, Y1_A∼N(f(X1−X4,Z1−Z3),1), where fX1-X4,Z1-Z3=5+2*X1+2*IX2=1+5*IX3=1+0.5*IX4=2+0.5*Z2-0.5*Z3, and I() denotes the indicator function for the event in the parenthesis. The coefficient for Z1 is set to zero. From Y1_A, we create Y1 based on the deciles of Y1_A, i.e. Y1=1 if Y1_A<=10th percentile, etc.

We select 100 samples independently using stratified Bernoulli sampling based on Z1–Z3, with an average sample size of 5,300. For each sample, we generate missing data in Y1 (42.5% missingness) and X4 (12% missingness) assuming missing at random (MAR), and in X2 and X3 assuming missing completely at random (MCAR). Two levels of missing data in X2 and X3 are generated, low level missingness (5% missing in X2, and 5% missing in X3 when X2 is observed and equal to 1) and high level missingness (20% missing in X2, and 20% missing in X3 when X2 is observed and equal to 1) to study the impact of the amount of missingness on the imputation results. Using the definition from Section 2, Ymiss=Y1,X2,X3,X4, Yobs=X1, Z=Z1-Z3, and the sampling weight for each subject in the sample). Details of the sample selection and missing data generation can be found in Appendix A.

Two approaches are used to impute missing data in the skip-pattern covariate X3 using the SAS PROC MI procedure [21]. The first approach, IAAC, imputes skip-pattern covariate X3 only among subjects who are applicable for this question, i.e. among those with X2=1. The imputed X3 will then be used as a covariate to impute missing data in other variables. For those with X2=0(X3 is skipped), X3 can be set as 0 or as a different category (such as ‘NA’) so that X3 is complete (i.e. no missing values in X3) when imputing missing data in other variables. In this paper, we set it as 0 which is consistent with the data generating process of Y1 described above. As a result, we expect this method performs well for the simulation study. The procedure is as follows,

impute X2,X4 and Y1 from their posterior predictive distributions fX2∣Y1,X1,X3,X4,Z,fX4∣Y1,X1,X2,X3,Z,fY1∣X1,X2,X3,X4,Z, respectively, i.e. conditioning on variable(s) without missing values and variables with missing values imputed from the previous iteration.

impute X3 among those with X2=1 from fX3∣Y1,X1,X2=1,X4,Z. After imputation, set X3=0 for those with X2=0.

Iterate (i)–(ii) until convergence (e.g. means of Y1 become stable). In each step, the variable to be imputed and the remaining variables with their missing values (if any) imputed from previous steps in the most recent iteration are included in a SAS PROC MI procedure.

The second approach, IWRNC, imputes skip-pattern covariates X3 among all subjects. Specifically, we first recode X3 among the non-applicable subjects (i.e. those with X2=0) and then treat the skip-pattern variable the same as the other variables, i.e. include the recoded X3 with the remaining variables Y1,X1,X2,X4,Z in one SAS PROC MI procedure. The missing data for X3 is imputed from fX3∣Y1,X1,X2,X4,Z among all subjects. Unlike IAAC where each imputation step calls a separate SAS PROC MI procedure, IWRNC requires only one SAS PROC MI procedure and thus does not require extra programming for the skip pattern variable(s). We use four recoding methods to recode X3 for the non-applicable cases as follows,

Method 1: recode X3 as missing values when X2 = 0 (i.e. after recoding, X3 is 1, 0, or missing).

Method 2: recode X3 as 0 when X2 = 0 (i.e. after recoding, X3 is 1, 0, or missing).

Method 3: recode X3 as 1 when X2 = 0 (i.e. after recoding, X3 is 1, 0, or missing).

Method 4: recode X3 as ‘NA’ when X2 = 0 (i.e. after recoding, X3 is 1, 0, ‘NA’, or missing).

For methods 1–3, missing values in X3 will be imputed as 0 or 1. While for method 4, the missing values in X3 will be imputed as 0, 1, or ‘NA’ since ‘NA’ is treated as a separate category. As a result, there may be some inconsistencies since missing values in X3 can be imputed as ‘NA’ when X2 = 1, where only 0 or 1 are ‘correct’ answers.

For both approaches, we use a cumulative logit model to impute Y1 and discriminant analysis models to impute X2, X3 and X4. We conduct 10 imputations for each replicate. Although Y1 is treated as an ordinal variable in MI, we derive marginal and conditional means of Y1 given X2, X3 and X4 treating Y1 as a continuous variable for easy presentation and interpretation. All analyses included the survey design features (strata) and were weighted by sampling weights. For each replicate, results of 10 multiply imputed datasets are summarized using the SAS PROCMIANALYZE procedure. We calculate bias (the average of the deviation of estimates from the true value over the 100 replications), empirical standard error (SE; the standard deviation of the estimates over the 100 replications) and root mean square error (RMSE; the square root of the mean of the squared deviation of the estimates over 100 replicates). Results are shown in Tables 1 and 2.

Simulation results

Table 1 shows biases, empirical SEs and RMSEs of the marginal and conditional means of Y1 when missing percentages in X2 and X3 are high (20% missing in X2 and 20% missingness in X3 when X2=1 and observed). The before-deletion analysis (BD), i.e. the original 100 replicates without missing values, yields estimates close to the target population with biases ranging from −0.02 to 0.00, RMSEs ranging from 0.04 to 0.13, and empirical SEs also ranging from 0.04 to 0.13. The complete-case (CC) analysis, where the generated missing values were removed from the analysis, yields estimates with large biases and RMSEs, with biases ranging from 0.45 to 0.95, RMSEs ranging from 0.45 to 0.96, and empirical SEs ranging from 0.04 to 0.18. IAAC, where the skip-pattern feature is considered during the imputation process, yields estimates close to the BD analysis with biases ranging from −0.09 to 0.10, RMSEs ranging from 0.06 to 0.16, and empirical SEs ranging from 0.05 to 0.14. For IWRNC, setting the not applicable cases as missing values (recoding method 1) yields estimates close to the BD analysis, with biases ranging from −0.04 to 0.04, RMSEs ranging from 0.06 to 0.14, and empirical SEs ranging from 0.05 to 0.13. Recoding method 2, which sets the not applicable cases as 0 and is consistent with the data generating model for Y1, yields estimates close to the BD analysis and recoding method 1, with biases ranging from −0.08 to 0.11, RMSEs ranging from 0.07 to 0.16, and empirical SEs ranging from 0.05 to 0.14. Recoding methods 3 and 4, which set not applicable cases as 1 and ‘NA’, respectively, yield relatively larger biases compared to the recoding methods 1 and 2 for conditional mean estimates given X2 and X3. For example, the biases for conditional mean of Y1 given X3=0 are −0.17 and 0.21 respectively for methods 3 and 4, compared with biases of 0.03 and −0.01 for methods 1 and 2, respectively. For the remaining estimates, marginal and conditional means given X4, methods 3 and 4 yield estimates similar to those of recoding methods 1 and 2.

When missingness percentages in X2 and X3 are low, both IAAC and IWRNC yield estimates with small biases and RMSEs (Table 2). Different recoding methods for IWRNC yield results close to each other.

Simulation 2- multiple imputation with a continuous skip-pattern covariate

In Simulation 2, we use the same setup as in Simulation 1 while assuming a continuous skip-pattern variable X3, with X3∼N(3,1). X3 is applicable when X2=1. To generate Y1, we first generate a latent variable Y1_A from a normal distribution as follows, Y1_A∼N(f(X1−X4,Z1−Z3),1), where fX1-X4,Z1-Z3=5+2*X1+2*IX2=1+2*IX2=1*X3+0.5*IX4=2+0.5*Z2-0.5*Z3. The coefficient for Z1 is set to zero. From Y1_A, we create an ordinal variable Y1 ranking from 1 to 10 based on deciles of Y1_A. We select 100 samples and generate missing values using the same methods as in Simulation 1. We apply the two approaches, IAAC and IWRNC, to conduct multiple imputation, missing values in X3 are imputed using a linear regression model. Since the skip-pattern variable is continuous, for IWRNC, we recode X3 among the non-applicable subjects in three ways: as missing, as 0, or as mean of X3 (i.e. 3). We derive the marginal mean of Y1 and the conditional means of Y1∣X2 and Y1∣X4, and regression of Y1 given X3. The results are shown in Tables 3 and 4.

When the missingness percentages in X2 and X3 are high, the CC analysis yield results with large biases (from −0.19 to 0.89) and RMSEs (from 0.20 to 0.89) (Table 3). IAAC, which imputes missing values in X3 only among the applicable subjects, yields estimates with small biases (from −0.07 to 0.08) and RMSEs (from 0.05 to 0.11). For IWRNC, all three recoding methods yield larger biases for the regression of Y1 given X3(0.29,-0.16, and −0.14 respectively for methods 1 to 3), although biases for the marginal mean of Y1 are small. Recoding the non-applicable cases as being missing also leads to larger biases for the conditional mean of Y1 given X2=0 (bias =0.16. When the missingness percentages in X2 and X3 are low, IAAC and IWRNC where X3 was recoded as 0 or as mean of X3 yield estimates with small biases and RMSEs (Table 4). IWRNC where X3 was recoded as missing yield relatively large biases for the conditional mean of Y1 given X2=0(bias=0.18) and regression of Y1 given X3 (bias =0.25).

Summary of the simulation studies

For both simulations, IAAC, which imputes missing skip-pattern variables only among the applicable subjects, yields estimates with small biases and RMSEs for both low and high levels of missing data and for both categorical and continuous skip-pattern variables. However, IWRNC yields small biases regardless of recoding methods when the skip-pattern variable X3 is categorical with a low percentage of missing data (5%); when the categorical skip-pattern variable X3 is missing for 20% of the values, setting the skip-pattern variable as missing or as 0 (the group which has a mean of Y1 closer to the non-applicable cases) yields estimates with small biases for the conditional mean of Y1 given X3=0, but not for the other recoding methods. On the other hand, for IWRNC, when the skip-pattern variable X3 is continuous, even when the missing percentage is low, simply setting X3 as missing doesn’t reduce the biases of complete-case analysis, though setting X3 as 0 or as the mean of X3 yields estimates with small biases and RMSEs; when the missing percentage is high, IWRNC yields large biases for regression of Y1 given X3 for all three recoding methods.

For IWRNC, Y is imputed across all cases conditioning on the covariates, which is consistent with the data-generating process; however, X3 is also imputed across all cases for the convenience of implementation, i.e. not limited to cases with X2=1. Thus, the imputation of X3 does not align with the actual data-generating process of X3. For both simulations, the correct imputation of Y1 based on a cumulative logistic regression model can be written as logit(P(Y1≤j∣X2,X3))=αj+β2×X2+β3×I(X2=1)×X3, where j=1,2,…,J-1, and J is the total number of categories of the ordinal variable Y1,I() is an indicator function with a value of 1 if the event in the parentheses is true. The remaining covariates are not shown for a simple illustration. While for IWRNC, the imputation of Y1 based on a cumulative logistic regression model is written as logit(P(Y1≤j∣X2,X3))=aj+b2×X2+b3×X3, where the indicator term IX2=1 is not included. The exclusion of the indicator term impacts the imputation of Y1. The recoding method on X3 has an essential impact on the imputation of X3, which in turn impacts the imputation of Y1. For example, when X2=0, Y1 is imputed from logit(P(Y1≤j∣X2=0,X3))=αj from the correct imputation model. However, in Simulation 1, when X2=0, Y1 is imputed from logit(P(Y1≤j∣X2=0,X3))=aj+b3×P(X3=1∣X2=0). Compared to the correct imputation model of Y1, there is an additional term b3×PX3=1∣X2=0, i.e. the imputation of Y1 depends on the proportion of subjects with X3=1 among those with X2=0. When PX3=1∣X2=0 is small, logit(P(Y1≤j|X2=0,X3))) would be close to aj. The smaller PX3=1∣X2=0, the closer the imputation model is to the correct imputation model. In Simulation 1, for the recoding method which sets X3 as 0 where X2 is observed and equal to 0, the imputation of X3 depends on both Y1 and X2. When X2 is 0, missing values in X3 are more likely to be imputed as 0 for two reasons: (1) X3 is set as 0 for all cases with X2=0 and observed; (2) distribution of Y1 for subjects with X2=0 is closer to that of subjects with X2=1 and X3=0 than subjects with X2=1 and X3=1. As a result, PX3=1∣X2=0 is much smaller than PX3=0∣X2=0, and the imputation model is close to the actual imputation model of Y1 and yields imputation results with small biases. The recoding methods which align with the actual data-generating process of Y1 and X3 result in the imputation models of Y1 being closer to the correct imputation model of Y1. Appendix B describes the imputation of Y1 and X3 from IWRNC for each recoding method in both simulation studies and discusses why some recoding methods yield results with smaller biases.

In summary, IAAC yields estimates with small biases and RMSEs for both categorical and continuous skip-pattern covariates, while IWRNC is more useful for categorical skip-pattern covariates with two recoding methods, i.e. recode the non-applicable cases to missing or to a certain category of the variable (such as natural groupings or to the category with similar means of the response variable).

Multiple imputation of missing income in RANDS with skip-pattern covariatesRANDS data description

RANDS is a series of cross-sectional and longitudinal online surveys from commercial survey panels, conducted by NCHS since 2015 (https://www.cdc.gov/nchs/rands/). RANDS 1 and 2 were conducted by Gallup in 2015 and 2016, respectively, using the probability-sampled Gallup Panel (https://www.gallup.com/174158/gallup-panel-methodology.aspx). The specific topics in RANDS 1 and 2 include access to healthcare and utilization, chronic conditions, food security, general health, health insurance, opioid use, physical activity, and psychological distress [10].

One important panel variable, household income, has around 21% missing values. In this section, we use multiple imputation to address missing household income in RANDS 1 and 2. Household income values range from 0 to 8, with 0 corresponding to under $15,000 household income and 8 corresponding to $200,000 or more household income. Around 60 variables related to income or the missingness of income are included in the imputation model (Table A1 in Appendix C) and missing values in these covariates are imputed during the imputation process. Most of the covariates are applicable to all survey participants, except several skip-pattern variables on smoking, health insurance, and working status which are only applicable to a subset of survey participants (Figure 1). For example, for the health insurance questions, the prior question is ‘are you covered by any kind of health insurance or some other kind of health care plan?’, for those who answered ‘yes’, a series of skip-pattern questions on specific health care insurance questions are asked, such as ‘are you covered by Private Health Insurance?’, ‘are you covered by Medicare?’, etc. Similar to non-skip-pattern variables, the skip-pattern variables may contain missing values. When the prior questions are missing, the skip-pattern question responses are missing since the participants won’t be asked the skip-pattern questions; in addition, when responses to the prior questions are not missing, survey participants may choose not to answer skip-pattern questions. In RANDS 1 and 2, the overall missing percentages in the skip-pattern variables included in the MI model are low (less than 5%). The responses for reasons the respondent was not working were grouped into two categories due to the small sample sizes of subjects within each response category and subjects with a similar mean household income were grouped together (i.e. 1 = Taking care of house or family / Retired / On a planned vacation from work/On family or maternity leave / On layoff; 2 = Going to school/Temporarily unable to work for health reasons/Have job or contract and off-season/Disabled/Other). Response categories for the current smoking question (‘every day’, ‘some days’ or ‘not at all’) and health insurance question (‘yes’, ‘no’) remained unchanged.

Two approaches to address missing data issues in skip-pattern variables for multiple imputation

We use the SAS PROC MI procedure to conduct multiple imputation for missing income in RANDS 1 and 2. The fully conditional specification (FCS) option is used to run SRMI. A cumulative logit model is used to impute missing income, linear regression models are used to impute continuous variables, and logistic regression and discriminant analysis models are used to impute the remaining categorical variables. Two approaches are used to address the missing values in the skip-pattern covariates.

The first approach, IAAC, imputes skip-pattern covariates (current smoking, specific health insurance coverage, and reason for not working) only among subjects who are applicable for these questions. The procedure is as follows,

impute skip pattern variables only among the applicable cases

impute missing current smoking status only among subjects who answered ‘yes’ to the question ‘have you smoked at least 100 cigarettes in your entire life?’. After imputation, recode the current smoking question response as ‘not at all’ for those who had a ‘no’ response to the ‘ever smoked’ question.

impute specific health insurance coverages (Medicare, private health insurance, etc.) only among those who had health insurance. After imputation, recode the specific health insurance coverages as ‘no’ for those who answered ‘no’ to the question ‘are you covered by any kind of health insurance or some other kind of health care plan?’.

impute the reason why the respondent was not working only among those not currently working. After imputation, recode the reason for not working as ‘1 = Taking care of house or family / Retired / On a planned vacation from work/On family or maternity leave / On layoff’ for those who answered they were working last week since the means of income for the non-applicable subjects (those who were working last week) are closer to this group of subjects.

impute income along with the remaining variables among all subjects. The imputed skip pattern variables from (i) are included as predictors and any missing values in the covariates are imputed during this imputation process. Iterate (i) – (ii) until convergence.

The second approach, IWRNC, imputes skip-pattern covariates (current smoking, specific health insurance coverage, and reason for not working) among all subjects, while recoding the values for the skip-pattern covariates among the non-appliable cases before imputation. Four recoding methods are tested, as follows,

Method 1: set the values of the skip-pattern variables as missing among the non-applicable subjects.

Method 2: recode the values of the skip-pattern variables among the non-applicable subjects as a specific category of the applicable subjects, where the category is selected based on natural groupings or the group with a similar mean household income. Specifically, set the values of the current smoking question as ‘not at all’ for non-applicable subjects since those who answered ‘no’ to the ever smoked question would be expected to answer ‘not at all’ to the current smoking question if they were asked this question; set specific health insurance coverages as ‘no’ for those who answered ‘no’ to the overall health insurance question, and set the reason for not working as ‘1 = Taking care of house or family / Retired / On a planned vacation from work/On family or maternity leave / On layoff’ for the non-applicable subjects since the mean income for the non-applicable subjects is closer to the mean income for these applicable cases.

Method 3: recode the values of the skip-pattern variables among the non-applicable subjects as the ‘opposite’ category to what was chosen in Method 2. Specifically, set the values as ‘every day’ for the current smoking question, set the values as ‘yes’ for the specific health insurance coverages questions and set the values as ‘2 = Going to school/Temporarily unable to work for health reasons/Have job or contract and off-season/Disabled/Other’ for the reason for not working question.

Method 4: set the values of the skip-pattern variables as ‘NA’ among the non-applicable subjects, i.e. ‘NA’ is treated as a valid category of the variable, so the missing values will be imputed as the one of the possible values of the skip-pattern covariates or as ‘NA’, which may lead to inconsistencies of the skip-pattern variables. For example, the ever smoked question is answered or imputed as ‘yes’ and the current smoking question is imputed as ‘NA’, although ‘every day’, ‘some days’ or ‘not at all’ are correct answers.

We conduct 10 imputations for RANDS 1 and 2 data. Though income is imputed as an ordinal variable in MI, it is treated as a continuous variable in the subsequent analyses for easy presentation and interpretation. Table 5 shows the means of household income, overall and by skip-pattern covariates and their prior questions. In general, the two MI approaches yield similar results; and the four recoding methods for IWRNC yield results close to each other. In addition, the marginal and conditional means of household income after MI are close to those of the complete-case analysis, while slightly larger differences in conditional mean estimates are shown in the categories where the sample sizes are small. Since the missingness percentages in the skip-pattern covariates are low, it is expected that results based on RANDS data would be similar to those from Simulation 1 with a low percent of missingness in the categorical skip-pattern variables. Although we do not expect to see large differences among alternative methods in this analysis, the application of alternative options can be used as a sensitivity analysis to evaluate the final results in the real data analysis.

Discussion

To meet a growing demand for faster data collection, web-based panel surveys have becoming increasingly popular during the last decade or so [31,32]. However, there are some limitations for web-based surveys, including potential coverage bias and survey nonresponse. Multiple imputation is a widely used technique to address missing data issues due to nonresponse. To incorporate data with different distributional forms, the sequential regression method is one of the most popular methods to construct an imputation model. Many software packages have been developed for MI which greatly improve the application of multiple imputation in practice. However, no software packages can incorporate all possible data features, thus adjustments are needed when conducting MI using available software packages. Missingness in skip-pattern variables is one of such examples.

Skip-pattern variables are common in surveys. For MI models, when skip-pattern variables with missing data exist, extra care is needed. In this paper we explore two approaches to address missing data issues with skip-patten variables for MI. IAAC imputes the skip-pattern variables only among the applicable subjects, which requires additional programming thus increasing the complexity of imputation programmes. IWRNC recodes the skip-pattern covariate before imputation and treats the recoded variable the same as other variables. It is easy to implement especially for MI with high-dimensional data and multiple skip-pattern variables, such as in national surveys. Simulation studies show that when the skip pattern variable is categorical, it yields estimates with small biases and RMSEs with proper recoding methods; however, when the skip pattern variable is continuous and the missing percentage is high, it yields biased estimates for the regression of the response variable on the skip pattern covariate regardless of the recoding methods. Nonetheless, when constructing a multiple imputation model with multiple variables with different distributional forms, using IWRNC could reduce programming complexity and save computing time.

Though IWRNC has been used in many applications, no formal guidance exists on how or when this approach should be applied. In this research, we conducted simulation studies to provide some guidance. When the missingness percentages in the skip pattern variables are low, IWRNC can be applied for both continuous and categorical skip-pattern variables. Recoding the continuous variable as 0 works for continuous skip-pattern variables, which is equivalent to including the interaction term of the skip-pattern variable and their prior questions. When the missingness percentage is high, IWRNC can still be applied to categorical skip-pattern variables; however, none of recoding methods tested worked well for continuous skip-pattern covariates. In that case, IAAC is preferred for imputation.

We tested several different recoding methods for IWRNC. When the skip pattern variable is categorical, recoding the non-applicable cases to missing or to a certain category of the variable (such as the natural grouping or to the category with similar means of the response variable) yields results with small biases. Recoding non-applicable cases as ‘NA’ seems a likely choice, however, it could yield inconsistencies after imputation. For example, 6.1% subjects among the applicable cases X2=1 were imputed as ‘NA’ for Simulation 1 with a high percentage of missing values (results not shown); additional imputation would be needed to address these inconsistent cases. When the skip pattern variable is continuous, recoding the non-applicable cases to 0 or the mean seems to be fine when the missingness percentage is low, while simply setting non-applicable cases to missing leads to biased results.

Our study has some limitations. First, we generated the skip-pattern variable X3 as a binary variable in Simulation 1 and a continuous variable in Simulation 2. Other data types, such as count or truncated data, were not investigated. Second, we simulated one level of skip-pattern variables. In practice, it is common to have multiple-level skip-pattern variables. Applying IWRNC for multiple-level skip-pattern variables is complicated and requires thorough analysis to choose the proper recoding methods for each level of skip-pattern variables. Third, we generated 5% and 20% missing data in the skip-pattern variable X3. The missing data percentages in skip-pattern variables could be much higher in many applications. In this case, the performance of IWRNC deserved further evaluation, especially for the method which set the categorical skip-pattern variable as missing values.

In summary, both IAAC and IWRNC are useful strategies for addressing missing data issues with skip-pattern variables. IAAC may require some extra programming, especially for large complex survey data with many skip-pattern variables. IWRNC saves programming and imputing time, however, extra effort is needed to investigate the data, to evaluate different recoding methods, and to check for inconsistencies. For data with multiple skip-pattern variables with various amounts of missing data, combining IAAC and IWRNC for different skip-pattern variables may be necessary to construct a MI model.

Disclosure statement

No potential conflict of interest was reported by the author(s).

AppendicesDetails of simulation 1 – sample selection and missing data generationSampling from the target population using a stratified Bernoulli sampling method

From the target population generated in Section 3.1, we conduct stratified Bernoulli sampling to select samples. The sampling probability (SP) is based on a logistic regression model, as follows, SP=exp(str∗β)/(1+exp(str∗β)), where str is the stratum number ranging from 1 to 50, and β=-4,-3,-1,-0.5,-0.3,-0.2,-0.15 and −0.1 for stratum numbers 1, 2, 3 to 5, 6–10, 11–20, 21–30, 31–40, and 41–50 respectively. For example, for stratum 10, the sampling probability is exp⁡(-0.5*10)/(1+exp⁡(-0.5*10))=0.006693. The sampling weights are the inverse of SP. We repeated the sampling procedure 100 times to create 100 samples. On average, the sample size in each sample is around 5,300.

Generating missing data

In each sample, we generate missing data in Y1 and X4 assuming missing at random (MAR) and in X2 and X3 assuming missing completely at random (MCAR). In addition, we generate two levels of missing data in X2 and X3, low level missingness (5% missing in X2, and 5% missing in X3 when X2 is observed and equal to 1) and high level missingness (20% missing in X2, and 20% missing in X3 when X2 is observed and equal to 1) to study the impact of the amount of missingness on the imputation results. The propensity models for generating missing data are described below: P(Y1missing)=1/(1+exp(0.5+2∗X1)); P(X2missing)=20%(high-level missingness)or 5%(low-level missingness); P(X3missing|X2=1)=20%(high-level missingness)or 5%(low-level missingness); P(X4missing)=1/(1+exp(2−0.2∗X1)).

On average, around 42.5% of Y1 and 12% of X4 are missing. The missingness in X3 is due to: (1) X2 is missing; (2) when X2 is observed and equal to 1,20% or 5% of X3 is missing. When X2 is observed and equal to 0,X3 is skipped.

Imputation of Y<sub>1</sub> and X<sub>3</sub> from IWRNCImputation of Y<sub>1</sub> and binary X<sub>3</sub> from IWRNC in simulation 1

In Simulation 1, the correct imputation of Y1 based on a cumulative logistic regression model can be written as logit(P(Y1≤j∣X2,X3))=αj+β2×X2+β3×I(X2=1)×X3, where j=1,2,…,J-1, and J is the total number of categories of the ordinal variable Y1, I() is an indicator function with a value of 1 if the event in the parentheses is true. The remaining covariates are not shown for a simple illustration. The combination of X2 and X3 forms three groups (group 1 X2=1,X3=1; group 2 X2=1,X3=0; group 3 (X2=0,X3 is skipped)). The imputation of Y1 across the three groups is as the following: logit(P(Y1≤j∣X2=1,X3=1))=αj+β2+β3; logit(P(Y1≤j∣X2=1,X3=0))=αj+β2; logit(P(Y1≤j∣X2=0))=αj. The correct imputation model of X3 given Y1 and X2 is: logit(P(X3=1∣Y1,X2))=γ0+γ1×I(X2=1)×Y1. When X2=0,X3 is not applicable.

For IWRNC, Y1 is imputed across all cases conditional on X2 and X3, which is consistent with the data-generating process; X3 is also imputed across all cases, i.e. not limited to cases with X2=1. When missing values in X2 are imputed as 0, missing values in X3 are imputed from the corresponding imputation models instead of assigning them as not applicable. Thus, the imputation of X3 by IWRNC does not align with the actual data-generating process of X3. The recoding methods which impute missing X3 closer to the actual distribution X3 would yield smaller biases. The following describes how Y1 and X3 are imputed for each recoding method using IWRNC.

Recoding methods which set X<sub>3</sub> as 0, as 1, or as missing when X<sub>2</sub> is observed and equal to 0

When setting X3 as 0, as 1, or as missing, X3 is still a binary variable with two levels (0,1), and the missing data in X3 are imputed as 0 or 1 from the corresponding imputation models. On the other hand, Y1 is imputed from a cumulative logistic regression model: logit(P(Y1≤j∣X2,X3))=aj+b2×X2+b3×X3. The imputation of Y1 across the three groups is as the following, logit(P(Y1≤j|X2=1,X3=1))=aj+b2+b3; logit(P(Y1≤j|X2=1,X3=0))=aj+b2; logit(P(Y1≤j|X2=0))=logit(P(Y1≤j|X2=0,X3=1)×P(X3=1|X2=0)+logit(P(Y1≤j|X2=0,X3=0)×P(X3=0∣X2=0)=aj+b3×P(X3=1|X2=0). Compared to the actual imputation model of Y1, there is an additional term b3×PX3=1∣X2=0, i.e. the imputation of Y1 for group 3 depends on the proportion of subjects with X3=1. When PX3=1∣X2=0 is small, logit PY1≤j∣X2=0 would be close to aj. The smaller PX3=1∣X2=0, the closer the imputation model is to the actual imputation model. The recoding method of X3 has an essential impact on the imputation of X3, which in turn impacts the imputation of Y1.

For the recoding method which sets X3 as 0 when X2 is observed and equal to 0, the imputation of X3 depends on both Y1 and X2. When X2 is 0, missing values in X3 are more likely to be imputed as 0 for two reasons: (1) X3 was set as 0 for all cases with X2=0 and observed; (2) distribution of Y1 in group 3 X2=0 is closer to that in group 2 X2=1,X3=0 than in group 1 X2=1,X3=1. As a result, PX3=1∣X2=0 is much smaller than PX3=0∣X2=0, and the imputation model is close to the actual imputation model of Y1 and yields imputation results with small biases.

For the recoding method which sets X3 as 1 when X2 is observed and equal to 0, missing values in X3 are more likely to be imputed as 1 when X2=0 since X3 is set as 1 for all observed cases with X2=0; thus, after imputation, PX3=1∣X2=0 is much larger than PX3=0∣X2=0. Thus logit⁡PY1≤j∣X2=0 is not close to the actual imputation model of Y1 and yields biased results.

When X2 is observed and equal to 0, the recoding method which sets X3 as missing creates additional missing data in X3 in addition to the original two sources of missingness in X3 (X3 is missing when X2 is observed and equal to 1, and X3 is missing when X2 is missing). In this case, the imputation of missing X3 depends mainly on Y1 since X3 is ‘observed’ only among subjects with X2=1. Among subjects with X2=0, missing values in X3 are more likely to be imputed as 0 since the distribution of Y in group 3 X2=0 is closer to that in group 2 X2=1,X3=0 than in group 1 X2=1,X3=1. Thus, after imputation, among group 3, PX3=1∣X2=0 is smaller than PX3=0∣X2=0. In our simulation study, with a high percentage of missing data in X2 and X3 (20% missing data in X2 and an additional 20% missing data in X3 when X2 is observed), the imputation model of Y1 yields small biases. With a higher percentage of missing values, the impact of PX3=1∣X2=0 may not be negligible.

Recoding method which sets X<sub>3</sub> = NA when X<sub>2</sub> is observed and equal to 0

Setting X3 as NA creates several different groups since X3 is treated as a categorical variable with three levels (1, 0, and NA). Though missing values in X3 are more likely to be imputed as NA when X2=0, missing values in X3 can also be imputed as NA when X2 is 1, leading to an extra group X2=1, X3=NA. In addition, X3 would also be imputed as 0 or 1 when X2=0. Thus Y1 is imputed from logitPY1≤j∣X2,X3=ej+e2×X2+e3×IX3=1+e4×IX3=0, as the following, logit(P(Y1≤j|X2=1,X3=1))=ej+e2+e3; logit(P(Y1≤j|X2=1,X3=0))=ej+e2+e4; logit(P(Y1≤j|X2=1,X3=NA))=ej+e2; logit(P(Y1≤j|X2=0))=logit(P(Y1≤j|X2=0,X3=1))×P(X3=1(X2=0))+logit(P(Y1≤j∣X2=0,X3=0))×P(X3=0∣X2=0)+logit(P(Y1≤j∣X2=0,X3=NA))×P(X3=NA∣X2=0)=ej+e3×P(X3=1∣X2=0)+e4×P(X3=0∣X2=0)

The imputation of Y1 is not close to the actual imputation model of Y1 given the covariates and yields biased results.

Imputation of Y<sub>1</sub> and continuous X<sub>3</sub> from IWRNC in simulation 2

For continuous X3, the correct imputation of Y1 given the covariates can also be written as logit (PY1≤j∣X2,X3=αj+β2×X2+β3×IX2=1×X3, which imputes Y1 as the following: logit(P(Y1≤j∣X2=1,X3))=αj+β2+β3×X3, logit(P(Y1≤j∣X2=0,X3))=αj. The correct imputation of X3 given the covariates is logit PX3=1∣Y1,X2=γ0+γ1×IX2=1×Y1. When X2=0, X3 is not applicable.

For IWRNC, Y1 is imputed from logit(P(Y≤j∣X2,X3))=aj+b2×X2+b3×X3, where logit(P(Y1≤j∣X2=1,X3))=aj+b2+b3×X3, and logit(P(Y1≤j∣X2=0,X3))=aj+(b3×X3)×P(X3≠0∣X2=0) Setting X3 as missing does not align with the actual imputation model of Y1, as X3 is imputed as a normally distributed continuous variable and PX3≠0∣X2=0 is not negligible; thus, it yields biased imputation results. On the other hand, setting X3 as 0 when X2=0 seems to align with the actual imputation model of Y1; however, it violates the normality assumption of X3, which impact the imputation of X3. Similarly, setting X3 as the mean of X3 also violates the normality assumption of X3. When the missing percentage is low (5% missingness; Simulation 2), the impact of imputation on X3 is small, and both recoding methods yield small biases; however, when the missing percentage is high (20% missingness; Simulation 2), both recoding methods yield biased results.

Variables included in the multiple imputation model of household income, RANDS 1 and 2

Variables included in the multiple imputation model of household income, RANDS 1 and 2.

Variable descriptionCategories or range
1Respondent’s Household Income0 = Under $15,0001 = $15,000 to $24,9992 = $25,000 to $34,9993 = $35,000 to $49,9994 = $50,000 to $74,9995 = $75,000 to $99,9996 = $100,000 to $149,9997 = $150,000 to $199,9998 = $200,000 or more
2Would you say your health in general is excellent, very good, good, fair, or poor?ExcellentVery goodGoodFairPoor
3In the last 30 days, I worried whether my food would run out before I got money to buy more.Often trueSometimes trueNever true
4In the last 30 days, the food that I bought just didn’t last, and I didn’t have money to get more.Often trueSometimes trueNever true
5In the last 30 days, I couldn’t afford to eat balanced meals.Often trueSometimes trueNever true
6In the last 30 days, did you ever cut the size of your meals or skip meals because there wasn’t enough money for food?YesNo
7In the last 30 days, did you ever eat less than you felt you should because there wasn’t enough money for food?YesNo
8In the last 30 days, were you ever hungry but didn’t eat because there wasn’t enough money for food?YesNo
9In the last 30 days, did you lose weight because there wasn’t enough money for food?YesNo
10During the past 12 months, did you receive care from doctors or other health care professionals 10 or more times? Do not include telephone calls.YesNo
11Are you covered by any kind of health insurance or some other kind of health care plan?YesNo
12Do you have private health insurance?YesNoSkip
13Do you have Medicare?YesNoSkip
14Do you have Medicaid?YesNoSkip
15Do you have a state-sponsored health plan?YesNoSkip
16Do you have an other government program?YesNoSkip
17Do you have a single service plan (e.g. dental, vision, prescriptions)?YesNoSkip
18Which of the following were you doing last week?Working for pay at a job or businessWith a job or business but not at workLooking for workWorking, but not for pay, at a family-owned job or businessNot working at a job or business and not looking for work
19What is the main reason you did not work last week?Taking care of house or familyGoing to schoolRetiredOn a planned vacation from workOn family or maternity leaveTemporarily unable to work for health reasonsHave job or contract and off-seasonOn layoffDisabledOtherSkip
20Have you ever been told by a doctor or other health professional that you had hypertension, also called high blood pressure?YesNo
21Have you ever been told by a doctor or other health professional that you had asthma?YesNo
22Other than during pregnancy, have you ever been told by a doctor or other health professional that you have diabetes or sugar diabetes?YesNoBorderline
23Have you ever been told by a doctor or other health professional that you had chronic bronchitis?YesNo
24Have you smoked at least 100 cigarettes in your entire life?YesNo
25How often do you now smoke cigarettes?EverydaySome DaysNot at AllSkip
26In any one year, have you had at least 12 drinks of any type of alcoholic beverage?YesNo
27Have you delayed healthcare in the past 12 months because you couldn’t get through on the telephone?YesNo
28Have you delayed healthcare in the past 12 months because you couldn’t get an appointment soon enough?YesNo
29Have you delayed healthcare in the past 12 months because once you get there you have to wait too long to see the doctor?YesNo
30Have you delayed healthcare in the past 12 months because the clinic or doctor’s office wasn’t open when you could get there?YesNo
31Have you delayed healthcare in the past 12 months because you didn’t have transportation?YesNo
32During the past 12 months, was there any time when you needed prescription medicines but didn’t get them because you couldn’t afford it?YesNo
33During the past 12 months, was there any time when you needed mental health care or counselling but didn’t get it because you couldn’t afford it?YesNo
34During the past 12 months, was there any time when you needed dental care (including checkups) but didn’t get it because you couldn’t afford it?YesNo
35During the past 12 months, was there any time when you needed eyeglasses but didn’t get them because you couldn’t afford it?YesNo
36During the past 12 months, was there any time when you needed to see a specialist but didn’t because you couldn’t afford it?YesNo
37During the past 12 months, was there any time when you needed follow up care but didn’t get it because you couldn’t afford it?YesNo
38During the past 12 months, have you ever used computers to look up health information on the internet?YesNo
39During the past 12 months, have you ever used computers to schedule an appointment with a health care provider?YesNo
40During the past 30 days, how often did you feel so sad that nothing could cheer you up?All of the TimeMost of the TimeSome of the TimeA Little of the TimeNone of the Time
41During the past 30 days, how often did you feel nervous?All of the TimeMost of the TimeSome of the TimeA Little of the TimeNone of the Time
42During the past 30 days, how often did you feel restless or fidgety?All of the TimeMost of the TimeSome of the TimeA Little of the TimeNone of the Time
43During the past 30 days, how often did you feel hopeless?All of the TimeMost of the TimeSome of the TimeA Little of the TimeNone of the Time
44During the past 30 days, how often did you feel that everything was an effort?All of the TimeMost of the TimeSome of the TimeA Little of the TimeNone of the Time
45During the past 30 days, how often did you feel worthless?All of the TimeMost of the TimeSome of the TimeA Little of the TimeNone of the Time
46How often do you feel worried, nervous, or anxious?DailyWeeklyMonthlyA Few Times a YearNever
47Do you take medication for these feelings?YesNo
48Respondent’s GenderMaleFemale
49Respondent’s Marital StatusSingleMarriedSeparatedDivorcedWidowedNever MarriedLiving with a Partner
50Respondent’s Employment StatusEmployed Full TimeEmployed part-time, but not a full-time studentA full-time studentRetiredHomemakerNot employed
51Respondent’s Renting StatusOwnRent
52Respondent’s Race and ethnicityWhiteOtherBlackAsianHispanic
53Respondent’s Educational AttainmentLess than High SchoolHigh School GradTechnical or Vocational SchoolSome CollegeFour Year Bachelor’s DegreeSome Postgraduate or Professional School or Degree
54BMI12–79
55Respondent’s Age18–95
56How often do you do light or moderate leisure time physical activities for at least 10 min that cause only light sweating or a slight to moderate increase in breathing or heart rate? times per year0–1460
57Respondent’s Sampling Weight (standardized)0.07–5.40
ReferencesHeY, ZhangG, HsuCH. Multiple imputation of missing data in practice: basic theory and analysis strategies. 1st ed. Chapman and Hall/CRC Press; 2021.LittleRJA, RubinDB. Statistical analysis with missing data. 2nd ed. New York: Wiley; 2014.RubinDB. Multiple imputations in sample surveys—a phenomenological Bayesian approach to nonresponse. In Proceedings of the survey research methods section of the American Statistical Association; 1978. p. 20–34.AßmannC, WürbachA, GoßmannS, Nonparametric multiple imputation for questionnaires with individual skip patterns and constraints: the case of income imputation in the national educational panel study. Sociol Methods Res. 2017;46(4):864–897. doi:10.1177/0049124115610346ArslanturkS, SiadatM, OgunyemiT, Skip pattern analysis for detection of undetermined and inconsistent data. In: 2012 5th international conference on biomedical engineering and informatics; 2012. p. 1122–1126. doi:10.1109/BMEI.2012.6513221LinTH. Estimating latent class model parameters for filter questions with skip patterns. Qual Quant. 2012;46:545–552. doi:10.1007/s11135-010-9385-xManskiCF, MolinariF. Skip sequencing: a decision problem in questionnaire design. Ann Appl Stat. 2008;2(1):264–285. doi:10.1214/07-AOAS13420419066 IrimataKE, HeY, CaiB, Comparison of quarterly and yearly calibration data for propensity score adjusted web survey estimates. Surv Methods Insights Field. 2020. Special issue ‘Advancements in Online and Mobile Survey Methods’. doi:10.13094/SMIF-2020-00018externaliconParkerJ, MillerK, HeY, Overview and initial results of the national center for health statistics’ research and development survey. Stat J IAOS. 2020;36(4):1199–1211. doi:10.3233/SJI-20067835923778 HeY, CaiB, ShinH-C, The national center for health statistics’ 2015 and 2016 research and development surveys. National Center for Health Statistics. Vital Health Stat. 2020;1(64). https://www.cdc.gov/nchs/data/series/sr_01/sr01-64-508.pdfpdficon.HapfelmeierA, HothornT, UlmK. Recursive partitioning on incomplete data using surrogate decisions and multiple imputation. Comput Stat Data Anal. 2012;56:1552–1565. doi:10.1016/j.csda.2011.09.024DavidM, LittleRJA, SamuhelME, Alternative methods for CPS income imputation. J Am Stat Assoc. 1986;81(393):29–41. doi:10.1080/01621459.1986.10478235National Center for Health Statistics. multiple imputation of family income in 2020 national health interview survey: methods; 2021. September 2020. Available from https://ftp.cdc.gov/pub/Health_Statistics/NCHS/Dataset_Documentation/NHIS/2020/NHIS2020-imputation-techdoc-508.pdfSchenkerN, RaghunathanTE, ChiuP-L, Multiple imputation of missing income data in the national health interview survey. J Am Stat Assoc. 2006;101:924–933. doi:10.1198/016214505000001375ZarnochSJ, CordellHK, BetzCJ, Multiple imputation: an application to income nonresponse in the national survey on recreation and the environment. In: Research paper SRS-49. Asheville (NC): U.S. Department of Agriculture Forest Service, Southern Research Station; 2010. p. 15.WatsonN, StarickR. Evaluation of alternative income imputation methods for a longitudinal survey. J Off Stat. 2011;27:693–715.RaghunathanTE, LepkowskiJM, van HoewykJ, A multivariate technique for multiply imputing missing values using a sequence of regression models. Surv Methodol. 2001;27:85–95.RaghunathanTE, SolenbergerPW, Van HoewykJ. IV eware: imputation and variation estimation software user guide. Ann Arbor (MI): Institute for Social Research; 2002.HeY, ZaslavskyAM, HarringtonDP, Multiple imputation in a large-scale complex survey: a practical guide. Stat Methods Med Res. 2010;19:653–670. doi:10.1177/096228020810127319654173 DengY, ChangC, IdoMS, Multiple imputation for general missing data patterns in the presence of high-dimensional data. Sci Rep. 2016;6(6):21689), doi:10.1038/srep2168926868061 SAS Institute Inc. SAS/STAT 14.1 user’s guide Cary (NC): SAS Institute Inc; 2015.van BuurenS, Groothuis OudshoornK. Mice: multivariate imputation by chained equations in R. J Stat Softw. 2011;45:1–67.MengXL. Multiple-imputation inferences with uncongenial sources of input: rejoinder. Stat Sci. 1994;9:566–573.RubinDB. Multiple imputation after 18+ years. J Am Stat Assoc. 1996;91(434):473–489. doi:10.1080/01621459.1996.10476908KimJK, BrickJM, FullerWA, On the bias of the multiple-imputation variance estimator in survey sampling. J R Stat Soc Ser B Stat Methodol. 2006;68(3):509–521. doi:10.1111/j.1467-9868.2006.00546.x Available from: http://www.jstor.org/stable/3879288.ZhouH, ElliottMR, RaghunathanTE. A two-step semiparametric method to accommodate sampling weights in multiple imputation. Biometrics. 2016;72:242–252. doi:10.1111/biom.1241326393409 QuartagnoM, CarpenterJR, GoldsteinH. Multiple imputation with survey weights: a multilevel approach. J Surv Stat Methodol. 2020;8(5):965–989. doi:10.1093/jssam/smz036SterneJA, WhiteIR, CarlinJB, Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls. Br Med J. 2009;338:b2393. doi:10.1136/bmj.b2393.19564179 Hayati RezvanP, LeeKJ, SimpsonJA. The rise of multiple imputation: a review of the reporting and implementation of the method in medical research. BMC Med Res Methodol. 2015 Apr 7;15:30. doi:10.1186/s12874-015-0022-1.25880850 van BuurenS. Multiple imputation of discrete and continuous data by fully conditional specification. Stat Methods Med Res. 2007;16:219–242. doi:10.1177/096228020607446317621469 TourangeauR, FrederickGC, CouperMP. The science of web surveys. Oxford: Oxford University Press; 2013.BakerR, BrickJM, BatesNA, Summary report of AAPOR task force on non-probability sampling. J Surv Stat Methodol. 2013;1:90–143. doi:10.1093/jssam/smt008

Skip-pattern questions included in the multiple imputation model, RANDS 1 and 2. (1) Smoking related questions; (2) Health insurance related questions; (3) Working status related question.

Biases, empirical standard errors (SE), and root mean square errors (RMSE) of marginal and conditional means of Y1 by imputation method: results of Simulation 1 when missing percentages in the categorical skip-pattern covariates are high (20%).

EstimandsMeasurementBefore-Deletion AnalysisComplete-Case AnalysisIAAC (Imputation among applicable cases)IWRNC (Imputation with recoded non-applicable cases)
Method 1 (set as missing)Method 2 (recode as 0)Method 3 (recode as 1)Method 4 (recode as NA)
Marginal mean of Y1Bias  0.00  0.95−0.06−0.03−0.06−0.06−0.08
SE  0.05  0.07  0.05  0.05  0.06  0.05  0.06
RMSE  0.05  0.96  0.08  0.06  0.08  0.08  0.10
Conditional means
mean Y1 |X2 = 0Bias  0.00  0.86−0.09−0.04−0.04  0.09−0.06
SE  0.05  0.09  0.05  0.05  0.05  0.05  0.05
RMSE  0.05  0.87  0.11  0.07  0.07  0.11  0.08
mean Y1 |X2 = 1Bias−0.01  0.62−0.01  0.01−0.02−0.14−0.01
SE  0.05  0.07  0.06  0.06  0.06  0.06  0.06
RMSE  0.05  0.62  0.06  0.06  0.07  0.15  0.06
mean Y1 |X3 = 0Bias−0.01  0.74  0.01  0.03−0.01−0.17  0.21
SE  0.06  0.08  0.07  0.07  0.07  0.07  0.07
RMSE  0.06  0.74  0.07  0.08  0.07  0.18  0.22
mean Y1 |X3 = 1Bias  0.00  0.45  0.10  0.04  0.11  0.13  0.13
SE  0.04  0.04  0.05  0.05  0.05  0.05  0.05
RMSE  0.04  0.45  0.11  0.06  0.12  0.14  0.13
mean Y1 |X4 = 1Bias−0.02  0.88−0.09−0.04−0.08−0.07−0.10
SE  0.13  0.18  0.14  0.13  0.14  0.14  0.14
RMSE  0.13  0.90  0.16  0.14  0.16  0.15  0.17
mean Y1 |X4 = 2Bias  0.00  0.94−0.05−0.03−0.05−0.05−0.08
SE  0.07  0.08  0.07  0.07  0.07  0.07  0.07
RMSE  0.07  0.94  0.09  0.07  0.09  0.09  0.11
mean Y1 |X4 = 3Bias  0.00  0.92−0.07−0.03−0.06−0.05−0.08
SE  0.10  0.14  0.11  0.11  0.11  0.11  0.11
RMSE  0.10  0.93  0.13  0.11  0.12  0.12  0.13

Bias: empirical bias, the average of the deviation of estimates from the true value over the 100 replications.

SE: empirical standard error, the standard deviation of the estimates over the 100 replications.

RMSE: root mean square error, the average of the square root of the squared deviation of the estimates from the true value over the 100 replications.

Biases, empirical standard errors (SE), and root mean square errors (RMSE) of marginal and conditional means of Y1 by imputation method: results of Simulation 1 when missing percentages in the categorical skip-pattern covariates are low (5%).

EstimandsMeasurementBefore-Deletion AnalysisComplete-Case AnalysisIAAC (Imputation among applicable cases)IWRNC (Imputation with recoded non-applicable cases)
Method 1 (set as missing)Method 2 (recode as 0)Method 3 (recode as 1)Method 4 (recode as NA)
Marginal mean of Y1Bias  0.00  0.95−0.02−0.01−0.02−0.01−0.02
SE  0.05  0.07  0.05  0.05  0.05  0.05  0.05
RMSE  0.05  0.96  0.06  0.05  0.06  0.05  0.06
Conditional means
mean Y1 |X2 = 0Bias  0.00  0.87−0.02−0.01−0.01  0.03−0.01
SE  0.05  0.09  0.05  0.05  0.05  0.05  0.05
RMSE  0.05  0.87  0.06  0.05  0.05  0.06  0.05
mean Y1 |X2 = 1Bias−0.01  0.62−0.01  0.00−0.01−0.04−0.01
SE  0.05  0.06  0.06  0.06  0.06  0.06  0.06
RMSE  0.05  0.62  0.06  0.06  0.06  0.07  0.06
mean Y1 |X3 = 0Bias−0.01  0.73−0.01  0.00−0.01−0.05  0.04
SE  0.06  0.07  0.07  0.07  0.07  0.07  0.07
RMSE  0.06  0.73  0.07  0.07  0.07  0.08  0.08
mean Y1 |X3 = 1Bias  0.00  0.44  0.02  0.01  0.02  0.03  0.03
SE  0.04  0.04  0.05  0.05  0.05  0.05  0.05
RMSE  0.04  0.45  0.05  0.05  0.05  0.05  0.06
mean Y1 |X4 = 1Bias−0.02  0.88−0.04−0.02−0.03−0.03−0.03
SE  0.13  0.18  0.14  0.13  0.13  0.13  0.13
RMSE  0.13  0.90  0.14  0.13  0.14  0.13  0.14
mean Y1 |X4 = 2Bias  0.00  0.94−0.01−0.01−0.01−0.01−0.02
SE  0.07  0.08  0.07  0.07  0.07  0.07  0.07
RMSE  0.07  0.94  0.07  0.07  0.07  0.07  0.08
mean Y1 |X4 = 3Bias  0.00  0.92−0.02  0.00−0.01−0.01−0.02
SE  0.10  0.14  0.11  0.11  0.11  0.11  0.11
RMSE  0.10  0.93  0.11  0.11  0.11  0.11  0.11

Bias: empirical bias, the average of the deviation of estimates from the true value over the 100 replications.

SE: empirical standard error, the standard deviation of the estimates over the 100 replications.

RMSE: root mean square error, the average of the square root of the squared deviation of the estimates from the true value over the 100 replications.

Biases, empirical standard errors (SE), and root mean square errors (RMSE) of marginal and conditional means of Y1 by imputation method: results of Simulation 2 when missing percentages in the continuous skip-pattern covariates are high (20%).

EstimandsMeasurementBefore-Deletion AnalysisComplete-Case AnalysisIAAC (Imputation among applicable cases)IWRNC (Imputation with recoded non-applicable cases)
Method 1 (set as missing)Method 2 (recode as 0)Method 3 (recode as mean)
Marginal mean of Y1Bias  0.00  0.89−0.01  0.02  0.00  0.01
SE  0.05  0.06  0.05  0.06  0.05  0.05
RMSE  0.05  0.89  0.05  0.06  0.05  0.05
Conditional means
mean Y1 |X2 = 0Bias  0.00  0.54−0.04  0.16  0.01  0.02
SE  0.04  0.05  0.04  0.05  0.04  0.04
RMSE  0.04  0.54  0.05  0.17  0.04  0.05
mean Y1 |X2 = 1Bias  0.00  0.54  0.08−0.02  0.00−0.02
SE  0.04  0.06  0.04  0.05  0.05  0.04
RMSE  0.04  0.54  0.09  0.05  0.05  0.05
mean Y1 |X3*Bias  0.00−0.19−0.07  0.29−0.16−0.14
SE  0.03  0.06  0.04  0.07  0.05  0.05
RMSE  0.03  0.20  0.08  0.30  0.17  0.15
mean Y1 |X4 = 1Bias  0.01  0.85−0.01  0.02  0.00  0.01
SE  0.11  0.14  0.11  0.11  0.11  0.11
RMSE  0.11  0.86  0.11  0.11  0.10  0.11
mean Y1 |X4 = 2Bias  0.00  0.88−0.01  0.02  0.00  0.01
SE  0.07  0.08  0.06  0.07  0.07  0.07
RMSE  0.07  0.88  0.06  0.07  0.06  0.07
mean Y1 |X4 = 3Bias  0.00  0.84−0.02  0.01−0.01  0.00
SE  0.11  0.15  0.11  0.11  0.11  0.11
RMSE  0.11  0.85  0.11  0.11  0.11  0.11

Bias: empirical bias, the average of the deviation of estimates from the true value over the 100 replications.

SE: empirical standard error, the standard deviation of the estimates over the 100 replications.

RMSE: root mean square error, the average of the square root of the squared deviation of the estimates from the true value over the 100 replications.

Mean of Y1 given X3 evaluated as the regression coefficient of Y1 given X3

Biases, empirical standard errors (SE), and root mean square errors (RMSE) of marginal and conditional means of Y1 by imputation method: results of Simulation 2 when missing percentages in the continuous skip-pattern covariates are low (5%).

EstimandsMeasurementBefore-Deletion AnalysisComplete-Case AnalysisIAAC (Imputation among applicable cases)IWRNC (Imputation with recoded non-applicable cases)
Method 1 (set as missing)Method 2 (recode as 0)Method 3 (recode as mean)
Marginal mean of Y1Bias  0.00  0.89  0.00  0.03  0.00  0.00
SE  0.05  0.06  0.05  0.06  0.05  0.05
RMSE  0.05  0.89  0.05  0.07  0.05  0.05
Conditional means
mean Y1 |X2 = 0Bias  0.00  0.53−0.01  0.18  0.00  0.00
SE  0.04  0.05  0.04  0.05  0.04  0.04
RMSE  0.04  0.54  0.04  0.19  0.04  0.04
mean Y1 |X2 = 1Bias  0.00  0.53  0.02−0.05  0.00−0.01
SE  0.04  0.05  0.04  0.05  0.04  0.04
RMSE  0.04  0.54  0.04  0.07  0.04  0.04
mean Y1 |X3*Bias  0.00−0.16−0.01  0.25−0.03−0.03
SE  0.03  0.06  0.04  0.08  0.04  0.04
RMSE  0.03  0.17  0.04  0.26  0.05  0.05
mean Y1 |X4 = 1Bias  0.01  0.85  0.00  0.04  0.00  0.01
SE  0.11  0.14  0.11  0.11  0.11  0.11
RMSE  0.11  0.86  0.11  0.11  0.11  0.11
mean Y1 |X4 = 2Bias  0.00  0.88  0.00  0.03  0.00  0.00
SE  0.07  0.08  0.06  0.07  0.06  0.06
RMSE  0.07  0.88  0.06  0.07  0.06  0.06
mean Y1 |X4 = 3Bias  0.00  0.84−0.01  0.03−0.01  0.00
SE  0.11  0.15  0.11  0.11  0.11  0.11
RMSE  0.11  0.85  0.11  0.12  0.11  0.11

Bias: empirical bias, the average of the deviation of estimates from the true value over the 100 replications.

SE: empirical standard error, the standard deviation of the estimates over the 100 replications.

RMSE: root mean square error, the average of the square root of the squared deviation of the estimates from the true value over the 100 replications.

Mean of Y1 given X3 evaluated as the regression coefficient of Y1 given X3

Mean and standard error estimates of household income from complete-case analysis and two imputation approaches, aRANDS 1 and 2 data.

VariablesCategoriesOriginal (Complete-Case Analysis)
IAAC (Imputation among applicable cases)
IWRNC (Imputation with recoded non-applicable cases)
bRecode Method 1
cRecode Method 2
d Recode Method 3
eRecode Method 4
fNMeanSEEstimateSEEstimateSEEstimateSEEstimateSEEstimateSE
gMarginal mean37714.260.054.180.054.190.054.190.054.190.054.170.05
Are you covered by any kind of health insurance or some other kind of health care planYes35474.360.054.280.054.290.054.300.054.290.054.280.05
No1962.870.202.830.222.820.192.800.182.860.202.700.18
Private Health InsuranceYes26334.740.064.670.064.690.064.680.064.670.054.690.06
No7783.310.113.230.103.230.103.240.103.240.103.220.10
hMedicareYes7753.800.093.820.103.830.093.810.103.810.093.800.09
No24044.490.074.450.064.460.064.470.064.460.064.450.06
Which of the following were you doing last week?Working for pay at a job or business23564.550.074.480.064.500.064.490.064.490.064.480.06
With a job or business but not at work603.530.553.730.503.720.493.760.503.750.493.690.48
Looking for work1233.220.303.000.272.950.272.980.283.000.262.930.27
Working, but not for pay, at a family-owned job or business893.930.303.910.273.820.283.910.273.890.263.860.30
Not working at a job or business and not looking for work11063.900.093.830.083.840.083.830.093.840.093.800.09
What is the main reason you did not work last week?1 (Taking care of house or family/Retired/On a planned vacation from work/On family or maternity leave/On layoff)9224.150.094.110.094.110.084.110.084.110.084.080.09
2 (Going to school/Temporarily unable to work for health reasons/Have job or contract and off-season/Disabled/Other)4393.110.173.050.143.050.153.080.153.090.153.030.15
Have you smoked at least 100 cigarettes in your entire life?Yes16744.170.074.120.074.140.074.130.074.130.074.100.07
No20644.310.074.230.074.230.074.240.074.240.074.220.07
How often do you now smoke cigarettes?Every day3793.840.163.750.163.790.163.730.163.710.153.670.15
Some days1483.480.333.360.293.320.303.310.303.300.303.320.29
Not at all11464.360.084.340.084.370.084.370.084.370.084.340.08

Research and Development Survey.

Recode method 1: set the values of the skip-pattern variables as missing among the non-applicable subjects.

Recode method 2: recode the values of the skip-pattern variables among the non-applicable subjects as a specific category of the applicable subjects, where the category is selected based on natural grouping or based on the group with similar means of the household income.

Recode method 3: recode the values of the skip-pattern variables as ‘opposite’ category to those in Method 2.

Recode method 4: set the values of the skip-pattern variables as ‘NA’ among the non-applicable subjects.

N: nominal sample size.

Though income is imputed as an ordinal variable in MI, it is treated as a continuous variable in the analyses for easy presentation and interpretation (the income categories correspond to: 0,Under$15,000;1=$15,000 to $24,999;2=$25,000 to $34,999;3=$35,000 to $49,999;4=$50,000 to $74,999;5=$75,000 to $99,999;6=$100,000 to $149,999;7=$150,000 to $199,999;8=$200,000 or more).

Results for other health care coverage types show similar patterns (results not shown).