Data came from electronic health record hosted by OCHIN, Inc. (not an acronym), a provider of a centralized electronic health record system for community-based health centers across the U.S. The population for this study was adults with T2DM seen in community-based health centers. We included all adults (age ≥ 18 years) in the OCHIN dataset with T2DM (determined by previously validated algorithm—Technical Appendix) who had at least one food insecurity assessment^11,19-21 in the period of study (June 24^th, 2016 to April 30^th, 2023).

The exposure of interest was reporting food insecurity, or reporting food security, at initial food insecurity assessment. In the OCHIN clinics’ shared electronic health record, several food insecurity assessment tools are used.¹¹ The most common include items from the PRAPARE (Protocol for Responding to & Assessing Patients’ Assets, Risks & Experiences) and Accountable Health Communities Health-Related Social Needs screening tools.^22,23 Both have been validated as performing well in comparison with the 18-item USDA household food security survey module.²⁴ The USDA module is considered the gold standard of food security assessment, but is rarely used in routine care given its length and respondent burden. Thus, the brief food insecurity assessments used in this study are those that are commonly used in routine clinical care to determine the need for food insecurity interventions.

The primary outcome for this study was hemoglobin A1c (HbA1c). Secondary outcomes were systolic blood pressure (SBP), diastolic blood pressure (DBP), and LDL. The timepoint of interest was 12 months after the initial food insecurity assessment during the study period. To operationalize this within electronic health record data, outcomes were computed as the mean of all assessments for that outcome made in the 6-12 months after the ‘index date’ (date of initial food insecurity assessment). Individuals with no outcome recorded within this time window were not included in analyses of that outcome, but could be included in analyses of another outcome recorded in that period (e.g., if an individual had a blood pressure but not an LDL cholesterol recorded in the time window, they would be included in the blood pressure analyses but not in the LDL analyses).

We considered several covariates that could affect the relationship between food insecurity and study outcomes. We used electronic health record data for following variables as of the date of initial food insecurity assessment: age, sex (recognizing that electronic health record assessment often imperfectly distinguishes between sex and gender), race and ethnicity, health insurance, income, an ICD-10 based comorbidity index,²⁵ and the mean value of the outcome in the year prior to food insecurity assessment. Using address data, we created a variable for the social vulnerability index (SVI) score of the census tract of residence, which reflects area-based socioeconomic exposures.²⁶ In sensitivity analyses, we also included body mass index (BMI) as a covariate.

For statistical analysis, our goal was to estimate, using observational data, the difference in study outcomes at 12 months that may have occurred if individuals in the study had versus had not experienced food insecurity at their initial assessment. Moreover, we sought to explore whether and how those estimates varied across different combinations of covariates that could affect the relationship between food insecurity and the study outcomes. To do this, we used three different approaches that estimate ‘potential outcomes’ (outcomes that may have occurred under different counterfactual exposure scenarios) and ‘conditional average treatment effects’ (CATEs—the estimated difference in outcomes under different exposure scenarios conditional on the specific combination of individual covariates). Under the assumptions of conditional exchangeability, positivity, and consistency, these estimates can have a causal interpretation, and when discussing the methods used here we use terms like ‘HTE’ and ‘CATE’ to retain consistency with the broader literature on this topic. However, because we do not know that causal inference assumptions were met in this observational study, in interpretation of this study’s results we consider findings to be associations. We view these as exploratory, hypothesisgenerating findings to be studied further.

To aid in exploring potential heterogeneity in the association between food insecurity and the study outcomes, we used three different analytic approaches. We reasoned that if different methods found similar results, it would be a stronger signal of true heterogeneity, while if different methods had different results, it may be more suggestive of chance differences without clear heterogeneity. The three methods used were generalized random forests²⁷, X-learner¹⁷, and DR-Learner.²⁸ We describe each of these methods briefly below, and provided references include more technical details.

Generalized random forest (GRF) is a non-parametric, tree-based approach to HTE estimation.²⁷ In brief, the approach randomly samples observations and variables within a data set to grow a ‘tree’ based on splitting the sample at a value of a variable that minimizes a loss function (e.g., mean squared error), in order to model the relationship between exposure, covariates, and the outcome. This process is then repeated many times to grow a ‘forest’ of such trees. This ‘forest’ can then be used to estimate what an individual’s outcome would have been under the scenario in which they experienced food insecurity, and the scenario in which they did not, which can then be contrasted. We fit forests of 2,000 trees.

The X-learner approach starts by fitting two separate models predicting the outcome—one using data from the exposure group (i.e., those who experienced food insecurity) and one using data from the comparison group (i.e., those who did not experience food insecurity).¹⁷ It then uses those models to impute the unobserved potential outcome (i.e., for those who experienced food insecurity, an estimate of what their outcome would have been if they had not experienced food insecurity, and vice versa). Next, an estimate of the treatment effect is created by taking the difference of two outcomes—the observed outcome under the exposure the individual experienced (food insecure or secure), and the estimated outcome had they experienced the other exposure. Then another two models are created, one using treatment group data and one using comparison group data, but this time the treatment effect estimated in the prior step is modeled. From these models, two new estimates of the treatment effect for each individual are produced, one using the treatment group model and one using the comparison group model. Next, a propensity model is created to estimate the probability of the exposure (i.e., experiencing food insecurity). Finally, the two treatment effects from the second step are combined as a weighted average to create a single estimate per individual, with the weights being a function of the propensity score.

DR-learner is an estimator of HTE that, like X-learner, relies on estimating two quantities of interest—the outcome under different scenarios, and the propensity score.²⁸ DR-learner allows for an approach to estimating the outcome and propensity score called ‘SuperLearning’ which creates a weighted average of estimates produced using multiple machine learning methods.²⁹ With DR-learner, estimates of the outcome and propensity score can be made using several machine learning methods; then cross-validation is used to find a weighted average of estimates, each made using a different approach, that minimizes a loss function (such as mean squared error). This study implemented a SuperLearner approach that combined estimates from generalized linear models, generalized additive models, ridge regression, random forest implemented by R package ranger,³⁰ and extreme gradient boosting trees (XGBoost³¹) approaches.

To avoid overfitting, for each algorithm we employed sample splitting, randomly selecting 80% of the cohort for the training set (i.e., the part of the overall sample used to train the models), with the remainder allocated to the test set (i.e., the part of the overall sample in which the results we present are calculated). To address data missing in the clinical record, we used multiple imputation by chained equations,³² fitting 10 imputation datasets. Results from different datasets were pooled using Rubin’s Rules.³³

GRF, X-learner, and DR-learner algorithms all have variable importance metrics which can aid in understanding what factors, if any, contribute to heterogeneity in associations between food insecurity and study outcomes. GRF and X-learner report variable importance metrics on a similar scale, while the scale for DR-learner is different. This makes direct numeric comparison between all three methods challenging, but the ranking of the variables, a relative metric of importance, can facilitate such comparisons. Overall, if the methods produce relatively similar rankings for individual variables, it would be interpreted as evidence of consistency of the association between the variable and heterogeneity in outcomes. Or if the methods do not rank variables consistently, it may indicate lack of strong associations between the variable and outcome heterogeneity, with chance variation playing a larger role.

Analyses were conducted in R version 4.2.1 and Python 3.9.13, including use of R packages mice, grf, ranger, tidyhte, and Python package causalml.^30,34-37