Data came from the Rhode Island Department of Behavioral Healthcare, Developmental Disabilities, and Hospitals (BHDDH), which oversees SUD treatment and regulates OTPs in the state. BHDDH administrative data captures admissions to and discharges from licensed inpatient, outpatient, and detoxification programs, as well as OTPs in Rhode Island. Rhode Island has 17 OTPs servicing both the state’s urban cores and suburban and rural outlying areas,³¹ and methadone access is robust across the state.³² As such, we consider Rhode Island an ideal jurisdiction to measure the impact of methadone access policy on treatment outcomes. Data do not include office-based buprenorphine treatment, which is not the focus of this study. We limited the data to admissions to and discharges from OTPs in Rhode Island, which dispense methadone to treat OUD. Patient OTP admissions and discharges were captured as part of routine patient care and administrative data collection. Use of BHDDH data and all study procedures were approved by the Brown University School of Public Health Institutional Review Board.

Data captured admissions to and discharges from OTPs in Rhode Island between January 1, 2016 and December 31, 2020. Admissions and discharges were identified using exact dates. To determine sample eligibility, we identified patient admissions to programs or services of all types, including new intakes and transfers from non-OTP programs, from January 1, 2016 through December 31, 2020, using data obtained from BHDDH. We excluded admissions related to non-OTP treatment modalities. Next, we restricted the sample to OTP patients newly admitted between March 18 and June 30, 2019 and March 18 and June 30, 2020 to exclude those with prior histories of OTP enrollment during our lookback period. Using a two-year lookback period restriction isolated changes in retention for newly admitted patients without longer histories of methadone treatment to reduce the potential for selection bias due to left censoring. These narrow enrollment windows allowed us to capture six-month retention in both our treated (2020) and control (2019) group periods while ensuring that the control group’s six-month retention did not overlap with the start of the pandemic.

We assessed one primary exposure: the March 18, 2020 implementation of Rhode Island’s expanded methadone take-home policy. Our exposed period was March 18 to June 30, 2020. Our unexposed period was March 18 to June 30, 2019. Notably, our policy exposure coincides with the start of the COVID-19 pandemic and, as such, reflects Rhode Island’s methadone take-home policy as implemented within the broader context of the COVID-19 pandemic.

Our primary outcome was six-month retention in methadone treatment after first admission, comparing patients newly admitted during the exposed and unexposed periods. We selected a six-month window, as prior literature indicates that this duration may be considered successful retention in OUD treatment and is associated with improved long-term outcomes.³³ We defined first admission as the first recorded admission date for a given individual in BHDDH administrative records during the two time periods of interest. We defined six-month retention as six months of continuous treatment enrollment after admission date without a recorded discharge date in BHDDH administrative records, permitting seven-day gaps between discharges and subsequent admissions for patients internally or externally transferred between programs.³⁴ Due to federal regulatory requirements and BHDDH administrative oversight of all OTP data, admissions and discharges are considered complete in their recording. That is, we assume that patients without a recorded discharge date remain enrolled in treatment and are not necessarily lost to follow up.

Multivariable models were adjusted for a selection of demographic, socioeconomic, and social/structural characteristics that we anticipated may differ between the exposed and unexposed periods.^24,35 Our analysis was restricted to the variables collected at admission and discharge. Demographic variables included: age group (18–34 years, 35–54 years, and 55 years and older), gender (female or male, recorded as self-reported gender identity at admission with only two response options), and race/ethnicity (coded as non-Hispanic white, non-Hispanic Black, Hispanic/Latino, or other/multi-racial). Socioeconomic variables included: educational attainment (completion below high school, high-school completion, or completion beyond high school), housing status (non-institutionally housed, institutionally housed, correctionally housed, or unhoused or housed in shelter), employment status (full-time employed; part-time employed, student, or homemaker; unemployed, retired, disabled; or institutionalized), and payment source (private insurance; Medicare, Medicaid, or other public insurance; or uninsured). Social/structural variables included: veteran status (history of military service or not), referral source (self or voluntary; health care system; behavioral health system; social service or shelter; employer, school, or community service; or the criminal justice system), primary drug at admission (heroin; fentanyl; opioid analgesic or other opioid; cocaine, methamphetamine, or other stimulant; or other non-opioid drug), and whether the patient had been arrested within 30 days of the admission (yes or no). Data for covariates were relatively complete, ranging from fully complete to 6.7% missing.

First, we tabulated characteristics for patients in the cohort overall and by exposure period. Differences between exposure periods was assessed using standardized mean differences (SMDs).³⁶ Counts fewer than five were suppressed from reporting in compliance with BHDDH policy.

Second, we estimated the overall and sub-group stratified odds and risks of six-month retention after first OTP admission before and after the implementation of Rhode Island’s expanded methadone access program using, respectively, logistic and Poisson regression.^37,38 We selected the features for stratification in collaboration with BHDDH to identify those patient characteristics most salient to treatment providers. Models were adjusted for the aforementioned patient and admission characteristics using a complete case analysis with listwise deletion.³⁹ To assess model robustness to missingness, we conducted two sensitivity analyses: a range sensitivity analysis for the variable with the most pronounced discrepancy in missingness between the pre- and post-intervention periods, arrest within 30 days of admission, and an analysis using multiple imputation to correct for missingness across all variables.^40,41 For the range sensitivity analysis, we reran our overall and stratified analyses assuming that either all participants missing in the post-intervention time period were arrested and all participants missing in the pre-intervention time period were not arrested or vice versa. For the multiple imputation, we reran our overall and stratified analyses with complete data imputed using multiple imputation with chained equations with predictive mean matching.⁴² Data management, descriptive analyses, logistic regression models, and sensitivity analyses were conducted using Stata version 16 (StataCorp LLC, College Station, TX).

Third, to assess whether any subgroups of patients were differentially affected by Rhode Island’s expanded methadone take-home policy, we used the HTE-Scan machine learning approach, which is an extension of the Treatment Effect Subset Scan.⁴³ HTE-Scan iteratively searches through subgroups, defined by combinations of covariates (eg, non-Hispanic white females aged 35–54), and identifies subgroups with the most significant heterogeneity in treatment effect, as determined by the highest value of a penalized log-likelihood ratio score. Subgroups were not compared to each other; rather, HTE-Scan analyzed the change in each subgroup’s retention rate from the control to the treatment period. The log-likelihood ratio score of a subgroup S can be interpreted as a measure of how much the data in S deviates from its expectation under the null hypothesis H₀, which is that there is a uniform treatment effect for all individuals. The alternative hypothesis for a given subgroup, H₁(S), is that individuals in S experience an increased treatment effect, larger than the uniform multiplicative shift in odds observed among the treatment group. The log-likelihood ratio score F(S) = log(Pr(Data | H₁(S))/Pr(Data | H₀)) is penalized by a constant factor (set at a value of 0.5 in this analysis) times the number of specified attribute values in the subgroup. This helps minimize the risk of overfitting the data or identifying small, uninterpretable groups. We selected 0.5 as a default value for HTE-Scan to utilize a penalty that is small but greater than zero, to avoid including irrelevant attributes that lead to overly complex subgroup descriptions, which is a risk for the unpenalized scan. However, very large penalty values would result in no heterogeneity being found. To account for differences in exposure groups in this observational study when modeling expectations under the null hypothesis, propensity weighting is used to obtain the average treatment effect on the treated (ATT). Thus, the steps for running the HTE-Scan algorithm and assessing the significance of its results are as follows:

Compute the maximum likelihood estimate of the uniform multiplicative shift in odds of the outcome under H₀, which assumes a uniform treatment effect for individuals.

HTE-Scan requires the standard causal assumptions of exchangeability, positivity, consistency, and non-interference for unbiased causal inference.^44,45 In our case, this means that effect estimates can potentially be biased by unobserved confounders that affect both selection into treatment and retention in treatment. The HTE-Scan algorithm as implemented only identifies subgroups with excessive increases in treatment effect. To explore subgroups that may have decreased retention due to Rhode Island’s expanded methadone take-home policy, we re-ran the HTE-Scan algorithm on the dataset with the retention outcome reverse-coded as discontinuation. We utilized 100 iterations of HTE-Scan for each run of the algorithm, assessing heterogeneity of the take-home policy effect across subgroups defined by gender, age groups, race/ethnicity, educational attainment, arrest within 30 days of admission, and primary drug at admission. We selected these features a priori for inclusion based on their known relevance to treatment retention and overdose risk.^20,46,47 For significance testing, we performed a permutation test using 1000 simulations. This permutation test correctly adjusts for multiple testing, bounding the overall proportion of type-I errors (ie, the familywise error rate). The maximum subgroup score of the actual data is compared to the distribution of the maximum subgroup score for data generated under the null hypothesis of no treatment effect heterogeneity, where the same search over subgroups is used for each null dataset as the actual dataset. While this procedure is computationally expensive, multiplying the runtime by the number of simulations used for the permutation test, it is necessary to correctly adjust for the multiple testing being performed by the search procedure. To assess the sensitivity of our results to alternative penalty values, we conducted a sensitivity analysis repeating the scan using penalty scores of 0.25 and 1.00. The HTE-Scan algorithm and permutation testing was implemented using Python version 3 (Python Software Foundation, Wilmington, DE) with results summarized using R version 4.2 (R Core Team, Vienna, Austria).

The cohort included 1248 patients who were newly admitted into an OTP in Rhode Island during the study time periods. Table 1 presents characteristics of admissions for the full cohort and for the pre- and post-intervention periods. Overall, most admissions were for patients aged 18–34 years (584; 47%) or 35–54 years (582; 47%), male (833; 67%), non-Hispanic white (1001; 80%), with education at the high-school level (510; 41%). Most admissions were for patients who were non-institutionally housed (717; 58%), unemployed, retired, or disabled (614; 49%), without history of military service (1171; 94%), voluntarily self-referred to treatment (1152; 92%), and not arrested in the month prior to admission (887; 71%). Most admissions were paid using public insurance (1121; 90%). The proportion of admissions recording heroin as the primary drug decreased from 64% to 46% between the pre- and post-intervention periods, while the proportion of admissions recording fentanyl as a primary drug increased from 17% to 28% between the two periods.

Figure 1 presents levels of six-month retention overall and among patient subgroups before and after the take-home methadone policy change. Overall, retention increased from 43% in the pre-policy period to 47% in the post-policy period. For most patient subgroups, changes were minor. However, among non-Hispanic Black patients, six-month retention decreased from 41% in the pre-policy period to 21% in the post-policy period, although this change was not statistically significant. Among patients with a past-month history of arrest prior to admission, six-month retention increased from 15% in the pre-policy period to 34% in the post-policy period.

Table 2 presents adjusted overall estimates and estimates stratified by gender, age, race/ethnicity, education, arrest history, and primary drug at admission. These models estimated an overall adjusted odds ratio (OR) and risk ratio (RR) for the change in the odds of six-month retention after the first OTP admission between the pre- and post-intervention periods of, respectively, 1.08 (95% CI, 0.80, 1.45) and 1.03 (95% CI 0.90–1.18). Subgroup estimates generally were consistent across patient and admission characteristics, with most stratified analyses suggesting no detectable change in the odds or risks of six-month retention before and after the take-home policy intervention.

Among patients with past-month arrest prior to OTP admission, we estimated an adjusted OR of 3.91 (95% CI, 1.24, 12.37) and adjusted RR of 2.21 (95% CI, 1.07, 4.58), indicating that the odds and risks of retention among patients with recent arrest were substantially elevated in the post-intervention time period. Our gender-stratified results detected differences in the change in retention across women and men. Among women, retention decreased in the post-intervention period, with an adjusted OR of 0.62 (95% CI: 0.35, 1.10) and adjusted RR of 0.84 (95% CI: 0.67, 1.05), whereas among men, retention increased, with an adjusted OR of 1.37 (0.95–1.97) and adjusted RR of 1.13 (0.97, 1.34). Unadjusted estimates are presented in Table S1.

Table 3 presents covariate profiles of patients identified by HTE-Scan for whom six-month retention significantly increased after the implementation of Rhode Island’s expanded methadone take home policy in excess of the cohort-wide estimates. Subgroups of patients for whom retention significantly increased in excess of the overall cohort were: (1) patients with education below the high-school level who had a history of past-month arrest prior to admission (P = 0.005) and (2) male patients who were non-Hispanic white or Hispanic/Latino, reported heroin or fentanyl as the primary drug at admission, and had a history of past-month arrest prior to admission (P = 0.041). We identified overlap between clusters: 64% of the first treatment cluster were included in the second treatment cluster and 50% of the first control cluster were included in the second control cluster. Overall, 51% of the patients in the first cluster are included in the second cluster. There were no groups of patients identified by HTE-Scan whose retention was significantly decreased by Rhode Island’s expanded methadone take-home policy.

This study has several limitations. First, BHDDH records contain comprehensive dates of OTP admissions and discharges but limited information about the quality of care, and how or why patients discontinued treatment. Second, while all patients initiated OTP during the observation period, it is possible that some had entered and exited OTP prior to the study. However, given our original look-back period of January 1, 2016, over three years prior to our observation start date for our main analysis, we assume that left censoring is not a major cause for concern. Despite this, as many individuals with OUD cycle in and out of treatment, future research leveraging longitudinal administrative data could help determine whether prior treatment experience influences retention outcomes under expanded take-home policies. Third, our study was restricted to Rhode Island and assessed the change in OTP retention before and after the enactment of the expanded take-home policy without a true control group. Thus, we are unable to interpret estimates as causal. Fourth, our study relied on administrative admission and discharge data to assess retention but does not include direct measures of opioid use, such as urine toxicology screening or self-reported substance use. Future research incorporating these measures is needed to evaluate whether expanded methadone take-home policies influenced continued opioid use during treatment and other patient-centered outcomes to better understand patient engagement beyond retention. Fifth, we estimated the short-run odds of OTP retention associated with a single policy enacted at the start of the COVID-19 pandemic in the United States (March 2020), a time when the national health policy landscape was in flux. Thus, our findings reflect the impact of expanded methadone take-home policies as implemented within the broader context of the COVID-19. While our study suggests that the policy did not negatively impact retention overall, we cannot disentangle its effects from other pandemic-related disruptions in healthcare access and social conditions. Some subgroups may have experienced lower retention during this period due to factors beyond the policy itself, highlighting the need for further research on the long-term effects of these regulatory changes in non-crisis contexts.

Sixth, the clusters identified through our application of HTE-Scan were small in size. This may be due to two factors: (1) the overall year-on-year reduction (by a factor of 2×) in opioid treatment program admissions; and (2) the year-on-year reduction (by a factor of 2.5×) in the proportion of individuals arrested in the 30 days prior to admission. However, since the control observations are used to model the expected retention rate in the treatment group under the null hypothesis of no treatment effect, the larger control group size makes our estimates of expected retention rate more reliable. Despite these larger control clusters, the small treatment cluster sizes decrease statistical detection power, and thus a small treatment cluster will only be detectable if it has a large difference in retention rate as compared to its corresponding control cluster. Seventh, while HTE-Scan identified two distinct clusters of patients with increased retention, we observed substantial overlap between them, with 51% of individuals in the first cluster also included in the second. This overlap suggests that the identified subgroups are not entirely independent, and while they highlight key patient characteristics associated with increased retention (ie, recent criminal legal involvement), they should be interpreted as complementary rather than strictly separate groups. In sum, while hypothesis generating, our findings regarding heterogeneity in OTP retention should be confirmed through further studies.