One of us (SLB) is currently conducting a study to optimize a fixed tobacco treatment regimen for adult smokers seen in the ED. The study protocol has been previously reported.⁹ The study extends work previously done by our group and others demonstrating the efficacy of ED-initiated treatment for tobacco dependence,^10,11 and is supported by the National Cancer Institute (R01CA201873). In this optimization trial, four intervention components are examined: (1) a Brief Negotiated Interview (BNI), which is a shortened version of a motivational interview; (2) 6 weeks of nicotine replacement therapy (NRT, nicotine patches and gum), with the first patch applied in the ED; (3) an active referral to the state smokers’ quitline, made by the research assistant faxing a consultation form; and (4) enrollment in a 6-week texting program. Participants are contacted by telephone at 1 and 3 months. Participants who self-report abstinence from smoking at 3 months are asked to return to the ED for biochemical validation, via measurement of exhaled carbon monoxide. Study enrollment and follow-up are completed.

Table 1 illustrates the design of the optimization trial for the tobacco dependence study. In a factorial optimization trial there is a factor (independent variable) corresponding to each intervention component to be examined, and the levels of the factors correspond to options for each component (e.g. on/off; high/low). All of the factors are manipulated simultaneously in the experiment. If there are k factors each with 2 levels, the design is a 2^k. Because in this experiment there are four factors, each with 2 levels (offered/not offered), this is a 2⁴ factorial experiment, with 16 experimental conditions.

We note that although this design has 16 experimental conditions, it should not be considered a 16-arm RCT. A 16-arm RCT would likely require an enormous sample size, whereas an advantage of factorial designs is their efficiency in the use of experimental participants. Rather than conducting a series of two-armed trials, comparing a single intervention component to usual care (or another intervention component), all components are be assessed simultaneously, along with their interactions, in a single trial. This saves both time and resources. The chief reason is that, in a traditional RCT, effects are estimated by directly comparing the means of individual experimental conditions. By contrast, in a factorial trial main effects are estimated by comparing means aggregated from combinations of experimental conditions. All main effects are estimated based on all experimental conditions, with the conditions aggregated in different ways for different main effects. For example, the main effect of the Brief Negotiated Interview is estimated by comparing the mean of experimental conditions 1—8 in Table 1 to the mean of experimental conditions 9—16, and the main effect of Nicotine Replacement Therapy is estimated by comparing the mean of experimental conditions 1—4, 9—12 to the mean of experimental conditions 5—8, 13—16. Hence, a more modest number of study participants can be recruited, as compared to conducting four separate RCTs to examine each component.^1,12,13 This experiment is well-powered, with 66 participants per cell, for an overall sample size of 1056. Another noteworthy feature of the design, evident in Table 1, is that fully 15/16 of all study participants will receive at least one active intervention component. It is even possible to design a factorial optimization trial so that all participants receive at least some active treatment.

Based on prior work on the cost effectiveness of tobacco treatment interventions,^14,15 we prespecified that we would select components for the optimized intervention within an economic constraint of no more than $5000 per quality-adjusted life-year gained. Hence, we are collecting data on the costs of the intervention and subsequent treatment related to tobacco use.

A novel feature of our optimization trial is the addition of semi-structured qualitative interviews conducted by phone with a subsample of 65 participants, drawn from 15 of the 16 possible study conditions (participants in condition 16, in which none of the components is offered, are exempted from the qualitative work). This will allow us to assess the feasibility and acceptability of individual intervention components and combinations of intervention components. Analysis of the qualitative data is ongoing.

At the conclusion of all follow-up, we will analyze results for clinical efficacy, cost-effectiveness, and feasibility/acceptability to participants, as assessed by the phone interviews. In addition, we will examine two- and three-way interactions among study components. Components that are clinically efficacious, fit within the cost constraint, and are feasible and acceptable to participants will be retained, and packaged for evaluation in a follow-on clinical trial, which will require additional grant support.

Data from a factorial optimization trial are generally analyzed by means of factorial analysis of variance (ANOVA). When properly conducted using effect coding, this approach keeps correlations between effects to a miminum, even to zero in a perfectly balanced experiment. A generalized linear models (GLM) approach may be used to enable ready inclusion of covariates and use of different link functions, although the latter calls for care in interpretation of effect estimates, particularly interactions.

Several of this report’s co-authors (MW, PC, RC, KK) are developing an ED-based adaptive intervention to reduce risky drinking and violent behavior among adolescents. The study, called the SafERteens M-Coach study and funded by the National Institute on Alcohol Abuse and Alcoholism (R01AA024755), builds on prior work from our research group demonstrating that a selective 30-minute brief in-person therapy intervention (SafERteens intervention) for alcohol and violence delivered during emergency care reduces severe peer violence, dating violence, and alcohol consequences.^16–18 A post hoc analysis identified that response and non-response at 3 months post-intervention depended on several factors, including greater problem severity (e.g., alcohol use, violence), delinquent behaviors, mental health concerns, and negative peer influences, suggesting that although a single brief in-ED intervention session may be sufficient for many youth, a subset of youth require a more intensive and longer-term intervention.¹⁹ Pilot studies indicated that participants are amenable to extended interventions delivered via text messages and/or remote coaching.^19,20 These findings suggested that a productive direction would be development and optimization of a time-varying adaptive intervention.

Adaptive interventions are made up of decision rules that assign treatment options to participants at decision points based on tailoring variables. The treatment options may vary in approach, intensity, dose, delivery method, and/or cost (e.g., daily e-health delivered therapy messages vs. in-person weekly behavioral therapy).^5,6,21–23 These treatment options may all be available at each decision point in a patient’s course, or some may be available only if an earlier treatment option has failed or been successful. The tailoring variables are one or more individual (e.g., treatment response, adherence, side effects) or environmental (e.g., family/social characteristics) variables used to evaluate patient response on their currently assigned treatment pathway in order to tailor the treatment approach (i.e., the treatment decision). For example, tailoring variables may be used to identify early signs that a treatment is not, or is no longer, effective or sufficient for that specific patient. Finally, decision rules are a series of guidelines governing how a patient’s treatment pathway will change, depending on their tailoring variables and/or response to prior treatments. Adaptive interventions mirror “real-life” medical practice by developing a series of treatment approaches to be tried in response to how a patient responds to prior approaches. Such an approach is particularly useful for managing chronic diseases (e.g., obesity, substance use, diabetes, interpersonal violence) as they are often characterized by multiple relapses or recurrences, a series of individual and environmental factors influencing treatment outcomes that often require a specialized approach, and significant heterogeneity in treatment responses, both within the individual person and between people with similar conditions.

When the objective is to optimize a time-varying adaptive intervention, as it is in the SafERteens M-Coach study, the SMART is often a good choice. The SMART was developed by Murphy²⁴, and informed by earlier work by Lavori and Dawson.^25,26 SMARTs are multi-stage randomized trials^{5,21,22,24,27,28} in which randomization of study participants occurs sequentially, and may depend on the outcome of a prior treatment phase in the study. However, although the SMART is useful in optimization of adaptive interventions, it is not an adaptive experimental design. Unlike adaptive experimental designs, in SMARTs the treatment arms and randomization parameters do not vary during the conduct of the study.¹ SMARTs facilitate testing the efficacy of individual components of adaptive interventions (e.g., what are the best tailoring variables, key decision rules) to aid in constructing an overall efficacious adaptive intervention that can then be tested against the current standard of care or a control condition.

The SafERteens M-Coach SMART tests two novel mechanisms for increasing intervention intensity: 1) an automated text messaging program; and 2) an ongoing intensive remote health coach delivering weekly post-ED behavioral therapy sessions. This experiment addresses three research questions. The first is which is the better main first-stage approach, the single-session SafERteens intervention augmented with the automated text messaging program, or the single-session SafERteens intervention augmented with the ongoing intensive remote weekly health coach counseling? The second is, which is the more efficacious second-stage strategy for youth who are responders and those who are non-responders based on measures of binge drinking and physical aggression, assessed one month after their ED visit? Should responders continue with the same treatment or step down to a less intensive treatment? Should non-responders continue with the same treatment or step up to more intensive treatment? The third is, are there baseline or time-varying moderators of efficacy? Such moderators may make useful tailoring variables in future versions of the intervention.

Similar to the original SafERteens trial, youth (age 14-20) are enrolled in the study based on screening positive for binge drinking and physical aggression within the prior 4-months, including having a cell phone with a text messaging plan (e.g., not a messaging app only). Figure 2 illustrates the design of the SMART. After screening and consenting, youth are randomly assigned at the time of their ED visit to receive either the SafERteens brief intervention with the post-ED automated, tailored daily text messaging (two messages per day; Brief Intervention (BI) + Text Message (TM) or the SafERteens brief intervention with continued weekly post-ED remote therapy delivered by a Health Coach (HC) (one ~20-minute session per week; BI+HC). Youth involvement in drinking and violence is assessed throughout the intervention period of 4 weeks via weekly surveys. After their fourth weekly survey (4 weeks following their ED visit), responses on the four weekly surveys are used to determine whether the participant is a responder (defined as no longer binge drinking, or no moderate or severe physical aggression behaviors^29,30) or a non-responder (defined as continued binge drinking or moderate or severe physical aggression behaviors).^29,30 Specifically, a responder could have some of these behaviors in weeks 1 or 2, but none in weeks 3 and 4; in other words, those reporting any binge drinking or physical aggression behaviors in weeks 3 or 4 are considered non-responders. Importantly, those with missing data (failure to complete a weekly survey) in weeks 3 or 4 are considered non-responders for conservative purposes. As Figure 2 shows, responders are then re-randomized either to stay the course with the current treatment (where the current treatment could be either BI+TM or BI+HC, depending on initial assignment), or to step down to a resource brochure (link to study website containing resources). Non-responders are re-randomized to either stay the course with the current treatment (where again the current treatment could be either BI+TM or BI+HC, depending on initial assignment) or to step up their treatment. Non-responders who were originally assigned to the BI+TM condition (receiving automated tailored text messages) switch to receive HC (weekly remote therapy) as their treatment. Non-responders originally assigned to BI+HC receive an increased intensity of contact with the HC, including scheduled weekly sessions and in between-session personalized text messages from the HC, with option to text back and forth.

Participants are contacted at 4- and 8-months post-baseline to assess, via self-report, alcohol and violence behaviors. Anticipated sample size is 700; at present, the study is screening and enrolling eligible youth. At study conclusion, analysis of results, which will include the cost of the pathways, will form the basis for constructing an optimized, adaptive treatment algorithm for reducing alcohol and violent behaviors among at-risk youth seen in ED settings.

Several analytic considerations must be taken into account for SMART designs.^5,21,22,31 For certain research questions, analysis of a SMART is similar to analysis of a traditional factorial experiment. The first research question in the SafERteens M-Coach study, concerning which is the better first-stage approach, can be addressed by comparing the mean of the four leftmost experimental conditions in Figure 2 (A, B, C, D) to the mean of the four rightmost experimental conditions (E, F, G, H). The second set of research questions is answered in a similar manner, but must be conditioned on the 4-week outcome, which determines whether a youth is considered a responder or non-responder. The question of which is the better second-stage approach for responders, whether to stay the course or step the treatment down, can be addressed by comparing the mean of the two conditions for responders labeled Stay the Course (A, E) to the mean of the two conditions labeled Step Down (B, F). Non-responders are not included in this analysis. Similarly, the question of which is the better second-stage approach for non-responders, whether to stay the course or step the treatment up, can be addressed by comparing the mean of the two conditions for non-responders labeled Stay the Course (C, G) to the mean of the two conditions labeled Step Up (D, H). Responders are not included in this analysis.

Analysis of data from a SMART is usually performed using standard longitudinal analytic techniques (e.g., Generalized Linear Mixed Models – GLMM). Depending on the design, some research questions may require weighted regression models and/or participant replication to account for under-representation of all necessary participants in the data. For research questions involving moderation, investigators may consider standard moderator analyses, as well as more advanced Q-learning regression analytic techniques that allow for a determination of which sequence of decision rules produces the best expected outcome on targeted clinical outcome measures.^5,21,22,31