This study utilized aggregated state-level data on the prevalence of SPT in public secondary schools from the Centers for Disease Control and Prevention’s (CDC’s) School Health Profiles (SHPs) biennial reports from 2010 to 2022 [12]. The SHP is a self-administered survey of principals and lead health education teachers used to assess school health policies and practices across secondary schools in the United States. The SHP survey uses random, systematic, equal-probability sampling strategies to select representative samples of public secondary schools in each jurisdiction. The SHP reports are produced by CDC with each new survey data release and provide a detailed description of the methods, new trends in outcomes, and state-level summary statistics that are representative of public secondary schools within each state. For each survey-year, data from some states were excluded in the reports if they were not surveyed or if representative state-level data were not able to be produced. Table 1 outlines state participation, the number of lead health educators surveyed, and response rates for each state.

From these reports, we collected state-level data from each survey-year on the percentage of public secondary schools in a state in which the lead health education teacher received professional development in suicide prevention during the 2 years before the survey (hereafter referred to as SPT prevalence). The data do not include specifics on the type or delivery of the training. This survey outcome is specific to health education teachers, which limits applicability to a broader generalization of the impact on all school staff; however, health education teachers would be impacted by GTLs that target all staff and most GTLs apply broadly to school employees, including and beyond teachers [13]. Additionally, with the exception of data focused on lead health educators in the SHP, longitudinal data sufficiently able to facilitate cross-state comparative analyses on SPT in schools are limited. For these reasons we believe this outcome to be the best SPT prevalence proxy available to evaluate the effectiveness of GTLs at increasing training uptake. In addition to data on SPT, we also collected data from the SHP reports on similar variables that reported professional development prevalence among lead health education teachers at the state-level across other health topics for falsification tests; these topics included nutrition and dietary behavior, physical activity, pregnancy prevention, and sexually transmitted disease prevention.

Data on GTLs from all 50 states were collected from previous research that used a legal epidemiology approach [10]. Legal data collected included passage dates, effective dates, whether training was mandated or encouraged, and duration and frequency of trainings. Laws were included for analysis if they pertained to training related to suicide prevention for school staff for any grade level. Strict mandates were defined as laws that required some or all school staff to be trained or included mechanisms to require training. Conditional mandates were defined as laws that appeared to mandate training but with conditions that may limit real world implementation of the mandate (e.g., mandatory training conditional on availability of funding). Encouraged laws were defined as laws that did not mandate training, but rather expressly encouraged staff to be trained or included mechanisms through which training would be encouraged [10].

Finally, we collected yearly data from 2005 to 2022 on whether states received funding through the Garrett Lee Smith (GLS) Youth Suicide Prevention Program from the Substance Abuse and Mental Health Services Administration’s grant portal [14,15]. The GLS program was rolled out across jurisdictions starting in 2004 and supports states and tribes in implementing youth suicide prevention strategies in schools, juvenile justice systems, substance abuse programs, and more [16]. Previous research has highlighted wide-reaching distribution of GLS grants across the United States with gatekeeper trainings as one of the most commonly implemented suicide prevention strategies among grant recipients (though not necessarily the only strategy) [17,18], making it important to account for the program’s potential impact in our modeling approach.

Data on GTLs and GLS funding were merged with the SHP report data to create an unbalanced state-by-year panel dataset with 291 state-year observations across 48 states (Table 1) (West Virginia and North Dakota were excluded from analysis. West Virginia repealed its GTL over the study period and North Dakota instituted a law encouraging training that replaced a conditional mandatory law.). This activity was reviewed by CDC, deemed research not involving human subjects, and was conducted consistent with applicable federal law and CDC policy (45 C.F.R. part 46.102(l) (2), 21 C.F.R. part 56; 42 U.S.C. Sect. 241 (d); 5 U.S.C. Sect. 552a; 44 U.S.C. Sect. 3501 et seq.)

We utilized a difference-in-difference (DID) empirical design to evaluate the association between GTL mandates and state SPT prevalence. DID is a commonly used empirical strategy to evaluate the impact of policy changes across a wide range of disciplines [19,20]. DID makes comparisons in outcomes between an “intervention” and “control” group, before and after a policy change to evaluate the policy’s impact. For the purposes of our study, “intervention” states are those that instituted strict or conditional GTL mandates (hereafter referred to as “GTL mandates”) throughout the study period, and “control” states are those that did not institute GTL mandates (“Instituting” GTLs refers to the date the legal provision on training went into effect, which may differ from the law’s passage date). The control states included states that passed laws “encouraging” SPTs for school staff, as these laws did not mandate trainings. We empirically implemented the DID strategy using a linear regression model, regressing the state SPT prevalence variable on our policy variable indicating if a state mandated gatekeeper training. Additionally, we controlled for state fixed effects, survey-year fixed effects, and whether a state received funding through the GLS program (formal model presented in the technical appendix). Standard errors were clustered at the state level. To correct the presence of potential bias arising from heterogeneous treatment effects [21–23], we estimated additional models using the Callaway and Sant’Anna (CS) approach (We estimate the Callaway and Sant’Anna model under two separate specifications: the not yet treated and never treated analyses. In the not yet treated analysis, the control group consists of states that never instituted a GTL mandate and states that have yet to institute a GTL mandate for the years prior to the mandate’s effective date. For the never treated analysis, the control group consists of just states that never instituted GTL mandates.), which accounts for heterogeneity in treatment effects across multiple time periods by identifying and then aggregating group-time average treatment effects, and has become a commonly used methodology when policies have a staggered effective date across states over time [22,24].

Furthermore, a main assumption of the DID model is that of parallel trends in outcomes which posits that trends in SPT prevalence would have been the same between the “intervention” and “control” states in the absence of GTL mandates. We indirectly tested this assumption by estimating an event study model with CS bias corrections. The event study model is an extension of the DID model that evaluates lead and lag effects of the GTL mandate (further described in the technical appendix). Coefficient estimates that are not statistically different from zero on the lead policy variables are a necessary condition for the validity of DID models and would suggest that the parallel trend assumption is not violated [20].

Finally, we conducted a sensitivity analysis evaluating strict GTL mandates (i.e., conditional mandates included in the control group). We also conducted a set of falsification tests in which the DID models specified previously were estimated with other nonrelated health training outcomes (e.g., state prevalence of sexually transmitted disease prevention trainings) as the dependent variable. If GTL mandates were the primary mechanism leading to changes in the SPT prevalence across states, we would not expect to see that the mandates were also associated with changes in the prevalence of other nonrelated trainings. All analyses were conducted using Stata/MP 17.0.

This research has several limitations. First, the specific scope of this study precludes conclusions about school-level results and the impact of GTLs on suicide outcomes. Second, given that implementation of gatekeeper training may fluctuate and is more closely tied to the timing of the school year, the biennial timing of the SHP reports may lead to an underreport of training prevalence. Third, generalizability of results is limited as the outcome measure is specific to lead health education teachers. Fourth, the lead health education teachers were posed retrospective questions looking back 2 years, which may introduce recall bias into the prevalence estimates. Additionally, the model accounts for the presence of a GTL in a state at the end of the 2-year retrospective, indicating short-term intervention effect estimates may include some noise. For example, if a GTL was instituted just before a survey took place, it is possible some teachers may have received training within the survey question window prior to GTL institution which could overestimate short-term effects of the policy change. Conversely, some teachers may not have had enough follow-up time to receive the training post mandate which could underestimate the short term effect of the policy change. Fifth, this evaluation did not explore how the impacts of GTLs may vary based on specific components of the law, such as frequency of training or delivery methods. Nor did the evaluation explore retention of suicide prevention material or knowledge among trainees or information about the characteristics of schools that saw increases in training. The SHP data do not include detailed measures of the quality of SPT implemented in schools (e.g., the specific types of training teachers received or the proportion of staff trained). These factors may account for the effectiveness of GTLs in reducing suicide rates and warrant future investigation. Finally, the SHP reports included state-level representative data for just public secondary schools, thus results may not be generalizable to other types of secondary schools (e.g., private).