The study was conducted in two K-8 inner-city public schools situated in a large city in the Northeast section of the United States. School A served 648 students (75% Latino, 18% African American, 1% Caucasian, 1% Asian, 5% Other). School B served 1,134 students (65% Latino, 16% African American, 11% Asian, 4% Caucasian, 4% Other). One hundred percent of students in both schools were eligible for free or subsidized lunch.

One hundred fourteen children (63% male) participated in one of the three group interventions over a span of 3 years. There were no differences between the groups regarding gender and ethnic composition. Children were grouped according to developmental level; children in Grades 4 to 6 and Grades 7 and 8 were assigned to a younger and an older group, respectively. Typically, three to five children were assigned to each group. Children who were absent for a session received an individual make-up session. The group sessions were conducted in the school setting at a time that did not interfere with academic instruction (e.g., free period or lunch period). All sessions were video-recorded to assess for implementation fidelity. All pertinent investigational review boards approved the study.

School counselors and project staff conducted in-service training with teachers on how to recognize children who might have problems with externalizing behavior, anxiety, and depression and how to go about referring children to the groups. School staff identified children for possible participation in the study following standard procedures for children in need of services. Children who had exhibited behavioral or emotional problems in the classroom were referred following the school district’s Comprehensive Student Assistance Process (CSAP) in which the referral is discussed by the school counselors and other members of the CSAP. If members of the CSAP agreed that the children met inclusion criteria, they were referred to study staff for an eligibility evaluation. School staff contacted the parents initially to provide a brief overview of the study. They also obtained verbal consent for research staff to contact them to provide a more thorough description of the study to the parents and to obtain written consent. Parents gave informed consent and children gave assent. Children who met primary positive or intermediate diagnostic criteria for an externalizing disorder based on the NIMH C-DISC IV and IT scale were assigned to CPP. Children who met primary positive or intermediate diagnostic criteria for an anxiety disorder were assigned to FRIENDS. Those who met primary positive or intermediate diagnostic criteria for a depressive disorder were assigned to PASCET. Once the groups started, parents received periodic updates regarding the progress children were making in the groups.

CPP (Curry, Wells, Lochman, Craighead, & Nagy, 2003; Lochman & Wells, 2002a, 2004) is a social-cognitive, multi-component group intervention for elementary and middle school students at risk of externalizing behavior disorders. In addition to anger management, CPP includes units on goal setting, emotional awareness, relaxation training, social skills training, problem solving, and handling peer pressure (Lochman, Powell, Boxmeyer, Deming, & Young, 2007). CPP offers eight sessions in the first year of intervention and 25 sessions in the second year of intervention. Most of the content is taught during the first eight sessions. Studies using an earlier, briefer (12 session) version of CPP (Anger Coping) reported significant reductions in aggressive behavior at post-intervention among targeted aggressive boys, compared with untreated aggressive boys and normal controls (Lochman, 1985; Lochman & Curry, 1986). For this study, we provided CPP in 12 weekly, 45-min sessions to make CPP more feasible for implementation in underresourced urban schools. In making program adaptations, we preserved all main components of the protocol.

FRIENDS addresses physiological, learning/behavioral, and cognitive processes that interact in the development and perpetuation of excessive anxiety (Barrett, 1998). These processes include (a) understanding the physical expression of anxiety and using relaxation skills; (b) problem-solving skills, graded exposure, and self-reward; and (c) cognitive restructuring and self-talk. The FRIENDS protocol consists of 10 weekly 70-min sessions and two booster sessions. For the present study, we included the booster sessions in the regular protocol and reduced the length of each session to fit them into the typical class period (45 min) for a total of 12 sessions.

PASCET was developed taking into account “real-world” implementation contexts (Szigethy et al., 2007; Weisz et al., 2009). The group version (Bearman et al., 2010) addresses symptoms of mild to moderate depression by teaching coping skills through the use of video-recorded vignettes and group discussion. Through the video-recordings, youngsters learn nine coping skills that can be used to gain control of their mood when stressful events make them feel sad or upset. Two primary control skills are emphasized: (a) identifying and consciously engaging in activities that the child finds mood enhancing and (b) skill building through goal setting and practice in activities that the child values. Three secondary control skills are emphasized: (a) identifying and modifying depressive thoughts, (b) cognitive techniques for mood enhancement, and (c) relaxation and positive imagery. PASCET was delivered in 12, 45-min sessions.

External consultants and project staff conducted an initial workshop with school counselors and members of the research team on intervention implementation. Trainers associated with the developers of FRIENDS and PASCET provided training on those protocols. Project staff who had previously received training by the developers of CPP conducted training for CPP. The training structure consisted of a one-and-a-half-day workshop for each program that included discussion of the theoretical background (identification of symptoms, prevalence rates, treatment efficacy), the development of each intervention (theoretical rationale, key components, efficacy/effectiveness findings), and a detailed review of the intervention sessions (content, structure, process, implementation challenges). Training included both didactics and active learning activities such as small group discussions, role-plays, behavior rehearsals, watching video-recorded sessions, and demonstration of techniques.

Following the initial training, members of the research team conducted weekly consultation with all staff participants on content delivery and implementation barriers. The principal investigator and two postdoctoral fellows in applied child psychology led the consultation sessions. The consultants, two of whom were English/Spanish bilingual, provided support to the counselors by examining intervention progress, addressing implementation barriers using problem-solving strategies, and preparing for the next sessions. Counselors were involved in decisions about every aspect of content delivery, including preparing handouts, determining of how much time to dedicate to each section of the manual, and determining how many make-up sessions to be held for children who missed some of the sessions.

The data analysis strategy focused on assessing whether the three group interventions were effective at decreasing level of diagnostic severity (i.e., from clinical to intermediate, clinical to negative, and intermediate to negative) for children who had post-treatment data as measured by the NIMH C-DISC-IV. We wanted to assess whether CPP was an effective treatment for externalizing disorders in general and for ODD, CD, and ADHD, specifically. Moreover, we wanted to determine whether FRIENDS was an effective intervention for internalizing disorders in general, and separation anxiety disorder (SAD), social phobia (SOPHOB), and generalized anxiety disorder (GAD), in particular. These disorders share an underlying construct of anxiety which is somewhat distinct from other anxiety disorders, such as obsessive-compulsive disorder (OCD) and specific phobias (Beesdo et al., 2009). Finally, we wanted to assess whether PASCET was an effective intervention for internalizing disorders in general, and major depressive episode (MDE), specifically.

Prior to statistical analysis, the diagnostic status of each participant at each time point (pre-, post-, and follow-up) was determined using the following ordinal scale: 0 = Negative (no diagnosis), 1 = Intermediate (at risk of diagnosis), 2 = Positive (presence of diagnosis). In evaluating the effect of each intervention on specific disorder status, a 2 × 2 McNemar test was utilized to compare pre and post diagnostic status and pre and follow-up status. At pre-intervention, participants were grouped as either not having the diagnosis or being at risk or positive for a diagnosis. At post-intervention, participants were differentiated by whether their diagnostic status remained the same as baseline (yes) or changed (no). Crossing pre-intervention and post-intervention status created four cells: (a) no change for positive or intermediate diagnostic status, (b) no change in negative diagnostic status, (c) decline in diagnostic status (i.e., from negative to intermediate status, or from intermediate to positive status), and (d) improvement in diagnostic status (i.e., from intermediate to negative, or from positive to intermediate).

In evaluating the effect of each intervention on a broad spectrum of disorders, a total score was derived at pre-intervention, post-intervention, and follow-up by summing the scores obtained for each specific disorder within the broad spectrum (e.g., ODD, CD, and ADHD for the externalizing disorders). The sum of scores at pre-intervention was crossed with the sum scores at post-intervention to create a 2 × 2 table for McNemar test comparisons.

Given the high comorbidity level between anxiety and depressive disorders, we examined whether FRIENDS was effective for children diagnosed with MDE. We also examined whether PASCET was effective for children diagnosed with anxiety disorders. Due to the exploratory nature of this study, significance was examined at p < .05. Changes from pre-intervention to follow-up are presented using descriptive statistics due to the small sample sizes.

Content fidelity was scored as a sum of covered content (i.e., a “yes” response) divided by total content to be covered per session. Total fidelity for the group was the sum of fidelity levels for all sessions divided by the total number of sessions. Total fidelity for the program was the average fidelity for all groups. Interrater reliability between two independent coders was assessed for 25% of the sessions chosen at random. The assessment of interrater reliability was evaluated using kappa coefficients, which represents agreement between two observers taking into account the fact that observers sometimes agree or disagree simply by chance (Viera & Garrett, 2005).

Average number of regularly scheduled group sessions attended by participants was 6.4 (3.4 SD) for CPP, 6.8 (2.7 SD) for FRIENDS, and 5.6 (4.0 SD) for PASCET. School counselors and project staff reviewed material with absent participants individually, or in groups of two, prior to the upcoming session. Using this procedure, 83% of participants in CPP, 85% in FRIENDS, and 82% in PASCET received the entire content of the interventions.

Fidelity data revealed that 88% of the elements of CPP were implemented as intended. Interrater reliability between the two raters for CPP was κ = .43 (p < .001), 95% confidence intervals [CI] = [0.353, 0.497]. Fidelity for FRIENDS was 87% and interrater reliability was κ = .63 (p < .001), 95% CI = [0.58, 0.68]. Fidelity for PASCET was 94% and agreement between the two raters was κ = .63 (p < .001), 95% CI = [0.55, 0.72]. The kappa statistics across the three groups indicated a moderate degree of agreement between the raters (Viera & Garrett, 2005).

The encouraging results with regard to implementation fidelity and child outcomes need to be considered in relation to the implementation strategy used in the study. This investigation was designed as an effectiveness and not an efficacy study, meaning that all three interventions were implemented in the “real world,” specifically, in underresourced urban schools and with children presenting multiple comorbidities. It should be noted that the implementers received high levels of implementation support, including participation in an initial training workshop, ongoing consultation on session preparation, and problem-solving barriers to implementation. In addition, graduate students, who had varying levels of exposure to EBIs, supported school counselors through co-therapy. This implementation strategy may not be feasible for most schools. However, the child outcomes represent a valid comparison for future studies that use lower levels of support for school implementers. Indeed, studies are needed to identify feasible and cost-effective implementation strategies that can achieve reasonably successful child outcomes.

Past research has shown that there are many barriers that make it difficult for low-income minority parents to participate in research studies (Gross, Julion, & Fogg, 2001). Low parent engagement affected both recruitment (e.g., difficulty with consent and eligibility evaluations) and subsequent data collection (e.g., post-treatment and follow-up interviews). This reduced the number of evaluable participants for analyses, especially for children participating in CPP. As some studies have demonstrated, lack of parent participation in research studies conducted in underresourced schools does not have to be endemic (Gross, Breitenstein, Eisbach, Hoppe, & Harrison, 2014). Studies have shown that effective engagement strategies with low-income families include (a) acknowledging parents’ values and their expertise about their children, (b) acknowledging that they want to be good parents, (c) reinforcing parents for their efforts to change, and (d) giving them options for achieving intervention goals (Eiraldi et al., 2015; Gross et al., 2014). Motivation strategies include eliciting self-motivating change statements and identifying, developing, and executing plans for dealing with barriers to treatment adherence and continued participation in treatment sessions (Nock & Kazdin, 2005).

The study has a number of implications for clinical practice. First, GCBT can be effectively delivered in urban schools serving children from low-income families presenting externalizing and internalizing disorders. Children can be included in GCBT if they have a combination of externalizing and internalizing disorders. However, school-based clinicians should determine which disorder causes the most impairment and choose a protocol for that disorder. As demonstrated in the study, choosing a protocol for a specific disorder within a broad spectrum (i.e., externalizing or internalizing) can also lead to reductions in other comorbid disorders within the spectrum.

Most participants in the study received all 12 sessions. Children who were not present for a given group session received the content material in brief individual sessions. Clinicians planning on using one of these protocols in their schools should ensure that children who miss a session are given the opportunity to cover the material for the missed session. Finally, clinicians who use one of these treatments must ensure that the manuals are implemented with relatively high levels of fidelity.

There are several limitations to the study. First, the clinical effectiveness of the interventions was examined without using control groups. Even though no control groups were used in the study, we showed similar effects using three different EBIs for treating three different types of psychiatric problems. Prior studies, including studies conducted in school settings, have demonstrated the efficacy and effectiveness of these treatments. Furthermore, student behavioral and emotional functioning typically worsens over the course of a school year (Kellam, Ling, Merisca, Brown, & Ialongo, 1998). The findings of this study demonstrate that group interventions generally appear to be successful in altering this expected decline. Second, the sample size for all three groups was small, raising questions about the adequacy of the sample to capture the variability among students attending similar types of schools. Nonetheless, the impact of the relatively small sample size is that it reduced the power of the analyses to detect changes in diagnostic severity level in response to intervention. It is quite possible that a larger sample size may have resulted in sufficient power to detect statistically significant change for comparisons that had alpha levels that were marginal. Third, post-treatment data were not collected for some students in the group interventions (ranging from 12% for PASCET to 35% for CPP). It is possible that attrition had an effect on the findings although the pattern of results across intervention approaches was remarkably similar. Third, only one instrument was used to measure change. Although inclusion of other methods of assessment is highly desirable, the parent-structured interview is considered the “gold standard” in clinical trials, and as such, lends credibility to study findings (Hughes et al., 2000). Fourth, counselors and graduate students conducted most of the group sessions together. We were not able to determine whether outcomes differed depending on whether counselors or graduate students conducted the groups. In addition, analyses of program effectiveness by gender and ethnic group were not conducted due to the relatively small sample size.

Research including suitable control groups and random assignment to condition are needed to verify the effectiveness of these interventions. Studies are also needed for testing various implementation strategies to identify feasible and cost-effective approaches for implementing EBIs in underresourced urban schools. For example, research is needed to find cost-effective strategies for creating sustainable internal capacity in schools and in mental health agencies that provide services in schools. One such strategy is internal clinical supervision. As the majority of clinicians and supervisors who provide services in schools have not received adequate training on EBIs and paying external consultants is not financially feasible for many school districts and agencies, efforts must be focused on finding effective ways to train clinical supervisors to become effective supervisors of EBIs. Although numerous challenges exist in transporting EBIs to urban schools, the school setting offers a tremendous opportunity to increase access to quality mental health care for underserved children.