This review focuses on four parent training interventions that have been given the highest possible rating as “well supported by research evidence” by the California Evidence-Based Clearinghouse for Child Welfare (http://www.cebc4cw.org): Parent-Child Interaction Therapy (PCIT), The Incredible Years (IY), Parent Management Training-Oregon Model (PMTO^R), and the Positive Parenting Program (Triple P). Our sample was selected from the list of 22 interventions indicated by SAMSHA's National Registry of Evidence-based Programs and Practices (NREPP) to have focused on mental health promotion and treatment in early childhood, to have been funded by the National Institutes of Health, and have been evaluated in comparative effectiveness research studies. From there, we selected our sample based on the ratings from the California Evidence-Based Clearinghouse, as programs that have (a) no case data suggesting a risk of harm; (b) a well-defined treatment manual and strong empirical evidence demonstrating their ability to change parenting behaviors and reduce child behavior problems; and (c) demonstrated efficacy across a variety of populations and in multiple settings. Triple P is a continuum of parent support and training (Sanders, Markie-Dadds, & Turner, 2003), so we focus here on Level 4 Triple P, which is most comparable to the other interventions under consideration.

There are several types of intervention adaptation (e.g., adapting the training, the agency; Stirman, Miller, Toder, & Calloway, 2013). We focus here on cultural adaptation, defined as “the systematic modification of an evidence-based treatment (EBT) to consider language, culture, and context in such a way that it is compatible with the client's cultural patterns meanings and values” (Bernal, Jimenez-Chafey, & Domenech Rodríguez, 2009, p. 362). Cultural adaptation is an important part of the implementation process (Cabassa & Baumann, 2013), and considering the current diversity of the U.S. population, attention to how cultural factors affect the implementation of parent trainings in usual care is crucial.

Meta-analyses in the cultural adaptation field have indicated that adapting interventions to clients' cultural backgrounds by explicitly integrating cultural factors such as language, cultural beliefs, and explanatory models into the intervention improves the relevance, acceptability, effectiveness, and sustainability of the intervention by the providers and target populations (e.g., Benish, Quintana, & Wampold, 2011; Huey & Polo, 2008; Griner & Smith, 2006; Smith, Domenech Rodríguez, & Bernal, 2011). Care should be taken, however, as these meta-analyses reflect great variability in effect sizes, study designs, populations, and interventions sampled (Cabassa & Baumann, 2013).

There is no single, correct way to culturally adapt interventions and there is no rule that states that every EBT should be adapted (Domenech Rodríguez & Bernal, 2012). When considering whether to culturally adapt an intervention, one should carefully consider what evidence about the intervention is available (e.g., what information does the literature provide about the EBT?), the target population (e.g., who was the original target population? To whom will the intervention be delivered?), and what is the target domain of the intervention (e.g., changing parenting practices). Domenech Rodríguez and Bernal (2012) provide guidelines to support the decision of whether to adapt an intervention, which involve assessing whether (a) the EBT is accessible to the providers who will be delivering the intervention, (b) the underlying mechanism of change of the intervention is a good fit for the target population, and (c) the EBT is acceptable for the target population. If the decision is to culturally adapt an intervention, the next step is to decide which framework will guide the process.

There are two sets of cultural adaptation frameworks: those that inform modification to the content of the intervention and those that inform the process of adaptation (Ferrer-Wreder, Snudell, & Mansoory, 2012). One model that informs what to adapt in the delivery and content of the intervention is the Ecological Validity Model (EVM) by Bernal, Bonilla, and Bellido (1995). The EVM specifies eight domains: language, persons, metaphors, content, concepts, goals, methods, and context. Another content model is the cultural sensitivity model, which distinguishes deep versus surface adaptations (Resnicow, Soler, Braithwaite, Ahluwailia, & Butler, 2000).

The second set of frameworks focus on the process of adaptation, where decisions about when to adapt, how to adapt, and which stakeholders should be involved in the process are outlined. A number of frameworks fall into this category and vary in how prescriptive (i.e., have a set of a priori steps that guide the process) or specific (i.e., focused on the adaptation of one specific EBT) they are (Ferrer-Wreder et al., 2012). Several of them have been described elsewhere (Bernal & Domenech Rodríguez, 2012). Generally, these models recommend adaptations to be informed by the expertise of stakeholders, use formative research methods, and conduct formal evaluations of the adapted intervention (Cabassa & Baumann, 2013; Domenech Rodríguez & Bernal, 2012). It is important to assess the extent to which adaptations to parent training interventions have been guided by cultural adaptation frameworks in order to document the types of adaptation that are being conducted and examine how these adaptations impact intervention and implementation outcomes.

Implementation refers to the process of integrating the intervention within a setting (Rabin & Brownson, 2012). It extends efficacy and effectiveness research that focuses on discovering what works to understanding how the implementation works in specific contexts (Damschroder, Peikes, & Peterson, 2013). This paper focuses on empirical tests of strategies used to implement the aforementioned evidence-based parent training programs. We define an implementation strategy as a “systematic intervention process to adopt and integrate evidence-based health innovations into usual care” (Powell et al., 2012, p. 124). The literature reflects a wide range of different implementation strategies (Powell et al., 2012); yet, the evidence to support the use of specific implementation strategies in mental health and social service settings has lagged behind other fields such as nursing and medicine (Powell, Proctor, & Glass, 2013). The recent prioritization of implementation research by the National Institutes of Health (2013) and the Institute of Medicine (2009) will undoubtedly increase the number of empirical studies testing innovative approaches to implementation. Developing a robust evidence-base for specific implementation strategies and learning more about how they interact with contextual elements of the settings in which they are deployed will give implementers the tools they need to improve the quality of care in social service settings. In other words, the testing of implementation strategies will attempt to answer the question set forth by Asgary-Eden and Lee (2011): “So now we've picked an evidence-based program, what's next?” (p. 169).

Empirical tests of implementation strategies should be guided by available theories, conceptual models, and/or frameworks, as they can ensure that essential contextual and process elements related to implementation are not overlooked (Proctor, Powell, Baumann, Hamilton, & Santens, 2012; Tabak, Khoong, Chambers, & Brownson, 2012). Moreover, frameworks provide a systematic way of evaluating the interventions and facilitating replication of the implementation process in different settings. While there are over 60 dissemination and implementation frameworks and models (Tabak et al., 2012), evidence from healthcare suggests that theory is drastically underutilized in implementation studies (Colquhoun et al., 2013).

It is also critical that implementation studies evaluate implementation outcomes. Proctor and colleagues (Proctor et al., 2011; Proctor & Brownson, 2012) have suggested a taxonomy of implementation outcomes, including acceptability, adoption, appropriateness, cost, feasibility, fidelity, penetration, and sustainability. Evaluation of implementation outcomes can help investigators disentangle implementation effectiveness from treatment effectiveness and to know, for example, if an intervention failed because it was ineffective or it was implemented incorrectly (Proctor et al., 2011). Furthermore, assessing implementation outcomes may improve our understanding of which implementation strategies work best with given interventions, settings, and conditions.

Cultural adaptation and implementation processes are inextricably linked and both processes should be planned and documented to facilitate future replication (Cabassa & Baumann, 2013). The purpose of this article is to review and assess the literature to determine the extent to which studies of evidence-based parent training programs have (a) used cultural adaptation models to guide the adaptation process, (b) tested implementation strategies, and (c) evaluated implementation outcomes.

The database search yielded a total of 67 articles related to Triple P, 137 articles related to PCIT, 72 articles related to Incredible Years, and 62 articles related to PMTO. Additional articles were added from the authors' personal libraries, intervention websites, and intervention developer queries, yielding a total of 106 articles for Triple P, 192 articles for PCIT, 209 articles for IY, and 103 articles for PMTO.

The goals of this review were to determine the extent to which researchers and developers of evidence-based parent training programs (a) used cultural adaptation models, (b) tested implementation strategies, and (c) evaluated implementation outcomes. Our results were discouraging. Out of 610 published reports, only eight studies fit our criteria for cultural adaptation studies and two fit our criteria for implementation studies.

We used the Ecological Validity Model (EVM; Bernal et al., 1995) as a framework to code the cultural adaptation of content of the parent trainings as it provides researchers a comprehensive list of potential targets of cultural adaptation and identifies critical areas where cultural factors can play a role in psychosocial interventions. Another advantage of EVM is that it helps identify surface and deep level adaptations (Resnicow et al., 2000). Surface modifications, such as translation of the materials, increase the feasibility of the program. Deep modifications, such as changes in methods and content, enhance the program impact (Resnicow et al., 2000). Oftentimes the translation of the materials (surface modifications) could be sufficient to produce impact of the culturally adapted program but we advocate for these to be carefully thought and planned (Cabassa & Baumann, 2013).

There is, however, debate regarding deep adaptations. Some scholars advocate for a rationale for cultural adaptations prior to adapting EBTs given concerns regarding an absence of data that confirms that such adaptations are needed, as well as concerns regarding feasibility, cost, and other resources needed to properly test the effectiveness of adapted treatments (Domenech Rodríguez & Bernal, 2012). Others also advocate that making adaptations: (1) delays implementation/delivery of the intervention unnecessarily (while adaptations are made); (2) is based on a potentially false assumption that people will not benefit from an intervention; (2) results in delivering an untested form of an intervention (National Advisory Mental Health Council's Workgroup, 2010).

However, arguments have also been advanced on the need to assume that culture must be taken into account on the basis of ethical and professional standards on the premise that adaptations are not needed when they, in fact, could require researchers to conduct research that harms vulnerable (e.g., marginalized) individuals (Domenech Rodríguez & Bernal, 2012). Moreover, as expressed in the Surgeon's General report ([USDHHS], 2001), ‘culture counts’ in mental health care, as it shapes how people seek help and engage in health behaviors and shapes how providers communicate with clients and deliver services. Considering that the population of ethnic minorities in the U.S. is projected to be 50% of the U.S. population by 2060 (Colby & Ortman, 2015), and that service delivery in its current form has not been efficient in reaching those in need, a careful assessment of the methodologies currently being used to provide service delivery may be important so as to “shift” current thinking and promote greater reach to those in need (Kazdin & Blasé, 2011). Implementation science, with methodology that aims to promote the spread of EBTs can be a powerful resource for helping accelerate the uptake of evidence-based parent interventions in usual care.

One problem when deciding whether or not to adapt a parent training is that few clinical trials have included a sufficient number of ethnic minority families to permit generalization across cultures (McCabe et al., 2005), and the gap in utilization of EBTs by minority populations still exists (Institute of Medicine, 2009). Moreover, the field of cultural adaptation still needs to address questions such as “what (if any) are the adaptations necessary to achieve cultural relevance and treatment efficacy?” (italics added) and “What are the most relevant procedures that should be undertaken in any process of cultural adaptation?” (Parra Cardona et al., 2012, p.3). While the answers to these questions are not straightforward, one methodology to answer these questions is to use differential research designs. This would entail testing two differentially culturally adapted versions of an existing EBT to examine their efficacy, feasibility and cultural acceptability (Martinez & Eddy, 2005; Parra Cardona et al., 2012). An alternative way to address whether or not to adapt an intervention is to bridge the fields of cultural adaptation and implementation whereby the evaluation of whether, when, and how to adapt a parent training would be part of the implementation process (Cabassa & Baumann, 2013; Chambers, Glasgow, & Stange, 2013). To do so, one should carefully choose the frameworks that would guide the process.

Our review revealed that few scholars were using cultural adaptation frameworks. Cultural adaptation frameworks are important to promote effective implementation research as it helps maintain high fidelity and avoid decrements in intervention impact (Allen et al., 2012). We advocate for the explicit description of what has been adapted, why it was adapted, and how it was adapted. The documentation of the process is important to inform stakeholders on how to (a) identify the diversity of adaptations being made; (b) examine the impact of the intervention on different populations; (c) help provide technical assistance to community providers, and (d) scale up programs (Stirman et al., 2013).

While no IY study fit our criteria, we note that there is a large amount of work by Webster-Stratton and her team to implement IY with minority groups (Webster-Stratton, 2009; Reid, Webster-Stratton, & Beauchaine, 2001). Cultural adaptation efforts for multicultural groups may be difficult to implement as the very pluralism of the group creates challenges for specific tailoring efforts. Future research could provide clarity on what kinds of adaptations may be made in multicultural contexts to address the diversity within groups.

The surprisingly low number of studies (n = 2) that fit our criteria of implementation indicates that the field of parent training, to date, has focused primarily on determining the effectiveness of the interventions. That is not to say that investigators are not assessing implementation outcomes. For example, other types of articles described findings on fidelity (e.g., Sigmarsdóttir & Gudmundsdóttir, 2013) or cost effectiveness (e.g., Foster, Olchowski, & Webster-Stratton, 2004). It seems, however, that the field is poised to engage in more rigorous research involving comparative tests of implementation strategies and evaluating implementation outcomes.

Our review has several limitations. First, there was a discrepancy between the numbers or articles from database search and from our personal libraries and inquiries to treatment developers and their team. Some of the articles that were captured from our personal libraries and/or from indications from the treatment developers did not contain the names of the interventions in the titles, and one intervention had several articles on their website that were also indexed by database. This poses a challenge for database searches; thus, we call for scholars to carefully consider titles, key words, and abstracts when publishing study results.

Second, our criteria could have been too stringent considering current state of research on parenting intervention. For example, articles such as those using observational or case study designs were excluded. An analysis of the excluded articles may provide an interesting picture of the adaptation and implementation work being done in the field. For example, while Domenech Rodríguez et al. (2011) provide a detailed description of how they used the EVM model to adapt the PMTO intervention,; their study was excluded from our analyses because they did not include data from the randomized controlled trial. Third, we were also stringent on our selection of parent trainings. Future studies could assess whether other interventions would provide a different picture of the cultural adaptation and the implementation process. Finally, while our team reached consensus about articles to include in this review, others may have a different perspective about our selection criteria. In terms of treatment adaptations, we focused on cultural adaptation. However, we recognize that other types of adaptations may be necessary during implementation in order to fit the intervention to the setting, such as adaptions in context, training, and modifications at the agency level and system level (Stirman et al., 2013).

In summary, our data indicate that the field of prevention science is ripe for more studies documenting the content and process of cultural adaptations and empirically testing different approaches to implementation. As the Institute of Medicine report (2009) points out: much research is needed to identify essential aspects of implementation, especially when implementing interventions whose evidence may be “significantly affected or impeded by aspects of the ethnic, linguistic, and cultural environment in which they are implemented” (p.4). In short, efforts to implement evidence-based parent training programs must be complemented with evidence-based approaches to cultural adaptation if they are to be adopted, implemented, sustained, and scaled-up in community settings (Cabassa & Baumann, 2013).