The EBPH training is a 10-module course focused on building an individual’s skills to apply the principles of EBPH to their work (Figure 1). The typical in-person EBPH training lasts 3½ full days, whereas the virtual version has been delivered through 2-hour weekly sessions for 10 weeks. One site provided in-person EBPH training through half-day training sessions, once per week, for 7 consecutive weeks. Each module focuses on a different topic of the process by which programs and policies are implemented, from assessing the assets and needs of a community to evaluating a program or policy and disseminating findings. Some topics, such as health equity, are integrated within all modules; thus, even though they do not appear in the wheel, they are central to the training. The course was developed as an in-person training and modified to a virtual format during the COVID-19 pandemic. In the past, the course developers at Washington University in St. Louis trained public health staff directly, in a centralized manner. In an attempt to scale the training and make it available to more public health practitioners, train-the-trainer iterations of the course have been implemented over time, with the understanding that health department staff and their academic partners would take on the future delivery of the course; however, this model has been challenging to sustain due to the turnover in public health practice and the additional time and financial burden of implementing the course. In an effort to address these barriers, a new model was developed in 2021. This model of replication expands the delivery network by bringing in the complementary expertise of PRCs, health departments, and PHTCs.

SHDs oversee the public health operations within a state or region, leading a variety of initiatives related to disease surveillance, health promotion, emergency response, and policy and regulatory actions. Often, they serve as pass-through agencies to distribute funds to local health departments (LHDs) and community-based organizations that work to deliver programs and policies within a local area. Staff are typically well-connected within governmental health agency structures, including potential participants in the EBPH training. Also, staff have expertise in implementing public health programs and policies outside of research settings, ie, within the context of their state.

PHTCs are a network of “the nation’s most comprehensive resource for public health workforce development”¹⁴ who have the existing infrastructure to develop, deliver, and sustain trainings for public health professionals. Funded by the Health Resources and Services Administration, 10 regional training centers support different areas of the country by providing professional development to public health professionals, experiential learning to students via field placements, and consulting and technical assistance to community partners. They bring expertise in instructional design and adult learning, as well as administrative expertise in organizing in-person and virtual trainings.

PRCs are a network of 26 academic research centers across the United States that are funded by the Centers for Disease Control and Prevention to conduct community-based public health research to address chronic diseases and other leading causes of death and disability.¹⁵ Additionally, a central role of the PRCs is to provide capacity building opportunities to public health professionals and community members. Situated within academic institutions, PRC faculty have expertise in conducting high-quality community-engaged research and subject matter expertise in chronic and other diseases and health behaviors that are relevant to public health.

In this study, the Washington University team served as the coordinating center and recruited 3 other PRCs to participate, each bringing with them their PHTC and SHD colleagues. Health Department staff and PRC faculty from each site completed a train-the-trainer iteration of the course so there was a critical mass of facilitators for course replications. PRCs then worked with their health departments and PHTCs to recruit participants for each training. Participant recruitment was done through the existing relationships between the PRCs and the health departments. Once a key contact person was identified, eg, a Director of Chronic Diseases or of Training, the PRC staff worked with them as they reached out to our primary target audience—mid-level supervisory and managerial staff in chronic disease programs and other major program areas of the health department, as well as key partners. The PHTCs coordinated course enrollment, reproduction of course materials provided by Washington University, and served as course coordinators throughout the delivery of the course.

Course participants completed a pre- and postcourse survey measuring on a 5-point Likert scale their self-rating of 14 skills that comprise evidence-based decision making. We derived the survey items from previously administered surveys developed by the study team and practitioners using literature reviews and cognitive response testing with SHD practitioners.^7,16–20 Psychometric properties of these constructs (ie, test-retest reliability and Cronbach’s alpha) have been published elsewhere.²¹ Participants also reported the type of organization with which they were affiliated, eg, SHD and community-based organization.

We analyzed changes in pre- and postcourse individual skills using mixed effects linear models, accounting for within-person and within-state variation. We considered the participant’s organization type as a confounder, but its inclusion did not change the point estimates of the models. Thus, unadjusted models are presented, using an alpha of .05 to designate statistically significant changes.

We conducted in-depth interviews with PRC, PHTC, and SHD partners to understand the organizational-level factors that influence the success of implementing the EBPH course through this partnered model.

We developed an interview guide to understand: (1) what resources each partner contributed to implementing the EBPH course; (2) the barriers and facilitators to getting the partnership off the ground or its continuation; (3) the benefits of delivering EBPH training through partnerships between PRCs, SHDs/LHDs, and PHTCs, including other collaborative work the partnership has conducted or will conduct; and (4) what is needed to sustain the partnership over time (Supplemental Digital Content 1, available at http://links.lww.com/NE/B593). Then, we pilot-tested the interview guide with 2 former chronic disease directors who had implemented EBPH training and revised the guide based on their input to improve the clarity and specificity of the interview questions. By email, we invited key partners identified by each PRC to participate in an interview. Interviews took place after each site’s course offering. In total, we (S.M.-R. and C.A.B.) conducted 15 phone interviews and received written responses from 1 participant by email. Interviews lasted an average of 57 minutes (range: 40–74 minutes).

We made field notes after each call to guide the interpretation of interview coding. Data saturation was reached once no new information was provided by additional interviews, as decided by the research team. The 2 members of the team who conducted the interviews met after each interview to debrief about each interview. Toward the end of data collection, we reflected how close we were to saturation with a perspective of diminishing returns, that we were getting largely repetitive information after so many interviews (eg, no new barriers and facilitators were discussed).²² We conducted two additional interviews after we considered data saturation to be reached to confirm, and data collection ended after that. An online service (rev.com) was used to transcribe interviews verbatim, and we conducted a thematic analysis to identify major themes common across the interviews.²³ We used NVivo (version 11) and a content analysis approach to analyze the transcripts using a common codebook developed deductively based on the interview guide (Supplemental Digital Content 2, available at http://links.lww.com/JPHMP/B370). Two individuals trained in qualitative analysis (S.M.-R. and a research assistant) double-coded each transcript and synthesized coded comments into overall themes. We report findings by topic, using selected, anonymized quotes to illustrate key themes.

In the pre- and postcourse surveys, the highest-rated skills were learning about an issue through scientific literature (pre-course mean = 3.97, SD = 0.77) and describing health equity and its relevance to their work (pre-course mean = 3.97, SD = 0.68) (Figure 2). At both time points, the lowest-rated skill was understanding economic evaluation enough to use findings to inform decision-making (pre-course mean = 2.76, SD = 0.87). Statistically significant increases in all skills were observed from pre- to postcourse. The two skills with the largest increase were understanding economic evaluation enough to use findings to inform decision-making (mean change = 1.22, SE = 0.05) and developing an action plan for the program or policy (mean change = 1.07, SE = 0.07).

Illustrative quotes for each theme are found in Supplemental Digital Content 3, available at http://links.lww.com/JPHMP/B371.

Interviewees described several resources that were provided by the partner organizations: grant funding, a training course with proven effectiveness, credibility and expertise, faculty instructors for the course from the PRCs, and infrastructure for course delivery from the PHTCs. One PHTC staff member commented that “we’re the scaffolding . . . providing the structure . . . the technical side of it, the day-to-day management, as well as the coordination part. . ..” SHD staff contributed staff time and relationships with LHDs and thereby access to practitioners to recruit as course participants, demand for the course, recruitment coordination, and faculty instructors for the course who have on-the-ground experience.

The existing EBPH course materials and resources developed by the PRC at Washington University saved facilitators time on course planning (Table 2). Also, the funding provided for organizing and delivering the course was essential to pay for course materials, training spaces, accommodations for trainees (for in-person courses), and stipends for trainers, among other costs. While not a barrier specific to the collaborative delivery of the EBPH course, the COVID-19 pandemic created a great deal of uncertainty and slowed course planning, resulting in courses being delayed or rescheduled as public health practitioners continued to support COVID-19 mitigation efforts.

Specific to the partnership between PRCs, SHDs, and PHTCs, interviewees described some communication barriers that had varying degrees of impact. Some were small things such as not being kept in the loop about how the course went, whereas others noted bigger topics like progress on recruitment of course participants or timeliness of communication. It was also reported that the time demands of organizing the EBPH course are notably higher compared to other, short-term continuing education courses that are typically offered to public health practitioners.

Interviewees noted several benefits of delivering the EBPH course in partnership between the PRCs, PHTCs, and SHDs/LHDs. First, this approach leverages the unique strengths of each partner so that the EBPH course can be delivered efficiently and effectively. Interviewees discussed that tasks were assigned to the partner based on what each organization already does, eg, relying on the PRCs to present up-to-date, evidence-based information within the training, on the SHDs/LHDs to connect to trainees through their relationships with public health practitioners who can benefit from EBPH training, and on the PHTCs to manage the technical aspects of course delivery and apply strong instructional design approaches. This collaborative approach allows each partner to contribute according to their strengths and avoids unnecessary duplication of efforts. Additionally, the collaborative delivery of the EBPH course broadens the expertise of the PRC facilitation team to meet the training needs of participants and is an opportunity for the facilitators to learn from their academic or practice cofacilitators.

Interviewees also noted that collaborative delivery of the EBPH course can also have a positive influence on the reach and impact of the PRC/public health practice network, ie, the individuals and organizations in the region that are working to improve public health. Having a dedicated project to work on collaboratively allows PRCs, SHDs/LHDs, and PHTCs to create and strengthen connections among different sectors and regions, eg, among PRCs and practice organizations that are in their state or who work regionally with the training centers. With enough momentum, delivering EBPH training can create a community of public health researchers and practitioners with an evidence-based mindset that can shift the culture of public health to one of innovation and consistent application of evidence-based practices. Lastly, working together on the EBPH training offers SHDs/LHDs and PRCs an opportunity to move beyond the desire to collaborate to a tangible project and purpose for working together, which creates relationships that can lead to additional joint efforts between PRCs and SHDs/LHDs.

Interviewees shared several factors that would increase EBPH collaboration, as well as other academic-practice collaborations. Strong relationships are critical, as is stable funding to deliver the training, including funding to pay for dedicated staff to coordinate course planning, implementation, and evaluation and serve as liaisons to partnering agencies. More specifically related to funding, there was a discussion that funding has to be available at the right time to support collaborations beyond the EBPH course. Often, the funding for program implementation (led by the SHD) and evaluation (led by academics) of a program are funded by separate grant opportunities that do not align in their timing. Additionally, the lag time between applying for and receiving funding can mean that a program’s implementation may be completed by the time an evaluation grant is funded. Support from SHD/LHD leadership and EBPH champions throughout the public health network was also discussed; leaders normalize the expectation for building a strong culture of EBPH in their departments and champions spread information about the value of EBPH across their networks and serve as mentors and collaborators with their peers.