The changing landscape of public health demands a workforce that is well equipped to apply evidence-based principles and processes. This evidence-based public health (EBPH) approach requires applying the best science, engaging the community in prioritizing their needs, and evaluating each step.^1–4 With myriad paths for individuals to join the public health workforce,⁵ it is important that they have a shared understanding of EBPH. Recognizing the diversity of backgrounds among the workforce and the needs for training in evidence-based approaches, an EBPH course was originally developed in 1997 in Missouri for state health department staff.² The course continued to evolve and expand its reach to include more U.S. states, territories, and countries and has been successfully scaled up through train-the-trainer efforts.^3,6,7 The course has been held internationally 20 times since 2002. From 2002 to 2018, courses hosted in Austria reached people from 13 European and Eastern European countries. Since its inception, >3,000 public health professionals have been trained from all 50 U.S. states, 2 U.S. territories, and >8 countries across 4 continents.

The EBPH course has been extensively evaluated. With in-person presurveys and postsurveys, longer-term follow-up efforts, qualitative interviews with participants, and quasiexperimental efforts that included presurveys and postsurveys with comparison (or control) groups, evaluation has been robust and has shown the course’s effectiveness.^6–12 For example, significant pre–post increases were shown in all perceived skills, with the highest increase in understanding economic evaluation.⁶ In 2 studies, training participants were compared with control groups and were found to have significant improvements (pre–post) in several competencies for evidence-based decision making (e.g., evaluation, communicating with policymakers).^10,12 Qualitative evaluation highlighted the benefits of the course in providing a common base of knowledge for those with a formal public health training background as well as for those who are newer to the field or have varying educational backgrounds.¹¹ As is common for sponsored projects with limited funding periods, the various evaluations have been segmented to certain years and participants. Although this allows for real-time evaluation as the course evolves, it is challenging to examine the overall benefits and challenges of training efforts among subgroups of participants with small sample sizes.

Using data from EBPH trainings collected over a 15-year period, the goal of the current evaluation is to describe the benefits, application, and barriers to applying course content across settings and time periods. With pooled data, there is an ability to examine course evaluation data across important subgroups to inform future workforce development efforts.

Among U.S. participants (n=583), the largest group was from state health departments (48.2%), followed by the group from local health departments (LHDs) (37.3%) and those from other organizations (community-based organizations, coalitions) (14.5%). For European participants (n=140), 49.2% were from other organizations (i.e., community health organizations, voluntary health organizations, medical facilities, universities), 34.8% were from federal/governmental health agencies, and 15.9% were from local/regional health agencies. Across all participants (N=723), the average number of years worked in the public health field was 11.3 (SD=8.2) years with a mean range of 9.5–12.8 years across the 5 surveys (Table 1). The overall proportion of those with a public health degree was 22.1% (ranging from 19.4% to 31.4%).

The most common benefits reported (agree or strongly agree) from participants were acquiring knowledge about a new subject (85.2%), identifying ways to apply this knowledge in their work (82.9%), becoming a better leader who promoted evidence-based decision making, and making scientifically informed decisions at work (81.4%) (Figure 1). The benefits score, calculated as the mean of 12 benefit scale items (α=0.91), had an overall mean of 3.71 (SD=0.62, range=1–5). On average, European participants reported higher benefit from the course than U.S. participants (mean difference=0.12, 95% CI=0.00, 0.23) (Table 2). No significant differences were found in benefit scores across course year groups, across groups with or without public health degrees, or across the length of years working in the public health field. Among U.S. participants, benefit scores were similar across position types. For European participants, benefit scores varied across position type (F[3,3.85]=4.09, p=0.008). European specialists reported fewer benefits than both executives (mean difference= −0.50, 95% CI= −0.92, −0.09) and program managers (mean difference= −0.45, 95% CI= −0.88, −0.03). There were no differences found in benefit score and organization type for both U.S. and European participants.

The course content was most frequently applied (at least quarterly) when searching the scientific literature (72.9%), when modifying a program or policy (66.2%), when planning a program or policy (65.3%), and when evaluating a program or policy (64.9%) (Figure 2). The frequency score for course application, calculated as the mean of 6 course application items (α=0.83), had an overall mean of 1.90 (SD=0.62, range=1–4). On average, European participants reported a higher frequency of course application than the U.S. participants (mean difference=0.17, 95% CI=0.06, 0.28). No significant differences were found in the frequency of application scores across course year, degree, or years in public health groups. No differences were found in application scores or position type among U.S. participants. For European participants, application scores varied significantly across position types (F[3,3.70]=3.73, p=0.013) and were higher for executives than for both specialists (mean difference=0.46, 95% CI=0.03, 0.88) and other positions (mean difference=0.38, 95% CI=0.03, 0.74). For U.S. participants, application varied across organizations (F[2,4.36]=5.89, p=0.003). State and LHD staff reported less application than those from other organizations (mean difference= −0.20, 95% CI= −0.38, −0.02 and mean difference= −0.27, 95% CI= −0.46, −0.09, respectively). For European participants, application scores were found to vary significantly across organization types (F[2,2.13]=3.17, p=0.045), although no significant post hoc comparisons were found.

The most common barriers reported (agree or strongly agree) to applying course content were lack of funding for continued training in EBPH (35.3%), not enough time to implement EBPH approaches (33.8%), and lack of other colleagues with EBPH training (33.1%) (Figure 1). The barriers score, calculated as the mean of 8 barrier scale items (α=0.79), had an overall mean of 2.49 (SD=0.65, range=1–5). No significant differences in barrier scores were found between U.S. and European participants. Participants from later cohorts (2012–2019) reported significantly higher barrier scores than those from earlier cohorts (2005–2011) (mean difference=0.15, 95% CI=0.05, 0.24). The more frequently reported barriers over time included not enough time and too much information to process (Appendix Table 2, available online). No significant differences were found in barrier scores among groups with or without public health degrees or length of years working in the public health field. No significant differences were found in barrier scores or position type among both U.S. and European participants. For U.S. participants, barrier scores were significantly different among organization types (F[2,5.34]=6.28, p=0.002), with LHD staff reporting greater barriers than both state health staff (mean difference=0.18, 95% CI=0.04, 0.33) and other organizations (mean difference=0.25, 95% CI=0.04, 0.45). Barrier scores did not differ significantly among organization types for European participants.

Participants reported many benefits from the EBPH course over the 15-year evaluation period. Learning a new subject and identifying ways to apply the information were the commonly reported benefits, which is encouraging given that only 14% of U.S. health department staff at all levels have formal public health training.⁵ The current sample showed that between 19% and 31% of respondents held public health degrees. No differences were shown in overall benefits from the course among those newer to the public health field or those with no formal training versus seasoned public health professionals with formal training, suggesting relevance across experience.

The most frequently applied EBPH concept is searching the scientific literature. Using online databases for searching for the latest evidence crosses several of the course modules, beginning with the planning phase of a program or policy. The current results suggest that for some organizations, course concepts may be less frequently applied overall, which may be due in part to the diversity of public health programming functions performed by different organizations. For the European participants, local or more regional-based public health organizations reported higher application scores, whereas in the U.S., LHDs reported less frequent application of course concepts. U.S. LHDs often cover smaller jurisdictions with less funding and rely on state resources for programming functions.^14,15 Participants from U.S. LHDs also reported higher barriers overall than those from state health departments and other organizations. Nonetheless, U.S. LHD participants rated overall benefits similar to those rated by participants from other organizations.

The smaller proportions of participants who reported barriers to applying course content (compared with the proportion reporting benefits) is encouraging. However, one of the most common barriers reported by more than one third of respondents was not having enough funding for continued training. Previous studies have established the importance of continued training in EBPH to maintain evidence-based processes within health departments.¹⁶ The challenge of keeping trained employees with limited funds and inevitable turnover emphasizes another commonly reported barrier of having too few coworkers trained in EBPH.¹⁷

In general, later cohorts reported more barriers to applying the course concepts in their work. In the U.S., the size of the public health workforce and its funding has decreased over the past several years.^18–20 For LHDs, there was an estimated 22% decrease in workforce size from 2008 to 2016 after the recession.^14,15 With fewer workers to do the work, less time may be available for public health professionals to integrate evidence-based processes into their usual work flow, a barrier commonly reported in this study and elsewhere.^8,21 Even finding the time to attend an in-person training over consecutive days can be challenging.

In light of barriers identified in this evaluation and in related literature,^3,22 there are several lessons to inform future workforce capacity-building efforts. First, to overcome the cost barrier, virtual options can more efficiently use resources and increase accessibility to participants.⁷ With the growth in the science of online education,²³ virtual versions of the training will be increasingly available. In recent years, virtual and hybrid models of the EBPH training have been developed and have shown training effects similar to those of more traditional, in-person formats.^7,22,24 For example, in June 2020, public health partners in Connecticut adapted the in-person EBPH course to an online version offered weekly in 2-hour blocks over 10 weeks. A second major barrier is the lack of time to apply EBPH concepts. To address time constraints, organizational leaders can prioritize workforce development.^25,26 In doing so, they can set aside protected time for staff to re-engage with content informally (e.g., searching the literature) and formally (e.g., monthly learning sessions with staff members).²⁷ Third, the lack of trained colleagues is a barrier. Leaders need to continually stress the importance of life-long learning and the need to maintain a critical mass and a social network in support of applying EBPH concepts.^26,28 Finally, the barriers and context for applying EBPH concepts differ in every public health agency. To further the application of key principles and allow tailoring to local priorities,²⁹ active brainstorming sessions are conducted at the end of each EBPH training to explore ways to tailor and apply evidence-based processes and embed them within participants’ organizational units.

To the authors’ knowledge, this is the first long-term evaluation of EBPH training—it provides an overview of the benefits, application, and barriers to the application of the EBPH course concepts to public health practice. The EBPH course represents 1 important strategy for increasing the capacity (individual skills) for evidence-based processes within public health practice. Training programs such as the EBPH course cannot be treated as one-time events—they should be funded, integrated into ongoing workforce development efforts, and tailored to local priorities. In addition, organization-level approaches (e.g., supportive climate and culture, leadership training, funding for training, staff to coordinate courses) are needed to scale up and sustain capacity-building efforts.^22,30,31