86103431792J Cancer EducJ Cancer EducJournal of cancer education : the official journal of the American Association for Cancer Education0885-81951543-015422528632342259610.1007/s13187-012-0348-xNIHMS374652ArticleAssessing Awareness and Use of Evidence-Based Programs for Cancer Control in Puerto RicoCaloWilliam A.JD, MPH1FernándezMaría E.PhD2RiveraMirzaMS3DíazElba C.DMD, MSD, MPH45Correa-FernándezVirmariePhD6PattatucciAngelaPhD3WetterDavid W.PhD6The University of Texas School of Public Health, Division of Management, Policy, and Community Health, Houston, TX, USAThe University of Texas School of Public Health, Center for Health Promotion and Prevention Research, Houston, TX, USAUniversity of Puerto Rico, Medical Sciences Campus, School of Public Health, Center for Evaluation and Sociomedical Research, San Juan, PR, USAUniversity of Puerto Rico, Medical Sciences Campus, School of Dental Medicine, San Juan, PR, USAUniversity of Puerto Rico Comprehensive Cancer Center, San Juan, PR, USAThe University of Texas MD Anderson Cancer Center, Department of Health Disparities Research, Houston, TX, USACorresponding author: William A. Calo, JD, MPH, Postdoctoral Fellow in Cancer Prevention and Control, The University of Texas School of Public Health, 7000 Fannin #2510G / Houston, TX 77030, Phone: (713) 500-9626 / Fax: (713) 500-9750, William.A.Calo@uth.tmc.edu1252012620120162013273486493

The Community Cancer Control Outreach Program (CCCOP) is a community-academic partnership aimed at developing and implementing a cancer control outreach, research, and training program in Puerto Rico. The CCCOP surveyed 56 partners to assess their awareness, training needs, and use of resources related to evidence-based programs (EBPs). Despite relatively high levels (70%) of confidence in adopting EBPs, there were low levels of awareness (37%) and use (25%) of existing EBPs resources. Respondents’ who had used EBPs resources were more likely to have positive beliefs about EBPs than nonusers (p<0.05). Training needs were high among respondents and no significant differences were found between those who had and had not used existing EBPs resources. These findings can guide the development of training tools and technical assistance to increase the use of EBPs for Latino audiences.

Evidence-based programsCancer controlDisseminationImplementationPuerto Rico
INTRODUCTION

Dissemination and implementation of evidence-based programs (EBPs) for cancer prevention and control is essential to achieve public health outcomes [12]. EBPs are available through web-based resources such as the Guide to Community Preventive Services, Cancer Control P.L.A.N.E.T., Research-tested Intervention Programs, and Using What Works, all of which provide users with free access to a variety of cancer control recommendations, programs, and materials [36]. Although cancer control planners are encouraged by their organizations and funders to use EBPs, several barriers hinder their widespread adoption and implementation [7]. Organization-level constraints, lack of knowledge about EBPs and where to find them, and limited training to effectively adapt EBPs are among the most common barriers to using EBPs in regular practice [811]. In addition, barriers faced at the community-level, such as failing in adapting existing EBPs to “real-world” conditions [9], low awareness of local preferences [12], and failure to use local data during adaptation efforts [13] impede the success of effective dissemination strategies.

The Puerto Rico Community Cancer Control Outreach Program (CCCOP) was funded by the National Cancer Institute to develop and implement a cancer control outreach, research, and training program guided by community-based participatory research methods [14]. The CCCOP was the first Island-wide cancer control community-academic partnership established in Puerto Rico and is led by the University of Puerto Rico Cancer Center and The University of Texas MD Anderson Cancer Center. The CCCOP consists of a Network of community members and cancer control planners from various areas around the Island. The purpose of the CCCOP is to help identify and prioritize cancer control issues in the community (e.g. low access to breast cancer screening, Pap test screening, HPV vaccination, and tobacco cessation programs) and develop strategies for addressing those problems. Among the goals identified by the Network was to increase the adoption of EBPs for cancer control and to adapt these programs to fit their community context and values. Because the CCCOP sought to increase the Network’s capacity to deliver effective and culturally appropriate EBPs, understanding the Network members’ knowledge and attitudes about finding, choosing, adapting, and implementing EBPs in their communities was necessary.

As such, the CCCOP surveyed Network partners to assess their awareness of and experience using EBPs resources, as well as any type of training that might be needed to increase the use of EBPs in communities. Thus, the study goal was to identify gaps in EBPs awareness, use, and capacity to inform the further development of EBPs training tools and technical assistance to accelerate and expand the use of EBPs in Puerto Rico.

METHODSParticipants

The Network’s cancer control partners included government agencies, healthcare delivery organizations, research and higher-education institutions, health-related organizations, and community-based organizations. Some partner organizations had more than one representative in the Network primarily because they had multiple areas of cancer focus within their organizations. All Network partner’s representatives were invited to complete the survey.

Measures

A survey developed by the Cancer Prevention and Control Research Network (CPCRN) was adapted for use in Puerto Rico. This survey measured cancer control planners’ awareness of and willingness to use EBPs [8]. This survey had previously been administered to CPCRN partner organizations in eight states and is available on the CPCRN web site (http://www.cpcrn.org/ebasurvey). Several of the original items were omitted (e.g. race/ethnicity of the population primarily served by the organization) and several additional items were added (e.g. participation in the Puerto Rico Comprehensive Cancer Control Plan) to better fit the contextual realities of cancer control in Puerto Rico. The survey was translated and back translated and decentering techniques were used [15] to develop the Spanish version of the survey. This revised version was sent to key CCCOP collaborators to obtain feedback on the clarity of the items and response options. After receiving comments, the CCCOP research team held several meetings to discuss the questionnaire items. Once the questionnaire was finalized, an on-line version was created. Items assessing the following key topics were included: 1) characteristics of the Network partner organization and their representatives, 2) awareness and use of existing web-based resources for EBPs, 3) organization’s current cancer control development processes, 4) perceived importance of program characteristics when choosing a program, 5) beliefs about the importance of using EBPs, and 6) perceived needs for training on how to find and use EBPs.

Procedures

The survey was conducted during February 2009 to January 2010. The CCCOP staff contacted Network partners by e-mail, telephone, and in-person. An initial e-mail was sent to all Network partners’ representatives that included: 1) an invitation to complete the survey, 2) information explaining the purpose of the survey, 3) information about how to access the survey on-line or to request a paper version, and 4) a statement about the confidentiality of their responses. Five weeks later, a second e-mail was sent and a follow-up call was made as a reminder to those representatives who had not yet responded. Paper copies of the survey were sent out by mail upon request. Partners were also encouraged to respond to the survey during the CCCOP face to face meetings held during the data collection time period. No monetary incentive was offered to complete the survey. The study was approved by the Institutional Review Board of the University of Puerto Rico, Medical Sciences Campus.

Data analysis

Descriptive statistics were used to describe partner organization and respondents’ characteristics, awareness and use of existing web-based resources for EBPs, organization’s current cancer control development processes, and beliefs about the importance of using EBPs, and perceived training needs related to EBPs. To assess whether previous experience using existing EBPs resources was associated with beliefs about EBPs, responses of respondents who had never used EBPs resources were compared to those who had used EBPs. We also assessed whether prior EBPs use was associated with 1) respondents’ perceived importance of program characteristics when choosing such programs, and 2) perceived training needs.

RESULTSRespondents’ characteristics

Table 1 summarizes the characteristics of the 56 Network representatives who volunteered to complete the survey. Overall, respondents were from a variety of organizations that provide health promotion and direct health services related to cancer control and prevention. Most of the respondents had access to the Internet at work (79%), reported confidence in their ability to adapt EBPs (70%), and about half reported participating in the Puerto Rico Comprehensive Cancer Control Plan.

Respondents’ awareness and use of EBPs resources

Respondents reported low levels of awareness and use of existing resources for EBPs (Figure 1). Overall, only 37% of respondents were aware of at least one of the two EBPs resources surveyed, and only 25% had used at least one of them. Respondents reported almost the same level of awareness and use for the two web-based EBPs resources (the Guide to Community Preventive Services and Cancer Control P.L.A.N.E.T.).

Reasons for choosing cancer control programs

Approximately 60% of respondents reported having developed a new program in the past 12 months and using a program developed by someone else; less than a half reported adapting a cancer control program (Table 2). Among organizations who had used or adapted existing programs, less than half (42.9%) reported that scientific evidence of effectiveness was a reason for their choice.

Perceived importance of program characteristics

Table 3 compares EBPs resource users and non-users on their perceived importance of program characteristics when choosing a cancer control program. Programs’ cost-effectiveness was perceived as the most important characteristic among EBPS users but not among non-users. Respondents’ who had used any EBPs resources rated as less important the characteristics of program innovation and consistency with organization’s mission (p<0.05) when compared to EBPs non-users.

Respondents’ beliefs about EBPs

Compared to respondents who had not used EBPs resources, respondents who had used EBPs reported greater agreement with statements indicating that EBPs are “easy to implement” and “easy to find and get” (p<0.05) (Table 4). However, EBPs users were also more likely to agree with the statement that “EBPs don’t come with very much information about how to implement them” than were those respondents who had not used any EBPs (p<0.05).

Perceived training needs

Table 5 shows that perceived training needs for implementing EBPs were high among respondents. There were no significant differences in perceived training needs between respondents who had and had not used existing EBPs resources.

DISCUSSION

This is the first study to assess awareness of and experience using EBPs among cancer control planners in Puerto Rico. The study yielded several important findings. First, despite relatively high levels of confidence in adapting EBPs, there were low levels of awareness and use of existing resources for EBPs. Second, respondents’ prior use of EBPs resourceswas associated with several positive beliefs about EBPs and with their ratings for several program characteristics when choosing cancer control programs. Third, although prior use of EBPs resources was not associated with the need for any given training topic, perceived training needs across topics were high among all respondents. These findings underscore the importance of providing such training to address cancer control planners’ needs for finding, adapting, and implementing EBPs in communities in PR.

The high levels of confidence in adapting a program to fit local needs reported in this study could be the result of a misconception that adaptation simply means the translation of materials from English to Spanish rather than a systematic process using information from the community and available needs assessment data [16]. Additionally, since there were low levels of awareness and use of existing EBAs, it is likely that most respondents had never attempted to adapt existing EBPs in the past. The levels of awareness and use of EBPs resources found in this study were much lower (by almost half) than the levels reported by Hannon and colleagues [8] in a sample of 240 CPCRN cancer control planners across the United States. This does not seem to be due to access to these resources since most of the respondents had Internet access at work to search for EBPs web-based resources. Respondents’ positive beliefs about EBPs were also quite low as compared to the CPCRN study [8]; therefore, it might explain the low use of EBPs.

Among those respondents who had adopted or adapted a program in the last year, scientific evidence saying that the program works was the most cited reason for choosing such programs. Nonetheless, among all respondents (EBPs users and non-users) the percentage of who noted that scientific evidence of effectiveness was an important consideration was much lower than the percentage who endorsed this characteristic among CPCRN cancer control planners [8]. This finding could suggest the lack of awareness among CCCOP partners that using EBPs is a best practice. Another important finding is that several factors reported in the literature [8, 1718] as important reasons for selecting programs, such as fitting into the organization’s budget, ease of implementation, and pressure from funder agencies were not highly mentioned among respondents when choosing programs. Thus, assessing all these factors before adapting cancer control programs is essential for achieving successful implementation [19].

Compared to EBPs users, non-users placed a higher value on program innovation and compatibility with the organization’s mission when choosing cancer control programs. Noonan and colleagues [20] have suggested that EBPs must be disseminated as complementary to an organization existing practices rather than competing products. As expected, and consistent with findings from the CPCRN study [8], respondents who had used any EBPs resource reported greater agreement with the statements that they know where to find EBPs and that EBPs are easy to find and get. In addition, compared to non-users, EBPs users were also more likely to agree with the statement that EBPs don’t come with very much information about how to implement them. These findings suggest that to facilitate the adoption and implementation of EBPs in communities, planners need materials such as implementation protocols or manuals that describe key components of the program and how it should be delivered [8].

Consistent with data from the CPCRN study [8], prior use of EBPs resources was not associated with endorsing the need for any given training topic. More than half of all respondents indicated a need for training in nine out of the ten topics assessed in the survey. Most respondents indicated that their organizations need training on identifying what program aspects can and cannot be changed and how to adapt programs and materials for cultural appropriateness. Implementing EBPs beyond the communities and contexts for which they were originally developed presents a major challenge that includes balancing program fidelity and adaptation to improve fit [21]. EBPs are often poorly adapted or implemented without their essentials components; this can weaken the program and compromise effectiveness [22]. Training programs aimed at supporting cancer control planners should address the issues of fidelity and adaptation. Intervention Mapping has been proposed as a tool to identify what program components must be retained and what of its elements can be adapted to better fit new settings and populations [21].

Current research on dissemination and implementation of EBPs has suggested the need for interactive training programs and technical assistance to guide cancer control planners through the process of finding, choosing, adapting, implementing, and evaluating EBPs [8]. Based on the findings from this study, such training should include messages to emphasize the importance of choosing programs backed by scientific evidence, increase positive attitudes toward EBPs, and address intervention and organizational characteristics when choosing EBPs. In addition, because successful dissemination of EBPs does not typically result from passive diffusion of programs into routine practice [9], training programs can increase the demand for EBPs [23].

Finally, research has shown a widening gap between the skills necessary to effectively deliver cancer control interventions and the current set of skills that public health practitioners have [2425]. Any training program aimed at increasing EBPs dissemination and implementation must be tailored to assess users’ competences and skills. The CPCRN is in the process of developing an interactive tailored decision support and technical assistance tool to increase the capacity of both individuals and organizations for adopting and implementing EBPs [26].

An important strength of this study is that it captured a broad and diverse community-based perspectives related to cancer-related EBPs use in Puerto Rico. The findings from this study can help the cancer control movement in Puerto Rico develop training tools and to offer technical assistance to accelerate and expand the use of EBPs. A limitation of this study is that the sample size was too small to perform additional statistical analysis for some variables such as organizational and representative characteristics. Another limitation is that the findings are based on self-reported data. Due to the relationship of the partner organizations with the CCCOP Network; there is a possibility that current awareness and use of, and beliefs about EBPs were overestimated if respondents were influenced by social desirability [8].

Although the survey did not measure the influence of language as a potential barrier for accessing EBPs resources, since the majority of these resources are in English, Spanish-speaking cancer control planners may face additional challenges in identifying and adapting EBPs. Thus, existing and new resources for promoting EBPs must be developed in different languages to increase their penetration and acceptance among communities whose primary language is not English such as Puerto Ricans.

CONCLUSIONS

An understanding of current awareness and use of EBPs among cancer control planners in Puerto Rico is necessary to accelerate the use of effective cancer control programs in PR. Although current awareness and use of EBPs was low, respondents had some positive beliefs for EBPs and expressed the need for training. These results underscore the need for a robust training program and the offering of technical assistance to enhance the capacity of both individuals and organizations to successfully adopt and implement cancer control EBPs in their communities.

We thank the CCCOP Network for its collaboration throughout all the phases of this study. Research for this publication was supported by the National Cancer Institute and the Center for Disease Control and Prevention cooperative agreements for the Cancer Prevention and Control Research Networks at The University of Texas School of Public Health (1-U48-DP-000057), the University of Puerto Rico Cancer Center / MD Anderson Cancer Center, Partners for Excellence in Cancer Research (U54CA96297), and The University of Texas School of Public Health, Cancer Education and Career Development Program (2R25-CA057712).

KernerJRimerBEmmonsK2005Introduction to the special section on dissemination: dissemination research and research dissemination: how can we close the gap?Health Psychol2444344616162037BowenDJSorensenGWeinerBJCampbellMEmmonsKMelvinC2009Dissemination research in cancer control: where are we and where should we go?Cancer Causes Control2047348519224380Guide to Community Preventive Serviceswww.thecommunityguide.orgAccessed 3 January 2012Cancer Control P.L.A.N.E.TLinks to Comprehensive Cancer Control Resources for Public Health Professionalshttp://cancercontrolplanet.cancer.gov/index.htmlAccessed 3 January 2012Research-Tested Intervention Programs (R-TIPs)http://rtips.cancer.gov/rtips/index.doAccessed 3 January 2012Using What Works: Adapting Evidence-based Programs to Fit Your Needshttp://cancercontrol.cancer.gov/use_what_works/start.htmAccessed 3 January 2012ChinmanMHunterSBEbenerPPaddockSMStillmanLImmPWandersmanA2008The getting to outcomes demonstration and evaluation: an illustration of the prevention support systemAm J Community Psychol4120622418278551HannonPAFernandezMEWilliamsRSMullenPDEscofferyCKreuterMWPfeifferDKeglerMCReeseLMistryRBowenDJ2008Cancer control planners’ perceptions and use of evidence-based programsJ Public Health Manag Pract16E1E820357600GlasgowREMarcusACBullSSWilsonKM2004Disseminating effective cancer screening interventionsCancer1011239125015316911DreisingerMLeetTLBakerEAGillespieKNHaasBBrownsonRC2008Improving the public health workforce: evaluation of a training course to enhance evidence-based decision makingJ Public Health Manag Pract1413814318287919BrownsonRCBallewPKitturNDElliottMBHaire-JoshuDKrebillHKreuterMW2009Developing competencies for training practitioners in evidence-based cancer controlJ Cancer Educ2418619319526405AndersonLMBrownsonRCFulliloveMTTeutschSMNovickLFFieldingJLandGH2000Evidence-based public health policy and practice: promises and limitsAm J Prev Med18226230LauA2006Making the case for selective and directed cultural adaptations of evidence-based: Treatments examples from parent ratingClinical Psychology-Science and Practice13295310Puerto Rico Community Cancer Control Outreach Programhttp://www.mdanderson.org/education-and-research/education-and-training/schools-and-programs/research-training/programs-and-courses/partnership-for-excellence-in-cancer-research/outreach-program/index.htmlAccessed 3 January 2012PrietoAJ1992A Method for Translation of Instruments to Other LanguagesAdult Educ Quarterly43114BartholomewKParcelGSKokGGottliebNHFernándezMEBartholomewKParcelGSKokGGottliebNHFernándezME2011Intervention Mapping Step 1: Need AssessmentPlanning Health Promotion Programs: An Intervention Mapping Approach3Jossey-BassSan Francisco, CAGreenhalghTRobertGMacfarlaneFBatePKyriakidouO2004Diffusion of innovations in service organizations: systematic review and recommendationsMilbank Q8258162915595944MihalicSFIrwinK2003Blueprints for violence prevention: from research to real-world settings-factors influencing the successful replication of model programsYouth Violence Juvenile Justice1307329WandersmanADuffyJFlaspohlerPNoonanRLubellKStillmanLBlachmanMDunvilleRSaulJ2008Bridging the gap between prevention research and practice: the interactive systems framework for dissemination and implementationAm J Community Psychol417118118302018NoonanRKEmshoffJGMoossAArmstrongMWeinbergJBallB2009Adoption, adaptation, and fidelity of implementation of sexual violence prevention programsHealth Promot Pract105970LeerlooijerJJamesSReindersJMullenPDBartholomewKParcelGSKokGGottliebNHFernándezME2011Using Intervention Mapping to Adapt Evidence-Based Programs to New Settings and PopulationsPlanning Health Promotion Programs: An Intervention Mapping Approach3Jossey-BassSan Francisco, CABackerTE2001Finding the balance: Program fidelity and adaptation in substance abuse prevention: A state of the art reviewDepartment of Health and Human Services, Substances Abuse and Mental Health Services Administration, Center for Substance Abuse PreventionRockville, MDOrleansCT2007Increasing the demand for and use of effective smoking-cessation treatments reaping the full health benefits of tobacco-control science and policy gains--in our lifetimeAm J Prev Med33S340S34818021909AllegranteJPMoonRWAuldMEGebbieKM2001Continuing-education needs of the currently employed public health education workforceAm J Public Health911230123411499109BrownsonRCBrightFS2004Chronic disease control in public health practice: looking back and moving forwardPublic Health Rep11923023815158101KernerJElwoodJMSutcliffeSB2010Integrating science with service in cancer control: closing the gap between discovery and deliveryCancer ControlOxford University PressOxford, UK

Respondents’ awareness and use of EBPs resources

Partner organization and respondent characteristics

CharacteristicsFrequency (N)%
Partner organization type
 Charitable organizations2239.3
 Government agency1425.0
 Healthcare delivery and for profit organizations916.0
 Universities or research center814.3
 Community/state coalitions and others35.4
Respondents’ position in partner organization
 Program planner or manager1730.4
 Healthcare provider, non-physician1221.4
 Health educator1221.4
 Physician610.7
 Researcher or program evaluator35.4
 Other610.7
Health promotion offered on
 Tobacco prevention and control4376.8
 Breast cancer screening2850.0
 Pap test screening2137.5
 HPV vaccination1933.9
Health services offered on
 Tobacco prevention and control1832.1
 Breast cancer screening2341.1
 Pap test screening1628.6
 HPV vaccination1017.9
Access to internet at work
 Yes4478.6
 No1221.4
Participate in the Comprehensive Cancer Control Plan
 Yes2648.1
 No3051.9
Confidence level to adopt EBPs
 Confident3969.6
 Neutral59.0
 Not confident1221.4

Respondents could select more than one health promotion and service activity, so percentages sum to more than 100%

Characteristics of the cancer control programs chosen

Frequency (N)%
Sources of cancer control programs in past 12 monthsa
 Respondent developed own program3562.5
 Respondent used a program someone else developed3460.7
 Respondent adapted a program that someone else developed2442.9
Reasons for choosing cancer control programs adopted in past 12 monthsb
 There was scientific evidence saying the program works1542.9
 We had used it (or something like it) before1131.4
 We felt it was better than the alternatives1028.6
 Our funding agency encouraged us to use this program822.9
 People in our community requested this type of program822.9
 The program fit our budget720.0
 It was available for free or low cost720.0
 It was easy to implement822.9
 Other organizations like ours are using this program720.7
 Technical assistance was available to help us with this program411.4
 We did not know of any alternatives38.6

N=56; respondents could select more than one source of cancer control programs, so percentages sum to more than 100%

Analysis restricted to respondents who had used or adapted a program someone else developed (combined n=35). Respondents could select more than one reason for choosing a program, so percentages sum to more than 100%

Respondents’ perceived importance of program characteristics when choosing them

The Program isUsed any resource, Mean (SD)Never used resources, Mean (SD)
Cost-effective4.2 (0.7)4.4 (0.8)
Available for free4.2 (1.0)3.8 (1.2)
Addresses our organization’s needs4.1 (1.4)4.5 (0.8)
Easy to use4.1 (1.1)3.9 (1.0)
Consistent with our organization’s image4.0 (1.4)4.5 (0.8)
Technical assistance available4.0 (1.4)3.8 (1.2)
Used in population like ours3.9 (1.3)4.2 (1.0)
Consistent with our organization’s mission3.8 (1.7)4.6 (0.8)*
Easy to evaluate3.8 (1.4)3.9 (1.1)
Other organizations are using it3.5 (1.5)3.4 (1.2)
Innovative3.3 (1.7)4.2 (0.9)*

Items were rated on a 5-point scale from 1 = not at all important to 5 = extremely important

p < 0.05

Respondents’ beliefs about EBPs

Beliefs about EBPsUsed any resource, Mean (SD)Never used resources, Mean (SD)
I know where to find EBPs3.7 (1.1)2.7 (1.7)
The research that shows that an EBPs works is reassuring3.6 (1.2)2.7 (1.9)
Our funding agency encourages us to use EBPs3.5 (1.7)2.3 (2.0)
EBPs are easy for us to adapt for use in our community3.4 (1.0)2.6 (1.6)
EBPs are easy to implement3.2 (0.9)2.1 (1.5)*
EBPs are easy to find and get3.0 (1.2)2.0 (1.5)*
People in our community have more confident in a program that has worked somewhere else2.8 (1.2)3.1 (1.6)
EBPs don’t come with very much information about how to implement them2.7 (0.9)1.8 (1.4)*
People in our community would not respond well to an EBPs developed somewhere else2.6 (1.0)2.2 (1.6)
EBPs require more resources than other programs2.4 (1.1)2.3 (1.6)
EBPs are too costly2.4 (1.2)2.1 (1.6)
Considering the time it takes to adapt an EBPs for our service population, we might as well develop our program2.2 (1.0)2.0 (1.4)
EBPs will not work better than what we are doing already2.1 (0.9)1.5 (1.3)
EBPs lack real world evidence2.0 (1.4)1.5 (1.2)
Using an EBPs keeps our organization form getting the credit we could get for a new program1.6 (1.0)1.8 (1.4)
Scientists don’t agree about what is evidence-based1.6 (1.0)1.7 (1.2)

Items were rated on a 5-point scale from 1 = strongly disagree to 5 = strongly agree

p < 0.05

Perceived training needs for implementing EBPs

Training needsaUsed any resource, N=14 (%)Never used resources, N=42 (%)
How to assess and utilize current available resources10 (71.4)31 (73.8)
How to obtain program materials9 (64.3)28 (66.7)
How to find and secure additional resources10 (71.4)25 (59.5)
How to identify what program aspect can and cannot be changed8 (57.1)27 (64.3)
How to develop an implementation and evaluation plan7 (50.0)27 (64.3)
How to involve other stakeholders/partners9 (64.3)25 (59.5)
How to adapt a program/materials for cultural appropriateness8 (57.1)26 (61.9)
How to implement and evaluate a program8 (57.1)27 (64.3)
How to pilot test a program with the intended audience8 (57.1)25 (59.5)
How to recruit participants7 (50.0)16 (38.0)

Respondents could select more than one training need, so percentages sum to more than 100%