This is a multi-phase dissemination study funded by the NCI to learn which dissemination strategies best support uptake and application of EBDM processes among state health department practitioners and their key partners that work in cancer and other chronic disease prevention program areas. This multi-phase study is guided by an international advisory group of university-based researchers, former public health practitioners from state health departments, as well as collaborators from Canada with experience in dissemination research. The two study phases overlap and complement each other. Phase one involves development and testing of a self-report survey and archival report abstraction instrument. Phase one also includes collection of self-report data from a national representative sample of state health department practitioners working in chronic disease prevention. Phase two is a group randomized evaluation study. Phase two involves stratified random selection of six dissemination and six pair-matched comparison state health departments to test acceptance and usefulness of the identified dissemination strategies to state-level practitioners working in chronic disease prevention. Dissemination strategies may include training in evidence-based public health, technical assistance, and provision of brief user-friendly evidence summaries. Study collaborators include the National Association of Chronic Disease Directors (NACDD) and the Centers for Disease Control and Prevention (CDC) Division of Cancer Prevention and Control, with collaboration with other units at CDC as well. The study was approved by the institutional review board of Washington University in St. Louis. Some aspects of phase one have been completed, while phase two is in the planning phase. Phase two is registered as a cluster randomized trial (clinicaltrials.gov NCT01978054).

The dissemination conceptual framework for the study is depicted in Figure  1. It is adapted with permission from Kramer and Cole’s [36]. Conceptual Framework for Research Knowledge Transfer and Utilization [37,38]. The study model is also informed by Diffusion of Innovations [39] and Institutional Theory [40-42]. Dissemination in this study is the process of enhancing the capacity of the target audience of state-level practitioners to apply EBDM processes to promote statewide and local planning, adaptation, implementation, and evaluation of specific EBPPs for chronic disease prevention [43]. The workplace context is hypothesized as a key determinant of how knowledge is received, used, and incorporated into the organization’s usual day-to-day operations. In knowledge transfer and knowledge exchange, there is a flow of information that affects not only the target audience of practitioners but also the researchers. Researchers use the terms knowledge translation and exchange or knowledge exchange to denote an interactive process in which practitioners and researchers together problem solve how to apply research knowledge in specific contexts [36,37,44-48]. As in this study’s framework (Figure  1), some researchers make a distinction between knowledge transfer that is largely unidirectional from researcher to practitioner and knowledge transfer and exchange that involves a social interactive process dependent on the quality of researcher-practitioner relationships [45,46]. In this study, researchers will learn from practitioners about key contexts that affect application of research knowledge, such as organizational climate and political influences. Practitioners will learn knowledge, skills, and evidence-based organizational practices from researchers. Skill development may address common public health workforce skill gaps, such as use of data including economic data for planning and evaluation, interpretation of intervention research findings, collaboration across disciplines for environmental changes, communication of evidence to policy-makers, and documentation of use of evidence-based approaches [49]. Together, practitioners and researchers will determine how best to enhance modifiable contextual elements to support evidence-based state level chronic disease prevention.

The study team will work in partnership with state-level practitioners from the dissemination states to develop user-friendly evidence materials and EBDM trainings; assess organizational and other factors that influence acceptance of EBDM; support state-level practitioner application of EBDM processes to enhance evidence-based chronic disease prevention; and design strategies to embed EBDM processes within ongoing practices. Evaluation will include process evaluation in dissemination states, and pre-post evaluation in dissemination and comparison states [38,43]. (See Figure  2).

The target audience is state health department practitioners working in comprehensive cancer prevention and control, cancer screening, tobacco control, physical activity, nutrition, obesity prevention, school health, diabetes prevention, and cardiovascular health. State health departments typically provide funding, informational resources, and guidance for the implementation of EBPPs by state and local coalitions, local public health departments, and other agencies more than directly implementing policies and programs. The study involves the provision of several EBDM dissemination strategies, identified by the study team targeted to participating state health department chronic disease units [35,43].

In phase one the study team has developed and tested a survey instrument and collected self-report survey data nationally. In addition, the study team will develop a tool to abstract archival state health department plans and progress reports as an objective source of data on planning and implementation of EBPPs in chronic disease prevention.

The main objective of the self-report online survey are to obtain a national snapshot of practitioner views on EBDM, training and informational needs for EBDM, organizational support for EBDM, barriers to the application of EBDM, and EBPP implementation among state health department mid-level program managers and staff working in chronic disease prevention across the United States.

The survey was developed from previous research conducted by Dr. Brownson et al.[32,49], a literature review [50], and five rounds of study advisory group input from August-November 2012. The 68-item survey contains eight sections and was designed for completion in about 15 minutes. Table  2 describes the survey domains and types of items included in the survey, as well as their sources. A state-level global score for EBPPs being implemented in the state was developed for use as either a dependent or independent variable. From the EBDM skill importance and availability scales, a gap score subtracts perceived availability from perceived importance [49]. Two items on perceived benefits and challenges of coordinated chronic disease prevention were pilot tested by 10 Missouri state department of health coordinated chronic disease committee members and revised based on feedback received. The survey instrument was programmed in Qualtrics online survey software and underwent several periods of trial and refinement internally with research staff prior to cognitive response and reliability testing.

We randomly selected 150 practitioners from contact lists collected from NACDD, CDC, and the Tobacco Technical Assistance Consortium. The 106 respondents that completed the survey the first time were each emailed an invitation to take the survey again within 14 to 24 days after their initial survey. Efforts were made to distribute the sample across states and program areas and all 50 states were represented in the reliability sample. Replacement sampling was done as needed to get 150 eligible invitees. Of the 150 eligible practitioners invited, 106 (70.7%) completed test one, and 75 completed test two (70.8% of test one). Respondents completed the second survey 10 to 30 days after the first survey. Among those that kept the online survey open less than 40 minutes, the median time to complete the survey was 18.5 minutes, with a mean time of 19.4 (SD = 7.3) minutes.

Test-retest statistical analyses included calculating intra-class correlation coefficients (ICC) for Likert scale items, and percent agreement and Cohen’s kappa statistic for dichotomized Likert-scale items (strongly agree and agree vs. other responses) [52,53]. To test internal consistency of the domain and influence of individual items on a domain, for each continuous variable, the Cronbach’s alpha was calculated. For ranking items, the percent agreement of the three items chosen in the top three in Test1 and Test2 was calculated. Landis and Koch [52] kappa categories of almost perfect (1.0 – 0.8), substantial (0.8 – 0.6), moderate (0.6 – 0.4), fair (0.4 – 0.2), and low (0.2 – 0.0) were used as qualifiers for interpretation of results. For ICCs and percent agreement, >0.70 were considered desirable and >0.80 were best [53]. Test-retest results showed that overall the percent agreements were typically ≥0.70 and ICCs, the appropriate statistic for most sections and items, were mostly ≥0.70. The majority of kappa coefficients were in the moderate range (0.40 – 0.60). Most of the scales showed adequate internal consistency (Cronbach’s alpha ≥0.70). Two questions were deleted after review of test-retest results and the wording of three items was modified. Because the survey was only slightly modified, test one completed surveys will be combined with the full survey sample data described below for nearly all items.

The study team created a list of eligible individuals through exhaustive searching of US state health department websites and updated lists from NACDD and CDC. State health department practitioners working in primary and secondary cancer prevention and screening, physical activity, nutrition, tobacco, obesity, diabetes, cardiovascular health, healthy aging, and general chronic disease prevention in the US or a US territory were invited to participate in the survey. State health department employees of all ages, genders and educational backgrounds were included. Administrative assistant staff members were excluded. Invitations containing information on the survey and a survey link were emailed to state health practitioners in March 2013. Pre-invitations informing survey respondents about the purpose of the study were sent one week prior to invitations. Initial non-respondents received two follow-up calls and three email reminders, which resulted in a response rate of 75.5%. Respondents were offered an optional $20 Amazon.com gift card for completion of the survey.

National survey data collection resulted in a total of 923 completed surveys from state health department employees in all 50 states, the District of Columbia, and five of the eight US territories. Of the 1,443 invited into the survey, 221 were ineligible because they no longer worked at a state health department, were on an extended leave of absence, or now worked outside of chronic disease prevention. Of the 1,222 eligible invitees, 923 completed the survey for a response rate of 75.5%. The 19 surveys from health department staff from the U.S. territories will be excluded from initial analyses, because three of the eight territories did not participate, and only 36.5% (19 of 52) eligible invitees from the territories completed the survey. Therefore, a total of 904 completed surveys will be included in data analyses. Among the 50 states, the response rate was 77.3%.

Phase one includes development of an abstract tool and codebook for archival record abstraction. The purpose of abstracting state health department plans and reports for various program areas in chronic disease prevention is to corroborate with and expand on self-report information on EBPPs being planned and implemented in chronic disease prevention in the twelve participating states. The study team will abstract health department progress reports and plans, as well as statewide coalition strategic plans, in cancer prevention and control, tobacco control, obesity prevention, physical activity, nutrition, cardiovascular health, and diabetes before and after dissemination strategies are applied. This will provide an objective gage of EBPP uptake before and after dissemination strategies are applied. The record abstraction tool will also be made available to other users online upon finalization. The Community Guide [54] will be the basis for EBPP inclusion on the tool for most program areas, while systematic reviews not yet incorporated into The Community Guide will be utilized to identify evidence-based nutrition EBPPs. To draft the record abstraction tool and codebook, the study team initially abstracted 32 state health department plans and program reports from 10 states. Additional interventions in updated Community Guide systematic reviews will be added as they become available.

Study staff then documented types of plans and reports publicly available from state health department websites in a random sample of six states. Publicly available information contained mostly plans for future strategies rather than information on what was currently being implemented. Of 84 documents found, 57 contained future plans only, and 27 contained future plans plus some information about implementation of current plans. From these findings along with professional consult, phase two’s abstraction form will be completed using state progress reports from the participating chronic disease prevention programs.

In each dissemination state, a core group of chronic disease unit members and study team members will work together to identify, select, and refine dissemination strategies pertinent to the work unit’s situations, priority topics, and broader agency and state government contexts. The purpose is to enhance capacity of state-level public health practitioners and work units to plan, promote, and evaluate local and statewide implementation of EBDM and EBPPs for the prevention of chronic diseases. Dissemination strategies will emphasize electronic modes of knowledge transfer and interactive knowledge exchange among each state’s core group. Knowledge transfer and exchange strategies will be informed by lessons learned from Canadian research with the help of the Canadian consulting investigators [44,55-59]. Key principles for dissemination strategy selection are the strategy will build chronic disease prevention practitioner and work unit capacity for EBDM; the strategy will be sustainable by state health departments to maintain after this grant-funded study ends; and the strategy will be developed and applied through participatory engagement [48,60-62]. Content will be targeted to the chronic disease risk factor prevalence and disease burden in each state and priority topic areas selected by the chronic disease unit [63,64]. Priority topic areas may include tobacco control, obesity prevention, physical activity, nutrition, cancer screening, skin cancer prevention, or coordinated chronic disease prevention. The initial dissemination strategy in each of the six states will be a targeted multi-day in-person dissemination workshop [63]. Potential additional dissemination strategies include:

1. Targeted electronic messaging across sites choosing similar topic areas;

2. Online discussion groups across sites;

3. Webinars;

4. Providing links to pre-existing evidence sources for easy access;

5. Conducting specific evidence searches in response to state requests and teaching state staff how to do this themselves;

6. Electronically-delivered issue briefs that provide public health evidence in user-friendly, one to two page formats with a combination of statistical information and narrative examples, which public health practitioners can share with state agency leaders, elected officials, and private funders [65-67];

7. Technical assistance on how to document use of EBDM and EBPP implementation;

8. Strategies to foster agency support for evidence-based chronic disease prevention in partnership with public health practitioners. Examples include finding ways to help agency leaders prioritize EBPPs, finding ways to persuade supervisors to expect EBDM use by staff, developing feasible incentives in state health departments for application of EBDM, and incorporating EBPPs in contracts with local partners;

9. Strategies to embed EBDM knowledge acquisition into ongoing work unit processes such as new employee orientation, job descriptions, and performance reviews.

Figure  2 depicts the paired cluster randomized design for state selection. The paired design is the most appropriate for this study because it is not feasible for us to obtain more than 12 states (clusters) in total. By using state population as the main matching criterion, we will balance some important state-level factors (e.g., state population is highly correlated (0.72) with chronic disease funding from the CDC) potentially affecting the trial outcomes. As a result, the between-cluster variation will be reduced, resulting in a gain in the statistical power [68,69]. Participating states and individuals will be aware of state status as dissemination or comparison; there is no blinding.

Based on our preliminary studies and values of ICC in the literature [32,44,70-74], we have estimated a range of effect sizes and ICCs. ICC estimates are the most difficult to obtain; we calculated a median ICC from similar studies and developed a range based on a 50% decrease and increase around the median (range 0.009 to 0.027). The sample size requirement is based on testing three hypotheses with a power of >90% and the overall type I error of 5% given six paired clusters (states). The null hypotheses involve the change in the scores from baseline of three outcomes— EBDM resources (awareness phase), supervisor support and expectations for EBDM use (adoption), and evaluation for maintenance of EBDM—in both the intervention and control arms (no change). Drawing from our previous work [32,44,74], the corresponding three alternative hypotheses for the change in scores in the intervention arm are 17%, 20%, 14%, higher for EBDM resources needed for awareness of the evidence, supervisory support and expectations for EBDM adoption/use, and evaluation for maintenance of EBDM, respectively. Following Donner [75] and Thompson [76], and our previous ability to obtain high response rates [32,74], we estimated the number of subjects needed in each state as 59 (total = 708). We calculated the number of subjects needed using calculated ICCs of 0.055 and 0.051 for EBDM resources and supervisory support, respectively, from the phase one national survey data, with a resulting number of participants needed in each state of 62 (total = 744). This assumes a 74% response rate (0.74 × 84). All individual chronic disease unit public health practitioners in participating states will be invited into the study (complete enumeration), and a purposive sample of coalition and local health department partners identified by the states will also be invited into the study.

Using IBM SPSS 20, eligible states will be de-identified, assigned random case numbers, and stratified by state population tertiles. Selection, pair-matching, and enrollment will be staggered by two pairs at a time over years two and three of the study for feasibility of conducting phase two activities. By the end of the staggered selection and enrollment, two state health departments will be randomly selected from each of the three state population strata and paired with the state closest in population. The principal investigator (last author) will invite chronic disease directors from each selected state health department to have their chronic disease units participate in the study and agree to be randomly assigned to dissemination or comparison conditions. If any state health department declines, the state next closest in population will be selected as a replacement match. After achieving a complete consented pair, the third author will de-identify states and randomly assign dissemination or comparison using the random case selection function in SPSS. After random assignment occurs, the principal investigator will communicate the results to the pair.

State eligibility among the 51 state health departments (50 states and the District of Columbia) for selection will be based on four criteria: an aggregate state EBDM capacity index factor score derived from the national survey, with five outlier low or high states excluded; a state cancer and chronic disease excess mortality index derived from archival data, with three outlier low burden states excluded; extent of EBDM training and technical assistance received by the principal investigator and teams, which excluded seven states with recent trainings; and availability of a logical pair-matched state, with three states excluded.

The two calculated indices for state eligibility criteria are described below. The EBDM capacity index will include five variables aggregated to states from individual respondents’ self-reported survey responses: resources for EBDM scale (eight items), organizational climate scale (five items), availability of EBDM skills scale (ten items), use of EBDM (a single item), and supervisor expectation of EBDM use (a single item). Individual scores will be aggregated to each state based on median scores. Confirmatory factor analysis will be used to examine the construct validity of each of the scales (e.g., organizational climate). The few states with outlier high and low EBDM capacity will be excluded on the rationale of lack of need or lack of readiness for the study. The state chronic disease excess mortality and risk index will be derived from national mortality rates of lung cancer, breast cancer, cervical cancer, colorectal cancer, coronary heart disease, stroke, diabetes, and chronic obstructive pulmonary disease, and prevalence of adult tobacco use, physical inactivity, unhealthy diet, and obesity. States with outlier low excess mortality and risk will be excluded. Additionally, several states with recent extensive training and technical assistance support from our research center will also be excluded, and only the contiguous 48 states will be considered for eligibility due to travel costs.

In dissemination states, state health department cancer and other chronic disease prevention statewide partners will be recruited into the study, as well as key state level partners from statewide coalitions and other governmental and non-governmental organizations. Additional key local public health partners will be recruited as well, with the help of the chronic disease directors, comprehensive cancer program managers, and other practitioners. In each state, we anticipate an average of 20 participants in the state health department, 10 partners from non-profit organizations such as the American Cancer Society, and 30 local public health system participants, for an average of 70 per state. Additional state pairs will be enrolled in years two and three of the study and then randomized to active dissemination or comparison status, and partners identified with the help of the chronic disease directors and comprehensive cancer program managers.

The evaluation of the dissemination strategies will track the knowledge transfer, exchange, and utilization as depicted in Figure  1 in combination with the dissemination stages of awareness, adoption, implementation, and maintenance [38,63]. The primary outcomes will be pre-post change scores in knowledge use and organizational supports for EBDM examined between dissemination and comparison states. Primary outcomes will be measured through individual and organizational level self-report survey items determined through phase one (see Table  2). Primary outcomes include: perceived resources for EBDM, resource use, supervisory support and expectations for EBDM use, knowledge transfer and exchange, evaluation maintenance, agency leadership support for EBDM, and self-reported use of EBDM. Data will be collected from individual state public health practitioners pre-initiation of dissemination activities in their respective work unit and 18 months post-initiation. Additionally, a secondary outcome, EBDM skill gaps pre-post, will be measured and examined across dissemination and comparison states. Following Phase one’s instrument tool, EBDM skill gaps are measured as the difference in importance and availability of specific skills required for EBDM. These gaps are measured at the individual practitioner level and will be aggregated to the state level for comparison between dissemination and comparison states.

Independent variables include participant characteristics, characteristics of the state health department, and state contextual variables such as rurality of the state and political affiliations of the governor and state legislatures. Quantitative data will come from three instruments: the national survey instrument, the archival record abstraction form, and a social network analysis instrument to be developed with input from the study international advisory group. Additional information will be gathered in qualitative interviews (face-to-face or by phone), recording project costs (e.g., labor, supplies, equipment, travel), and process evaluation. Process indicators will also be measured to assess on an ongoing basis throughout phase two, variables such as staff time needed to develop and coordinate the trainings, participation rates across sites, how effectively partners are involved across sites, and how the state health department promotes EBPP implementation among local entities.

From the cross-sectional phase one national survey data, confirmatory factor analyses will be conducted to test and refine the hypothesized dissemination stage variable groupings from observed data. Multilevel structural equation modeling will be conducted if needed. Descriptive statistics will be calculated for all variables, and chi-square and t-tests will be applied to compare subgroups of participants. Multivariate linear and logistic regression modeling will be conducted to test for hypothesized associations. For phase two, we will conduct bivariate analyses to explore associations and multivariate analyses within and across time allowing for adjustment. Multilevel regression modeling will be utilized for the analysis of pre-post change scores to account for variance across and within clusters.

Qualitative interview recordings from key informant interviews will be transcribed and the transcripts reviewed for completeness and accuracy. Each interview will then be coded by two coders [77]. This method will use the interview guide questions to establish major categories such as organizational factors. Matrices and tables will be created to facilitate comparisons between and within states [78].

The study is currently ongoing. Some phase one activities have been completed (as documented above), including survey cognitive response testing, reliability test-retest data collection, and data collection of the national survey. The development of an abstract tool to record uptake of EBPPs from state health department program area plans and reports is close to completion.

Phase two, the cluster randomized trial, is in the planning phase. Planning is in progress with development of the state chronic disease excess mortality and risk index and a state EBDM capacity index for the 50 states and the District of Columbia. State selection criteria have been established. State pair-matched randomization procedures have been reviewed with a statistician and the full study advisory group and revised. An August 2013 in-person meeting furthered phase two planning.