The evaluation team was led by university-based researchers and included members from the state's cancer registry and cancer control program. To ensure the evaluation plan was rooted in the reality of ongoing CCCP implementation activities of the South Carolina Cancer Alliance (SCCA), the organization charged with devising and implementing the plan, the evaluation team communicated with SCCA leadership, workgroups, and general membership throughout the 18-month evaluation process (July 1, 2009, to December 31, 2010). In the first 6 months of the evaluation, the evaluation team worked with SCCA leadership groups to ensure that the evaluation methodology was consistent with the organization's needs and expectations. In months 7 to 12, more than 100 key stakeholders with content area expertise and interest were contacted by phone or e-mail. In months 10 to 16, after it became clear that consensus within the organization's workgroups would be necessary to accurately describe SCCA activities and accomplishments, the evaluation team convened these workgroups with the sole purpose to collectively describe their progress toward the CCCP objectives. Input from broader statewide constituents was received at 4 town hall meetings and at 1 statewide SCCA meeting during months 12 to 18. Immediately after new information was obtained from stakeholders, the new information was recorded in an evaluation-tracking database. Stakeholder feedback informed all aspects of the evaluation, which included the needs and expectations for the evaluation, insight into stakeholder activities undertaken to achieve CCCP objectives and barriers/facilitators they encountered, history of how the CCCP had evolved over time, and suggestions for future direction for the upcoming 5-year CCCP.

The initial evaluation task was to dissect the CCCP's goals, objectives, and strategies. A database constructed to consolidate key data was populated with information such as current/historical goals, objectives, and strategies; history of modifications; data sources for tracking indicators; benchmarks; outcomes; and key words. This database was the central repository for tracking evaluation findings. To account for objectives and strategies that were modified, added, or removed since the initial CCCP, a numeric classification system was developed to track each indicator's initial and current location in the CCCP. During this process, 2 of the 8 chapters were identified as outliers: (1) the Genetics chapter had never been directly addressed and (2) the Health Disparities chapter was primarily a compilation of objectives/strategies from other CCCP chapters, so its content overlapped with content elsewhere in the plan. These 2 chapters were thus excluded from subsequent evaluation activities. These initial evaluation steps of organizing the fluid cancer plan took place in the first 3 months of the evaluation process.

Each objective was rated on 2 dimensions: the extent to which it (1) was measurable as written and (2) had available outcome data. This step took place in months 7 to 12 of the evaluation process. The first dimension measured the objective's clarity and precision. An “A,” “B,” or “C” rating referred to objectives that could be evaluated directly as written, with minor changes, or only with major revisions, respectively. Examples of “minor changes” were defining baseline and benchmark values or more precisely characterizing the target population. An example of a “major change” was splitting an objective that comprised multiple objectives into separate components. Objectives were also rated on the availability of outcome data. A “1,” “2,” or “3” rating meant outcome data were readily available, available with secondary data analyses, or would require new primary data collection, respectively.

These 2 separate ratings were combined into a single summary measure to jointly characterize the extent that an indicator was clear/precise and had available outcome data to assess achievement. An “A1” rating meant an objective was clear, precise and had available data to evaluate progress. In contrast, a “C3” rating lacked both clarity/precision and available outcome data. Each objective was rated independently by 2 reviewers, with differences resolved by consensus.

The rating system was used to prioritize the objectives evaluated for outcomes. Because of resource constraints, outcome data were obtained and summarized solely for the indicators rated “A1,” namely, those that were clearly written and could be linked to available outcome data. Indicators that could be measured (ie, those rated A1) were then categorized as follows: “little or no progress,” no evidence of measurable progress to meet the objective; “partially met or some progress,” evidence of progress that could result in achievement of the objective; and “met,” the objective was accomplished.

In South Carolina, a primary method of CCCP implementation is via allocation of grant funds for local projects. These funds are allocated by the South Carolina Department of Health and Environmental Control (SC DHEC) Cancer Control Program in 2 separate funding streams. One funding stream runs through the SCCA, which awards competitive grants to SCCA partners for projects that address CCCP objectives (SCCA grants). The other funding stream is to the state's 8 public health regions through a regional mini-grant mechanism (SC DHEC grants) focused exclusively on prevention and early detection activities. These funding streams generate “on the ground” cancer control activity. Therefore, stakeholders felt it was important to map the funded projects back to the CCCP indicators to assess the effectiveness of distributing funding across the CCCP elements.

Projects funded through the SCCA and SC DHEC mini-grant mechanisms were ascertained for the period July 1, 2005, to June 30, 2010. Data collected from the funding agency, grantee, or both included title, funding period, region served, statement of work, and a description of project components. The scarce data available for most grants precluded a summary of project outcomes. To describe the distribution of funding across the CCCP, the topics of the funded grants were mapped to pertinent CCCP indicators. The mapping of resource allocation to pertinent CCCP indicators was coded by 1 rater. If the most precise indicator(s) that a grant mapped back to was unclear, 2 raters discussed mapping until consensus was achieved.

Funded projects could address more than 1 CCCP objective, either through separate activities directed at different objectives or through a single activity that cut across multiple objectives. An example of a crosscutting activity is advocacy for increasing the tobacco tax. This activity addressed objectives in both the Advocacy and Prevention chapters; thus, this type of grant contributed toward both the chapters. For DHEC mini-grants that addressed both prevention and early detection, the funding amount was uniformly allocated evenly between the 2. The total number of unduplicated grants was calculated, as was the total number of grants that addressed prevention and that addressed early detection. The process of mapping the allocation of grant projects back to the CCCP objectives took place in months 13 to 18 of the evaluation process.

Of the 64 objectives, 47% were measurable as written with the balance evenly divided between requiring minor and major changes (Table 1). Data were readily available for 36% of the objectives, but secondary and primary data collection was required for 28% and 37%, respectively. At the extremes of the composite ratings, 33% were A1 and 16% were C3. As a proportion of total objectives, these were the 2 largest cells in Table 1 and reflected a positive correlation between clarity and data availability.

The outcome evaluation was limited to the 33% (n = 21) of A1-rated objectives that were measurable as written with available outcome data. Of these, 38% showed little or no progress, 38% were partially met, and 24% were met (Table 2). The percentage of objectives achieved (yes/no) ranged across the CCCP chapters from 0% (Prevention, Patient Care) to 67% (Research). These results were partly a function of how ambitiously the target was set. An example of a very ambitious objective was “by June 2010, decrease the rate of tobacco use among South Carolinian adults (older than 18 years) from 26% to 12%.” Thus, although smoking prevalence decreased substantially to 21%, this objective was not met. An example of a more modest target that was met was “by June 2010, increase the proportion of newly diagnosed prostate cancers that are detected at the localized stage from 73% to 75%.” These examples illustrate why precise quantification of progress toward objectives is preferred.

During the 5-year evaluation period, a total of $1.73 million was awarded to grants for local cancer control projects: 29% via the SCCA grant mechanism and 71% via the DHEC mini-grant mechanism (Table 3). Due partly to the fact that the DHEC mini-grants focused exclusively on prevention and early detection, these 2 topic areas received by far the largest allocations: 47% and 43%, respectively.

There were 56 SCCA grants totaling $500 642 distributed to partners, ranging from $500 to $30 000 each. Projects addressing objectives in Early Detection comprised 36% of the total funding, followed by Prevention (28%) and Advocacy and Policy (27%; this overlapped with other chapters), whereas Patient Care (17%), Survivor and Family (14%), and Research (6%) received substantially less. The funding was strongly aligned with CCCP objectives in areas such as tobacco cessation, legislative advocacy, colorectal cancer awareness, and cancer education.

A total of 41 SC DHEC grants totaling $1 230 323 were awarded to public health regions to carry out cancer prevention and control projects. Funds awarded ranged from $160 000 to $315 000 per year; most grants ranged from $20 000 to $35 000. These grants addressed solely prevention (22%), solely early detection (34%), or both (44%). These grants were strongly aligned with the CCCP in the areas of cancer education and screening, nutrition and physical activity education, tobacco control, and sun safety education and policy.