Investigators in South Carolina and Alabama assessed the availability of data for measuring 31 policy and environmental indicators for heart disease and stroke prevention. The indicators were intended to determine policy and environmental support for adopting heart disease and stroke prevention guidelines and selected risk factors in 4 settings: community, school, work site, and health care.
Research teams used literature searches and key informant interviews to explore the availability of data sources for each indicator. Investigators documented the following 5 qualities for each data source identified: 1) the degree to which the data fit the indicator; 2) the frequency and regularity with which data were collected; 3) the consistency of data collected across time; 4) the costs (time, money, personnel) associated with data collection or access; and 5) the accessibility of data.
Among the 31 indicators, 11 (35%) have readily available data sources and 4 (13%) have sources that could provide partial measurement. Data sources are available for most indicators in the school setting and for tobacco control policies in all settings.
Data sources for measuring policy and environmental indicators for heart disease and stroke prevention are limited in availability. Effort and resources are required to develop and implement mechanisms for collecting state and local data on policy and environmental indicators in different settings. The level of work needed to expand data sources is comparable to the extensive work already completed in the school setting and for tobacco control.
Beginning in 1998, the Centers for Disease Control and Prevention (CDC) received federal funding to support state heart disease and stroke prevention programs. The purpose of these state programs is to develop comprehensive programs emphasizing community-based policy and environmental strategies to reduce risk factors related to heart disease and stroke, such as physical inactivity, poor nutrition, tobacco use, and hypertension. The CDC recommends that assessment and policy development be included within the 10 core public health services to support individual and community health efforts. To monitor their progress on developing community-based policy and environmental strategies, state programs require intermediate evaluation measures of policy and environmental factors. Community-level indicators have been used to measure such intermediate policy and environmental outcomes for other community-based disease prevention programs (
The Cardiovascular Health Branch of the CDC, in collaboration with other units within the National Center for Chronic Disease Prevention and Health Promotion, used literature searches, expert recommendations, and a Delphi process to identify policy and environmental indicators associated with physical activity, nutrition, tobacco control, and national heart disease and stroke prevention guidelines. A draft list of 31 pilot policy and environmental indicators was developed with the intention of revising the list upon feedback from this study. The indicators were selected, in part, because they were thought to be feasible for consistent measurement across 50 states. For example, one indicator can be used to track the number of states that have policies requiring daily physical education for grades K–12. The indicators were categorized by community, school, work site, or health care setting (
Because literature on community-level indicators was limited, little was known about the availability of data sources for use by state heart disease and stroke prevention programs. Hence, the Cardiovascular Health Branch staff asked the Alabama and South Carolina heart disease and stroke prevention program directors to assess the availability of data sources for the 31 pilot indicators in those 2 states and to provide their perspectives on the feasibility of using these indicators. These 2 states were selected because of their proximity to the CDC in Atlanta for technical assistance and because each state program has a close relationship with its Prevention Research Center. Each state program collaborated with its Prevention Research Center (the Center for Health Promotion at the University of Alabama at Birmingham and the Prevention Research Center at the University of South Carolina) to carry out the assessment. This paper summarizes the findings and provides recommendations for collecting data and refining community-level indicators for the surveillance of heart disease and stroke prevention.
Between October 2000 and October 2001, research teams at the South Carolina and Alabama Prevention Research Centers worked in tandem to identify and examine possible data sources and to assess sensitivity and specificity for each indicator. To identify possible data sources, the research teams completed a systematic search within each of 4 settings: community, school, work site, and health care. They identified individuals in state departments of health and education, other state agencies, and private organizations who might have access to or be aware of relevant data sources (
Individuals were identified using a snowball technique that began with people or organizations known to research team members as well as contacts identified from Web sites. As individuals were identified, a team member contacted them by telephone. A conversational interview was used to ask respondents if they collected any data related to a given indicator, and if so, they were asked to provide details about the data source. If the agency or organization did not collect relevant data, the research team requested names of other potential informants or sources of data. These new informants were contacted and the process was repeated until all identified individuals or agencies were contacted.
Additionally, the research teams completed literature and on-line searches using keywords from each indicator (e.g., sidewalks, mixed-use, bicycle) to identify additional data sources and possible contacts. Once data sources were identified, the research teams reviewed each data source, taking note of the degree to which the data fit the indicator; the frequency and regularity with which data were collected; the consistency of the data collected across time; the costs (time, money, personnel) associated with data collection and/or data access; and the accessibility of data.
In addition to evaluating the data sources, the research teams made a general assessment of the sensitivity and specificity of each indicator. Sensitivity refers to the extent to which an indicator allows for documentation of incremental change. Indicators were flagged as lacking sensitivity if they referred only to the presence or absence of a policy rather than the extent to which a policy addressed an issue. Indicators were also flagged as lacking sensitivity if they measured change at an inappropriate level (i.e., if an indicator asked about state policy when policy is set at the local level). Specificity refers to the extent to which an indicator precisely and accurately describes an environmental feature or policy being measured. Indicators were flagged as lacking specificity if they were ambiguous or failed to define key terms.
During this project, research teams participated in regular conference calls with personnel from the CDC's Cardiovascular Health Branch and the state program managers in Alabama and South Carolina to review progress, clarify issues, and share protocols and information. Although each research team completed tasks independently and had a different contractual relationship with its state program, efforts were made to ensure that working protocols (including evaluation criteria and reporting formats) were consistent.
Among the 31 pilot indicators, 11 (35%) had readily available data sources and 4 (13%) had data sources that could provide at least partial measurement. Data sources were available for most indicators in the school setting and for indicators related to tobacco policies across all settings. Data sources were least available in the work site and health care settings. Most data sources identified were maintained by a national agency or organization (e.g., CDC, U.S. Department of Agriculture [USDA], National Transportation Enhancements Clearinghouse). State agencies often report data to these national data sources. Neither research team found a data source unique to its state.
The list of indicators was in draft form at the time of this assessment; thus, many pilot indicators were found to lack specificity. Ten (37%) indicators were flagged as lacking specificity because of ambiguous or imprecise definitions. In addition, 9 (29%) indicators were flagged as lacking sensitivity because they considered only the presence or absence of state legislation, not the quality or degree to which recommendations were included in the legislation. More detailed results are presented about the data sources found in each of the 4 settings.
Two of the 8 pilot indicators in the community setting — clean indoor air laws and smoking in the home — have readily available data sources (
The legislative database in the State Tobacco Activities Tracking and Evaluation (STATE) system summarizes state tobacco legislation, including smoke-free indoor air ordinances for restaurants, day care centers, and public places (
Data sources also are available that partially measure 2 other community indicators: highway funding of transportation alternatives and the number of farmers' markets. The National Transportation Enhancements Clearinghouse maintains a database of transportation enhancements funds allocated and spent by each state under the Transportation Equity Act for the 21st Century (TEA-21). This searchable, on-line database is updated annually (
The USDA maintains a list of farmers' markets searchable on-line by state (
Although regional milk production figures are available, no state data were found on milk production or sales. The research teams also noted that this indicator is not a measure of environment or policy but a community-level indicator of purchasing behavior.
Ten pilot indicators for heart disease and stroke prevention were identified in the school setting (
All 7 of these indicators can be assessed using data from the School Health Policies and Programs Study (SHPPS), which is conducted every 6 years. The study surveys all state departments of education and a nationally representative sample of districts and schools (
The School Health Education Profile (SHEP) collects data that provide partial measurement of school health councils and tobacco-free schools. SHEP is a survey completed every 2 years by a sample of school principals and lead health educators in public schools containing classrooms at the sixth-grade level or higher (
Only one of the 8 pilot work site indicators — clean indoor air laws for work sites — has a readily available data source (
The STATE system contains information that measures state clean air laws that apply to work sites (
Questions from the National Worksite Health Promotion Survey could be used to assess on-site physical activity programs and nutrition or weight management programs (
Among the 5 pilot indicators identified in the health care setting, only one has a readily available data source: smoking cessation advice delivered by health care professionals (
In Alabama and South Carolina, the school setting has data to measure — at least partially — all but one of the pilot indicators for heart disease and stroke prevention. The community, work site, and health care settings have data sources for fewer than half of the indicators.
Given the overall lack of data in most settings assessed in this study, consideration should be given to designing and implementing new data collection processes. Vehicles for new data collection efforts are likely to be surveillance efforts now supported by the CDC (e.g., BRFSS, Youth Risk Behavior Surveillance System, SHPPS, SHEP). The SHPPS and SHEP are designed to collect policy data and are updated regularly to include more complete information. For example, SHEP 2002 included questions related to 2 school indicators: the percent of schools that provide health education instruction that includes the physical education topics listed in CDC's School Health Index and the proportion of schools that have adopted tobacco-free policies that meet CDC recommendations (
Systems similar to the legislative database of the STATE system could be developed to monitor other state policies. In fact, in late 2003, the CDC Division of Nutrition and Physical Activity launched an on-line searchable database containing bill information related to physical activity and nutrition from all 50 states (
Although the research teams made extensive efforts to consult with a wide range of organizations, other data sources might exist. The research teams restricted their exploration to data that are collected either nationally or within their states. While this project did not complete an exhaustive review of data sources in other states, it did identify some noteworthy examples, such as New York's Heart Check (
An additional challenge of data collection is assessing the impact of policy and environmental changes on behavior and health. Policy and environmental indicators provide only one part of the equation. For example, assessing the impact of school policies on children's behavior presents challenges in obtaining informed consent from the children, school administration, and/or parents.
To be useful to state programs, indicators for heart disease and stroke prevention examined in this study need to be refined to improve specificity and sensitivity. Including clear definitions would improve the specificity of the indicator and the accuracy and consistency of data collected. Sensitivity for many indicators could be enhanced by establishing criteria for evaluating policies and laws beyond consideration of their presence or absence at the state level. Some data sources like STATE and SHPPS already collect detailed information that could be used to evaluate the content and quality of policies in addition to tracking their presence or absence.
While it may be sufficient to look at states' policies for national surveillance, state programs might need additional surveillance data that show progress in meeting prevention goals within their own states. In some cases, particularly within school and community settings, it might be more relevant — albeit more costly — to assess the percentage of local jurisdictions (counties, municipalities, school districts) that implement a given policy.
The health care indicators provide the greatest challenge for surveillance. As worded, the indicators look at the percentage of insurers that provide a specific type of coverage. Knowing this information might not reflect the percentage of the population covered by those companies. For example, South Carolina currently has only 5 health maintenance organizations, which cover less than 10% of the state's population (
The results of this investigation support the need for more attention, resources, and research to provide a consistent, documentable system for measuring indicators for heart disease and stroke prevention. It also will be important to improve the sensitivity and specificity of each indicator and to evaluate how each indicator corresponds to risk factors and health outcomes. These recommendations are consistent with the new
This journal article was supported by grant numbers U50/CCU416128 and U50/CCU416100 from the Centers for Disease Control and Prevention. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the Centers for Disease Control and Prevention.
The opinions expressed by authors contributing to this journal do not necessarily reflect the opinions of the U.S. Department of Health and Human Services, the Public Health Service, Centers for Disease Control and Prevention, or the authors' affiliated institutions. Use of trade names is for identification only and does not imply endorsement by any of the groups named above.
Examples of Agencies and Organizations Contacted for Information on Data Sources for Heart Disease and Stroke Prevention, South Carolina and Alabama, 2001
| Federal and state departments of transportation | |
| State departments of education | |
| Better Business Bureau | |
| State insurance commissioners |
Pilot Indicators and Data Sources for Heart Disease and Stroke Prevention, Community Setting, South Carolina and Alabama, 2001
| 1. | Percent of highway funds devoted to transportation alternatives (e.g., bicycle lanes linked to public transportation, mass transit systems, facilities and roadway changes; supports such as parking hubs and bicycle racks). | 1. | National Transportation Enhancements Clearinghouse
(http://www.enhancements.org). Includes only data on funding spent under the federal Transportation Enhancements
Program. |
| 2. | Percent of counties or municipalities with policies requiring sidewalks in all new and redeveloped residential and mixed-use communities. | 2. | No data source found. |
| 3. | Percent of counties or municipalities with policies that promote recreation facilities (e.g., bikeways, parks, fields, gyms, pools, tennis courts, and playgrounds) in new and redeveloped residential and mixed-use communities. | 3. | No data source found. |
| 4. | State policies and percent of counties or municipalities with policies and strategic plans to promote bicycle use for transportation purposes. | 4. | No data source found. |
| 5. | Percent of low-fat milk sales in the state (1= or less). | 5. | No data source found. Regional milk production data are available but do not reflect state sales. |
| 6. | Number of farmers' markets per capita in the
state. | 6. | U.S. Dept. of Agriculture Farmers' Market database
(http://www.ams.usda.gov/ farmersmarkets/). Incomplete due to inconsistent reporting and definition of farmers' markets across states. |
| 7. | State with laws on smoke-free indoor air that prohibit smoking or limit it to separately ventilated areas in restaurants, day care centers, and other public
places. | 7. | State Tobacco Activities Tracking and Evaluation (STATE) System
( |
| 8. | Proportion of smokers who report that smoking is not allowed anywhere inside their homes. | 8. | Behavioral Risk Factor Surveillance System (BRFSS), optional Tobacco Indicators module
( |
2 indicators (25%) lack specificity (ambiguous, lack precision).
1 indicator (12%) lacks sensitivity (unable to measure incremental change, measured at inappropriate level).
2 indicators (25%) have data sources that partially measure indicator.
2 indicators (25%) have adequate data sources.
Pilot Indicators and Data Sources for Heart Disease and Stroke Prevention, School Setting, South Carolina and Alabama, 2001
| 1. | State policies that require daily physical education or its equivalent in minutes per week, for all students in K–12, with no substitution of other courses or activities for physical education. | 1. | School Health Policy and Programs Study (SHPPS) ( |
| 2. | State policies that require schools to assess students on the knowledge and skills specified by the state's physical education standards, frameworks, or guidelines. | 2. | SHPPS. |
| 3. | State policies requiring that the foods and beverages available at schools outside of school meal programs reinforce the principles of the | 3. | SHPPS. |
| 4. | State policies that require newly hired school food service managers to have a nutrition-related baccalaureate or graduate degree and certification/credentialing in food service from either the state or the American School Food Service Association. | 4. | SHPPS. |
| 5. | State policies that require all newly hired staff who teach physical education to be certified, licensed, or endorsed by the state to teach physical education. | 5. | SHPPS. |
| 6. | State policies that require all newly hired staff who teach health education to be certified, licensed, or endorsed by the state to teach health education. | 6. | SHPPS. |
| 7. | States policies that require schools to assess students on the knowledge and skills specified by the state's health education standards, frameworks, or guidelines. | 7. | SHPPS. |
| 8. | Percent of schools that provide health education instruction that includes the physical education, nutrition, and tobacco use prevention topics listed in | 8. | No data source found. Questions from |
| 9. | Proportion of schools with School Health Councils. | 9. | School Health Education Profile (SHEP)
( |
| 10. | Proportion of schools that have adopted tobacco-free school policies that meet CDC recommendations. | 10. | SHEP. See 9 above. SHEP does not include questions to thoroughly assess if tobacco policies meet recommendations. |
7 indicators (70%) lack sensitivity (unable to measure incremental change, measured at inappropriate level).
2 indicators (20%) lack specificity (ambiguous, lack precision).
7 indicators (70%) have adequate data sources.
2 indicators (20%) have data source that could partially measure indicator
Pilot Indicators and Data Sources for Heart Disease and Stroke Prevention, Work Site Setting, South Carolina and Alabama, 2001
| 1. | Percent of work sites that have policies supporting the engagement of all employees in physical activity during work time (e.g., flexible scheduling, relaxed dress codes). | 1. | No data source found. |
| 2. | Percent of work sites that provide showers and changing facilities to support physically active employees. | 2. | No data source found. |
| 3. | Percent of work sites that provide and promote on-going, on-site employee physical activity programs (e.g., walking, stretching, aerobics) during the previous 24 months. | 3. | No data source found. National Worksite Health Promotion Survey measures this indicator at the national level, but the sample is too small for state analysis. |
| 4. | Percent of work sites with vending machines and/or snack bars that offer heart-healthy food and beverage choices, including water or flavored water, 1% or less milk products, 100% juice products, fruits, vegetables, and products labeled low or reduced calorie, low or reduced sodium, and those labeled 3 grams or less of fat per serving. | 4. | No data source found. |
| 5. | Percent of work sites with cafeterias that offer heart-healthy food and beverage choices including water or flavored water, 1% or less milk products, 100% juice products, fruits, vegetables, and products labeled low or reduced calorie, low or reduced sodium, and those labeled 3 grams or less of fat per serving. | 5. | No data source found. |
| 6. | Percent of work sites that offer nutrition or weight management classes or counseling. | 6. | No data source found. National Worksite Health Promotion Survey measures this indicator at the national level, but the sample is too small for state analysis. |
| 7. | States with laws on smoke-free indoor air that prohibit smoking or limit it to separately ventilated areas in government and private work sites. | 7. | State Tobacco Activities Tracking and Evaluation System (STATE) ( |
| 8. | Proportion of work sites (segmented by number of employees) that cover smoking cessation programs. | 8. | No data source found. |
Two indicators (25%) lack specificity (ambiguous, lack precision).
One indicator (12%) lacks sensitivity (unable to measure incremental change, measured at inappropriate level).
One indicator (12%) has adequate data source.
Pilot Indicators and Data Sources for Heart Disease and Stroke Prevention, Health Care Setting, South Carolina and Alabama, 2001
| 1. | Percent of managed care organizations that adopt a policy to incorporate nationally accredited guidelines (e.g., the | 1. | No data source found. |
| 2. | Percent of managed care organizations that adopt a policy to incorporate nationally accredited guidelines (e.g., the | 2. | No data source found. |
| 3. | Percent of managed care organizations (e.g., health maintenance organizations, independent provider organizations, and preferred provider organizations) that have policies or guidelines to routinely provide or reimburse for assessments and counseling for physical activity, medical nutrition therapy, and tobacco cessation to plan members as part of their standard care package, according to the | 3. | No data source found. |
| 4. | Percent of health insurance plans that have policies or guidelines to routinely provide or reimburse for assessments and counseling for physical activity, medical nutrition therapy, and tobacco cessation to plan members as a covered benefit, according to the | 4. | No data source found. |
| 5. | Proportion of current and recent smokers who received advice to quit smoking from a health professional. | 5. | Behavioral Risk Factor Surveillance System (BRFSS), optional Tobacco Indicators module ( |
4 indicators (80%) lack specificity (ambiguous, lack precision).
1 indicator (10%) has adequate data source.