IntroductionWorldwide, road crashes are a leading cause of death and disability among persons younger than 40 years of age and the leading cause of death among 15- to 29-year-olds (World Health Organization [WHO] 2013). More than 1.24 million people die and an additional 20–50 million are injured each year as a result of road crashes (WHO 2013). An over-whelming majority of road traffic deaths (92%) occur in low- and middle-income countries, yet these countries have just over half (53%) of the world’s registered vehicles (WHO 2013).
A rapid increase in the number of motorized vehicles in Southeast Asia has added to the traffic injury burden in the region. In Cambodia, registered vehicles have more than doubled from 714,000 in 2006 to 1.6 million in 2010, of which more than 80 percent are motorcycles (Department of Public Work and Land Transport, unpublished data, 2010). Motor-cycle riders are considered “vulnerable” road users, along with bicyclists and pedestrians, because they navigate the road system without a “protective shell” (WHO 2009).
The WHO identifies reliable road crash surveillance data as a critical public health tool for assessing the burden of road traffic injury, targeting responses, and evaluating the effectiveness of road safety interventions (WHO 2009). For low- and middle-income countries, where the burden of road crashes is growing, establishing quality data systems is critical for identifying the most important road safety issues and tracking progress. In Cambodia, the Road Crash and Victim Information System (RCVIS) combines police and health facility data to monitor road crash injuries and fatalities nationwide. In August 2011, the U.S. Centers for Disease Control and Prevention’s (CDC) Division of Unintentional Injury Prevention conducted a formal evaluation of RCVIS to understand whether road crash injuries are captured efficiently and effectively and to help identify ways to strengthen the system.
MethodsThis project used the CDC Guidelines for Evaluating Public Health Surveillance Systems as a framework for evaluating RCVIS (CDC 2001). The CDC guidelines identify 9 attributes with which to evaluate the performance of a surveillance system—simplicity, flexibility, data quality, acceptability, sensitivity, predictive value positive, representativeness, time-liness, and stability. We developed a semistructured questionnaire to guide interview discussions with key stakeholders. Questions focused on each system attribute, along with basic characteristics of RCVIS, such as system purpose, data flow, and data usefulness. We used the interview guide to conduct an in-depth group discussion with 6 representatives from the National Road Safety Committee and Handicap International (HI), the organizations responsible for developing and implementing RCVIS. We also conducted interviews with a representative from the Cambodia Ministry of Health and a representative from Cambodia Ministry of the Interior, the governmental organizations responsible for each of the data sources. Follow-up questions were answered through e-mail correspondence with HI.
To supplement system descriptions provided by stakeholders, we used information from RCVIS documents, including RCVIS annual reports and the system guidelines report compiled in 2007 (HI 2007). We also used data presented in RCVIS annual reports to provide key surveillance findings and evidence for data representativeness and sensitivity. We further assessed sensitivity by comparing RCVIS counts to estimates from the 2010 Demographic and Health Survey, a nationally representative household-based survey that included questions on injuries and deaths among Cambodian house-holds over the prior 12 months (National Institute of Statistics 2011). Finally, we assessed data quality by determining the percentage of missing values for variables in the 2010 police and health facility data, based on customized reports provided by HI. We present evaluation results on system usefulness and 6 performance attributes—flexibility, acceptability, sensitivity, representativeness, data quality, and timeliness.
ResultsRCVIS System DescriptionAn official agreement to merge data among the Ministry of Health, Ministry of Interior, and the Ministry of Public Works and Transport provided the foundation needed to combine data on road crash victims from police and health facilities into a single road crash surveillance system. The system, initially called the Road Traffic Accident and Victim Information System (RTAVIS), was piloted in the capital city of Phnom Penh in 2004. The pilot’s success resulted in the inclusion of nation-wide surveillance into the 2006–2010 National Road Safety Action Plan, and by 2006 RCVIS expanded to all provinces. HI, an international aid organization, played a key role in the development and implementation of RCVIS and was initially responsible for overseeing the entire system. HI has slowly transitioned the work to the Cambodian government, moving data collection responsibilities to the Ministry of Health and the Ministry of Interior in 2008–2009 and transferring system oversight to the National Road Safety Committee (NRSC) in 2010. NRSC is an interministerial body tasked with the overall coordination and implementation of road safety action plan activities in Cambodia. HI continues to play a role in RCVIS through technical assistance to the NRSC.
A case in RCVIS is any person who is injured or killed in a road crash, defined as a collision involving at least one moving vehicle on a public road. Cases are collected by the Ministry of Interior (police) and the Ministry of Health (health facilities). The Ministry of Interior collects data on cases when the police are called to the site of a road crash and the Ministry of Health collects data when a person receives care at a health facility for a crash-related injury (Figure 1). Police send hard-copy data forms to the Ministry of Interior, where officers enter the data and send them electronically to the NRSC monthly. Health facilities send hard-copy forms to the Ministry of Health, which forwards the forms quarterly to the NRSC for data entry. The NRSC combines the data sources, using an algorithm to identify and remove duplicate entries based on crash details, name, and demographic information.
Key Surveillance FindingsIn 2010, RCVIS reported 1816 fatalities and 16,471 nonfatal road crash injuries. With a population of 14.3 million, that translates into an annual rate of 12.7 deaths per 100,000 population. Males accounted for 79 percent of fatalities, and 48 percent of road crash fatalities were among persons 15 to 29 years of age. Most injuries in 2010—67 percent of fatalities and 72 percent of severe nonfatal injuries—occurred to motorcyclists. Only 21 percent of motorcyclist fatalities were wearing a helmet at the time of the crash, and helmet use was higher among driver fatalities (26%) than passenger fatalities (8%). Pedestrians accounted for 8 percent of all road traffic casualties in 2010, including 12 percent of fatalities and 9 percent of severe injuries. Forty-two percent of fatality cases were deemed hit-and-run incidents. Speeding (49% of crashes) and drunk driving (13% of crashes) were most commonly indicated as contributing factors.
Evaluation ResultsUsefulnessRCVIS data are used and distributed widely within Cambodia. Every year, 250 to 500 Cambodian road safety stakeholders receive the RCVIS annual report. The report highlights many important aspects of the road crash injury burden; for example, the count of road crash casualties (including fatal and nonfatal injuries), road user type (e.g., motorcycle, pedestrian, passenger vehicle), and traffic injury risk factors (e.g., speeding, drunk driving, lack of motorcycle helmet use). Road safety stakeholders across Cambodia use RCVIS reports and data to understand and address road traffic safety issues. For instance, traffic police find the data useful to monitor when and where traffic crashes occur most frequently (“black spots”) and to plan where to focus enforcement efforts (e.g., drunk driving and motorcycle helmet roadside checkpoints). Road engineers also use black spot data to improve the safety of road construction projects. The NRSC and the Ministry of Interior use the data for strategic planning purposes, including the development and monitoring of the 2011–2020 National Road Safety Action Plan. The Cambodian components of 2 global road safety campaigns—The Global Helmet Vaccine Initiative and Road Safety in 10 Countries Project—use RCVIS data to evaluate campaign activities (Craft 2010; Peden 2010). Additionally, Cambodia uses RCVIS data for the WHO Global Status Report (WHO 2013).
A limitation to the usefulness of RCVIS data is that the database contains personally identifiable information, so the NRSC and HI cannot distribute the data to end-users in raw form. Users interested in information not available in annual reports must request customized reports. Notably, though health facilities contribute data to the system, the Ministry of Health did not identify specific uses of the data for health facilities.
FlexibilityThe flexibility of a surveillance system refers to its ability to adapt to changing circumstances (CDC 2001). Since the inception of the RCVIS, the Ministry of the Interior has made several minor changes to the data collection process, including the addition of GPS data in 2007. The Ministry conveys these changes during annual training sessions held within each province. The ability to make changes is aided by the strong hierarchical structure of the Ministry of Interior, which helps to ensure police compliance.
The Ministry of Health reported more difficulty in making changes to their data collection process. The Ministry of Health implemented a major change to their data collection form in 2008, by expanding the form to collect data on all injuries (e.g., falls, poisonings, animal bites). This change was implemented nationwide in 2009 after provincial trainings occurred in 2008. The Ministry of Health described the modification as difficult to implement given resistance from the health facilities, which view the new form as more work.
AcceptabilityAcceptability is the willingness of organizations and individuals to participate in the surveillance system (CDC 2001). Overall, RCVIS participation is much higher among the police districts than among health facilities. The Ministry of the Interior reported that all 210 (100%) police districts participate. The Ministry of Health reported difficulty in getting the health facilities to report data, and only 43 (65%) hospitals participated in 2010 (HI and NRSC 2010). Furthermore, hospital reporting was inconsistent—on average, reporting hospitals only provided data for 8 months over the course of the year.
The Ministry of Health identified many possible reasons for lower participation, including (1) the lack of time among health facility workers, who are already overworked; (2) the increasing reporting burden associated with the transition to a general injury surveillance form in 2008–2009; (3) the removal of financial incentives to report, which until 2008–2009 were given to health facility workers in 4 of the largest provinces and the capital city; and (4) the concern among health facility workers regarding patient confidentiality. HI reported that a 2008 law change, which prohibited private ambulances from transporting patients, also contributed to reduced participation among private hospitals, which may not have fully complied with this restriction. The perceived usefulness of the data may also explain differences in participation between the Ministry of Interior and Ministry of Health.
SensitivityThe sensitivity of a surveillance system is the proportion of cases that are captured by the system (CDC 2001). WHO recognizes the importance of including multiple sources of data in collecting and reporting road traffic injury data to minimize the issue of undercounting (WHO 2009). The combination of police and health facility data increases the sensitivity of RCVIS, because there is minimal case overlap between health facility and police records. Adding health facility cases to police cases increases the number of fatalities and injuries by 61 and 6078, respectively (Table 1).
It was not possible to calculate sensitivity for RCVIS because the true number of road traffic casualties is unknown. One single time point source of comparison for the RCVIS data is the 2010 Demographic and Health Survey (DHS), a nationally representative household-based survey, which included questions on injuries and deaths among Cambodian households over the prior 12 months. Results from the 2010 DHS indicate that RCVIS, which is intended to capture all road crash injuries nationwide, may largely undercount the total road crash burden in Cambodia (Table 2). In addition to nonreporting health facilities, underreporting of cases by police or health facilities (not quantified in this evaluation) may also contribute to this difference. Crashes that are not attended to by police and road crash injuries that do not receive treatment or are treated outside a hospital are unlikely to be captured by RCVIS, because health facility data collection focuses on hospitals with few health centers contributing cases to the surveillance system (HI and NRSC 2010).
RepresentativenessThe representativeness of RCVIS refers to how accurately the system captures road crash injuries and fatalities over time and the geographic and demographic distribution of victims (CDC 2001). According to the Ministry of Interior, all police districts report to RCVIS. However, it is possible that some groups are systematically excluded from this reporting. For example, police may be less likely to go to the scene of crashes in areas that are farther from police stations (i.e., rural areas) or for less severe crashes.
Reporting by health facilities is less consistent and reduces RCVIS’s representativeness. Though RCVIS covers all 23 provinces and Phnom Penh, 6 provinces did not report a single case from a hospital in 2010, and 4 did not report a case from any type of health facility. All but one of these provinces had at least one reporting health facility in 2007, suggesting that road crash causalities in those provinces are underrepresented. Hospital reporting was also inconsistent month to month, with reporting hospitals only providing data for an average of 8 months over the course of the calendar year in 2010.
Representativeness over time is also a concern. Between 2007 and 2008, there was a large decline in the total number of hospitals reporting to RCVIS (Table 3). This reduction was particularly apparent in the capital city of Phnom Penh, which had a substantial 1-year decrease in reporting hospitals (from 10 to 6). This trend continued through 2010, when only 3 hospitals reported. Changes in reporting and representativeness over time present difficulties for using the RCVIS to interpret time trends in road crash injuries and fatalities. From 2007 to 2010, RCVIS reported a large decrease in road crash injuries, led by a notable reduction in slight injury cases (Figure 2). Given the increase in fatalities over that same time period and a near doubling of new vehicle registrations (HI and NRSC 2010), this trend seems likely to be related to changes in reporting rather than an actual decrease in the occurrence of road crash injuries.
Data QualityData quality is related to the completeness and validity of the data collected (CDC 2001). We assessed data quality by determining the percentage of missing values for variables in the 2010 police and health facility data. In the police data, 4 of 17 variables had more than 20 percent missing. A particular limitation of police data were high levels of missing data for variables related to safety—wearing seat belt/helmet (58% missing) and whether alcohol use was suspected at the time of crash (45% missing). In the health facility data, 7 of 19 variables had more than 20 percent missing, including seat belt/helmet use (28% missing). Data quality was highest for name, age, and gender, which had less than 1 percent missing in both data sources.
TimelinessThe timeliness of a surveillance system is related to the speed between data collection steps and how quickly the data can be disseminated for use in public health action (CDC 2001). HI and NRSC distribute annual reports to stakeholders approximately 6 to 10 months after the end of the calendar year. Timeliness has declined since the beginning of RCVIS, with lags coinciding with a shift in data entry responsibilities to the Ministry of Interior for police data in 2008 and again with the transition of RCVIS oversight from HI to the NRSC in 2010. Nonetheless, the timeframe for annual reports remains reasonable for the types of public health responses associated with road crashes.
DiscussionAs the only country-wide, comprehensive source of road crash fatalities and injuries in Cambodia, RCVIS data are highly useful to the government for understanding and addressing road safety issues. Frequent requests for data from both government and nongovernmental stakeholders—from road builders to the prime minister—point to the value of RCVIS. Moreover, the prompt distribution of RCVIS annual reports adds value to the system because decision makers are able to use data to plan and evaluate road safety policies and interventions in a timely manner. Nonetheless, there are limitations to the RCVIS data in its current form and lessons to be learned that are applicable to other road crash surveillance systems in low- and middle-income countries. Primary concern with the RCVIS data is inconsistent reporting from health facilities, which affects both the sensitivity of the system and the representativeness of time–trend data. Reporting issues in- cluded health facilities not reporting at all during recent years and health facilities that reported incidents for some months but not others. These inconsistencies make it difficult to use RCVIS as a tool for evaluating road safety initiatives, which are highly dependent on examining data trends over time.
To address the issue of nonreporting, we recommend that the NRSC and Ministry of Health take steps to increase RCVIS acceptability in health facilities. Such steps include increasing buy-in from senior level administrators at health facilities, formalizing and standardizing RCVIS reporting pro- cedures, and showing health facilities how RCVIS data can be useful to them. The current data collection process involves having injury surveillance forms filled out by medical students, nurses, or physicians, depending on a facility’s preference and the availability of personnel. To simplify the process and clarify roles and responsibilities, health facilities should identify a single person or small group that is responsible for over-seeing and ensuring data collection at each facility. Given the burden of collecting data, buy-in from senior-level health facility administrators will likely be critical in setting expectations for participation among health facility workers and providing the necessary resources. The NRSC and the Ministry of Health should monitor incoming data and actively follow-up regarding data anomalies and inconsistent data reporting. Formally recognizing health facilities and staff for participation in RCVIS through certificates of appreciation, plaques, or other means may also encourage participation.
This evaluation highlights the relationship between usefulness and acceptability in surveillance systems. One major difference between the police and health facilities was that police used the data they collected, whereas health facilities saw the data collection as an added burden without an added benefit. To increase the acceptability of RCVIS, the data collection must be viewed as useful by the Ministry of Health and the health facilities. One way to make the data more useful to health facilities is to provide them with facility-specific reports, including insights from the data. Adding clinical variables to the data collection form to allow hospitals to calculate and monitor injury severity over time and providing training on data analysis may also increase data usefulness for health facilities.
The factors that contributed to the decline in health facility participation in RCVIS can provide insight to other countries that are building and sustaining surveillance systems. A 2008 law that prohibited private ambulance transport to hospitals created a disincentive for private hospitals to report road traffic injuries, and RCVIS participation declined. Surveillance system developers should be aware of policies such as this that provide deterrents to reporting. On the other hand, providing incentives was also an issue for RCVIS. RCVIS initially gave a monetary incentive of $0.50 for each completed data collection form at health facilities in certain locations. By early 2009, RCVIS stopped all incentives as part of the transition of data collection responsibilities from HI to the Ministry of Health. Combined with an expanded injury surveillance reporting form, the change had the effect of asking health facility workers to do more work for less compensation and corresponded with a decline in reported road crash injuries. This suggests that developers of surveillance systems should consider the long-term implications of using incentives to promote participation, particularly if incentives cannot be maintained indefinitely. These events underscore the potential fragility of surveillance systems and the importance of acceptability for maintaining system integrity as circumstances change.
This surveillance evaluation is subject to some limitations. First, because we did not have direct access to RCVIS data, we relied on aggregate summaries for analysis. This may hide some underlying trends in reporting among data sources. In addition, we did not have the opportunity to observe data collection for either the police or hospital data sources, which would be useful for identifying problems with the data collection process and finding underreporting (WHO 2004). Finally, the Ministry of Health and Ministry of Interior self-reported their RCVIS activities.
Road crash surveillance systems are critical in low- and middle-income countries for identifying the most significant road safety issues, determining where to put limited resources, and tracking progress. Linked data systems, including data from multiple sources (e.g., police and health facilities), are especially valuable because they provide better coverage of fatal and nonfatal injuries, which may be differentially captured between sources (WHO 2009). Linking police and health facility data allows RCVIS to capture more road crash casualties than either system alone, so linking the sources increases system sensitivity. Notably, there is little overlap between the data sources in RCVIS. Though it is common for those who die at the scene to not make it to a hospital for treatment and thus be reported in police data only, it is less clear why so few non-fatal health facility cases are captured in police reports. These cases likely come from a combination of crashes where police responded but did not report the cases, crashes where injured cases left the scene before police arrived, and crashes where police did not respond at all. Directly observing data collec- tion and following up on cases in each system would provide more insight into the lack of overlap between data sources.
Overall, the Cambodia Road Crash and Victim Information System provides a strong foundation for the surveillance of road crash casualties in Cambodia. The objective of RCVIS is to gather accurate and complete data on road crashes and victims in order to increase the understanding of the current road safety situation, plan appropriate responses and policy, and evaluate the impact of current and future initiatives. In its current form, the RCVIS provides the infrastructure needed to meet these objectives. Nonetheless, there are areas for improvement, particularly in terms of acceptability and representativeness among health facilities. Addressing these issues will make the RCVIS an even more useful tool for understanding and reducing the burden of road crashes in Cambodia.