One United Nations Millennium Development Goal (MDG) is to halve, by 2015, the proportion of the world’s population that is without access to safe drinking water and basic sanitation. The world reached the MDG drinking water target in 2010, five years ahead of schedule. However, reaching the MDG sanitation target by 2015 is unlikely (UNICEF/WHO 2012). One major obstacle for both water and sanitation interventions is long-term sustainability.

Projects often focus on providing basic infrastructure, rather than on ongoing functionality. Moe & Rheingans (2006) reported on many examples of ‘failed water and sanitation projects supported by well-intentioned but ill-informed agencies’. When assessing sustainability, projects often fail to consider community capacity and needs. Various estimates have documented somewhere between 30 and 60% of existing water supply systems do not provide adequate service (Brikke & Bredero 2003; Davis 2011; Lockwood & Smits 2011). ‘Water-supply and sanitation projects should not be viewed as an end in themselves, but as the initiators of benefits that continue long after the projects have been handed over to the communities’ (Brikke & Bredero 2003).

Organizations around the world have made significant progress in providing access to improved water and sanitation worldwide. However, limited evidence is available on sustainability of rural water and sanitation interventions (Montgomery et al. 2009). The focus of this evaluation is sustainability over the medium to long term. Our evaluation has reconfirmed that sustainable water, sanitation and hygiene (WASH) interventions should address several elements, including:

In this evaluation, we do not propose a universal definition for sustainability but, rather, we assess sustainability using WASH indicators. The indicators integrate the sustainability elements so that if water and sanitation systems continue to function and people continue to practice positive hygiene practices, then communities will meet at least some of the sustainability elements.

We documented the first 7 years of a 10-year WASH sustainability evaluation in rural communities in Central America. At the time of data collection in 2009, 19% of the rural population in the four Central American countries in our evaluation had no access to improved drinking water sources; 38% of this population had unimproved sanitation facilities, with 14% of that population not using any type of sanitation facility (WHO/UNICEF 2013).

In 1998, Hurricane Mitch struck Central America. It caused major damage to infrastructure especially throughout El Salvador, Guatemala, Honduras and Nicaragua and was recognized as the deadliest Atlantic hurricane since the Great Hurricane of 1780 (NOAA 2009). The devastation left many without water, sanitation and other services, affecting an estimated 3.6 million persons. Some 10,000 died and nearly 100,000 homes were destroyed (USAID 1999). The American Red Cross (ARC) responded to the disaster by providing community- and household-level WASH interventions to 110 communities.

The ARC’s goal for the program was to ‘decrease health risks associated with water and sanitation to Hurricane Mitch survivors’. The objectives were for communities to have sustainable water systems, to have access to potable water and to learn how to improve sanitation and hygiene practices. In providing interventions, the ARC took a participatory approach and integrated community participation in project development and implementation at the beginning of these projects. A minimum of 80% of the population in each community had to be willing and able to participate through labor and willingness to pay a water fee. Hurricane-affected communities also provided input on the level of services they were able to support and selected their interventions based on the costs, benefits and feasibility of each option (Moll et al. 2007). Water system designs included projected population growth rates and per capita water needs. Sanitation design considered local geologic factors such as depth to water table and soil type.

In collaboration with ARC, the U.S. Centers for Disease Control and Prevention (CDC) evaluated the effectiveness and the health impacts of the ARC WASH program. The evaluation occurred in eight study areas located in four Central American countries. ARC defined the study areas as either a single community or two adjacent communities that shared similar demographics, geographic region and intervention type. The ARC WASH program’s health objective was a 25% reduction in childhood diarrhea from baseline (2000) to final (2002). The 2002 final evaluation showed that ARC’s WASH reconstruction program met this goal on a regional basis (i.e. across all study areas). Results from the 3-year health impact study have been previously documented (Moll et al. 2007).

Because of the short 3-year period, however, the health impact study was limited in its ability to address longer-term intervention sustainability. The CDC recommended follow-up evaluations every 3 or 4 years over a decade to assess the long-term sustainability of the WASH interventions in these communities. After 2002, the study areas received no technical or financial assistance from ARC. This paper documents the first 7 years of that 10-year sustainability evaluation.

For the sustainability evaluation, we revisited six study areas from the health impact study (Table 1). We excluded two of the original eight study areas for logistical reasons (e.g. inaccessibility), Waspam, Nicaragua and Huitzitzil, Guatemala. The sustainability evaluation consisted of:

The U.S. Agency for International Development (USAID) Food and Nutrition Technical Assistance (FANTA) Project ‘Water and Sanitation Measurements Guide’ (Guide) (Billig et al. 1999) provided the basis for the original health impact study and was used in the sustainability evaluations for consistency. ARC requested use of the eight performance indicators for the health impact study. We continued the sustainability evaluations using four of eight performance indicators as we could reliably collect data for those four. Performance indicators were a consistent way to evaluate WASH interventions over time (Table 2).

We used monitoring indicators to evaluate the progress of the intervention toward achieving its programmatic goal. Monitoring indicators assess both water access and access to sanitation facilities. We based the water access indicator on the FANTA Guide definition that includes connection to a piped system, distance to water and reported year-round water availability. Access to a sanitation facility means the presence of a private/shared facility in close proximity to the home. This indicator does not measure whether families use the facility, but rather the physical presence of the facility. An improved sanitation facility in rural areas is a dry pit latrine, ventilated improved pit (VIP) latrine, pour-flush latrine or composting latrine.

We used impact indicators to assess the effect the interventions have on behavior, such as handwashing and use/maintenance of sanitation facilities. Appropriate handwashing knowledge is based on both the interviewee’s self-reported ability to recite – unprompted – critical times at which handwashing occurs and the interviewee’s ability to demonstrate specific handwashing techniques. Sanitation facilities were assessed using a standard checklist in the household survey to determine use and if they were hygienic.

We determined sample size by region rather than by community. We calculated the number of households needed to conduct statistical analyses of handwashing behaviors. The handwashing behavior indicator required the largest sample size. The target sample size for the region was 94 households, a range of 14–16 households in each of six study areas, with a probability of alpha = 0.05 and 80% power. We collected data using a form developed in EPI INFO 2002 (CDC 2003) and did additional statistical analyses using SAS software versions 9.1 and 9.2 (SAS Institute, Inc. 2002–2003). We compared survey results using Chi-square odds ratios to determine differences between the years.

We collected water quality samples from stored household drinking water, community water sources and tap water from the distribution system. Given that the focus of this paper is on sustainability in terms of the indicators described above, we report only generalizable water sampling results.

The ‘Sustainability of WASH interventions’ section contains results of the sustainability evaluation using the performance indicators. The household participation rate was 100% in six study areas (eight communities). The ‘Factors influencing sustainability of WASH interventions’ section describes the factors we identified through evaluation of community and household surveys.

Several limitations in this sustainability evaluation could influence our results. Owing to limited resources, the CDC designed overall sample sizes for the 2006 and 2009 sustainability evaluations to detect expected differences in the USAID WASH indicators (Table 2) over the entire region rather than at the study area level. Sample size calculations did not account for clustering and the design effect could be large. Thus, sample sizes only allowed for statistical analyses at a regional level, across all study areas combined, whereas only trends in the USAID WASH indicators are observable at the study area level.

Additionally, limitations due to the random selection of households within the study areas might have changed some of the evaluation results. New households might never have received certain ARC WASH interventions (e.g. water system access, sanitation facilities, hygiene promotion) before or after 2002. When conducting the surveys for the 2006 and 2009 sustainability evaluations, these households were included in the random selection. These households may not have known who provided WASH interventions to their communities. If these households did not receive certain interventions and were unaware of who provided them, it would be difficult to assess accurately the sustainability of these interventions.

Self-reporting of data is also a limitation in this evaluation. CDC included self-reported knowledge and observed practice of appropriate handwashing behavior in each household according to the FANTA Guide. Ram (2010) recognized that self-reporting of handwashing knowledge is not a valid measure of actual behavior; however, we collected data in this manner to be consistent from survey to survey. This indicator introduces bias since observed and evaluated respondents may have modified their normal handwashing techniques. To lessen the impact of this limitation, interviewers received thorough training in the proper way to conduct this survey.

Finally, we did not initially use these indicators to measure sustainability. CDC used the same indicators from the health impact study and added questions to the survey to identify the factors associated with long-term sustainability. Two indicators, for example, measure sanitation access and use. Our results show that although access to sanitation appears sustainable (monitoring indicator), use of sanitation facilities did decline (impact indicator) as we have noted in our results. We must consider indicator results collectively to get a better picture of the sustainability of each intervention. Despite such limitations, our evaluation was a unique opportunity to follow the same study areas over a number of years.

The goal of this evaluation was to observe and measure the sustainability of infrastructure and hygiene interventions once communities began the operation and maintenance of these interventions, with no additional ARC follow-up. We wanted to identify possible factors that influenced WASH intervention sustainability. Community participation and input did not vary widely across the region and would not have a differential impact on sustainability in this study.

The primary factors that influenced the sustainability of water systems were severe storm events, population changes and active water committees. Our results suggest that improving water system design by accounting for such weather events could promote sustainability. If available, using localized population growth estimates rather than standard rules of thumb or national growth estimates, could also help promote sustainability. Active water committees also enhanced water system sustainability, indicating that developing or identifying appropriate community-level capacity to manage water systems is an important consideration (Gelting & Ortolano 1998).

We can best evaluate sanitation interventions by looking at indicators for both access and use. ARC initially provided latrines to households, which require proper maintenance and use. Simply looking at access did not provide an adequate measure of sustainability. In some areas, latrines were still accessible and in use but were damaged or past their usable life, and could not therefore be considered sustainable. Thus, organizations beginning a sanitation intervention should consider what would happen at the end of the sanitation facilities’ design life. Planning for eventual replacement or pit emptying could be included at project initiation. Replacement could result from new interventions in the future or by training community members in techniques for constructing new latrines when needed, or making repairs to existing latrines. Most helpful to these communities will be developing training materials that emphasize the use of locally obtained materials for repairs or construction.

Rural communities, with limited resources and expertise, can benefit most from follow-up by local organizations once the initial WASH program is complete. The follow-up hygiene promotion that occurred in some of these communities, appeared in every instance to improve WASH intervention sustainability (appropriate handwashing behavior). Hygiene promotion programs involve teaching materials and/or technical assistance as opposed to capital investment in infrastructure. Organizations can accomplish ongoing follow-up hygiene promotion programs with relatively little expense using local resources available from municipalities, local health clinics or NGOs.

Our sustainability evaluation results can potentially guide future program start-up toward ensuring and enhancing the long-term sustainability of these types of projects and lead to programmatic change in international development organizations. The ARC has incorporated our results to make changes in their current WASH programming to reflect this. In addition, our results provide the background research for future studies to explore sustainability further, such as investigating which WASH interventions are most sustainable and identifying additional factors contributing to sustainability. If interventions are not sustainable, health and other benefits derived from them are lost. Thus for all water and sanitation interventions, future investigators need to address sustainability issues.