Tables 1 and 2 present the number of participants per country that were included in the initial contact lists for key informant interviews and web-based surveys, the number of valid contacts (i.e., person could be reached by available phone or email), and the number of completed interviews and surveys. As expected, not all information from the original contact lists was current and valid, and not all those with a valid email or other contact information provided a response. In one country, staff turnover between the time of the workshop and the time of the interviews made identifying an appropriate representative or alternate difficult, and several countries indicated that a single representative could provide sufficient feedback on behalf of the planning team. We conducted an interview with representatives of four out of the five country planning teams; in cases where more than one representative provided feedback, we combined country-level feedback.

For the web-based surveys, we obtained 92 valid email addresses from the original participant lists and sent each an invitation with up to two reminders to participate. No alternates to the original participants were sought or further tracking of contact information was conducted. A total of 38 survey respondents represented 42.1% of those who received an invitation to participate. The majority of respondents (52.6%, n = 20) represented an MOH or other national agency, followed by a PEPFAR-funded implementing partner organization (28.9%, n = 11) and PEPFAR agencies (15.7%, n = 6). The majority of respondents worked in strategic information (57.9%, n = 22), followed by HTS program managers (23.7%, n = 9) and other HTS service providers (7.9%, n = 3).

The purpose of these data use workshops was to build capacity among HTS program providers and SI staff on using their own data to inform program decision-making at relevant national, province, district, program, or site levels. Through demonstration, discussion, and hands-on small group exercises, the processes for strengthening data use included collation and review of available data; data management; integration into a unified dataset; data exploration and analysis; and interpretation and presentation of tables, graphs, and maps created during the workshop. The program decision-making component of the workshops included facilitated discussion to identify recommendations and next steps for refining program strategies and data monitoring. During these discussions, we encouraged the group to specify timelines and persons responsible for those next steps. We specifically designed the workshops to include both program and SI staff so that all would have opportunities to learn how to answer specific, relevant questions about their programs in relation to the health burden in the communities they serve.

In the KI interviews, we were told that the data use tools were generally easier for SI staff to use than they were for other HIV prevention staff, but that even those whose jobs may not require frequent production of data illustrations found it was helpful to understand what the tools could offer. Non-SI participants indicated that this helped them in communicating to SI staff the types of program questions they would like to use data to answer. Since it was not the objective of the workshop to turn all participants into “SI folks,” we feel that both of these points underscore successes in the types of capacity building the workshops aimed to achieve.

KIs also indicated that one particularly useful—and new to them—type of analysis that resulted from these data reviews included identifying HTS modalities that yielded higher proportions of HIV-positive results (relative to the overall number of tests delivered). The UNAIDS 90-90-90 targets (UNAIDS, 2014a) have increased the spotlight on programs increasing the proportion of PLHIV who are aware of their status and linked to care: critically examining which modalities are yielding higher numbers of HIV diagnoses is essential to achieving this first 90. Unfortunately, understanding which HTS programs are most successful at linking those who were diagnosed with HIV to care is not yet achievable in many countries given the limitations on this type of data, although recent recommendations on how to do this have become available (World Health Organization, 2015).

Importantly, most survey respondents indicated that their participation in the workshops increased their confidence in both interpreting data and in assessing target achievements. This is especially important given that the majority of the respondents represented the MOH or other government agencies. While the United States President’s Emergency Plan for AIDS Relief (PEPFAR) has introduced an extensive set of new tools and guidelines for analyzing data to set and monitor PEPFAR program targets in 2015 (PEPFAR, 2014), the audience most likely to benefit from ongoing capacity building in this realm continues to be national program staff. Relatedly, several KIs indicated that in order to improve future use of data for decision-making it would be helpful to have a more ‘automated’ process. It may benefit program and SI staff to work together to identify outputs that would be most relevant to their contexts and create templates or dashboards to auto-populate quickly and easily as data are updated over time. Such tools are increasingly a component of PEPFAR data monitoring systems and have been made available in other contexts as well (Nutley et al., 2013).

During the time that these workshops were implemented, it was difficult to integrate expenditure or other program cost data into the data use tools. This restricted the kinds of analyses that could be conducted and conclusions that could be drawn related to best use of limited resources. When PEPFAR (2015) shifted its approach to country program planning in 2015, major improvements for data use in regards to HIV testing programs included (1) site-level yield analyses that were not available before; (2) use of expenditure data, which had previously been difficult to share between country and headquarters offices; and (3) alignment of testing targets (i.e., numbers testing HIV positive) to achieve 80% ART coverage, which had not been formally built into prior planning cycles. The extent to which this shift in approach yields more efficiencies and impact within PEPFAR-supported programs will be observable in the years to come. Greater efforts to incorporate financial data into decision-making tools for national health programs and program components supported by other donors may also be needed.

Finally, participants indicated that program decision-makers may not have always changed program strategies based on epidemiologic data such as geographic disease burden or program success in case identification. In our experience, discussions on next steps for HTS programs were typically left as preliminary and pending further discussion with larger decision-making bodies. Previous efforts to increase data use in decision-making have similarly acknowledged that hard data is often only one element in the types of inputs that guide decision-making by health program managers (Mutemwa, 2006). In Zambia, for instance, Mutemwa (2008) found that use of HMIS data for program decision-making at the health district level was typically integrated with other data sources such as verbal or observational reports from program staff, program manager’s personal experience or training, and the organizational culture or structure. Without a clear mandate to weight these kinds of data analyses above potentially many other factors that drive local program decisions, it was often difficult to draw a clear line from the implementation of a data use workshop and a specific programmatic shift. It will continue to require in-country leadership to determine how best to act on evidence to improve the overall effectiveness and impact of HTS and other HIV prevention programs.

Our evaluation of the data use workshop process was limited in the availability of contacts, given the delay between the workshops and the interviews and surveys. Depending on when the most recent workshop was held in a country, some of the interviews and surveys may have occurred from 1 to 3 years post-workshop. Therefore, in addition to the potential biases from self-report or recall bias, incomplete or outdated contact information from original participant lists may have yielded a potentially biased sample of past workshop participants. However, the rate of responses among those with valid email addresses was acceptable in the context of web-based surveys more generally (Cook, Heath, & Thompson, 2000). The HTS workshops that were delivered during this period were also focused primarily on questions of program targeting and alignment with populations with disproportionate burden. While scaling up HTS programs certainly require capacity to align those services with client needs, it is also important to monitor program quality, i.e., providing correct test results and providing high quality IEC materials, post-test counseling, and referrals and/or linkages to other health services as appropriate for the client. To the extent that other relevant data sources are available, they could be integrated into the data use workshop structure to address questions of program quality.

These data use workshops provided accessible tools that could be readily learned by the majority of HIV testing program staff, including those with limited prior experience using spreadsheet or mapping software. The framing of the workshop around a series of program questions created a flexible platform to analyze data at any administrative level for which data were available, and which could be easily adapted to address questions for other health programs. Indeed, staff from Zambia and South Africa both reported interest in using these tools to examine other areas of their HIV programs and services, and staff from Malawi expressed interest in applying the approach to their maternal and child health programs (Spindler, personal communication, July, 2014). We developed a set of examples and instructions on the UCSF website (http://globalhealthsciences.ucsf.edu/prevention-public-health-group/global-strategic-information-gsi/gsi-courses-and-materials/data-use) as reference materials for workshop participants, which may also foster the adaptation of these tools for other program purposes.

Data use for program decision-making is increasingly a skill needed in resource-limited health programs, and it is likely to be most successful in guiding evidence-based changes to national or donor-funded programs when there are more tools, capacity, and formal processes to do so. Such tools should also be applied to routinely monitoring programs in order to identify areas where expected outputs are not being met or where quality may need improvement. While processes for doing this have recently been more formally integrated into PEPFAR’s approach to HIV program planning (PEPFAR, 2014), governments with fewer resources or lower quality data systems may still require significant ongoing support to build capacity for data use in health program decision making. This may especially be true where much of the decision-making still occurs at lower levels in decentralized national health systems. We hope that these findings can be useful for future capacity-building efforts in HTS as well as other health program areas.