Background:

Despite widespread interest in computerized vaccination information systems, evaluation of the data quality in these systems and their acceptability to frontline healthcare workers in low and middle-income countries aren’t well addressed in the literature.

Objectives:

Evaluation of vaccination data quality and facility-level staff perspectives on the strengths and challenges of a vaccination data module in a widely used electronic health record (EHR) system in Zambia.

Methods:

After a desk review of data from two provinces, a cross-sectional mixed methods study was designed, including quantitative analysis of data quality and qualitative analysis of the module’s acceptability to facility staff, using the Information System Success model as the framework for evaluation of system quality, service quality, and information quality. Data were collected from 10 purposively sampled health facilities.

Results:

There was low current use of the vaccination module by facilities in the study area (2%). Daily power outages presented a practical challenge. Staff who had used previous EHRs had concerns about sustainability.

System quality:

While the module was user-friendly, there were concerns about EHR compatibility with vaccination workflow and outreach settings, where vaccines are commonly administered to older children.

Service quality:

The module was viewed as dependable; perceptions were influenced by computer literacy.

Information quality:

The database contained incomplete and incongruous data. Staff perceived data as accurate but incomplete; easy access to data was a strength.

Conclusions:

Potential benefits of the vaccination module were frequently unrealized due to infrastructure, workflow, and data flow challenges that resulted in low module use and poor information quality. Elements to optimize vaccination information system implementation could include robust engagement of facility-level staff in system design, system suitability to the vaccination setting and workflow, and comprehensive planning for data flow, sustainability, data monitoring and feedback. Adaptability to the outreach setting might be increasingly important as vaccination schedules extend past infancy.