Although much progress has been made in the last decade, 7.6 million children worldwide still die each year before their fifth birthday, and half of the 4.5 million deaths among children between one and 59 months old are due to pneumonia, diarrhea, and malaria [1]. These three diseases all have effective treatment that can be delivered within community- or facility-based settings [2–4]. Evidence from field studies of routine care support the importance of correct case management for preventing child deaths, with an analysis of outpatient visits for sick children in western Kenya finding that appropriate health facility-based treatment reduced mortality by 78 % [5].

The Integrated Management of Childhood Illness (IMCI) strategy was designed to address the major causes of childhood mortality in a comprehensive manner and was first introduced the 1990s [6, 7]. IMCI has been recommended by the World Health Organization (WHO) and the United Nations Children’s Fund (UNICEF) and implemented in more than 100 low- and middle-income countries [8]. As the symptoms of childhood illnesses can be non-specific and overlapping, with co-morbidities common, IMCI provides an algorithmic framework for systematically assessing, classifying, and treating common childhood illnesses [9]. The cornerstone of IMCI, along with strengthening health system supports and community health practices, is improving the skills of facility-based health workers through IMCI training, an 11-day course focused on syndrome-based management of children for identifying and treating illnesses. Evaluations of IMCI have found mixed results, with improvements in disease classification, treatment, and counseling in several countries [10, 11], but inadequacies in overall implementation, health system supports and community guidelines and practices [9, 12].

Despite significant improvements in quality of care after IMCI training, important performance gaps remain, with some studies indicating that about one-third of sick children still do not receive correct treatment for potentially life-threatening conditions after health workers receive IMCI training [11, 13]. Significant decreases in mortality after IMCI training have been found in a few studies [11, 14–18] but not in others [14, 15], likely due to both low use of the formal health care system, lack of health system infrastructure, and gaps in health worker performance.

To help close gaps in case-management quality, it is important to understand more fully the factors influencing health worker performance, but definitive evidence on determinants of health worker performance is lacking [19]. We had a unique opportunity to examine an unusually large number of outpatient consultations performed in Benin by IMCI-trained health workers, who were part of a larger trial of additional health worker supports after IMCI training. Drawing from more than 9,000 register records of consultations with children ages 2–59 months, we were able to examine multiple treatment outcomes, including three cross-cutting quality of care outcomes and seven disease-specific outcomes. Whereas previous analyses have typically focused on a single disease outcome (e.g., malaria), this analysis of 10 outcomes allowed us to assess factors that were consistently related to case-management quality across a range of potentially life-threatening illnesses (PLTI). This approach enabled us to determine the most important predictors of health worker performance for treatment of child illnesses. A previous analysis using this dataset focused on the trends in health worker performance over time following initial IMCI training [21] but not on other predictive factors. This analysis focuses on a wide array of other factors that might influence health worker practices. Although the IMCI guidelines have been updated since the time of data collection in 2001–2002, it is still relevant and informative to understand the reasons behind health worker non-adherence to guidelines, a problem that persists today [22, 23].

The 32 IMCI-trained health workers were all nurses working at 21 different health facilities: 14 smaller primary care clinics and 7 larger referral facilities. Altogether 13,324 visits were recorded in the 12 months following IMCI training, and of these 9,402 initial consultations were abstracted. Nine initial consultations were excluded because of implausible values, and 63 were excluded because they occurred on the day of a supervisor’s post-IMCI training follow-up visit (about four to six weeks after the IMCI course) and were not thought to be representative of typical consultations. The remaining 9,330 initial consultations were screened for eligibility in the analysis. Each health worker had a median of 252 consultations in the dataset (range: 31–704). As a measure of data completeness, we calculated that 98.4 % of required IMCI assessment tasks were documented by health workers in the registers.

Among the 9,330 initial consultations, 8,277 (88.7 %) involved a PLTI according to gold standard IMCI algorithms, including malaria/fever (6,872, or 73.6 %), anemia (3,133 or 33.6 %), pneumonia (2,787 or 29.9 %), and diarrhea (1,680 or 18.0 %) (Table 1 and Box 1). Among children with a PLTI, 41.0 % had one PLTI, 41.4 % had two, 16.3 % had three, and 1.3 % had four.

For children with at least one PLTI, 63.6 % were treated exactly according to IMCI guidelines, and 77.8 % were treated adequately (i.e., effective treatment even if it did not exactly match IMCI guidelines) (Table 1 and Box 2). Performance varied widely by health worker, ranging from an average of 15 % to 88 % of children receiving recommended treatment (Fig. 1). The mean percentage of total IMCI tasks (assessment, classification, and treatment) done per patient was quite high, at 96.5 %. This result was primarily driven by the large number of assessment tasks (maximum of 39) that were very thoroughly performed, and by the generally correct classifications (mean of 93.2 %), despite much lower levels of correct treatment (74.2 %).Fig. 1

Graph of recommended treatment by individual health worker. Note: Vertical lines represent confidence intervals

To understand what might have influenced health worker performance after IMCI training, we assessed a broad set of factors across the most common child illnesses in Benin. Mean health worker performance, though significantly improved after IMCI training, especially for health workers who received additional supports [20], was still insufficient, with only 63.6 % of children with PLTI receiving recommended treatment. Health workers performed thorough assessment and classification tasks, but fell short on providing recommended treatments to children. Our analyses showed that a range of factors were associated with case management quality, which has been seen in other studies of health worker performance [24–26]. In general, factors largely outside managers’ direct control, such as patient-level factors and health worker age, were the most important predictors of case-management quality. Factors more directly within program managers’ control, such as frequency of supervisory visits or in-service training, were found to have lesser effects on quality of care.

Increasing health worker age was associated with consistently worse performance (for 7/10 outcomes), suggesting that older health workers should be prioritized for quality improvement strategies. To our knowledge, only one previous study, also from Benin, found worse performance among older health workers [27]. Health worker age and experience were highly correlated in our analysis, and previous studies have found improved performance with greater health worker experience [28]. While it is possible that health worker age might be a proxy for another unmeasured characteristic besides experience, several plausible explanations exist for this finding. Older health workers, who also had more years of experience, may have relied more on their clinical experience and intuition and been less apt to follow guidelines, in line with other studies that have found that higher-level cadres of health workers tend to perform less well at following new guidelines than lower-level cadres [29, 30]. Alternatively, older health workers may be more likely to follow guidelines learned during initial pre-service or early in-service training, when they were more open to new ideas. It is also possible that older health workers were older than IMCI trainers and perceived the trainers to be less knowledgeable.

Children ages one to four years old received worse care than infants for most outcomes, an association found in another study of IMCI-trained health workers in Morocco [24]. Two other studies, neither of which were of IMCI-trained health workers, found worse care for infants [25, 31]. This finding, which perhaps results from the perception that younger children are particularly vulnerable, supports the importance during initial IMCI training and routine supervision visits of emphasizing the vulnerability of all children under five years.

Increased case complexity, defined as the number of IMCI tasks required, was associated with worse performance in terms of recommended treatment, adherence to guidelines, and anemia treatment. A likely explanation is that with greater case complexity there are more opportunities for errors. A good example of this is the low scores for recommended anemia treatment, which generally involves prescribing three different medicines (antimalarial, iron, and folate), all in correct doses. For more complex treatments such as that for anemia, more time during initial IMCI training and post-training supervision should be devoted to reviewing the treatment and reminding health workers to consistently prescribe all three medications.

Presence of a danger sign was consistently associated with worse care except for anemia treatment, for which it was associated with better care. Correct care for children with a danger sign involved referral plus any required pre-referral treatment, which was often skipped. As there is no pre-referral treatment for anemia, referral was sufficient.

We found a complex relationship between patient complaints and quality of care. In some cases, complaints probably prompted health workers to give a needed medicine—for example, antimalarials for children with anemia and ORS for children with diarrhea. This positive relationship was found in several previous studies of fever treatment [32–34]. However, there is not always a simple correspondence between a complaint and correct treatment. In our study, fever complaint was not associated with better fever care. Furthermore, patient complaints can paradoxically be associated with worse care for other diseases. For example, a complaint of cough or difficult breathing, which may have prompted health workers to zero in on pneumonia diagnosis, was associated with worse diarrhea treatment.

Significant predictive factors of health worker performance that are potentially more modifiable (e.g., through quality improvement interventions) were less consistently related to better case management quality. The number of supervision visits the health worker had in the past six months was positively related to several outcomes, including anemia treatment, but was associated with worse pneumonia treatment. One possible explanation is that the IMCI training and subsequent supervision was focused primarily on fever/malaria treatment, which was a larger perceived problem at the time in Benin. For adequate treatment of fever, additional supervision visits also appeared to attenuate the worse performance with increasing health worker age (see Additional file 2: Web Appendix 2). A review of the effects of supervision in low- and middle-income countries has found limited high-quality evidence and uncertain results of its impact [35].

Finally, higher patient volumes the day of the child’s consultation was associated with significantly better care for some outcomes. Higher volumes of IMCI patients perhaps afforded health workers more opportunities to practice their new skills, although we cannot rule out another underlying factor, given the small number of consultations (6.5 %) that took place on days where the health worker saw at least 10 IMCI patients. It should also be noted that the volumes in these facilities in Benin are on average quite low, and thus higher volumes in this setting would not overwhelm health workers or result in extremely limited consultation time as they might in other settings [27, 36].

Despite its limitations, this study offers several important findings that have implications for case management in Benin and other settings. First, a range of factors at various levels can influence case-management quality, and many of these influences can be outside a manager’s direct control. For example, health worker age had one of the most consistently strong negative influences on case management quality. Although health worker age might be a proxy for another factor (e.g., experience or resistance to change) more directly related to poor performance, it is a measurable indicator and points to the importance of evaluating individual health worker performance on a regular basis and having targeted follow-up with poorly performing individuals.

Significantly poorer treatment for children with danger signs or anemia and some evidence of poorer treatment for more complex cases indicates potential gaps of IMCI training and supervision in addressing these types of patients. As real-life examples of children with danger signs or more complex patient presentations might be uncommon during initial IMCI practical training, IMCI program managers should take advantage of training modalities such as videos to expose health workers to more complex cases. In addition, supervisors should review documented cases of children with danger signs or anemia during supervision visits and refresher training and remind health workers about co-morbidities and the importance of assessing and addressing all signs and symptoms. This would also make consultations less driven by patient complaints, for which we saw some evidence, and more by comprehensive health worker assessment and classifications.

It is encouraging to see that additional supervision visits were associated with better performance across a number of outcomes, as previous evidence on supervision has been mixed [35]. A separate assessment found that the quality of supervision was generally good [45], and the supervision process was specifically tailored to improve performance for IMCI-recommended tasks. However, the low levels of performance of certain health workers (see Fig. 1), which did not change significantly over time [21], suggest that supervisors should focus more attention on underperforming health workers. In addition to more structured supervision, and regular on-the-job and refresher trainings, program managers should also consider creative strategies, such as group problem solving (including improvement collaboratives [46]) or text message reminders to health workers’ mobile phones [47, 48] or other forms of technology such as electronic job aids [49], to improve particularly challenging performance issues.