There were 800 CAP encounters during the study period, including 413 (52%) in the preintervention and 387 (48%) in the postintervention period. Children were slightly older in the postintervention period (36 vs 44 months, P < .01) and were less likely to have a viral ICD-10 code (17% vs 11%, P = .02). Otherwise, there was no significant difference in other clinical or demographic variables (Table 1). The proportion of CAP encounters with an appropriate antibiotic choice and duration increased from 52% in the preintervention period to 80% in the postintervention period (P < .01). This improvement was driven by increases in appropriate antibiotic duration (54% vs 83%, P < .01), whereas appropriate antibiotic choice was high in both the baseline and intervention periods (95% vs 94%). The decrease in antibiotic duration was driven by shortening discharge durations (5 vs 4 days, P < .01) (Table 2).

The interrupted time series analysis demonstrated that there was no significant change over time in the proportion of appropriate antibiotic use before the intervention. However, there was an immediate absolute increase of 18% in the proportion of encounters receiving the appropriate antibiotic choice and duration (95% confidence interval [CI]: 3%–33%) at the time of the intervention, with no further change in the time trend following the intervention (−0.3% per month, 95% CI: −2% to 2%). When adherence to the antibiotic choice and duration measures were analyzed separately, the intervention had no impact on antibiotic choice but was associated with an immediate absolute 20% increase in the proportion of encounters in which antibiotic duration was appropriate (95% CI: 5%–36%) (Figure 2).

In the modified Poisson model adjusted for site, age, sex, race, PICU admission, and season, the likelihood of a patient receiving the appropriate antibiotic choice and duration did not change significantly over time before the intervention. However, there was an immediate increase in the likelihood of appropriate antibiotic choice and duration after intervention start (RR 1.32, 95% CI: 1.12–1.56, P < .01), with no further change in trend. PICU admission (RR 1.16, 95% CI: 1.02–1.32, P = .03) and admission to the community-based hospital (RR 1.18, 95% CI: 1.07–1.30, P≤.01) were also associated with an increased likelihood of appropriate antibiotic prescribing (Table 3). When antibiotic choice and duration were analyzed separately, the intervention had no impact on the likelihood of appropriate antibiotic choice (RR 0.96, 95% CI: 0.91–1.02, P = .21) but was associated with an increase in the probability of receiving the appropriate antibiotic duration (RR 1.35, 95% CI: 1.16–1.58, P < .01). There was no significant difference in the preintervention versus postintervention periods in the proportion of CAP encounters in which there was any revisit (3% vs 5%, P = .15) or readmission (2% vs 4%, P = .15).

A sensitivity analysis excluding 39 patients who received three or fewer days of antibiotics produced results similar to the primary analysis (RR 1.37, 95% CI: 1.15–1.63, P < .01). Similar findings were demonstrated in a sensitivity analysis limited to those with bacterial CAP ICD-10 codes, excluding 116 encounters with viral diagnosis codes only (RR 1.37, 95% CI: 1.13–1.64, P < .01). Finally, when the 163 patients with β-lactam allergy, a positive blood culture, a respiratory culture positive for S. aureus, or concurrent otitis media or pharyngitis were excluded, the association between the intervention and appropriate antibiotic use was stronger (RR of 1.55, 95% CI of 1.25–1.93, P < .01).

We demonstrated the effectiveness of group-level clinician feedback reports in improving adherence to antibiotic use guidelines among academic general pediatrics clinicians within our pediatric health system. These electronically delivered feedback reports, coupled with clinician education, an established local guideline, and regular review of the metrics at divisional QI meetings, resulted in a 32% increase in the rate of appropriate antibiotic use for CAP over the 1-year postintervention period, driven by improved adherence to antibiotic duration recommendations. These findings support the use of clinician feedback reports as a stewardship tool for inpatient ASPs, an approach made more feasible through use of electronically derived measures of antibiotic appropriateness.

Consistent with other reports, antibiotic treatment duration was identified as the key opportunity for improved stewardship locally [5, 7, 26–31]. We observed an immediate improvement in adherence to appropriate antibiotic duration recommendations at the start of our intervention, with no significant change over time. This is likely due to a “ceiling effect,” as no additional new interventions to improve performance were implemented beyond the monthly feedback reports, by design. Similar improvements in adherence to guideline-recommended antibiotic duration have been demonstrated in other studies utilizing combinations of clinician education, local guidelines, and audit with feedback [10–13]. For example, Foolad and colleagues conducted a multicenter study in which infectious diseases pharmacists reviewed patients with an admitting diagnosis of CAP and provided verbal recommendations to stop antibiotics 72 hours after patients achieved clinical stability. The median treatment duration was 3 days shorter in the intervention group (6 vs 9 days, P < .001), and the proportion of patients receiving 5 days of therapy was higher in the intervention group (35% vs 5%, P < .001) [11]. Monday et al performed a single center QI study and demonstrated that prospective audit with feedback performed by a non-infectious diseases physician reduced treatment duration from a median of 8 days to 5 days, with no difference in mortality, readmission, or length of stay [13]. Although ultimately successful, these interventions require significant human resources, whereas our approach may offer a less resource intensive strategy.

Several reports support the use of prescribing feedback reports in the ambulatory setting. Outpatient feedback reports generally summarize individual prescribing performance for a given condition or conditions, often with comparisons to peer clinicians [14–18]. There are several important differences between the inpatient and outpatient setting that impact feedback report implementation. First, unlike the ambulatory setting where a single clinician is generally making antibiotic prescribing decisions, inpatients are often managed by multiple clinicians, such that attribution of an individual prescribing decision to one person is difficult. For this reason, we developed group-level feedback reports, which aggregate all CAP encounters cared for by the general pediatrics service and compare these to historical performance. Second, and in contrast to outpatients who generally have 1 diagnosis and 1 antibiotic prescription, hospitalized patients may carry multiple infectious diagnoses and receive different antibiotics over the course of the admission and at discharge. This contributes to challenges with developing reliable metrics of appropriate antibiotic use, with few pediatric studies published to date establishing validated measures that do not rely on manual chart review [19, 32]. The group level electronic feedback reports deployed in this study were both feasible and effective tools for driving improvement, despite these differences.

These strengths notwithstanding, our study has several limitations. First, despite the use of robust statistical methods, we cannot definitively conclude that the observed changes were the result of our intervention, as opposed to an unmeasured time varying confounder. This is unlikely, however, given that we worked closely with key stakeholders, such that we should be aware of any potential confounders. Second, our definitions for CAP as well as the appropriate antibiotic choice and duration measures used in the feedback reports were developed and validated within a single health system and require external validation. Third, the impact of clinician feedback reports within our health system may not generalize to other settings, both because baseline adherence to the choice and duration metrics may differ across centers and because the acceptability of the feedback reports may vary. Furthermore, these reports were implemented in the setting of a longstanding guideline as well as a forum for reviewing performance metrics monthly, implementation supports which may differ across centers. Fourth, these metrics prioritize being practical for electronic implementation and may therefore be more permissive in classifying antibiotics as appropriate than would be feasible if real time chart review were conducted. For example, although many patients with a β-lactam allergy label do not truly have allergies and could therefore receive first line, narrow-spectrum antibiotics, it was not practical to assess the nature of the reported allergy, so any alternative antibiotic were classified as appropriate. Fifth, we implemented the emailed feedback reports with clinician education, monthly review of the metrics, and with an established CAP clinical care guideline. Our study was not designed to evaluate the impact of each of these separately, but rather, each of these was considered a fundamental element of high-quality audit and feedback report implementation [22]. Finally, although we were not able to detect a statistical difference in revisits following the intervention, we cannot rule out the possibility that rare adverse outcomes would have been detected with a larger cohort or that revisits or readmissions could have occurred outside of our system. That said, even if a low rate of deleterious outcomes are detected with shorter antibiotic durations, these must be balanced against potentially shorter hospital length of stay and reduced risk of antibiotic adverse events [33].

Overall, we demonstrated the effectiveness of audit and feedback reports, implemented with clinician education, monthly review of metrics, and a long-standing guideline in improving compliance with evidence-based recommendations for antibiotic duration in children hospitalized with CAP. Future work should focus on evaluating the impact, feasibility, and acceptability of inpatient feedback reports focused on CAP at other institutions, as well as developing similar reports for other infections.