PE, a macroergonomic approach to promote the application of human factors and ergonomics to work system design, emphasizes the involvement of people in “planning and controlling a significant amount of their own work activities, with sufficient knowledge and power to influence both processes and outcomes in order to achieve desirable goals.”¹¹ A PE approach can vary along different dimensions, including permanency of the initiative, nature of involvement, level of influence, location of decision-making power, mix of participants, requirement of involvement, topics to be addressed, extent of involvement, and role of ergonomics specialists.¹⁷ While PE has been applied across a range of industries (for example, manufacturing,^{18, 19} construction,^{20, 21} mining²²), its application in health care has been limited, focusing on task and process design in individual health care settings.^{23, 24}

In this study, we broaden the application of PE to facilitate the dissemination of the program to the BrighT STAR collaborative. Two key PE principles were emphasized: (1) participation of different stakeholders in work system and process redesign and (2) application of human factors and ergonomics (HFE) knowledge and tools to work system and process redesign. The adaptation and implementation of the program at individual sites involved local stakeholders who could affect or be affected by the blood culture ordering process and applied different HFE design principles (for example, systems approach, shared mental model, usability) and implementation principles (for example, leadership support, stakeholder engagement, communication and feedback, learning and training, project management).²⁵ In addition, given the nature of the BrighT STAR collaborative, cross-site collaboration was emphasized to facilitate peer-to-peer learning²⁶ and create isomorphic pressures.²⁷

The BrighT STAR collaborative was guided by the coordinating team developing the program at JHH, which consisted of one pediatric infectious disease physician, two pediatric intensivists, one human factors engineer, one biostatistician, and three program coordinators. Fourteen teaching hospitals were recruited to participate in the BrighT STAR collaborative (Table 1). Each participating hospital convened a local QI team (Dissemination Process – Step 1), which worked with local stakeholders to develop and implement their own interventions, including CDS tools and other changes to the work systems (for example, institutional policy) and processes (such as daily rounds or huddles). The local QI teams routinely communicated with the coordinating team through various avenues, including an initial webinar, monthly individual site calls, monthly all-site calls, individual and group emails, and individual phone calls.

The dissemination process consisted of six steps over a period of 27 months (Figure 1). Table 2 describes each step of the dissemination process in detail.

To characterize the dissemination process, the coordinating team kept a diary for each participating hospital. Two program coordinators recorded real-time interactions (for example, emails, phone calls, conference calls) between each site and the coordinating team. Each entry included the date, purpose of the interaction, participants, and outcomes. Entries were jointly reviewed by the two coordinators on a weekly basis to ensure consistency.

Four surveys were conducted during individual site calls with local QI teams over time to examine their experiences with the dissemination process. A pre-implementation survey was conducted around each site’s kick-off with subsequent surveys approximately every six months. The four surveys consisted of similar questions assessing six domains of the dissemination process, including (1) leadership support (for example, How supportive is your unit leadership of the blood culture program?), (2) resources available (for example, How sufficient are the resources allocated for quality improvement in your ICU?), (3) engagement of champions (for example, To what extent is your blood culture program driven by individual champions who are influential on the unit?), (4) engagement of frontline staff (for example, How engaged is your frontline staff in the blood culture program?), (5) impact of the larger collaborative (for example, How useful has being part of the larger collaborative network been in facilitating your blood culture program?), and (6) impact of ongoing QI projects (for example, To what extent is your blood culture program hindered by ongoing quality improvement projects with competing goals?). All questions were assessed using 5-point Likert scales, with 5 representing the most positive response and 1 the most negative response. For each question, members of the local QI team were asked to discuss their opinions, achieve consensus, and provide a specific answer along the Likert scale. A copy of the pre-implementation survey is included in Appendix A. Minor revisions of the questions were made to align the surveys with the time point at which they were conducted.

A qualitative content analysis²⁹ of the site diaries was conducted. While conference calls included all-site calls and individual site calls, interactions through emails and phone calls were classified based on their purposes into five sub-categories: (1) data dialogue (for example, collecting monthly data on blood culture use and other clinical outcomes), (2) collection of site QI activities (for example, reminding or receiving site log detailing QI activities), (3) implementation facilitation (for example, addressing site questions about intervention design and implementation), (4) project administration (for example, providing support on regulatory logistics of running the project), and (5) scheduling (for example, setting up individual site calls). Using this node structure, two program coordinators coded the site diaries and swapped their coding for cross-checking. Discrepancies were discussed to achieve consensus.

The frequencies of different (sub-)categories of interactions between each site and the coordinating team during different phases of the participatory ergonomics process (pre-intervention, post-intervention, and total) were calculated. Poisson regression models were used to estimate the relative mean frequency of each (sub-)category, comparing post- to pre-intervention, as well as the relative mean frequency of each (sub-)category across sites, separately for each phase. Robust variance estimates were used to account for over- or under-dispersion in the data and clustering within site when comparing the different phases.

After each survey, each of the six domains of the dissemination process was scored (ranging from 1 to 5) by averaging the responses to all questions related to the domain. Next, a radar chart was created for each site to depict the changes of all six domain scores across the four surveys. The domain scores were then rescaled to a range of 1 (most negative and negative response), 2 (neutral response) and 3 (positive and most positive response); a hierarchical agglomerative cluster analysis with average linkage and Euclidean distance was conducted to define clusters of sites based on their responses to the pre- and post-intervention semi-annual surveys.

We correlated the site-specific change in blood culture rates, post- vs. pre-intervention, with dissemination process measures, including the number of interactions with the coordinating team, responses to the pre-intervention and post-intervention semi-annual surveys, and whether sites developed only CDS tools vs. making additional changes to other work system elements. For each process measure, a Poisson regression model was fit for the monthly blood culture rate as a function of a main effect of phase, post- vs. pre-intervention, a main effect of the process measure, and the interaction. Robust variance estimates were obtained to account for the clustering of monthly blood culture data within sites and for possible over-dispersion in the data.

The total number of interactions between a site and the coordinating team ranged from 120 to 205. Table 3 shows the frequencies of different (sub-)categories of interactions during the pre- and post-intervention phases. During the post-intervention phase, the sites had 50% fewer emails and phone calls focusing on implementation facilitation (relative mean frequency = 0.5, p = 0.002), but more emails and phone calls focusing on data dialogue (1.3, p < 0.001), collection of site QI activities (2.0, p < 0.001), and scheduling (2.4, p = 0.001); along with more conference calls in total (1.5, p = 0.002), and more individual site calls (1.8, p < 0.001) with the coordinating team compared to the pre-intervention phase.

In addition, across sites, there were significant variations in the total number of emails and phone calls (median = 113, interquartile range = [103, 140], p < 0.001) and the numbers of emails and phone calls focusing on data dialogue (33, [24, 39], p = 0.003), implementation facilitation (22, [17, 26], p = 0.002), project administration (23, [21, 40], p < 0.001), and scheduling (22, [18, 28], p = 0.004).

The radar charts and the cluster assignments for the 14 sites based on the pre-intervention and post-intervention semi-annual survey responses are shown in Appendixes B and C, respectively. At the pre-intervention survey, the vast majority of sites responded positively to the leadership support, engagement of frontline clinicians, engagement of champions, and larger collaborative domains. Variation was observed across sites in responses to the resources and ongoing QI domains, with five, four, and five sites responding negative, neutral, and positive, respectively, for both domains. When clustering sites according to the post-intervention survey responses, the vast majority of sites had stable positive responses to the leadership support, engagement of champions, larger collaborative, and ongoing-QI domains. For the resources domain, three clusters were identified as stable neutral (n = 7), worsening positive (n = 5), or stable positive (n = 2). For the engagement of frontline clinicians domain, four clusters were identified as stable positive (n = 7), improving neutral (n = 4), worsening positive (n = 1), and increasing then decreasing neutral (n = 1) sites.

The 14 sites developed different interventions for improving blood culture ordering practices (Table 4). All sites included CDS tools as part of their interventions: four sites developed a checklist to screen for signs of sepsis and potential reasons for fever, assess risk factors, and determine if a blood culture was indicated (see Figure 2a for an example); five sites developed a flowchart to illustrate their blood culture decision algorithm (see Figure 2b for an example); and five sites developed both a checklist and a flowchart. While most sites placed paper-based CDS tools in the clinician workroom and/or at patient bedside, four sites (sites 3, 8, 9, and 11) integrated their CDS tools into the electronic blood culture order set and/or electronic health records. In addition to the CDS tools, eight sites made additional changes to their local work systems and processes, including revising institutional policies to standardize blood culture ordering practices, integrating discussions about blood cultures into daily rounds or huddles, introducing a peripheral blood culture kit, developing educational materials for families, and requiring patient exam before ordering a blood culture.

The 14 sites applied various strategies to facilitate the implementation of their interventions (Table 4): all sites engaged hospital leadership and key stakeholders through division meetings, meetings with specialist groups, and/or direct emails; 11 sites provided training on the use of the CDS tools, the updated blood culture policies, and/or the practices for obtaining peripheral blood cultures to clinicians; 11 sites shared data on compliance measures and blood culture use and discussed specific cases with clinicians; nine sites collected feedback from key stakeholders to continuously improve intervention design and implementation; and 12 sites sent periodic reminders through emails, newsletters, and/or unit posters to keep key stakeholders engaged and informed of the interventions.

Overall, BrighT STAR collaborative sites reduced blood culture use (monthly number of blood cultures per 1000 patient days) by 32.7% when comparing the post- to pre-intervention phases (p < 0.001), with an estimated 149.4 blood cultures per 1000 patient-days/month pre-intervention and an estimated 100.5 blood cultures per 1000 patient-days/month post-intervention.¹²

There was a weak negative correlation between the total number of interactions during the pre-intervention phase and the change in blood culture rate (Appendix D). For every five additional interactions, the reduction in blood culture rates decreased by ~1% (p = 0.057). There was no association between change in blood culture rate and pre-intervention site assessment of the six domains of the dissemination process (Appendix C). When comparing the eight sites that made additional changes to local work systems and processes to the six sites that only developed CDS tools, we found no statistically significant difference in the reduction of blood culture use (34% vs. 31%, p = 0.650).