Geisinger Health System (GHS) is a non-profit physician-led, fully integrated health care delivery system in rural central and northeast Pennsylvania with a service area population base of 2.6 million people in 43 counties. The health system in 2010 consisted of 2 medical centers, an 829 physician multi-specialty group, and a 247,000 member health plan. The Community Practice Service Line (CPSL) of approximately 200 primary care providers located in 37 practices across the service area completed over 894,000 outpatient visits. GHS has been using a fully integrated EHR introduced in ambulatory care since 1996.

We conducted a 3-arm cluster randomized trial of different implementation strategies to reduce antibiotic use for uncomplicated acute bronchitis—a PDS strategy arm, a CDS strategy arm, and a control arm. We excluded 4 practices because the annual number of visits with a primary diagnosis of bronchitis (ICD9=466 or 490) was < 100 in a sample of three recent years. One site served as a test site for the CDS development, and was automatically assigned as a control site. Among the remaining practices, 9 large practices (with 9,000 to 15,000 annual patient visits) were randomly assigned to each study arm, and among the remaining 23 smaller practices (with 2,000 to 9,000 annual visits) we randomly assigned 8 practices to PDS and CDS arms, and 7 practices to the control arm.

Previous experience has shown that multidimensional implementation strategies are often more successful than strategies employing a single intervention tool.¹⁴ The PDS and CDS arms both received intervention components guided by the PRECEDE-PROCEED model, which included tools addressing predisposing factors (provider education and practice guidelines), reinforcing factors (clinical champions; audit/feedback) and enabling factors (patient education and decision support).¹⁵ Provider education was delivered by a clinical champion for each clinic who participated in a half-day training session (led by RG and JPM). Champions were provided with data on their specific clinic’s performance on the acute bronchitis HEDIS measure, and a teaching slide set to use when reviewing this information with the providers in their respective clinics. Patient education was provided through the provision of CDC brochures developed by the Get Smart: Know When Antibiotics Work program (“Cold or Flu. Antibiotics Don’t Work for You”) (www.cdc.gov/getsmart).

At PDS sites, patient educational brochures were provided by triage nurses to patients with cough illness as part of routine care, and a poster displaying the clinical algorithm decision support was placed in all examination rooms.¹⁶ (Figure 1) At CDS sites, when nurses entered “cough” as the chief complaint in the EHR, a “Best Practice Alert” for the nurse would appear, prompting the nurse to provide the educational brochure to the patient to read before being evaluated by the provider. At the CDS sites, the algorithm was programmed into the health system’s EHR (EpicCare®) by the health system’s information technology staff with input from doctors and nurses from a clinic site assigned to the control group, with the goals for the tool to enhance work efficiency, adapt to complex visits, and provide high-level documentation. The key features included a structured template for documenting relevant history and physical examination elements in patients with ARIs broadly. These elements provided the data necessary to categorize the patient’s probability of pneumonia based on the acute cough clinical algorithm. Groups of electronic order sets (Smart Sets) were created that simplified relevant testing and treatment options for bronchitis, pneumonia, sinusitis, URI and influenza.

Patients that were targeted by the intervention included all adolescents and adults (age ≥ 13 years) with an office visit for uncomplicated acute bronchitis during the baseline (2006–2007, 2007–2008, 2008–2009) and intervention (2009–2010) periods. We focused the intervention implementation during the periods of highest visit volumes for bronchitis; therefore, the analysis was restricted to visits occurring between October 1 through March 31 in each year. All clinicians caring for patients diagnosed with acute bronchitis were included in the analysis, including board-certified internal medicine and family practice physicians, nurse practitioners (NPs), physician assistants (PAs), and registered nurses (RNs). The clinicians at the control sites were not informed of the study’s objectives. Waivers of informed consent from patients and providers were obtained from the health system’s institutional review board.

Incident adult office visits with the diagnosis of acute bronchitis (International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) code 466.0) or bronchitis, not otherwise specified (ICD-9-CM code 490) during the specified periods were identified from the EHR. Although the HEDIS measure for acute bronchitis is based solely on ICD-9-CM code 466, we include both 466 and 490 because we previously found that significant coding shift toward the use of code 490 occurred within the study sites in efforts to improve performance on this measure.¹⁷ Incident acute bronchitis visits were defined as not having an office visit for any ARI during the previous 30 days. We defined a return visit as a subsequent visit occurring within 30 days of an incident visit. To calculate antibiotic prescription rates for uncomplicated acute bronchitis—the primary outcome for this study—we excluded visits by patients age less than 13 and greater than 64 years, or with secondary diagnoses of chronic heart and lung disease, HIV or malignancy, or with secondary diagnoses for ARIs for which antibiotic therapy might be indicated (sinusitis, pharyngitis, otitis media, pneumonia). (see Figure 2)

Information on electronic alert and Smart Set utilization is only captured at the record level, meaning that a count is registered each time an alert fires or Smart Set is opened. If a nurse or clinician closes the record in order to attend to another issue, then re-opens the record then the alert or Smart Set registers an additional count. Thus, alert and Smart Set counts will overestimate the utilization at the patient visit or provider levels. All office visit and prescription data were extracted from the EHR by individuals who were unaware of the allocation of randomization across the study sites. Other data collected for each visit from the medical record included patient age and gender, vital sign measures, smoking status and chest radiography ordering.

Generalized estimating equations and alternating logistic regression (PROC GENMOD in SAS) were used to control for clustering of antibiotic prescribing patterns by both practice site and provider using patient visits as the unit of analysis. These models also adjusted for patient and provider characteristics (at the visit level) that demonstrated a statistically significant change (p-value < 0.05) within study arms between baseline and intervention periods, including provider type, provider specialty, smoking status and vital sign abnormalities. Statistical significance of a change in antibiotic prescribing between periods was defined as p<0.05 for the interaction term between intervention status and baseline vs. intervention period. Adjusted odds ratios with 95% confidence intervals (CI) for antibiotic treatment during the intervention period were calculated from parameter estimates. As a secondary analysis, we also examined changes in antibiotic prescription rates at the provider-level in order to assess the variation in provider prescribing behavior change. We restricted this analysis to clinicians with at least 10 patient visits in each study period in order to achieve more precise estimates of baseline and intervention prescription rates [n=31 (46% of PDS providers); n=26 (63% of CDS providers); and n=27 (59% of control providers)]. All statistical analyses were performed using the Statistical Application Program (release 9.2, SAS Institute, Inc, Cary, NC).

This study was approved by the Institutional Review Board at the participating clinical site and at the University of Pennsylvania and University of California, San Francisco. The study was registered with Clinical Trials.Gov prior to enrolling patients (NCT00981994).