This project was conducted in the KPCO Denver-Boulder metropolitan area. Kaiser Permanente of Colorado is a not-forprofit, integrated health care delivery system that in 2017 had more than 700 000 members, approximately 600 000 of whom were in the Denver-Boulder area. Members of KPCO receive outpatient health care at KP-owned medical facilities; facilities additionally provide laboratory, radiology, and pharmacy services. All KPCO facilities have fully integrated ambulatory EHR in all patient care areas with access from any KPCO location.

We applied a systematic, multicomponent approach to identify preanalytic and postanalytic laboratory medicine QI opportunities in the KPCO ambulatory care environment. We convened a Gap Identification Core Team (the Team) composed of local and national laboratory leaders, clinical content experts with diverse knowledge of quality issues, clinical medicine and pharmacy content experts, health information technology professionals, medical education specialists, evaluators, researchers, MDW experts, and project managers. For example, KPCO laboratory leaders who participated in the Team included the director of chemistry and toxicology (organizational leader), the director of pathology (physician champion; topic suggestions), the manager of laboratory QI (expertise on current laboratory QI projects), and the LIS manager (LIS capabilities; EHR interface). Team member expertise was instrumental in identifying, prioritizing, and selecting quality gaps. Because many evidence-based findings are not successfully implemented into routine clinical care, by carefully planning the composition of the Team and engaging key stakeholders early, our plan was to maximize the potential for success of this work. We intended to identify high-priority gaps that had local and national public health significance through proactively seeking KPCO sponsors and collaborators, educating internal stakeholders, and focusing on sustainability from the time of project initiation.

The Team interviewed 11 KPCO organizational and opinion leaders about QI priorities. These interviews established the local importance of the candidate gaps the Team had identified, while also identifying new potential quality gaps. The interviews also helped determine whether the local health care system was conducive to engagement with laboratory professionals to reduce the identified gaps. The leaders interviewed were selected because they represented expertise in clinical and operational quality, value, resource stewardship, health and prevention, chronic care, nursing, primary care, risk management, patient safety, and chemistry/toxicology. Interviews were conducted by project managers and/or the project lead and began with an explanation of project goals and objectives. After the interviewers answered any general questions the interviewees had about the project, the interviewers asked the leaders about KPCO QI priorities, complementary or competing ongoing initiatives, and challenges the leaders believed should be addressed. Leaders were asked to prioritize (ie, high, medium, or low) several candidate potential laboratory medicine quality gaps. The interviewers also asked questions about the leaders’ knowledge of the KPCO VDW. The qualitative input gathered during these interviews was incorporated into Team deliberations around potential gaps for intervention.

The MDW at KPCO, the VDW, is a comprehensive data repository that contains the timely, electronically linked data needed to characterize preanalytic and postanalytic laboratory medicine quality gaps. Key VDW content areas include demographics, enrollment, encounters, diagnoses, procedures, death, cause of death, tumor, census, pharmacy, vital signs, social history, provider, and LIS data such as laboratory results. Data tables are typically updated monthly or quarterly, but can be updated as often as daily if needed and if sufficient resources are available. Virtual data warehouse data lags range from essentially no lag (eg, laboratory results and other clinical data elements are available the same day) to 1 to 3 months (eg, external claims) to more than a year (eg, state death data). Virtual Data Warehouse tables are linked by a common, unique patient identifier that is different from the patient’s health record number. The crosswalk between the VDW patient identifier and the patient’s health record number is maintained in a separate data table for additional privacy and confidentiality protection. Implementation and operations of the VDW are governed by an operations committee. Ongoing Health Care Systems Research Network and KPCO processes ensure VDW quality is assessed and improved through programming and crowdsourcing via the user base.²² Virtual data warehouse data were used to determine if current performance for each potential laboratory quality gap could be quantified, and, if so, whether those data indicated a gap. In addition, for each potential gap, the Team evaluated the feasibility of measuring patient or system outcomes using VDW data.

To select potential quality gaps, the Team used a standard decision aid tool (see examples in Table 1 and in Supplemental Tables 1 and 2 [see supplemental digital content at www.archivesofpathology.org in the April 2019 table of contents]). The decision aid included health care system, patient, and data considerations. Completion of the decision aid for each potential gap was assisted by data and information gleaned from published literature, Team members’ knowledge, leader input, and VDW data. The national significance of each potential quality gap was also considered with this tool.

This work is a public health activity. The KPCO Institutional Review Board determined that this project did not meet the regulatory definition of research involving human subjects.

Preanalytic and postanalytic laboratory medicine quality gap topics preliminarily considered are shown in Table 2. Based on knowledge developed using the decision aid, the preliminary list was narrowed to 3 topics for further consideration: (1) test ordering; (2) diagnosis, detection, and documentation; and (3) high-risk medication monitoring. The completed decision aid for high-risk medication monitoring is shown in Table 1. Completed decision aids for test ordering and diagnosis, detection, and documentation are available in Supplemental Tables 1 and 2.

Each of these 3 gap topics had several potential subtopics. Given available resources, the only subtopics examined within each topic were those of national importance identified by leaders as of high or medium interest at KPCO. For test ordering, subtopics included (1) tests that should be ordered only in certain circumstances (ie, prostate-specific antigen [PSA] testing,²⁶ 1,25-dihydroxyvitamin D testing,²⁶ and genetic testing), and (2) obsolete test ordering (ie, creatine kinase–MB isoenzyme for acute myocardial infarction).²⁶ Within diagnosis, detection, and documentation, the subtopics were conditions where laboratory testing was integral to diagnosis, including (1) serum creatinine (SCr) and glomerular filtration rate for chronic kidney disease,^27,28 (2) glycosylated hemoglobin and fasting glucose for diabetes and prediabetes,^29–31 and (3) hemoglobin for anemia. For high-risk medication monitoring, the subtopics considered were areas where laboratory testing was important to safe and effective medication use. These included (1) international normalized ratio monitoring among patients with atrial fibrillation receiving warfarin and (2) alanine aminotransferase (ALT), aspartate aminotransferase (AST), complete blood count (CBC), and SCr monitoring among patients with a rheumatologic condition receiving a disease-modifying antirheumatic drug (DMARD).³²

The Team examined and deliberated on data extracted from the VDW and other evidence related to these potential quality gaps during 4 to 5 months. At the completion of this process, high-risk medication monitoring was selected as the laboratory medicine quality gap topic for future intervention, with ALT, AST, CBC, and SCr laboratory monitoring for DMARD therapy as the subtopic. A major consideration in selecting this quality gap was the potential for clinically important adverse events associated with DMARD use. For example, approximately 9% of patients taking the DMARD leflunomide have ALT and/or AST elevations exceeding 2- to 3-fold the upper limit of normal, with 2 to 4.9 per 10 000 patients per year experiencing leflunomide-associated hepatotoxicity of sufficient severity to require hospitalization.^33,34 Similarly, the DMARD methotrexate is associated with a 15% occurrence of ALT and/or AST abnormalities during the first year of therapy, with 5% of patients discontinuing methotrexate because of hepatotoxicity.³⁵ Risks of DMARD-associated adverse events such as hepatotoxicity are minimized if laboratory test results are monitored in a timely manner and DMARD dosages adjusted or the DMARD discontinued if laboratory abnormalities are present or emerging.³⁶

The VDW data tables accessed and linked to characterize the high-risk medication monitoring QI opportunity included enrollment, diagnoses (all individuals had at least 2 diagnoses of one of the following: rheumatoid arthritis, ankylosing spondylitis, psoriatic arthritis, or plaque psoriasis), laboratory results (dates of completion, result values, and reference ranges), pharmacy (dispensing dates and DMARD generic names), procedures (for DMARDs administered by infusion or injection), and death (an exclusion). From these VDW data, a gap in DMARD laboratory monitoring was characterized in guideline-concordant completion of laboratory tests by patients (timeliness of testing) (Table 3). For example, in patients treated with leflunomide or methotrexate, ALT, AST, CBC, and SCr are recommended to be evaluated at least every 12 weeks (84 days).³² In preliminarily characterizing the gap, we used VDW pharmacy and laboratory data to estimate the proportion of patients with guideline-concordant completion of ALT, AST, CBC, and SCr, allowing a grace period of 16 days (ie, testing within 100 days). As shown in Table 3, among patients treated with leflunomide or methotrexate, many patients did not make themselves available for a blood sample to be drawn. More than 100 days elapsed between ALT test completions in 328 (65.5%) and 922 patients (60.7%) taking leflunomide or methotrexate, respectively. Similarly, more than 100 days elapsed between completion of CBC testing in 288 (57.5%) and 847 patients (55.7%), respectively.

Additional considerations that contributed to selecting this quality gap were organizational leadership, rheumatology, pharmacy, and laboratory departmental interest and the presence of DMARD regulatory labeling and national and local guidelines for laboratory testing and frequencies of testing. A final consideration in selecting high-risk medication monitoring as the gap for intervention was an intent to demonstrate that laboratory professionals are positioned to have leadership roles not only in identifying and characterizing quality gaps, but also in developing preanalytic or postanalytic interventions that link to health outcomes and process improvement when data through an MDW are available for analysis.

A test-ordering subtopic was not selected as the gap for intervention largely because the organization was already addressing several testing subtopics, including low-risk PSA testing, 1,25-dihydroxyvitamin D testing, and creatine kinase–MB isoenzyme ordering. For example, data from the VDW illustrated that an ongoing initiative to decrease low-risk PSA testing was effective: in 2012 (preintervention), 36 168 PSA results were documented in the VDW laboratory results table, whereas in 2015 (postintervention), 19 498 PSA results were documented. Further information is in Supplemental Table 1.

A diagnosis, detection, and documentation subtopic was not selected for intervention because the impact of a laboratory-led QI intervention would have been difficult to separate from the impact of ongoing initiatives led by other departments. For example, data extracted using VDW enrollment, death, diagnosis, and laboratory result tables indicated that an existing initiative was effective at ensuring diabetes was documented: among 6154 patients with initial and follow-up glycosylated hemoglobin result values of 6.5% or higher, 5817 (94.5%) had a diabetes diagnosis within 1 year. Further, because of multiple ongoing efforts within the organization aimed at improving diagnosis, detection, and documentation subtopics, organizational leaders encouraged the Team to pursue a different gap topic for this project. Further details are in Supplemental Table 2.

These results demonstrate the utility of an MDW for providing data to document the presence or absence of quality gaps in preanalytic and postanalytic phases of laboratory medicine. To develop the information needed to identify QI opportunities, we used linked MDW data content that included enrollment, demographics, diagnoses, pharmacy, laboratory results, and death. We then extracted relevant data and examined the results to confirm performance gaps.

Although production databases such as EHR and LIS usually have the capability to interrogate data within that database (eg, the LIS can be queried to provide data about laboratory testing), production databases typically cannot access and link data from other databases. This technical capability is a major strength of MDWs that enables their use for identifying preanalytic and postanalytic laboratory medicine QI opportunities. For example, because MDW data include dates of medication dispensing and laboratory results, temporal relationships relevant to patient testing are readily derived. Further, the quality of the data in MDWs such as the KPCO VDW and the VDWs at other Health Care Systems Research Network sites is maintained and improved through routine, ongoing characterization and quality checks of content areas. Also, all sites that participate in a multisite MDW use the same data structure, supporting conduct of the same or similar studies at other sites and enabling cross-site comparisons.

The innovative approach we applied to use MDW data to select laboratory medicine QI opportunities is generalizable and transferable, as health systems increasingly have MDWs as well as health information technology capabilities that can use MDW data. The potential scope of MDW utility in laboratory medicine QI is significant when placed within the context of the role that laboratory testing plays in medical care. Laboratory tests are ordered during one-third of primary care encounters³⁷ and laboratory results inform diagnosis, treatment selection, and monitoring of conditions and therapies.² Up to 54% of errors reported by primary care physicians and staff are related to laboratory medicine.³⁷

Timeliness of guideline-concordant testing during DMARD therapy is a potential gap involving appropriate test utilization that the laboratory can identify and characterize using data available in an MDW. Characterizing this gap enables the laboratory to collaborate with clinicians to bring patients in for specimen collection (preanalytic) so that timely testing can be performed and reported (postanalytic) to inform medication adjustment, if needed. Contributing to appropriate clinical test utilization is within the laboratory’s role,³⁸ but is part of the role that is difficult to fulfill without linking laboratory data to patient and outcomes data via an MDW. This linkage provides opportunities to advance the laboratory’s role in improving health outcomes, considering that the intervention will include identifying patients who are not guideline concordant (a laboratory role relative to the DMARD gap).

Notably, our method included convening a multidisciplinary team to identify quality gaps and to select a gap for future intervention. The Team’s work was supported by a decision aid tool where relevant information was compiled and reviewed. The expertise of Team members was instrumental in identifying performance gaps. Additionally, Team engagement was crucial to developing commitment to the project, agreement on gap topics to pursue, awareness around MDW utility in quality initiatives, and building the trust necessary for laboratory professionals to initiate and effectively engage with the local health care system. Furthermore, we believe that the project work now underway (developing and implementing the intervention to improve timeliness of laboratory monitoring of DMARD therapy) is progressing smoothly in large part because of the multidisciplinary relationships established during the gap selection activities.

Interviewing leaders to determine their knowledge and QI priorities served the dual purpose of also educating them about preanalytic and postanalytic laboratory QI opportunities. The interviews also aided in developing the organizational commitments necessary to identify a locally important laboratory medicine QI gap and to establish engagement for a future intervention targeting that gap.

One fact the Team learned from the leader interviews was that, in a large organization such as KPCO, no single individual is knowledgeable about all ongoing QI initiatives or all complementary or competing priorities. This learning underscores the importance of the multidisciplinary process when identifying and planning for a successful laboratorycentric QI intervention that impacts the preanalytic or postanalytic phase of the total testing process.

The multidisciplinary team approach and organizational leader engagement efforts are key strategies to, and activities for, local success and internal sustainability, as well as strategies to enhance external generalizability and dissemination. The multistakeholder involvement approach intersects with a recommendation regarding the utility of diagnostic management teams (and health care teams in general) made in the Institute of Medicine³⁹ (now National Academy of Medicine) report, Improving Diagnosis in Health Care. It also intersects with laboratory test utilization teams. It is potentially more accurate to consider our project’s multidisciplinary team as a combination of a diagnostic management team and a laboratory test utilization team (ie, to improve test utilization in conjunction with an intervention/outreach). The Institute of Medicine report also emphasizes the importance of health information technologies such as MDWs to support patients and health care professionals.

A limitation to the transferability of our work is that although all MDWs contain administrative data, some MDWs do not include the clinical data needed to identify laboratory QI opportunities.^20,40 However, even at organizations with this constraint, MDW tables such as diagnosis, pharmacy, and procedures (eg, whether a laboratory test was completed) can be linked to identify QI opportunities.

In conclusion, providing laboratory access to clinical and other data contained within an MDW provides novel and important opportunities to identify and characterize quality gaps that may otherwise not be addressed. The process described also highlights the need for the laboratory to engage with other departments because patient and system outcomes include broad components of a health care delivery system. Our example demonstrated that the combination of laboratory, clinical, and administrative health care data available in a comprehensive MDW, together with the expertise of a multidisciplinary team of professionals, supplemented by organizational stakeholder engagement, is effective at identifying, prioritizing, and characterizing quality gaps in the preanalytic and postanalytic phases of laboratory testing. The baseline data developed and the information learned to date will inform the development and implementation of a specific QI intervention.