We developed a decision support tool that can guide the development of heart disease prevention programs to focus on the interventions that have the most potential to benefit populations. To use it, however, users need to know the prevalence of heart disease in the population that they wish to help. We sought to determine the accuracy with which the prevalence of heart disease can be estimated from health care claims data.
We compared estimates of disease prevalence based on insurance claims to estimates derived from manual health records in a stratified random sample of 480 patients aged 30 years or older who were enrolled at any time from August 1, 2007, through July 31, 2008 (N = 474,089) in HealthPartners insurance and had a HealthPartners Medical Group electronic record. We compared randomly selected development and validation samples to a subsample that was also enrolled on August 1, 2005 (n = 272,348). We also compared the records of patients who had a gap in enrollment of more than 31 days with those who did not, and compared patients who had no visits, only 1 visit, or 2 or more visits more than 31 days apart for heart disease.
Agreement between claims data and manual review was best in both the development and the validation samples (Cohen’s κ, 0.92, 95% confidence interval [CI], 0.87–0.97; and Cohen’s κ, 0.94, 95% CI, 0.89–0.98, respectively) when patients with only 1 visit were considered to have heart disease.
In this population, prevalence of heart disease can be estimated from claims data with acceptable accuracy.
We developed a spreadsheet–based decision support tool that helps the user determine which heart disease prevention and treatment interventions would be expected to have the biggest effect on mortality in a population (
Populating the decision support tool with US data, we found that implementing more effective primary and secondary prevention services could prevent or postpone as many as 56% of all deaths among people aged 30 to 84 years (
The Institute of Medicine report on a nationwide framework for surveillance of cardiovascular and chronic lung diseases observed that, as electronic health records become more ubiquitous and health information exchanges become operational, they could become powerful tools for improving health and relieving the burden of chronic diseases (
To identify the magnitude of the opportunities among the population with chronic heart disease, disease prevalence must be known. Health plan claims data are a potential source of prevalence data, but disparities in length of enrollment and gaps in coverage are threats to the validity of prevalence calculations. To test the extent to which claims data are affected by these issues, we compared diagnosis based on claims data to diagnosis based on manual record review in a stratified random sample of a population enrolled in a health plan and treated by the associated medical group.
This record review was approved by the HealthPartners Research Foundation institutional review board on November 20, 2008, as protocol number 08–093. The study was completed on December 29, 2011, and was conducted in Minneapolis, Minnesota.
The study sample was drawn from people aged 30 years or older who were covered by HealthPartners insurance and received any type of care from HealthPartners medical group. We characterized these people by 3 attributes: any length of enrollment versus long enrollment, a gap in enrollment, and number of visits for heart disease. The “any enrollment” group comprised patients who were enrolled at any time from August 1, 2007, through July 31, 2008, and may have been enrolled on August 1, 2005. The “long enrollment” group comprised patients who were enrolled at any time from August 1, 2007, through July 31, 2008, and were also enrolled on August 1, 2005. Patients were considered to have a gap in enrollment if they had an enrollment gap of more than 31 days during August 1, 2007, through July 31, 2008. Three patterns of visits for heart disease (
A cardiologist (T.E.K. or C.J.B.) manually reviewed the record of each patient for evidence that the patient had heart disease. To increase the probability of detecting references to heart disease in the free-text portion of the record, we used the electronic health record text search function to search all records for the following words: angiogram, atherosclerosis, bypass, cardiac stress test, coronary, echocardiogram, ejection fraction, heart attack, heart disease, heart failure, infarct, ischemic heart disease, myocardial perfusion, and sestamibi. We also searched the records for the following acronyms: AMI (acute myocardial infarction), CABG (coronary artery bypass graft), CAD (coronary artery disease), CHD (coronary heart disease), CHF (congestive heart failure), CVD (cardiovascular disease), and MI (myocardial infarction). We accepted the following as evidence of heart disease: tests that were positive for heart disease, a clinic visit during which heart disease was treated, heart disease mentioned in the past medical history, a hospital discharge coded for heart disease, or heart disease on a problem list. We did not require objective evidence of heart disease. However, if the sole evidence for heart disease was a heart disease code for a test that was performed while the patient was hospitalized (eg, an echocardiogram), the code was not accepted as evidence that the patient had heart disease. For the cases in which the manual review disagreed with claims data, we manually reviewed the record a second time to determine whether a reference to heart disease in the text had been overlooked. To test the reproducibility of the manual record review, both cardiologists abstracted a stratified random sample of 48 records. They agreed on the classification of 47 of the 48 records.
We calculated Cohen’s κ and the 95% confidence interval (CI) for patients who had been enrolled any time from August 1, 2007, through July 31, 2008. We performed the calculations for both the development and validation samples. Because it has been suggested that accuracy requires 2 visits for heart disease (
We used logistic regression to test whether any of the variables used in the analysis were associated with agreement between claims data and manual review. Agreement was the dependent variable and age, sex, duration of enrollment, gap in enrollment, and number of visits for heart disease were the independent variables. The development data set and the validation data set were combined to perform a single data set for the analysis. Because agreement between claims data and manual record review for patients with no visits for heart disease was perfect, we excluded these patients from the multivariate analysis.
Just over half of the 474,089 people in the population were women, and the average age was slightly less than 50 years (
| Length of Enrollment/Gap in Enrollment >31 Days | No. of Visits for Heart Diseaseb Based on Claims Data |
|---|---|
|
| |
|
| |
| Gap: n = 146,939; female, 50.2%; age, 46.1 (16.0) | None: n = 140,440; female, 53.9%; age, 44.5 (15.0) |
| 1 visit: n = 3,918; female, 38.6%; age, 60.6 (21.0) | |
| ≥2 visits: n = 2,581; female, 40.5%; age, 64.6 (23.0) | |
|
| |
| No gap: n = 327,150; female, 53.4%; age, 51.0 (17.0) | None: n = 296,073; female, 56.3%; age, 45.8 (17.0) |
| 1 visit: n = 15,182; female, 47.1%; age, 62.5 (22.0) | |
| ≥2 visits: n = 15,895; female, 50.6%; age, 68.3 (20.0) | |
|
| |
|
| |
|
| |
| Gap: n = 40,174; female, 52.6%; age, 48.7 (18.0) | None; n = 37,179; female, 57.7%; age, 48.7 (18.0) |
| 1 visit: n = 1,601; female, 45.4%; age, 71.6 (24.0) | |
| ≥2 visits: n = 1,394; female, 47.0%; age, 75.7 (22.0) | |
|
| |
| No gap: n = 232,174; female, 54.2%; age, 52.6 (17.0) | None: n = 207,195; female, 58.1%; age, 53.1 (17.0) |
| 1 visit: n = 11,782; female, 43.7%; age, 69.1 (21.0) | |
| ≥2 visits: n = 13,197; female, 43.2%; age, 72.3 (19.0) | |
a Ages are presented in years and as mean (interquartile range).
b
Classification based on claims data agreed with the manual review classification for all but 9 of the 240 records examined in the development sample (
| Enrollment Category | Enrollment Gap >31 days | No. of Visits for Heart Disease Based on Claims Data | Agreement in the Development Sample (95% CI) | Agreement in the Validation Sample (95% CI) |
|---|---|---|---|---|
| Enrolled any time from August 1, 2007–July 31, 2008 | Gap | 0 | 1.00 (0.90–1.00) | 1.00 (0.90–1.00) |
| 1 | 0.90 (0.69–0.98) | 0.90 (0.69–0.98) | ||
| ≥2 | 1.00 (0.90–1.00) | 0.95 (0.76–0.99) | ||
|
| ||||
| No gap | 0 | 1.00 (0.90–1.00) | 1.00 (0.90–1.00) | |
| 1 | 0.65 (0.43–0.82) | 0.85 (0.63–0.96) | ||
| ≥2 | 1.00 (0.90–1.00) | 0.85 (0.63–0.96) | ||
|
| ||||
| Also enrolled on August 1, 2005 | Gap | 0 | 1.00 (0.90–1.00) | 1.00 (0.90–1.00) |
| 1 | 1.00 (0.90–1.00) | 0.90 (0.69–0.98) | ||
| ≥2 | 1.00 (0.90–1.00) | 0.95 (0.76–0.99) | ||
|
| ||||
| No gap | 0 | 1.00 (0.90–1.00) | 1.00 (0.90–1.00) | |
| 1 | 1.00 (0.90–1.00) | 1.00 (0.90–1.00) | ||
| ≥2 | 1.00 (0.90–1.00) | 1.00 (0.90–1.00) | ||
Abbreviation: CI, confidence interval.
a Twenty records were audited for each of the 12 categories for number of visits.
Cohen’s κ was 0.74 (95 % CI, 0.66–0.82) when patients with only 1 visit were considered not to have heart disease. The true number of cases of heart disease was underestimated by 13,395. Estimated prevalence of heart disease was 3.9% with this assumption, an underestimation of 42%. When record review was the gold standard, the sensitivity of ICD coding was 0.53 (95% CI, 0.45–0.61) and specificity was 1.00 (95% CI, 0.95–1.00). The predictive value of a positive test (PV+) was 1.0 and the predictive value of a negative test (PV−) was 0.08.
Cohen’s κ was 0.92 (95% CI, 0.87–0.97) when patients with only 1 visit for heart disease were classified as having heart disease. The true number of cases was overestimated by 5,706. Estimated prevalence of heart disease was 7.9% with this assumption, an overestimation of 18%. Because there were no disagreements between the claims data and manual record review in the long enrollment sample, the estimate of heart disease prevalence (10.3%) was the same with both assumptions. When chart review was considered the gold standard, sensitivity of the ICD codes was 1.00 (95% CI, 0.98–1.00) and specificity was 0.94 (95% CI, 0.97–1.00). PV+ was 0.59, and PV− was 1.0.
Classification based on claims data agreed with classification based on manual review in all but 12 of the 240 records in the validation sample (
If patients who had only 1 visit were considered not to have heart disease, Cohen’s κ was 0.65 (95% CI, 0.56–0.74). The prevalence of heart disease was 3.9% in the any enrollment population, an underestimation of 43%, and the prevalence of heart disease in the long enrollment population was 5.4%, an underestimation of 48%. When chart review was considered the gold standard, sensitivity of the ICD codes was 0.51 (95% CI, 0.42–0.59), specificity was 0.95 (95% CI, 0.87–0.98), PV+ was 0.29, and PV− was 0.07.
Conversely, if patients with only 1 visit for heart disease were classified as having heart disease, Cohen’s κ was 0.94 (95% CI, 0.89–0.98) and the estimated prevalence of heart disease in the any enrollment population was 7.9%, an overestimation of 16%. Estimated prevalence of heart disease in the long enrollment population was 10.3%, an overestimation of 1%. When chart review was considered the gold standard, sensitivity of the ICD codes was 1.00 (95% CI, 0.98–1.00), specificity was 0.87 (95% CI, 0.78–0.93), PV+ was 0.40, and PV− was 1.0.
When patients who had no visits for heart disease were excluded from a logistic regression analysis, long enrollment (
This study investigated the effect of length of enrollment, gaps in coverage, and number of visits for heart disease on the agreement of claims data with manual record review. Agreement was higher for patients who had longer enrollment and more than 1 visit for heart disease, but even so, accepting claims data at face value yielded the highest levels of agreement and the best estimates of true prevalence. Requiring at least 2 outpatient visits to classify a patient as having heart disease resulted in heart disease prevalence being underestimated by nearly 50%, an error that significantly underestimates the potential effect of secondary prevention initiatives.
Our conclusion that claims data acceptably represent the presence or absence of disease is generally consistent with other analyses of administrative data (
These conclusions must be accepted with some caveats. Perhaps foremost, the analysis is based on the records of only 1 medical group; other groups may have a different experience. Also, the purpose of this study was to determine the algorithm that most accurately estimates the prevalence of heart disease in an enrolled population. If the purpose were to identify patients for case management or to assess the quality of health care (
We also do not have an adequate explanation for the fact that heart disease prevalence was 50% higher in the long enrollment population than in the any enrollment population. It may be that patients who have heart disease are less likely to change health plans because of coverage limitations for pre-existing conditions or that they have a preference for a long-term relationship with a particular physician. The fact that the prevalence rates were nearly identical in the development and validation samples makes the observation less likely to be a data sampling error.
We acknowledge that the disease prevalence estimates based on claims data appear to overestimate true prevalence by 15% to 20%. However, assuming that the true prevalence of heart disease is 20% lower than the calculated prevalence does not substantively change the conclusion that improving secondary prevention (33% of the total opportunity) would have a far greater effect on deaths in the United States than would improving care for patients hospitalized for acute cardiac events (8% of the total opportunity) (
It is probably safe to assume that coding practices vary between medical groups and that any medical group should document that their billing codes acceptably reflect the medical record before they use them to estimate disease prevalence. However, the data presented here suggest that claims data can be used to estimate disease prevalence.
This work was supported by the following agencies: The HealthPartners Research Foundation (a partnership grant to T.E.K.); The Heart Disease and Stroke Prevention Unit at the Minnesota Department of Health from a Capacity Building — Cooperative Agreement grant from the Centers for Disease Control and Prevention, no. 5U50DP000721-04; and National Institutes of Health training grant no. T32 HL69764 (supporting C.J.B.).
The opinions expressed by authors contributing to this journal do not necessarily reflect the opinions of the U.S. Department of Health and Human Services, the Public Health Service, the Centers for Disease Control and Prevention, or the authors' affiliated institutions.