American Indian and Alaska Native (AI/AN) persons have had the second highest rates of chlamydia and gonorrhea (CT/GC) among all racial and ethnic groups in the United States since 1996.^1,2 As of 2021, 8.7 million persons self-reported their race(s) to the US Census as AI/AN³ and over 2.5 million AI/AN members of 574 federally recognized tribes are eligible for care from Indian Health Service (IHS).⁴ Previous reports have estimated CT/GC case rates among AI/AN persons served by the IHS using the laboratory-based US national surveillance data and US Census denominators for counties where IHS provides services.^5,6 However, this is not all sourced directly from IHS and there is a need for more regular and timely reporting. The use of IHS electronic medical records data could be a convenient source for reporting the burden of CT/GT where there is currently no dedicated IHS CT/GC surveillance program. This information could be utilized for assessing the needs of IHS with regards to identifying outreach focus areas and for projecting needs for testing and treatment, staffing, and budgeting.

IHS is a federal agency that provides healthcare at IHS, Tribal, and Urban Indian healthcare centers (I/T/U) in 37 states and by funding contracted care from other providers.⁴ IHS maintains routinely collected medical records data submitted by these I/T/U sites,⁷ which contain International Classification of Diseases, 10th Revision, Clinical Modification (ICD-10-CM)⁸ diagnostic codes. However, there is a need to assess the validity of using ICD-10-CM diagnostic codes to estimate diagnosed infections from this source. One approach to determine if the appropriate code was recorded is by using other relevant information, such as confirmatory test results. Previous studies combining state surveillance laboratory results and diagnostic codes from private insurance data sources in the United States have found the sensitivity of CT/GC ICD-CM 9^th and 10^th revision diagnostic codes to be very low at 6–25%, indicating that diagnostic codes are often not captured in administrative data after a positive laboratory test result.^9,10 The specificity and negative predictive values (NPV) in these studies were generally very high at over 96% each, and positive predictive value (PPV) ranged from 33–87%. They indicated that, while CT/GC ICD-9/10-CM diagnostic codes were found to be very accurate, they may grossly underestimate CT/GC infections when used alone.

The purpose of this study was to validate the use of ICD-10-CM CT/GC diagnostic codes for estimating CT/GC infections in IHS medical records. We estimated the sensitivity, specificity, PPV, and NPV of ICD-10-CM CT/GC diagnostic codes using CT/GC laboratory results as the reference standard. We describe select characteristics of persons with ICD-10-CM CT/GC diagnostic codes with and without linked laboratory results and for laboratory results with and without linked diagnostic codes.

The National Data Warehouse (NDW) is the IHS national data repository, which is comprised of medical records data submitted by I/T/U sites. These records contain ICD-10-CM diagnostic codes from care settings including hospitals, outpatient facilities, and emergency departments (ED), and testing results from laboratories.⁷ Demographic information including sex, age, and IHS region (location of residence) are included by date of discharge (hospital) or service (outpatient, ED, laboratory). The IHS regions are an aggregation of the 12 IHS administrative areas: Alaska (Alaska), East (Nashville), Northern Plains East (Bemidji), Northern Plains West (Great Plains, Billings), Southern Plains (Oklahoma City), Southwest (Albuquerque, Navajo, Phoenix, Tucson), and West (California, Portland).

For this analysis, we examined de-identified medical records from the NDW from all persons aged 15 years and older who had a documented CT/GC laboratory test or ICD-10-CM diagnostic code during calendar years 2016 to 2021. Diagnoses were defined using the ICD-10-CM codes A56 and A74 for chlamydial infections and A54 and O98.2 for gonococcal infections.¹¹ Repeat ICD-10-CM diagnostic codes within 28 days were excluded as some tests within four weeks may provide false-positive results.¹² Laboratory tests, henceforth referred to as tests, were defined by Logical Observation Identifiers Names and Codes (LOINC) as previously described^10,11 and by text strings containing appropriate labels indicating CT/GC tests when codes were not available. Tests were excluded from validation analyses when results were unknown, including indeterminate results, cancelled tests, results noted elsewhere in file and unavailable, or contained a value that could not be interpreted. Repeat CT/GC test results within 28 days were excluded, as test of cure is not usually recommended before four weeks;¹² the following hierarchy was used to select a result value from multiple results entered on the same date: positive, negative, unknown.

We attempted to “link” all ICD-10-CM diagnostic codes to a laboratory test result for the same person, where a laboratory test precedes an ICD-10-CM diagnostic code by 30 days or less. Diagnostic codes and tests were otherwise considered not linked. Tests 30 days before the beginning of 2016 were included, except where an ICD-10-CM diagnostic code from the same time period could be linked. The tests that could not be linked to a CT/GC diagnostic code were further examined by looking for any other hospital, outpatient facility, or ED visits within 30 days of the test for each person. This allowed us to see if any of the tests without linked CT/GC diagnoses could be linked to other with ICD-10-CM encounter codes⁸ used to indicate contact with health services that might be related to CT/GC testing.

We estimated the sensitivity, specificity, PPV, and NPV of the linked ICD-10-CM CT/GC diagnostic codes using positive or negative laboratory test results as the reference standard. Sensitivity of ICD-10-CM diagnostic codes was defined as the number of ICD-10-CM diagnostic codes linked to a positive test in the previous 30 days or less divided by the number of all positive tests and expressed as a percentage. Specificity is the number of negative tests without a linked diagnostic code divided by the number of all negative tests; PPV is the number of diagnostic codes that had a linked positive test divided by the number of diagnostic codes that had a linked positive or negative test; NPV is the number of negative tests without a linked diagnostic code divided by the number of positive and negative tests without a linked diagnostic code; and all expressed as percentages. Estimates were further stratified by age, sex, region, care setting, and calendar year, and 95% binomial confidence intervals (CI) were calculated. Differences in percentages by group were presented with 95% Wald asymptotic confidence intervals. Cochran-Armitage test for trend was used to assess association with linked status and age groups with p < 0.05 considered statistically significant. Small cell counts (n < 5) and surrounding cells were suppressed to avoid revealing exact small cell values. All analyses were completed using SAS^® 9.4M6 (SAS Institute, Cary, NC). This activity was reviewed by CDC, deemed not research, and was conducted consistent with applicable federal law and CDC policy. Authors have no conflicts of interest to report. The findings and conclusions in this manuscript are those of the authors and do not necessarily represent the views of the funding agencies.

During the study period of 2016 to 2021, 1.2 million AI/AN persons aged 15 years and older used healthcare services within the IHS system. An average of 673,000 persons had at least one laboratory test of any kind recorded in the NDW during 2.3 million hospital, outpatient, or ED visits each year.

We identified nearly 53,000 ICD-10-CM CT diagnoses, 20,000 ICD-10-CM GC diagnoses, and 820,000 tests each for CT/GC (Tables 1, 2). Nearly all CT (n=784,747; 95.7%) and GC (n=776,561; 94.8%) tests had a positive or negative result available. The majority of CT/GC diagnoses were among females and persons aged 15–39 years, and the largest number of CT/GC diagnostic codes were recorded in the Southwest IHS region (Tables 1, 2). Most ICD-10-CM diagnoses originated from an outpatient setting (CT: 95%, GC: 91%), and nearly all CT/GC tests (98%) were ordered in an outpatient setting.

The annual numbers of positive CT tests (minimum: 7,943 in 2020; maximum: 10,258 in 2019) were similar to, or slightly greater than, the numbers of ICD-10-CM CT diagnoses over time (minimum: 6,935 in 2020; maximum: 9,628 in 2019; Figure 1A). The annual percentage of positive CT tests among all CT tests remained steady (range: 6.6–7.1%). The numbers of positive GC tests (minimum: 2,597 in 2016; maximum: 4,505 in 2019) were similar to, or slightly greater than, the numbers of ICD-10-CM GC diagnoses over time (minimum: 2,703 in 2016; maximum: 3,906 in 2019; Figure 1B). The annual percentage of positive GC tests among all GC tests increased slightly over time (2.0% in 2016 and 3.6% in 2021).

The validation analyses used all the positive and negative CT/GC tests and 69% of the CT (Table 1) and 63% of the GC (Table 2) ICD-10-CM diagnoses, which were linked to a positive or negative test. Most linked ICD-10-CM diagnoses and tests occurred within 14 days (CT: 84%; GC 92%). Most tests linked to diagnostic codes were positive and most tests not linked to diagnostic codes were negative (Tables 3, 4).

The overall sensitivity of ICD-10-CM diagnoses was slightly higher for CT (54%) than GC (50%; Tables 3, 4). Sensitivity varied by characteristic for both CT and GC: higher among females than males; highest among the youngest age group and decreased with age. CT sensitivity was highest in the Southwest and lowest in the West regions, and GC sensitivity was highest in the Northern Plains East and lowest in the West regions (Tables 3, 4). Sensitivity decreased over time for CT (60.3%, 95% CI: 59.3–61.3 in 2016 and 49.0%, 95% CI: 48.0–50.1 in 2021) and GC (55.7%, 95% CI: 53.8–57.6 in 2016 and 49.2%, 95% CI: 47.7–50.7 in 2021; Figure 1). Specificity was > 99% for both CT and GC, and there was little variability by sex, age, or region. PPV was high, but slightly lower for CT (83%) than GC (91%; Tables 3, 4). For CT, PPV was similar between sexes; highest among the youngest age group; and highest in the Northern Plains West and lowest in the Southern Plains (Table 3). For GC, PPV was slightly higher among females; and highest among the youngest age group (Table 4). NPV was very high, but slightly lower for CT (97%) than GC (99%) and was slightly higher for females than males for both (Tables 3, 4).

Nearly one-third of CT/GC ICD-10-CM diagnoses were not linked to a test and could not be used in the validation analyses (Tables 1, 2). Males had a slightly larger proportion of ICD-10-CM diagnoses not linked to a test when compared to females for both CT (difference=6.8%, 95% CI: 5.8–7.8) and GC (difference=3.0%, 95% CI: 1.5–4.4) and the proportion of ICD-10-CM diagnoses not linked to a test increased with increasing age group for both CT/GC (p < 0.001; Tables 1,2). The Alaska region accounted for 42% of both CT/GC ICD-10-CM diagnoses not linked to a test (Tables 1, 2). On average, the Alaska region reported 65.0% (95% CI: 64.1–65.9) more CT and 64.7% (95% CI: 63.4–66.0) more GC ICD-10-CM diagnoses not linked to tests than the other regions (Tables 1,2).

Among all tests, 3% of CT tests and 1% of GC tests were positive and not linked to ICD-10-CM CT/GC diagnoses (Tables 1–4). Nearly all of these positive CT tests (91.3%; n=23,088) and positive GC tests (93.9%; n=10,703) not linked to CT/GC diagnosis codes could be linked to hospital, outpatient, or ED visits with other ICD-10-CM diagnostic codes. Among these, encounter codes were linked to 83.5% of CT and 80.1% of GC tests; the most common codes included encounter for issue of a repeat prescription (Z76.0), encounter for other administrative examinations (Z02.89), encounter for administrative examinations (Z02.9), and encounter for screening for infections with a predominantly sexual mode of transmission (Z11.3).

The results of this study indicate that ICD-10-CM diagnostic codes should be combined with positive laboratory test counts to inform more reliable estimates of diagnosed CT/GC infections among AI/AN persons using IHS healthcare services. The high PPV and very high specificity results indicate that ICD-10-CM codes align well with laboratory test results: most diagnostic codes that could be linked have positive tests and very rarely do negative tests have linked diagnostic codes. The high NPV shows that where no CT/GC diagnostic codes were linked to positive or negative tests, the tests were mostly negative, as expected. However, observed sensitivities show that CT/GC diagnostic codes were only linked to about half of the positive test results. We did find that the overall absolute number of CT/GC ICD-10-CM diagnostic codes and positive tests was similar, but after linking the two for the validation analysis, about a third of CT/GC diagnostic codes were not linked to any test results. Therefore, these validation results alone are also not enough to understand the true burden of CT/GC.

High specificity, PPV, and NPV values for CT/GC ICD-10-CM diagnoses in the NDW data were similar to previous reports.^9,10 The sensitivity level for CT/GC ICD-10-CM diagnoses was much higher than previous studies that combined state and private insurance data sources,^9,10 which may be a result of IHS clinical and laboratory records coming from a singular administrative data system with the capability to easily link person-based records. The highest sensitivity values for both CT/GC diagnostic codes were among those recommended for routine screening including females and younger age groups,¹³ and who may interface more with healthcare visits that encourage routine screening for CT/GC infections. While the sensitivity of ICD-10-CM CT/GC diagnostic codes did decrease over time, the number of tests reported was generally increasing until 2020 and could continue to increase in parallel with rising cases in the US general population, any increases in infections within IHS, increases in screening rates, or due to increased availability of local laboratory testing,¹⁴ therefore, validation of CT/GC ICD-10-CM diagnostic codes should be intermittently assessed to measure stability of these estimates. There were larger differences in sensitivity between regions for both CT/GC, which is likely related to varying medical record keeping practices or other data reporting practices from different facilities, so we recommend that this be considered when estimating CT/GC infections by region in future analyses.

A major strength of the IHS NDW data, that may not be available in other administrative data sources, is that person-records can easily be linked across multiple data sources and over time. This was necessary for linking test results to ICD-10-CM diagnoses and for removing duplicate diagnostic codes or tests in a defined period for a specific person. Unlike previous studies,^9,10 the IHS NDW could be used to enumerate and describe all CT/GC ICD-10-CM diagnoses that were unable to be validated because they could not be linked to tests. We were also able to link most positive tests without a corresponding CT/GC diagnostic code to other hospital, outpatient, or ED records; almost all were linked to other visits, and many contained encounter ICD-10-CM codes. Some of these encounter codes may be related to screening for sexually transmitted infections and could be considered for a similar validation study in the future.

Consistent with US general population surveillance and guidance,^1,2 females and younger age groups had the greatest absolute number of CT/GC diagnoses and tests, regardless of linked status. Females may have more frequent healthcare contact in the younger age groups seeking services for contraception or prenatal care. Males had slightly higher percentage of diagnostic codes without linked tests, which could be because they have symptomatic infections more often and may be receiving presumptive treatment at higher levels.¹³ The Southwest region had the highest levels of CT/GC testing and diagnosing, which is at least partially attributed to population size (approximately 32% of those aged ≥ 15 years reside in the Southwest). The distribution of IHS CT/GC diagnostic codes, regardless of linked test status, aligns with previous reporting of US laboratory-based case surveillance in counties where IHS provides healthcare services, including cases primarily among females and younger age groups, and higher CT/GC case counts in the Alaska, Northern Plains West and Southwest regions,^5,6 however, counts of diagnostic codes or positive tests cannot be directly compared until there is a more current surveillance report available.

The greatest challenge in this study was capturing tests and test results from the NDW and would be difficult to replicate frequently without more resources. Laboratory test names were obtained using a combination of LOINC codes and systematically searching editable text fields, which is time and labor intensive and requires specialized skills. Further, lack of uniform labeling introduces potential for misclassifying results, which is most likely to lead to underreporting of test results. We recommend an increased use of LOINC codes for laboratory documentation or lists of standardized laboratory test names with limited editing capabilities to simplify analyses for regular use. The relatively small percent of ICD-10-CM diagnoses in the Alaska region that could be linked to a test result indicates there may be systematic differences in reporting laboratory data to IHS NDW by region; this could be related to data reporting practices, such as where LOINC or other laboratory data is not required to be reported by IHS tribal facilities using non-HL7 file formats.⁴ Generally, missing linked tests may be a result of: empiric treatment based on clinical characteristics and known risk factors; patient follow up after expedited partner therapy for sexual partners; or if treatment is given within IHS after outside testing. The laboratory data was otherwise found to be very complete for CT/GC based on the relatively small number of tests with unknown results but, given the potential value and ease of using ICD-10-CM diagnostic codes for CT/GC surveillance, we encourage more robust case management of positive lab results that refer patients to follow-up appointments for further evaluation or treatment response assessment that would also improve the ICD-10-CM sensitivity.

We defined several assumptions for this validation analysis. We preferentially included positive test results if multiple test results were found for the same person, which risks including a small number of false positive test results in our analysis. We chose a 30-day cutoff between test results and diagnoses to represent a reasonable amount of time between test results and recording diagnoses in a real-world clinical setting. While it is possible that some CT/GC reporting of diagnostic codes occurred after the 30-day cutoff, which would likely artificially decrease our sensitivity, we found that most diagnoses were documented within 14 days of testing. Additionally, we used a 28-day window for exclusion of duplicate test results that precludes description of reinfection and test-of-cure diagnoses during the study period. However, true positives occurring in this 28-day window are difficult to identify without further information and could lead to misclassification if included. It is possible that we may not have identified all CT/GC-related visits and tests for our study population if someone obtained CT/GC-related testing or follow-up services outside the IHS system which may not be captured in the NDW. Our recommendation to combine diagnostic codes with laboratory results assumes that codes without links to tests are not fundamentally different than those with links and does not account for type of laboratory test, type of specimen tested, or imperfect testing results in the validation results.

Future work includes describing and estimating the burden of CT/GC infections among AI/AN persons who use IHS healthcare services to provide a more accurate assessment of needs for testing, treatment, outreach, staffing and funding that support of AI/AN persons who use IHS healthcare services. The results of this study should be considered when estimating diagnosed CT/GC infections, for which we suggest a more sophisticated approach, such as a capture re-capture analysis for the same time period. This type of analysis would combine the two data sources to estimate the CT/GC burden, which in turn will also provide a sense of the difference in magnitude between estimated burden and use of IHS ICD-10-CM diagnoses alone or the laboratory-based national surveillance estimates for counties where IHS provides service. Next, ICD-10-CM diagnostic codes can be used to create a cohort to describe CT/GC risk factors, co-diagnoses and sequalae, trends over time, and treatment.

This is the first study to validate the use of ICD-10-CM CT/GC diagnostic codes for estimating diagnosed CT/GC infections among AI/AN persons utilizing the IHS healthcare network. While we found diagnostic codes aligned well with test results, we also discovered that the use of diagnostic codes alone would underestimate the true burden of CT/GC. However, IHS epidemiologists could consider using diagnostic codes alone to describe diagnosed CT/GC infections, particularly in resource-limited circumstances where laboratory data are unavailable or difficult to analyze.