Estimation of cancer survival is an important part of assessing the overall strength of cancer care and success of prevention programs. Relative survival is a measure that can be used to describe the survival of a cohort of cancer patients by removing the effect of competing death events of a comparable general population. The measure is the ratio of observed survival among cancer patients divided by the expected survival of the general population that is comparable to the cancer patients with respect to covariates including age, sex, and year of diagnosis.^1,2 Population-based cancer relative survival rates are important for medical and public health efforts, including measuring the survivorship of cancer patients after diagnosis and monitoring the impact of intervention and early detection programs.³

Current U.S. cancer survival rates are estimated based on data from the National Cancer Institute’s (NCI’s) Surveillance, Epidemiology, and End Results (SEER) program, which covers approximately 28% of the U.S. population.⁴ The National Program of Cancer Registries (NPCR), established by Congress in 1992 and administered by the Centers for Disease Control and Prevention (CDC), is conducted in 45 states, the District of Columbia, Puerto Rico, and the Pacific Island Jurisdictions, and covers approximately 96% of the U.S. population. Data submitted annually to the NPCR-Cancer Surveillance System (NPCR-CSS) may also be used to calculate survival rates and provide greater coverage at national, regional, and state levels, so clinicians, public health practitioners, and researchers can effectively monitor cancer patient care and prognosis in the U.S.

NPCR-CSS collects data on the occurrence of cancer including the type, extent, and anatomic location of the cancer and the type of initial treatment by providing funding and technical assistance to the central cancer registries (CCRs) within the program.⁵ Population-based CCRs are data systems that collect, manage, and analyze data about cancer cases. In each state, medical facilities (including hospitals, physicians’ offices, therapeutic radiation facilities, free-standing surgical centers, and pathology laboratories) are required to report demographic and clinically-related data to their central cancer registry. Each year, CDC supports efforts to link registry data with the National Death Index (NDI), Indian Health Service data, and state vital records, receives data from NPCR registries, and assesses the completeness and accuracy of the data.⁵ The annual data submissions from CCRs add a new year of data, update data from prior diagnosis years, and include the variables needed to calculate survival rates (e.g., age, sex, year of diagnosis, date of last contact) as well as variables that are important surrogates of the quality of the follow-up information obtained (e.g., type of reporting source, follow-up source, ICD revision number, cause of death) or can be used to stratify analyses (e.g., stage, county or state, race, ethnicity).

Before using NPCR-CSS data to assess cancer survival, it is necessary to understand the coding completeness and coding quality of the data elements used in survival analyses across all the participating cancer registries. NPCR rigorously evaluates the completeness of case ascertainment and data quality for each annual data submission.³ Other studies have evaluated completeness, accuracy, and data quality of some, but not all, of the NPCR-CSS data required for conducting and validating survival analyses.^6–10 For example, German and colleagues looked at the quality of breast and prostate cancer variables⁶; Hall and co-workers compared SEER and NPCR incidence rates of cutaneous melanoma⁷; McDavid’s team assessed breast, prostate, and colon variables⁸; and Singh and co-workers investigated the quality of the census tract 2000 variable.⁹

Using 1998–2001 data from 34 CCRs, Thoburn and colleagues compared NPCR-CSS incidence data to medical record data for 13 data elements among 4 primary cancer sites (lungs/bronchus, colorectal, prostate, and female breast) to assess the case completeness and data accuracy.¹⁰ The elements investigated included date of birth, race, sex, state of residence at time of diagnosis, diagnosis date, primary site, histology, behavior, grade and SEER summary stage. The authors found data accuracy in 95% of the cases and case completeness in 96% of the cases; individual site-specific data element accuracy ranged from 81.2% to 100%, with a median accuracy of 98.1%.¹⁰

The purpose of the current study is to build upon these previous studies by evaluating the coding completeness and quality of 26 data elements, for all primary cancer sites, that are covariates to survival calculations and those used to assess the calculations across the 46 funded CCRs.

Using NPCR-CSS data from 46 CCRs inclusive of 1995 through 2008 diagnosis years, 26 data elements used in survival calculations and validation were examined for coding completeness and quality. Coding completeness was assessed by calculating the proportion of non-missing values by data element and by central registry; the numerator was the number of non-missing values and the denominator was the total number of values (Table 1). Coding quality was calculated through the proportion of known values; the numerator was the number of known values (excluding unknown or blank values) and the denominator was the total number of values (Table 1).

The data elements consisted of survival analysis variables, demographic variables, cancer identification variables, follow-up and death variables, and cancer stage and prognostic variables. They specifically included:

Survival analysis: date of birth, date of diagnosis, date of last contact/death, and sex;

Demographics: age at diagnosis, U.S. state of residence, county of residence at diagnosis, race, ethnicity (Hispanic), Indian Health Service (IHS) linkage (used to better classify cases of American Indian/Alaska Native heritage), North American Association of Central Cancer Registries’ (NAACCR) Hispanic Identification Algorithm (NHIA) derived Hispanic origin and NAACCR Asian/Pacific Islander Identification Algorithm (or NAPIIA, which is used to better classify cases of Asian Pacific Islander origin);

Cancer identification: behavior (benign, in situ, or malignant), diagnostic confirmation, histology, SEER primary site group, sequence number central (number of primary cancers), and type of reporting source (primary source from which original cancer incidence report received);

Follow-up, recurrence, and death: cause of death code, follow-up source central (2006–2008 diagnosis years for which the variable was captured by CCRs), follow-up source (1995–2005 diagnosis years for which the variable was captured by CCRs) (source from which follow-up information received), International Classification of Diseases (ICD) revision number (cause of death coding system version; in the analyses this variable was divided into separate groups as versions 7–9 and version 10 or combined into 1 variable, versions 7–10), and vital status;

Cancer stage and prognostic: SEER summary stage 1977 (1995–2000 diagnosis years for which the variable was in use), SEER summary stage 2000 (2001–2003 diagnosis years for which the variable was in use), and collaborative stage (CS) derived SEER summary stage 2000 (2004–2008 diagnosis years for which the variable was in use).

The mean percent coding completeness for each of the 46 central cancer registries was calculated and averaged over the 26 data elements combined and all diagnosis years (1995–2008) combined; the mean percent coding completeness was also calculated for each of the 26 data elements averaged over all the diagnosis years combined and all central cancer registries combined. Mean percent coding quality was assessed by each of the 26 data elements, taking the average of all diagnosis years and all central cancer registries combined. General linear modeling was performed to assess statistical differences for each data element by diagnosis year (year was coded as a categorical variable; the latest year available, 2008, was used as the referent year) and NPCR central cancer registry (assessed individually; the referent variable was a state that has maintained high coding quality and stability over time) (α=0.05). Coding completeness of each data element was modeled as the outcome variable with diagnosis year and NPCR central cancer registry as the independent variables, respectively, in a least squares linear model. All analyses were conducted using SAS (version 9.2, SAS Institute, Inc., Cary, NC).

The results show a high level of coding completeness and quality for the 4 survival analysis variables (date of diagnosis, date of birth, date of last contact or death, and sex) across central cancer registry sites and diagnosis years. The majority of variables used to assess quality of the follow-up information and to stratify analyses also had high averaged means of coding completeness and quality by central registry sites and diagnosis years. These findings may be indicative of the training and support CDC NPCR provides in monitoring and improving coding completeness.

The mean coding completeness percent for the relative survival data elements examined is relatively high, compared with the previous studies,^6–10 with an increase in the 2006 diagnosis year, and is similar among NPCR CCRs. This increase may result from the increase in NPCR CCRs conducting linkage processes with the National Death Index (NDI), including additional data editing and record updating, as well as the availability of training sessions and other resources. Demographic (some of which are evaluated annually to determine compliance with NPCR Program Standards), cancer identification, cancer stage/prognostic, and the majority of the follow-up/death data elements have high mean coding completeness percentages.

Even though the results are very promising, additional work may be needed for some of the data elements (e.g., date of last contact/death, follow-up source, IHS linkage). Some NPCR central cancer registries have concerns releasing the full date of last contact/death due to confidentiality while other central cancer registries may not be updating this data element following death certificate clearance procedures and/or NDI linkages. Additional discussions to assure confidentiality or resources to facilitate automatic record updates may be needed to improve the completeness of date of last contact /death. Our analysis also showed how competing priorities can affect coding completeness, as exhibited with the date of last contact/death variable. Starting in 2001, NPCR established a linkage agreement with NDI, which facilitated improved linkages and date of last contact/death information. However, in 2004, when Collaborative Stage activities became a priority for CCRs, the linkages were not completed as frequently and the completeness of the variable was affected.

The data element follow-up source (1995–2005 diagnosis years) has not been required by CDC for the NPCR registries, so the low level of completeness is not surprising. However, this data element is important; it makes it possible to identify records with information resulting from NDI linkages and, when necessary, release of that information can be recorded and reported back to NDI. The data item can also serve as a surrogate for the quality of the follow-up information.

As shown in the Results, the cause of death variable has a low percent of coding quality, 78% and 81% for the different ICD versions. The cause of death is dependent upon information recorded on death certificates, available through vital statistics linkages, and data quality issues have been identified in other evaluations. For this reason, researchers generally rely on relative survival rates for cancer rather than cause-specific survival rates.

Not all NPCR CCRs link with the IHS Administrative database on an annual basis, but all do link every 5 years. If a record is not sent for IHS Administrative database linkage, this data element is not coded. This most likely explains the lower percent completeness for the IHS linkage variable. Additional analyses may be needed, limiting the analyses to only those CCRs that conduct the IHS linkage annually or to the years where all CCRs conduct the linkage. The coding quality evaluation, however, showed that for records that were sent for linkage, the percent with a coded known value is very high.

Our results show that the NPCR-CSS data can be a complete source of information for researchers interested in using population-based cancer data to study cancer relative survival in the U.S. Another strength of the NPCR data is the potential to calculate relative survival by race and ethnicity, which may assist researchers and comprehensive cancer control coalitions in making decisions about the type of cancer care and cancer programs they provide to the various ethnicities in the U.S., thereby having the potential to reduce disparities in cancer incidence and survival.

More work is needed to improve coding completeness for cancer case follow-up, the information with the lowest mean coding completeness percentages in this database. A limitation of this study is that we did not assess data accuracy for these relative survival variables. Evaluating the data accuracy requires an audit of the source documents and the assigned codes. Other projects are conducting this evaluation and include some, but not all, of the data elements assessed in this project.

Survival analysis estimates are critical for many prevention, control, and treatment activities, including evaluation of the impact of screening and comprehensive cancer control programs and assessing the progress in cancer treatments. Because NPCR provides data for approximately 96% of the U.S. population, it has the potential to provide near-national estimates as well as regional and state-based measures that have not before been available to researchers, clinicians, and public health decision makers. Our analyses demonstrate the high coding completeness and quality of the NPRC-CSS variables that are needed to calculate relative survival estimates and variables used to validate and stratify the estimates.