pmc10115926230202J Womens Health (Larchmt)J Womens Health (Larchmt)Journal of women's health (2002)1540-99961931-843X415888591288459110.1177/15409996251405046HHSPA2143682ArticleRanked Severe Maternal Morbidity Index for Population-Level Surveillance by Mode of DeliveryWomackLindsay S.PhD, MPH1GalangRomeo R.MD, MPH1AdmonLindsay K.MD, MSc34ClarkElizabeth A.MD, MPH1EwingAlexander C.MPH1SattenGlen A.PhD, MA1KoJean Y.PhD12FerreCynthia D.MA1KroelingerCharlan D.PhD1Division of Reproductive Health, National Center for Chronic Disease Prevention and Health Promotion, Centers for Disease Control and Prevention, Atlanta, Georgia, USA.United States Public Health Service, Commissioned Corps, Rockville, Maryland, USA.Department of Obstetrics and Gynecology, University of Michigan, Ann Arbor, Michigan, USA.Institute for Healthcare Policy and Innovation, University of Michigan, Ann Arbor, Michigan, USA.

Authors’ Contributions

L.S.W.: Conceptualization, methodology, formal analysis, data curation, writing—original draft, writing—review and editing; R.R.G.: Conceptualization, methodology, formal analysis, writing—original draft, writing—review and editing; L.K.A.: Conceptualization, methodology, formal analysis, writing—original draft, writing—review and editing; E.A.C.: Conceptualization, methodology, formal analysis, writing—original draft, writing—review and editing; A.C.E.: Conceptualization, methodology, validation, formal analysis, data curation, writing—original draft, writing—review and editing; G.A.S.: Conceptualization, methodology, writing—original draft, writing—review and editing, supervision; J.Y.K.: Conceptualization, methodology, writing—original draft, writing—review and editing, supervision; C.D.F.: Conceptualization, writing—original draft, writing—review and editing, supervision; C.D.K.: Conceptualization, resources, writing—original draft, writing—review and editing, supervision, project administration.

Address correspondence to: Lindsay S. Womack, PhD, MPH, Division of Reproductive Health, National Center for Chronic Disease Prevention and Health Promotion, Centers for Disease Control and Prevention, 4770 Buford Hwy NE, MS S107-2, Atlanta, GA 30341, USA, nrc1@cdc.gov31120265202626120261022026355473481https://creativecommons.org/licenses/by-nc/4.0/This article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 License (https://creativecommons.org/licenses/by-nc/4.0/) which permits non-commercial use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access page (https://us.sagepub.com/en-us/nam/open-access-at-sage).Objective:

To determine which severe maternal morbidity indicators identify the most in-hospital mortality during delivery hospitalization by delivery mode.

Materials and Methods:

We obtained data from the 1993–2015 Healthcare Cost and Utilization Project’s National Inpatient Sample. Separate analyses were conducted for cesarean and vaginal deliveries. We ranked 22 severe maternal morbidity indicators by their overall population-attributable fraction of in-hospital mortality.

Results:

We identified 87,864,173 delivery hospitalizations; 27.9% were cesarean deliveries and 72.1% were vaginal deliveries. There were 6,686 records with a discharge disposition of “died,” with the most occurring for cesarean deliveries (71.2%). Most deaths had a severe maternal morbidity indicator (cesarean deliveries = 94.2%; vaginal deliveries = 73.5%). Among cesarean deliveries, the top five ranked indicators were as follows: cardiac arrest/ventricular fibrillation, conversion of cardiac rhythm, ventilation, temporary tracheostomy, and aneurysm. Among vaginal deliveries, the top five ranked indicators were as follows: conversion of cardiac rhythm, cardiac arrest/ventricular fibrillation, ventilation, temporary tracheostomy, and amniotic fluid embolism. The top three ranked indicators identified 78.8% of in-hospital mortality among cesarean deliveries and 66.0% of in-hospital mortality among vaginal deliveries.

Conclusion:

Severe maternal morbidity indicator rankings for cesarean and vaginal deliveries were similar; however, there were differences by delivery mode in the performance of the SMM indicators in identifying in-hospital deaths. Our findings underscore the need for the improved documentation and measurement of severe obstetric complications during pregnancy and the postpartum period at the population level.

maternal mortalitymaternal morbiditypopulation surveillancecesarean deliveryvaginal deliveryIntroduction

Surveillance of maternal mortality is important for monitoring temporal trends and identifying where maternal health outcomes could be improved1; however, pregnancy-related deaths are rare.2 Severe maternal morbidity (SMM) is defined as unexpected outcomes of birth that result in significant short- or long-term health consequences.3 SMM encompasses a range of serious conditions that may or may not result in death, and it is widely used in public health to monitor trends.4 Because pregnancy-related deaths are comparatively rare, SMM indicators are used to monitor severe complications that may lead to mortality.5–7 In this study, we examine the effectiveness of SMM indicators in identifying in-hospital mortality as one critical, but not exclusive, dimension of severe maternal outcomes. Surveillance of SMM offers a larger number of adverse health events to monitor maternal health and inform delivery of care in health care institutions.8 In the United States, hospital discharge data are an important population-based source for SMM surveillance. To identify SMM cases using hospital discharge data, indicators were developed based on International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM) diagnosis and procedure codes.9

Validation studies of the indicators as a composite measure show the suboptimal positive predictive value (0.44 and 0.48) of ICD-9-CM codes correctly identifying adverse health events compared with a medical record review.10,11 The precision and accuracy of the indicators and codes for severe events during delivery hospitalization could be improved. Recently, a ranked SMM index was developed based on in-hospital mortality, concluding that 15 indicators identified 80% of in-hospital mortality, and the remaining 7 indicators identified only 2%.6 The remainder of in-hospital deaths (18%) had no SMM indicators.

The ranked SMM index determines the indicators that identify the most in-hospital mortality; however, it is unknown if the ranking of the indicators varies by delivery mode. Previous studies show that the risk of maternal complications is higher in cesarean deliveries than in vaginal deliveries.12–16 A study assessing the associations of maternal characteristics and delivery mode with SMM found that the largest percentage of population-attributable risk for SMM was for cesarean delivery (37%).17 However, it is important to note that cesarean delivery may lead to SMM and vice versa. In hospital discharge data, this temporal relationship between cesarean delivery and SMM cannot be determined. Cesarean deliveries may occur for various life-threatening indications (e.g., placenta previa, obstructed labor, and severe preeclampsia), which may lead to confounding by indication when looking at the associations between mode of delivery and SMM. Women who experience cesarean delivery may represent a different population, with different risk factors and preexisting comorbidities, than women who experience vaginal delivery.

Therefore, understanding how the ranking of the indicators varies by delivery mode is important given that the risk of maternal complications is higher in cesarean deliveries compared with vaginal deliveries.12–16 The objective of this study is to determine which SMM indicators identify the most mortality during delivery hospitalization by delivery mode. As a secondary objective, we sought to identify the possible other factors related to in-hospital death, stratified by delivery mode. Based on the current literature of complications associated with cesarean delivery, we conducted a review of the discharge record for deaths that did not have an SMM indicator code. We sought to determine if discharge records for deaths without codes for an SMM indicator were proportionately greater in one delivery mode compared with the other.

Materials and MethodsStudy sample

We obtained data from the 1993–2015 Healthcare Cost and Utilization Project’s (HCUP) National Inpatient Sample (NIS),18 an all-payer database of hospital discharge records. The NIS is obtained annually from all states participating in HCUP, representing >97% of the U.S. population. Discharge records contained ICD-9-CM diagnosis and procedure codes, Medicare severity diagnosis-related group (MS-DRG) codes, and external cause of injury codes (E codes). We used data from the 1993–2015 period to be consistent with the study period used to develop the ranked SMM index. This period allowed for comparability of codes across years and the ability to obtain a sufficient sample size to produce reliable estimates of SMM and death, as these are rare events We did not examine the ICD transition because this was previously assessed for the ranked SMM index for all deliveries, indicating comparable SMM prevalence before and after the transition.6 Delivery hospitalizations were identified using a previously described algorithm.19 Analyses were restricted to hospitalizations for females aged 12–55 years. The cross-sectional nature of NIS data prevented the identification of multiple hospitalizations for the same woman during the study period. Therefore, the unit of analysis was the delivery hospitalization and not the woman. We suppressed estimates according to HCUP reporting guidelines (e.g., estimates based on cell size <11).20

Study measures for ranking of SMM indicators. In-hospital mortality was identified based on the disposition of the patient at discharge (died). Cesarean deliveries were identified using diagnosis codes (669.70, 669.71), procedure codes (740, 741, 742, 744, 749.9), and MS-DRG codes (370–371 for MS-DRG version 24 or lower; 765–766 for MS-DRG versions 25–32). We examined the 22 SMM indicators described by Kuklina et al. (Supplementary Table S1).6 Hospitalization characteristics included patient demographics (age, race/ethnicity, primary type of expected payer of the hospital stay, year of hospitalization) and hospital characteristics (urban/rural location and teaching status, region). Covariates were selected a priori based on the prior literature and clinical significance.6 Within our study sample, race/ethnicity was missing for approximately 24.1% of the sample in 1993. Race/ethnicity coding improved in more recent years, with 7.0% missing in 2015. Accordingly, for each year in the study sample, we used a fully conditional specification (FCS) method to generate five sets of imputed race/ethnicity values for delivery hospitalizations missing this information.21 Using a discriminant function, imputed race/ethnicity was modeled on patient age, primary type of expected payer of the hospital stay, NIS stratum, and a dichotomous indicator of a patient having at least 1 of the 22 SMM indicators included in the NIS discharge record (yes/no). This method adheres to the recommendations provided by HCUP for handling missing racial/ethnic data.22 The primary purpose of the imputation was to normalize population-based estimates and not to reliably identify the racial/ethnic classification of patients for any single delivery hospitalization. Observations for all other variables with missing data were excluded; each individual variable had <0.3% missingness, and the overall proportion of delivery hospitalizations excluded due to missingness in any covariate was 0.7%.

Ranking of indicators

We conducted separate analyses by delivery mode. Statistical analysis of the ranking of the SMM indicators is described elsewhere.6 Briefly, for each SMM indicator, we calculated the prevalence, in-hospital mortality rate, and population-attributable fraction (PAF) for all in-hospital deaths attributable to that indicator. We calculated the PAF using the formula: pd(RR−1RR) where pd is the proportion with the indicator among all the records with death, and RR is the relative risk of indicator-specific death.23 The indicator-specific RR was estimated using log-linear models, adjusting for patient demographics (age, race/ethnicity, primary type of expected payer of the hospital stay, year of hospitalization), hospital characteristics (urban/rural location and teaching status, region), and all other SMM indicators. Patient and hospital characteristics were included in each adjusted model to account for potential confounding. Each SMM indicator was modeled separately in a hierarchical manner, resulting in slight variations in the adjusted RR for these covariates. As they were not the primary focus, the covariate results are not shown; however, their inclusion improved the robustness of estimates and did not meaningfully change the relative ranking of the indicators. Parameter estimates from the five sets of imputed race/ethnicity values were pooled into a single set of estimates for the log-linear models.

For each delivery mode, an iterative process hierarchically ranked the indicators in order of their importance in identifying in-hospital mortality.6 To account for the indicators with relatively large PAFs due to high prevalence, we used a signal-to-noise ratio (SNR), described elsewhere, to determine the ranking of the indicators.6,24 A higher SNR value suggests that an indicator’s higher PAF is due to a higher mortality rate rather than a higher prevalence in the population. After selecting the indicator with the highest SNR, we forced the selected indicator to be mutually exclusive of the remaining indicators by removing all as-yet-unselected indicators from a record, if that record contained the selected indicator. This enabled the identification of how many additional deaths that specific indicator captures. The resulting PAF was recalculated and designated as the hierarchical PAF (hPAF). A cumulative hPAF was calculated by summing the hPAF values up to, and including, that iteration. The cumulative hPAF at each iteration represents the proportion of in-hospital deaths identified using the indicator selected on that iteration and all the previously selected indicators. We repeated this iterative ranking process, selecting the indicator with the next highest SNR until all the indicators were selected.

Secondary analysis: Discharge record review of in-hospital deaths

We conducted a secondary analysis for each delivery mode to review in-hospital deaths without an SMM indicator to (1) confirm that no SMM code was available, and (2) identify the possible other factors related to these deaths (e.g., trauma-related, metastatic cancer; Supplementary Table S2). We defined SMM as a patient having at least one of the 22 SMM indicators included in the NIS discharge record (yes/no). NIS data for each delivery hospitalization death without an SMM indicator were reviewed by two of three obstetricians (R.R.G., E.A.C., L.K.A.) to identify diagnosis, procedure, MS-DRG, or E codes for the documentation of the possible other factors related to the death.

All analyses were conducted using SAS 9.4 (SAS Institute Inc) and SAS-callable SUDAAN version 11.0 (RTI International, Research Triangle Park, NC, USA) to account for the survey design and weights. Because the NIS is de-identified, this study did not require approval by an institutional review board at the Centers for Disease Control and Prevention.

Results

Among the weighted 87,864,173 delivery hospitalizations (unweighted number of records = 18,198,934) identified in the NIS, 24,541,451 (27.9%) were cesarean deliveries and 63,322,722 (72.1%) were vaginal deliveries. There were 6,686 in-hospital deaths, with most deaths occurring among cesarean deliveries (4,758 [71.2%]). Most deaths had an SMM indicator; however, cesarean deliveries had a higher proportion of deaths with an SMM indicator (4,483 [94.2%]) compared with vaginal deliveries (1,417 [73.5%]).

Prevalence of indicators by delivery mode

Prevalence, in-hospital mortality rate, and PAF for the SMM indicators are summarized for cesarean and vaginal deliveries (Table 1). Severe preeclampsia; hemolysis, elevated liver enzymes, and low platelet count (HELLP) syndrome; or eclampsia was the most prevalent indicator for both cesarean (365.4 per 10,000 delivery hospitalizations) and vaginal (81.7 per 10,000) deliveries; however, the in-hospital mortality rate for this indicator was the lowest among all the indicators (0.1% for both cesarean and vaginal deliveries). For both delivery modes, indicator-specific mortality rates were the greatest for hospitalizations with conversion of cardiac rhythm (39.0% for cesarean deliveries; 31.5% for vaginal deliveries), cardiac arrest/ventricular fibrillation (37.5%; 32.8%, respectively), and amniotic fluid embolism (16.4%; 14.5%, respectively).

Ranking of indicators by delivery mode

Among cesarean deliveries, the top five ranked indicators were as follows: cardiac arrest/ventricular fibrillation, conversion of cardiac rhythm, ventilation, temporary tracheostomy, and aneurysm (Table 2). The lowest ranked indicators (ranked 13, 14, 15) were puerperal cerebrovascular disorders; blood transfusion without disseminated intravascular coagulation (DIC); and severe preeclampsia, HELLP syndrome, or eclampsia. In addition, 7 of the 22 SMM indicators did not identify any additional death for cesarean deliveries (acute myocardial infarction, shock, sickle cell crisis, heart failure/arrest during surgery or procedure, severe anesthetic complications, pulmonary edema/acute heart failure, DIC without blood transfusion). The top five ranked indicators among vaginal deliveries were as follows: conversion of cardiac rhythm, cardiac arrest/ventricular fibrillation, ventilation, temporary tracheostomy, and amniotic fluid embolism (Table 2). The lowest ranked indicators (ranked 11, 12, 13) were sepsis; DIC without blood transfusion; and severe preeclampsia, HELLP syndrome, or eclampsia. Moreover, 9 of the 22 SMM indicators did not identify any additional death for vaginal deliveries (aneurysm, acute respiratory distress syndrome, acute myocardial infarction, air and thrombotic embolism, puerperal cerebrovascular disorders, sickle cell crisis, heart failure/arrest during surgery or procedure, severe anesthetic complications, blood transfusion without DIC).

The top three ranked SMM indicators (conversion of cardiac rhythm, cardiac arrest/ventricular fibrillation, ventilation) identified 78.8% of in-hospital mortality among cesarean deliveries and 66.0% of in-hospital mortality among vaginal deliveries. These indicators had the three highest hPAFs for both cesarean deliveries (conversion of cardiac rhythm = 15.1%; cardiac arrest/ventricular fibrillation =33.4%; ventilation = 30.3%) and vaginal deliveries (conversion of cardiac rhythm = 25.8%; cardiac arrest/ventricular fibrillation = 12.6%; ventilation = 27.7%). Among cesarean delivery hospitalizations, 15 SMM indicators accounted for 90.4% of in-hospital mortality. Among vaginal delivery hospitalizations, 16 indicators accounted for 76.4% of in-hospital mortality.

Secondary analysis: Possible other factors related to in-hospital death for discharge records without an indicator

While most in-hospital deaths had an SMM indicator, the possible other factors related to death could not be determined for most deaths without an SMM indicator. Among in-hospital cesarean delivery deaths, 275 deaths (5.8%; unweighted number of records = 56) did not include coding for an SMM indicator. After a review of each discharge record by obstetricians, the possible other factors related to death could not be determined based on the available data for 205 deaths (74.5%) without an SMM indicator. Among in-hospital vaginal delivery deaths, 511 deaths (26.5%; unweighted number of records = 107) did not have coding for an SMM indicator based on the available data; the possible other factors related to death could not be determined for 456 deaths (89.4%). Among the deaths without an SMM indicator, the possible other factors related to death that could be determined based on the available data included: possible trauma, possible complications of metastatic cancer, possible cardiac condition, possible hemorrhage, multiple high-risk conditions, possible surgical complications, possible infectious cause, and possible DIC (data suppressed because cell count was <11). A list of all ICD-9-CM codes documented among the deaths without an SMM indicator is included in Supplementary Table S3.

Discussion

The top four ranked indicators were the same for both delivery modes. The top three ranked indicators identified the most in-hospital deaths for both cesarean (78.8%) and vaginal (66.0%) deliveries. While most deaths during delivery hospitalization had documented SMM indicators, vaginal deliveries (26.5%) had a higher proportion of deaths without an SMM indicator compared with cesarean deliveries (5.8%).

Our study provides rankings of the SMM indicators in identifying in-hospital mortality for cesarean and vaginal deliveries. These rankings are similar to the ranked SMM index for all delivery types described previously6; however, there were differences by mode of delivery in the performance of the SMM indicators in identifying in-hospital deaths. Specifically, 15 of the 22 SMM indicators used in the ranking process accounted for 90.4% of in-hospital mortality among cesarean delivery hospitalizations; 7 of the indicators did not identify any additional death. In addition, 13 of the 22 SMM indicators used in the ranking process accounted for 76.4% of in-hospital mortality among vaginal delivery hospitalizations; 9 of the indicators did not identify any additional death.

In our study, 94.2% of cesarean delivery in-hospital deaths and 73.5% of vaginal delivery in-hospital deaths were identified as having SMM. This finding is consistent with previous population-level studies from the United States and Canada that identified 78.7% and 68.0% of deaths at delivery as having SMM, respectively.5,25 Our study also aligns with earlier studies examining the association between maternal mortality and delivery mode, which show a higher risk of maternal mortality associated with cesarean versus vaginal deliveries.26,27

Data derived using ICD codes to identify SMM cases are routinely used to monitor maternal health at the population level and evaluate the impact of clinical quality improvement initiatives during delivery hospitalization.28–30 However, the measurement of SMM is limited by how accurately the indicators identify morbidity that could contribute to maternal deaths. Our study demonstrated that several indicators (e.g., DIC without blood transfusion, blood transfusion without DIC, severe preeclampsia, HELLP syndrome, or eclampsia) do not identify a substantial proportion of in-hospital deaths for either delivery mode after accounting for the other indicators, suggesting that these specific indicators alone may not be as effective for surveillance in capturing severe events such as death. Using only higher ranked indicators to identify morbidity that could contribute to maternal deaths could potentially improve the measurement of SMM by improving the positive predictive value for in-hospital death. However, using only these indicators could reduce the sensitivity of SMM measurement because fewer indicators and codes will be used to identify morbidity.

While death represents the most severe adverse outcome, it is not the only meaningful consequence of SMM.31–33 Surveillance of SMM aims to monitor and reduce severe pregnancy and delivery complications that can have lasting health impacts and increase health care use, regardless of mortality.31,32 In this study, we focused on mortality to assess how individual SMM indicators contribute to in-hospital death, recognizing that some indicators may have limited validity. For example, a validation study of ICD-10-CM codes for SMM at delivery hospitalization using 21 indicators reported low sensitivity (26%) and positive predictive value (28%) for identifying true clinical events.34 Our study along with the findings from this validation study suggests that there is a need for the improved documentation and measurement of severe complications at delivery. Future studies could examine SMM measurement compared with other adverse outcomes, including intensive care unit admission. As ICD-11-CM implementation advances, surveillance systems, in turn, will adapt to coding changes.

These rankings are intended to inform the refinement of population-level SMM surveillance strategies by identifying which indicators identify the most in-hospital mortality. The lower ranked indicators may contribute minimally to mortality in our analysis because of both limitations in ICD-coded administrative data and true differences in mortality risk across indicators. Some SMM indicators are associated with lower mortality and would remain lower-ranked. Still these indicators remain important for comprehensive SMM surveillance. Further studies could assess whether these conditions are accurately captured and determine their relevance in surveillance definitions or in facility-level morbidity reviews.

In our secondary analysis, we sought to identify the possible other factors related to deaths without an SMM indicator, particularly because the SMM indicators used in the ranking process accounted for <80% of deaths among vaginal delivery hospitalizations. After a discharge record review of deaths without an SMM indicator, the possible other factors related to death could not be determined for most deaths, especially among vaginal deliveries (89.4%). It is possible that these records could be missing codes that could have provided more information about why these deaths occurred. However, it is also possible that these records may have discharge dispositions that were miscoded. For instance, some records could have discharge dispositions coded as “died” instead of “routine” or vice versa. Our findings suggest that more work may be needed to identify the ranked indicators and the possible other factors related to in-hospital death among vaginal deliveries. Specifically, once there are enough years of data for a sufficient sample size to produce a reliable estimate, it will be important to re-examine deaths with ICD-10-CM codes because ICD-10-CM codes capture a higher level of detail than ICD-9-CM codes.

A major strength of the study is that the NIS is a national all-payer database that provides generalizable estimates and is widely used for health research in the United States. The NIS provided an opportunity to determine which SMM indicators, although rare outcomes, contributed the most to in-hospital mortality at delivery hospitalization by delivery mode.

Our study has several limitations. First, we were limited to using data with ICD-9-CM codes to compare codes across years and to obtain a sufficient sample size to produce reliable estimates of SMM and death. However, predictive modeling, using the ranked SMM index for all deliveries, determined that the SMM prevalence remained comparable after the transition to ICD-10-CM coding.6 Second, only SMM events during delivery hospitalization are analyzed in this study. The SMM indicators are combined to form an index generally used to capture severe events during delivery hospitalization and do not capture severe events that may contribute to mortality in the late postpartum period (43–365 days postpartum), such as mental health conditions and substance use disorders.9 Other metrics may be needed to capture SMM postpartum events after delivery hospitalization. Third, temporality cannot be determined from the data. The timing of the SMM events—either before or after delivery—cannot be established in the database; thus, this aspect of our results should be interpreted with caution. The purpose of this study was to assess SMM measurement for surveillance and not to identify the causes of individual indicators or causes of maternal death. Additional analyses are needed to examine the ranked SMM index and causes of morbidity and mortality, which include the factors contributing to death, using other data sources, such as data from maternal mortality review committees. Fourth, race/ethnicity values were missing for nearly a quarter of our study population in the earlier years, which are known to not be missing completely at random. To decrease the risk of bias, we performed multiple imputation using a FCS method according to the recommendations from HCUP.22 While this approach improves the representativeness of population-level estimates, imputed data are not equivalent to the observed values and may still introduce residual bias, particularly given the differences in maternal health outcomes across racial and ethnic groups that may not be fully captured in administrative data.35 Finally, after a discharge record review, our study was unable to determine the possible other factors related to deaths without coding for an SMM indicator for either delivery mode. Misclassification is possible because some records could have discharge disposition codes that were miscoded (e.g., coded “died” instead of “routine” or vice versa). Additionally, misclassification is also possible because some records may have SMM codes that were miscoded. The presence of in-hospital death may prompt a more thorough documentation of complications, particularly in cesarean deliveries, or conversely, under-coding of complications in vaginal deliveries, especially in cases without death, may underestimate the utility of certain indicators in that group. These differential documentation patterns, if present, could bias our results and complicate the interpretation of differences in indicator performance by delivery mode.

Conclusions

The rankings for cesarean and vaginal deliveries are similar to the ranked index for all deliveries. However, there were differences by mode of delivery in the performance of the SMM indicators in identifying in-hospital deaths. Most in-hospital deaths for both cesarean and vaginal deliveries were identified by the top three ranked SMM indicators (conversion of cardiac rhythm, cardiac arrest/ventricular fibrillation, ventilation). Future studies of the ranked index are needed and may inform its utility for public health surveillance. Our findings underscore the need for the improved documentation and measurement of severe obstetric complications during pregnancy and the postpartum period at the population level.

Supplementary Material

Supplementary Table S1

Supplementary Table S2

Supplementary Table S3

Funding Information

The author(s) received no specific funding for this work.

The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.

Corresponding preprint: Available on medRxiv. DOI: 10.1101/2023.10.17.23297168.

An abstract based on this analysis was presented at the 2024 Society for Pediatric and Perinatal Epidemiological Research (SPER) Annual Meeting held in June 17–18, 2024.

The abstract is available in the conference abstract book: SPER 2024 Abstract Book, https://sper.org/meeting/annual-meeting-2024/

This article cites an earlier, related paper (reference 6) for which a preprint exists. The details for that previously published study are as follows: Published article: Ranked severe maternal morbidity index for population-level surveillance at delivery hospitalization based on hospital discharge data, PLoS One, 2023. DOI: 10.1371/journal.pone.0294140

Author Disclosure Statement

No competing financial interests exist.

ReferencesWorld Health Organization. Maternal and perinatal death surveillance and response: Materials to support implementation; 2021. Available from: https://www.who.int/publications/i/item/9789240036666TrostS, BeauregardJ, ChandraG Pregnancy-Related Deaths: Data from Maternal Mortality Review Committees in 36 US States, 2017–2019; 2022. Available from: https://www.cdc.gov/reproductivehealth/maternal-mortality/erase-mm/data-mmrc.html [Last accessed: September 19, 2022]Obstetric Care Consensus No. 5. Severe maternal morbidity: Screening and review. Obstet Gynecol 2016;128(3):e54–e60; doi: 10.1097/AOG.000000000000164227548555 SnowdenJM, LyndonA, KanP, Severe maternal morbidity: A comparison of definitions and data sources. Am J Epidemiol 2021;190(9):1890–1897; doi: 10.1093/AJE/KWAB07733755046 RayJG, ParkAL, DzakpasuS, Prevalence of severe maternal morbidity and factors associated with maternal mortality in Ontario, Canada. JAMA Netw Open 2018;1(7):e184571; doi: 10.1001/JAMANETWORKOPEN.2018.457130646359 KuklinaEV, EwingAC, SattenGA, Ranked severe maternal morbidity index for population-level surveillance at delivery hospitalization based on hospital discharge data. PLoS One 2023;18(11):e0294140; doi: 10.1371/JOURNAL.PONE.029414037943788 KayemG, KurinczukJ, LewisG, Risk factors for progression from severe maternal morbidity to death: A national cohort study. PLoS One 2011;6(12):e29077; doi: 10.1371/journal.pone.002907722216171 CallaghanWM, MacKayAP, BergCJ. Identification of severe maternal morbidity during delivery hospitalizations, United States, 1991–2003. Am J Obstet Gynecol 2008;199(2):133.e1–133.e8; doi: 10.1016/J.AJOG.2007.12.020Centers for Disease Control and Prevention. How does CDC identify severe maternal morbidity? Available from: https://www.cdc.gov/reproductivehealth/maternalinfanthealth/smm/severe-morbidity-ICD.htm [Last accessed: August 24, 2022].MainEK, AbreoA, McNultyJ, Measuring severe maternal morbidity: Validation of potential measures. Am J Obstet Gynecol 2016;214(5):643.e1–643.e10; doi: 10.1016/J.AJOG.2015.11.004HimesKP, BodnarLM. Validation of criteria to identify severe maternal morbidity. Paediatr Perinat Epidemiol 2020;34(4):408–415; doi: 10.1111/ppe.1261031951038 SchutteJM, SteegersEAP, SchuitemakerNWE, ; Netherlands Maternal Mortality Committee. Rise in maternal mortality in the Netherlands. BJOG 2010;117(4):399–406; doi: 10.1111/J.1471-0528.2009.02382.X19943828 ZwartJJ, RichtersJM, ÖryF, Severe maternal morbidity during pregnancy, delivery and puerperium in the Netherlands: A nationwide population-based study of 371,000 pregnancies. BJOG 2008;115(7):842–850; doi: 10.1111/J.1471-0528.2008.01713.X18485162 LiuS, ListonRM, JosephKS, ; Maternal Health Study Group of the Canadian Perinatal Surveillance System. Maternal mortality and severe morbidity associated with low-risk planned cesarean delivery versus planned vaginal delivery at term. CMAJ 2007;176(4):455–460; doi: 10.1503/CMAJ.06087017296957 PallasmaaN, EkbladU, GisslerM, The impact of maternal obesity, age, pre-eclampsia and insulin dependent diabetes on severe maternal morbidity by mode of delivery-a register-based cohort study. Arch Gynecol Obstet 2015;291(2):311–318; doi: 10.1007/S00404-014-3352-Z25115277 PallasmaaN, EkbladU, Aitokallio-TallbergA, Cesarean delivery in Finland: Maternal complications and obstetric risk factors. Acta Obstet Gynecol Scand 2010;89(7):896–902; doi: 10.3109/00016349.2010.48789320583935 LeonardSA, MainEK, CarmichaelSL. The contribution of maternal characteristics and cesarean delivery to an increasing trend of severe maternal morbidity. BMC Pregnancy Childbirth 2019;19(1):16; doi: 10.1186/S12884-018-2169-330626349 HCUP Databases. Healthcare Cost and Utilization Project (HCUP). HCUP-US NIS overview. Agency for Healthcare Research and Quality, Rockville, MD. September 2021. Available from: https://www.hcup-us.ahrq.gov/nisoverview.jsp [Last accessed: August 23, 2022].KuklinaEV, WhitemanMK, HillisSD, An enhanced method for identifying obstetric deliveries: Implications for estimating maternal morbidity. Matern Child Health J 2008;12(4):469–477; doi: 10.1007/S10995-007-0256-617690963 Healthcare Cost and Utilization Project (HCUP). Publishing with HCUP Data.; 2023. Available from: https://hcup-us.ahrq.gov/db/publishing.jsp [Last accessed: February 9, 2024].BerglundP, HeeringaS. Multiple Imputation of Missing Data Using SAS. SAS Institute; 2014.HouchensR Missing Data Methods for the NIS and the SID. HCUP Methods Series Report # 2015–01 ONLINE.; 2015. Available from: http://www.hcup-us.ahrq.gov/reports/methods/methods.jspRockhillB, NewmanB, WeinbergC. Use and misuse of population attributable fractions. Am J Public Health 1998;88(1):15–19; doi: 10.2105/AJPH.88.1.159584027 GeorgeF, KibriaBMG. Confidence intervals for estimating the population signal-to-noise ratio: A simulation study. J Appl Stat 2012;39(6):1225–1240; doi: 10.1080/02664763.2011.644527FriedmanAM, AnanthCV, HuangY, Hospital delivery volume, severe obstetrical morbidity, and failure to rescue. Am J Obstet Gynecol 2016;215(6):795.e1–795.e14; doi: 10.1016/J.AJOG.2016.07.039HallMH, BewleyS. Maternal mortality and mode of delivery. Lancet 1999;354(9180):776; doi: 10.1016/S0140-6736(05)76016-5Deneux-TharauxC, CarmonaE, Bouvier-ColleMH, Postpartum maternal mortality and cesarean delivery. Obstet Gynecol 2006;108(3 Pt 1):541–548; doi: 10.1097/01.AOG.0000233154.62729.2416946213 The Alliance for Innovation on Maternal Health. AIM data overview. Available from: https://saferbirth.org/aim-data/overview/ [Last accessed: September 26, 2022].MainEK, ChangSC, DhurjatiR, Reduction in racial disparities in severe maternal morbidity from hemorrhage in a large-scale quality improvement collaborative. Am J Obstet Gynecol 2020;223(1):123.e1–123.e14; doi: 10.1016/J.AJOG.2020.01.026ConreyEJ, ManningSE, ShellhaasC, Severe maternal morbidity, a tale of 2 states using data for action-Ohio and Massachusetts. Matern Child Health J 2019;23(8):989–995; doi: 10.1007/S10995-019-02744-131055701 UkahUV, DayanN, PotterBJ, Severe maternal morbidity and long-term risk of cardiovascular hospitalization. Circ Cardiovasc Qual Outcomes 2022;15(2):e008393; doi: 10.1161/CIRCOUTCOMES.121.00839335098729 VescoKK, FerranteS, ChenY, Costs of severe maternal morbidity during pregnancy in US commercially insured and Medicaid populations: An observational study. Matern Child Health J 2020;24(1):30–38; doi: 10.1007/s10995-019-02819-z31655962 MengistuTS, TurnerJM, FlatleyC, The impact of severe maternal morbidity on perinatal outcomes in high income countries: Systematic review and meta-analysis. J Clin Med 2020;9(7):2035; doi: 10.3390/jcm907203532610499 BouletSL, StanhopeKK, Valdez-SinonAN, Validation of ICD-10 codes for severe maternal morbidity at delivery in a public hospital. Epidemiology 2024;35(4):506–511; doi: 10.1097/EDE.000000000000174338567907 ZhangY, KissinDM, LiaoKJ, Multiple imputation of missing race/ethnicity information in the National Assisted Reproductive Technology Surveillance System. J Womens Health (Larchmt) 2024;33(3):328–338; doi: 10.1089/jwh.2023.026738112534

Prevalence, in-Hospital Mortality, and PAF for SMM Indicators among Cesarean Deliveries (Weighted Sample = 24,541,451) and Vaginal Deliveries (Weighted Sample = 63,322,722) Using Existing SMM Indicators,6 nis, 1993–2015

Cesarean deliveriesVaginal deliveries
Prevalence among delivery hospitalizationsIn-hospital mortalityRRbPAFcPrevalence among delivery hospitalizationsIn-hospital mortalityRRbPAFc
SMM indicatorsa n d Per 10,000 deliveries (95% CI) n d % (95% CI)RRb% n d Per 10,000 deliveries (95%o CI) n d % (95%, CI)RRb%
Severe preeclampsia, HELLP syndrome, or eclampsia896,621365.4 (356.0, 374.7)1,0110.1 (0.1, 0.1)1.11.9%517,52981.7 (78.2, 85.3)3680.1 (0.1, 0.1)3.312.3%
Blood transfusion without DIC349,430142.4 (137.6, 147.2)8450.2 (0.2, 0.3)5.318.0%227,49935.9 (34.6, 37.3)2200.1 (0.1, 0.1)26.014.5%
DIC without blood transfusion83,93134.2 (32.8, 35.6)6300.8 (0.6, 0.9)4.78.9%86,71313.7 (13.0, 14.4)3220.4 (0.3, 0.5)29.315.5%
Hysterectomy58,00823.6 (22.8, 24.4)5140.9 (0.7, 1.1)0.9−1.3%15,1702.4 (2.3, 2.5)2651.8 (13,2.2)4.516.3%
Acute respiratory distress syndrome35,49214.5 (14.0, 14.9)1,8045.1 (4.6, 5.6)1.413.0%10,5771.7 (1.6, 1.8)5975.6 (4.6, 6.7)1.410.8%
Ventilation31,80113.0 (12.5, 13.4)2,5448.0 (7.3, 8.7)39.457.2%8,5181.4 (1.3, 1.4)8079.5 (8.0, 10.9)12.042.4%
Pulmonary edema/acute heart failure30,07212.3 (11.7, 12.8)2670.9 (0.6, 1.1)0.7−2.1%12,1841.9 (1.8, 2.1)1050.9 (0.5, 1.2)2.93.8%
DIC and blood transfusion26,68910.9 (10.4, 11.4)7502.8 (2.4, 3.2)7.417.7%12,3662.0 (1.8, 2.1)3282.7 (2.0, 3.3)51.927.2%
Renal failure25,68810.5 (10.1, 10.9)8353.3 (2.8, 3.7)1.23.8%9,7211.5 (1.4, 1.6)3773.9 (3.0, 4.8)1.811.5%
Sepsis20,3208.3 (7.9, 8.7)6323.1 (2.5, 3.7)3.715.1%13,0822.1 (1.8, 2.3)3442.6 (2.0, 3.3)5.220.3%
Puerperal cerebrovascular disorders16,8606.9 (6.6, 7.1)7414.4 (3.7, 5.1)5.311.3%11,8551.9 (1.8, 2.0)2271.9 (1.4, 2.4)3.49.2%
Severe anesthetic complications14,6866.0 (5.6, 6.4)1721.2 (0.8, 1.6)0.8−0.4%7,3931.2 (1.1, 1.3)DSDSDSDS
Shock12,7335.2 (5.0, 5.4)7145.6 (4.7, 6.5)0.9−2.1%7,5571.2 (1.1, 1.3)3134.1 (3.2, 5.1)1.12.0%
Heart failure/arrest during surgery or procedure8,8243.6 (3.3, 3.9)1161.3 (0.8, 1.8)0.3−2.3%1,3300.2 (0.2, 0.2)DSDSDSDS
Air and thrombotic embolism8,3743.4 (3.2, 3.6)4595.5 (4.4, 6.6)2.87.1%5,4920.9 (0.8, 0.9)691.3 (0.6, 1.9)0.7−1.9%
Sickle cell crisis4,5451.9 (1.7, 2.0)531.2 (0.5, 1.9)1.50.4%3,8150.6 (0.5, 0.7)DSDSDSDS
Cardiac arrest/ventricular fibrillation4,1791.7 (1.6, 1.8)1,56837.5 (34.3,40.8)7.933.4%1,1680.2 (0.2, 0.2)38432.8 (26.8, 38.9)6.221.6%
Conversion of cardiac rhythm3,8301.6 (1.4, 1.7)1,49539.0 (35.4,42.6)6.531.1%1,3900.2 (0.2, 0.3)43831.5 (25.9, 37.0)10.025.8%
Amniotic fluid embolism2,7351.1 (1.0, 1.2)45016.4 (13.3, 19.6)1.00.0%1,1230.2 (0.2, 0.2)16314.5 (9.7, 19.3)1.94.4%
Temporary tracheostomy1,6920.7 (0.6, 0.8)21712.8 (9.5, 16.2)0.6−3.3%4120.1 (0.1, 0.1)DSDSDSDS
Acute myocardial infarction1,1210.5 (0.4, 0.5)998.8 (5.2, 12.5)0.7−0.7%5120.1 (0.1, 0.1)DSDSDSDS
Aneurysm7710.3 (0.3, 0.4)DSDSDSDS3060.1 (0.0, 0.1)DSDSDSDS

Sorted by prevalence among delivery hospitalizations for cesarean deliveries.

The estimated RR associated with a specific indicator was obtained from log-linear regression models, adjusting for patient age, patient race/ethnicity, primary expected payer, hospital location and teaching status, hospital region, year, and all other SMM indicators.

PAFs for all SMM indicators were calculated using the formula pd ((RR – 1)/RR), where pd is the proportion with the indicator among all hospital discharge records with in-hospital death, and RR is the estimated relative risk of indicator-specific in-hospital death. The estimated RR associated with a specific indicator was obtained from log-linear regression models, adjusting for patient age, patient race/ethnicity, primary expected payer, hospital location and teaching status, hospital region, year, and all other SMM indicators.

Weighted count (n).

PAF, population attributable fraction; SMM, severe maternal morbidity; NIS, National Inpatient Sample; CI, confidence interval; HELLP, hemolysis, elevated liver enzymes, and low platelet count; DIC, disseminated intravascular coagulation; DS, data suppressed because of cell count of 10 or less.

Results of the smm indicator ranking, by delivery mode, using methods described by kuklina et al.,6 nis, 1993–2015

Cesarean deliveriesVaginal deliveries
Overall ranking for all delivery hospitalizations from Kuklina el al.6SMM indicatorsIteration/rankSNRaIn-hospital mortality nb,cCumulative in-hospital mortality nb,dhPAFeCumulative hPAFf,gIteration/rankSNRaIn-hospital mortality nb,cCumulative in-hospital mortality nb,dhPAFeCumulative hPAFf,g
1Conversion of cardiac rhythm255.97502,31815.1%48.5%150.543843825.8%25.8%
2Cardiac arrest/ventricular fibrillation125.01,5681,56833.4%33.4%2111.119162912.6%38.3%
3Ventilation314.01,4383,75630.3%78.8%343.45361,16527.7%66.0%
4Temporary tracheostomy4DSDSDSDSDS4DSDSDSDSDS
5Amniotic fluid embolism6DSDSDSDSDS5DSDSDSDSDS
6Aneurysm5DSDSDSDSDS———1,417—76.4%
7Acute respiratory distress syndrome101.5894,2030.6%87.3%———1,417—76.4%
SAcute myocardial infarction———4,483—90.4%———1,417—76.4%
9Shock———4,483—90.4%8DSDSDSDSDS
10Air and thrombotic embolism74.1623,9251.6%81.6%———1,417—76.4%
11Puerperal cerebrovascular disorders13DSDSDSDSDS———1,417—76.4%
12SepsisS3.3874,0122.3%83.9%11DSDSDSDSDS
13DIC and blood transfusion93.21014,1142.8%86.7%10DSDSDSDSDS
14Acute renal failure11DSDSDSDSDS6DSDSDSDSDS
15Hysterectomy12DSDSDSDSDS7DSDSDSDSDS
16Sickle cell crisish———4,483—90.4%———1,417—76.4%
17Heart failure/arrest during surgery or procedureh———4,483—90.4%———1,417—76.4%
ISSevere anesthetic complicationsh———4,483—90.4%———1,417—76.4%
19Pulmonary edema/acute heart failure——04,483—90.4%9DSDSDSDSDS
20DIC without blood transfusion———4,483—90.4%12DSDSDSDSDS
21Blood transfusion without DICh14DSDSDSDSDS———1,417—76.4%
22Severe preeclampsia, HELLP syndrome, or eclampsia15DSDS4,483DS90.4%13DSDS1,417DS76.4%

SNR is the selection criterion used to select the indicator that would maximize the ratio of the expected number of SMM cases detected (signal) to the increase in the standard deviation of the cumulative sum of indicators (noise).

Weighted count (n).

The number of additional deaths that the indicator identifies at this iteration of the ranking.

The cumulative count is based on the iteration/rank and not the position in the table rows. The row order is presented using the overall ranking for all delivery hospitalizations to enable comparison across the mode of delivery.

The hPAF value represents the PAF that was calculated for an indicator at the specified iteration/rank after previously selected indicators were made mutually exclusive. hPAFs for all SMM indicators were calculated using the formula pd ((RR – 1)/RR), where pd is the proportion with the indicator among all hospital discharge records with in-hospital death, and RR is the estimated relative risk of indicator-specific in-hospital death. The estimated RR associated with a specific indicator was obtained from log-linear regression models, adjusting for patient age, patient race/ethnicity, primary expected payer, hospital location and teaching status, hospital region, year, and all remaining (in hierarchical order) SMM indicators.

The cumulative hPAF is calculated as the sum of hPAF for the individual indicator and those above in ranking.

The cumulative hPAF is based on the iteration/rank and not the position in the table rows. The row order is presented using the overall ranking for all delivery hospitalizations to enable comparison across the mode of delivery.

Indicator fell out the ranking for vaginal deliveries because the indicator did not capture any additional death.

hPAF, hierarchical population attributable fraction; SNR, signal-to-noise ratio.