Hospital length of stay (LOS) and time for a patient to reach clinical stability (TCS) have increasingly become important outcomes when investigating ways in which to combat Community Acquired Pneumonia (CAP). Difficulties arise when deciding how to handle in-hospital mortality. Ad-hoc approaches that are commonly used to handle time to event outcomes with mortality can give disparate results and provide conflicting conclusions based on the same data. To ensure compatibility among studies investigating these outcomes, this type of data should be handled in a consistent and appropriate fashion.

Using both simulated data and data from the international Community Acquired Pneumonia Organization (CAPO) database, we evaluate two ad-hoc approaches for handling mortality when estimating the probability of hospital discharge and clinical stability: 1) restricting analysis to those patients who lived, and 2) assigning individuals who die the "worst" outcome (right-censoring them at the longest recorded LOS or TCS). Estimated probability distributions based on these approaches are compared with right-censoring the individuals who died at time of death (the complement of the Kaplan-Meier (KM) estimator), and treating death as a competing risk (the cumulative incidence estimator). Tests for differences in probability distributions based on the four methods are also contrasted.

The two ad-hoc approaches give different estimates of the probability of discharge and clinical stability. Analysis restricted to patients who survived is conceptually problematic, as estimation is conditioned on events that happen

Treating death as a competing risk gives estimators which address the clinical questions of interest, and allows for simultaneous modelling of both in-hospital mortality and TCS / LOS. This article advocates treating mortality as a competing risk when investigating other time related outcomes.

Traditionally, mortality or survival has been the outcome of clinical interest in evaluating new ways to combat community acquired pneumonia (CAP). More recently, outcomes such as a patient's length of hospital stay (LOS) and their time to reach clinical stability (TCS) have increasingly become outcomes of interest in patients with CAP, as these are directly relevant to patient management, quality of care, and hospital costs [

A key assumption with the Kaplan-Meier estimator is that the event of interest will eventually occur for

An alternative approach to handling mortality is to treat this event as a competing risk, which precludes the occurrence of the other events of interest (see [

The goal of this article is to evaluate what the two ad-hoc approaches to handling mortality are estimating, and demonstrate the potential disparity in results which can occur from naive use of these approaches. Results from the ad-hoc estimators are further compared with the advocated way of handling mortality as a competing risk. Differences between the estimators are illustrated using data from the international Community Acquired Pneumonia Organization (CAPO) database [

In this section, we define the functions to be estimated. The presentation given here follows in spirit to that given in Allignol _{t}_{t≥0 }represent the state that an individial is in for every time point, _{t }_{t }_{t }_{t }_{01}(_{02}(

_{01}(_{02}(

The quantity

The cause-specific hazards sum to give the _{0}.(_{0}. (_{0}.(

When competing risks are present, probabilities are described in terms of the cause-specific

The integrand on the right-hand side of the equation represents the joint probability of having survived (i.e., made neither transition) to time just prior to _{1}(_{2}(

In addition to the cause-specific hazards, another hazard function which has been developed in the competing risks literature is the

The quantity _{j}_{T }_{T }

For competing risks, the basic ingredients for estimation are the number of individuals who make a transition, and the number of individuals who are still _{1 }_{2 }< ... <_{k }_{0}(_{0}. (

The starting point for estimation are the Nelson-Aalen (N-A) estimators of the cause-specific cumulative hazard functions (see [

The N-A estimator of the all-cause hazard function is

The standard Kaplan-Meier (KM) estimator [

This estimator is a decreasing step function with "drops" at the observed transition times. The KM estimator estimates the probability that an event will eventually occur for

By substituting

(see [

The Aalen-Johansen estimates of the cumulative incidence functions [_{i }

This estimator can be derived in a straightforward fashion starting from the KM estimator of the overall survivor function [_{i }

The variability of the cumulative incidence estimator has received considerable attention in the recent literature [

The first ad-hoc approach evaluated in this study was to exclude individuals who died from the analysis (restricted analysis). With this approach, individuals who die are excluded entirely from the analysis, and their LOS data are essentially treated as being censored at day zero. Of significant note is that the restricted estimator suffers from a serious conceptual flaw, in that estimation at any time

The second ad-hoc approach was to assign individuals who died the "worst" outcome (worst outcome analysis). In this approach, individuals who die are right-censored at the longest possible follow-up time. This is essentially equivalent to assigning a discharge or clinically stable time of ∞ to the individuals who die in the hospital, and so coincides with the random variable which forms the basis of the subdistribution hazard in Equation (3). The complement of the KM estimator based on these censored subdistribution times has been previously shown to be equivalent to the cumulative incidence estimator (see [

Pointwise 95% confidence intervals for each estimator at time

To illustrate the differences between the four estimation methods from an established database, analyses were performed on data from the international CAPO dataset (see [

Time to clinical stability was defined using the American Thoracic Society criteria for switch therapy from intravenous to oral antibiotic therapy: 1) improvement in cough and shortness of breath; 2) afebrile status for ≥ 8 hours (

The study was approved by the Human Subject Protection Program Institutional Review Board at the University of Louisville. Additional approval was obtained from the local internal review board for each participating hospital. Patient consent was waived due to the retrospective and observational study design.

Competing risks data were simulated following the methodology outlined in Beyersmann

Conditional on an event happening at time

_{T }_{0i}(_{01}(_{02}(

Events were simulated for _{dis}

Description of methods evaluated using simulated data

Estimator | Description |
---|---|

Complement of the Kaplan-Meier estimator of hospital discharge, obtained by censoring those patients who died at their time of death | |

Estimate of cumulative incidence function of hospital discharge, obtained by treating in-hospital mortality as a competing risk | |

Complement of the KM estimator of hospital discharge, obtained by removing those patients who died | |

Complement of the KM estimator of hospital discharge, obtained by censoring those patients who died at the longest recorded LOS (30 days) |

We additionally compared the coverage of 95% linear confidence intervals for _{dis}

The probability of rejecting the null hypothesis of no differences in hospital discharge probability distributions between two patient groups was evaluated under several scenarios. First, simulations were conducted under the null hypothesis, with both the mortality and discharge hazard functions being equal between the two patient groups. Additionally, we ran simulations with the hazard ratio for mortality (HR_{mor}_{dis }_{mor }_{mor }_{mor }_{mor }

All the analyses presented in this paper were performed using R version 2.12.2 [

To illustrate the differences between the five estimators, consider the following artificial example of ten patients who are admitted to the hospital with CAP. Suppose that two of the patients die at days 3 and 5, five of the patients reach clinical stability on days 2, 2, 4, 5, and 7, and three patients do not reach clinical stability by day 7. After day 7, information regarding clinical stability is no longer collected, so that the three patients who did not reach clinical stability by day 7 are censored at day 7. The data are displayed in Tables _{i }_{i }_{i}_{i }_{i}_{i }_{i}

Kaplan-Meier and cumulative incidence estimators of probability of clinical stability for artificial data

_{i} |
_{i} |
_{i} |
_{i} | SE | SE | ||||
---|---|---|---|---|---|---|---|---|---|

2 | 10 | 2 | 0 | 1-2/10 = 0.80 | 0.20 | 0.126 | 1-2/10 = 0.8 | 2/10 = 0.2 | 0.133 |

3 | 8 | 0 | 1 | 0.80 | 0.20 | 0.126 | 0.8*(1-1/8) = 0.7 | 0.2+0/8*(0.8) = 0.2 | 0.133 |

4 | 7 | 1 | 0 | 0.8*(1-1/7) = 0.69 | 0.31 | 0.151 | 0.7*(1-1/7) = 0.6 | 0.2+1/7*(0.7) = 0.3 | 0.154 |

5 | 6 | 1 | 1 | 0.69*(1-1/6) = 0.57 | 0.43 | 0.164 | 0.6*(1-2/6) = 0.4 | 0.3+1/6*(0.6) = 0.4 | 0.167 |

7 | 4 | 1 | 0 | 0.57*(1-1/4) = 0.43 | 0.57 | 0.174 | 0.4*(1-1/4) = 0.3 | 0.4+0/4*(0.4) = 0.5 | 0.172 |

>7 | 3 | 0.43 | 0.57 | 0.3 | 0.5 |

_{i }_{i }_{i}_{i }_{i}_{i }_{i}_{i}

Ad-hoc estimators of probability of clinical stability for artificial data

_{i} |
_{i} |
_{i} |
_{i} | SE | SE | ||||
---|---|---|---|---|---|---|---|---|---|

2 | 10 | 2 | 0 | 1-2/8 = 0.75 | 0.25 | 0.153 | 1-2/10 = 0.8 | 0.2 | 0.126 |

3 | 8 | 0 | 1 | 0.75 | 0.25 | 0.153 | 0.8 | 0.2 | 0.126 |

4 | 7 | 1 | 0 | 0.75*(1-1/6) = 0.63 | 0.37 | 0.171 | 0.8*(1-1/8) = 0.7 | 0.3 | 0.145 |

5 | 6 | 1 | 1 | 0.63*(1-1/5) = 0.50 | 0.5 | 0.177 | 0.7*(1-1/7) = 0.6 | 0.4 | 0.155 |

7 | 4 | 1 | 0 | 0.5*(1-1/4) = 0.38 | 0.62 | 0.171 | 0.6*(1-1/6) = 0.5 | 0.5 | 0.158 |

> 7 | 3 | 0.38 | 0.62 | 0.5 | 0.5 |

_{i }_{i }_{i}_{i }_{i}_{i }_{i}

The estimate of clinical stability restricted to only patients who lived,

The 'worst outcome' estimator

The standard error of the cumulative incidence function is Gray's estimate [

Our analysis of the CAPO data focuses on length of hospital stay. We initially restrict our presentation to the subset of patients in the highest risk class calculated from the pneumonia severity index (PSI), Risk Class V (

The test-statistic for hospital discharge based on the cumulative incidence function has a p-value of 5 × 10^{-17}. Log-rank tests based on the ad-hoc estimators, restricted and worst outcome, give p-values of 8 × 10^{-4 }and 3 × 10^{-18 }respectively, which are several orders of magnitude different. The log-rank test based on the Kaplan-Meier estimator gives a p-value = 8 × 10^{-8}, but it is unclear what is the clinical interpretation of this test.

The competing risks analysis allows simultaneous comparison of both LOS and mortality incidence. In contrast, right censoring or removing patients who die prevents mortality information from being incorporated. Treating mortality as a competing risk provides a mechanism to view the incidence curves of both outcomes, so that multiple outcomes can be compared between patient groups. To illustrate, we view both the discharge and mortality incidence for patients in RC V (410 patients) versus RC IV (822 patients) in Figure

To more comprehensively evaluate the differences between the four methods, we simulated competing risks data using the methods in Beyersmann

As noted in the Methods, their are several estimators of the variance of cumulative incidence estimator that have been proposed in the literature. Previous research [

Coverage probabilities for 95% confidence intervals for

Sample Size | Estimator | 5 | 10 | 15 | 20 | 25 |
---|---|---|---|---|---|---|

1500 | Gray | 0.945 | 0.952 | 0.951 | 0.956 | 0.954 |

Greenwood | 0.945 | 0.954 | 0.951 | 0.957 | 0.954 | |

750 | Gray | 0.953 | 0.958 | 0.962 | 0.965 | 0.961 |

Greenwood | 0.960 | 0.963 | 0.962 | 0.965 | 0.961 | |

200 | Gray | 0.935 | 0.938 | 0.943 | 0.942 | 0.944 |

Greenwood | 0.935 | 0.938 | 0.953 | 0.942 | 0.944 | |

100 | Gray | 0.961 | 0.949 | 0.952 | 0.955 | 0.950 |

Greenwood | 0.961 | 0.949 | 0.949 | 0.955 | 0.943 | |

50 | Gray | 0.938 | 0.950 | 0.926 | 0.931 | 0.938 |

Greenwood | 0.938 | 0.950 | 0.926 | 0.931 | 0.938 | |

25 | Gray | 0.950 | 0.941 | 0.936 | 0.955 | 0.936 |

Greenwood | 0.950 | 0.941 | 0.936 | 0.955 | 0.936 |

Lastly, we evaluated the power for the four methods to reject the null hypothesis of no differences in discharge probability distributions between patient groups, under various scenarios. A sample size of 750 patients was used for each of the two patient groups, for each simulation. It should be noted that each method is testing a different null hypothesis, e.g. the test based on the cumulative incidence estimator is testing for differences between the cumulative incidence functions _{j}_{j}

The first row of Table

Proportion of times that the null hypothesis of no difference in discharge probabilities is rejected, using an

HR_{dis} | HR_{mor} | ||||
---|---|---|---|---|---|

1.0 | 1.0 | 0.048 | 0.047 | 0.049 | 0.054 |

1.15 | 0.064 | 0.061 | 0.055 | 0.062 | |

1.5 | 0.128 | 0.120 | 0.048 | 0.362 | |

1.15 | 1.0 | 0.553 | 0.544 | 0.583 | 0.505 |

0.67 | 0.772 | 0.772 | 0.601 | 0.137 | |

0.87 | 0.652 | 0.644 | 0.603 | 0.339 | |

1.15 | 0.470 | 0.462 | 0.577 | 0.681 | |

1.5 | 0.256 | 0.248 | 0.577 | 0.946 |

HR_{dis }_{mor }

The next five rows of Table _{mor }>_{dis}_{01}) to mortality hazard (_{02}) for this group is

The trend for

The way in which mortality is handled while investigating other time-related outcomes can have an impact on the results obtained and the interpretations conjectured. In the CAP literature, there is no consensus for how to properly handle mortality when evaluating TCS and LOS. Therefore, direct comparisons are difficult and conflicting conclusions can be reached depending on the approach that is used. Two ad-hoc approaches for handling mortality were investigated in this paper, analysis restricted to the subset of patients who survived and assigning patients who died the worst possible outcome. The two methods give different results for the same data and could lead to conflicting conclusions, unless investigators are aware of the differences between the estimators. The first approach conditions on the occurrence of future events, and the practical use of this estimator is questionable. The second approach coincides with the Kaplan-Meier estimator based on the subdistribution hazard, which has been proven to be equivalent to the cumulative incidence estimator [

It should be mentioned that although we considered the 'worst outcome' approach to be ad-hoc, the equivalence of this method to the random variables defined for the subdistribution function in [

The complement of the Kaplan-Meier estimator, which censors patients who die at their time of death, provides an estimate that is between the cumulative incidence function and restricted analysis estimator. As past authors have pointed out [

In contrast to the ad-hoc estimators and Kaplan-Meier estimator, the estimators based on treating in-hospital mortality as a competing risk have clearly defined interpretations. Recent applications of these approaches (and more general multi-state models) to the study of nosocomial (hospital-acquired) pneumonia infections have been conducted by Beyersmann

Simulations revealed that the power of each method to detect differences in underlying discharge rates between patient groups depended on the rate of the competing event (mortality). When the hazard ratios were in opposite directions, so that the patient group with the higher discharge rate also had a lower mortality rate, differences in the cumulative incidence of discharge were increased and the power based on Gray's test [

In contrast to the tests based on the cumulative incidence function, tests based on restricting the patient sample to those who survived had increased "power" when the group with the higher discharge rate also had a higher mortality rate. However, by removing patients who died from the study, the differences in mortality rates between the two patient populations results in a

In our modeling of TCS, we focused solely on the time from hospital admission to clinical stability and included in-hospital mortality as a competing risk. Hospital discharge was not included as an additional competing risk, which is reasonable given the definition of time of clinical stability as the time the criteria for switching from intravenous to oral antibiotic therapy was met. In the competing risks model, each outcome is treated as an absorbing state, so that transitions do not occur after reaching these outcomes. In interest is solely in the time that clinical stability is reached, then this model is reasonable. However, an extended modeling for clinical stability and subsequent discharge and/or in-hospital mortality can be constructed, using a multi-state model which allows transitions from the clinically stable state to these other two outcomes. The model would be similar to that used for modeling nosocomial infections in Beyersmann

Lastly, in our evaluation of methods for handling mortality we did not discuss the issue of incorporating additional risk factors, which can be done using a regression model. When competing risks are present, regression modeling focuses on either Cox models [

This paper investigates mechanisms for handling in-hospital mortality when analyzing length of hospital stay (LOS) or time to clinical stability (TCS), using data from patients admitted to the hospital with community-acquired pneumonia. Two currently used ad-hoc approaches, restricting analysis to those patients who lived and assigning individuals who die the worst outcome (longest LOS or TCS), gave disparate results when applied to the same data set and are discouraged from use. In contrast, estimators based on treating mortality as a competing risk have clinically relevant interpretations. Additionally, the incidence of mortality can be compared simultaneously with that of discharge / clinical stability, for more comprehensive comparisons between patient populations. These estimators have been readily available in the literature for over thirty years, yet still are frequently overlooked when analyzing time-to-event outcomes in the presence of competing risks. With the ready availability of software for these estimators and their straightforward interpretation, there is no reason to eschew them in favor of other ad-hoc estimators that may be considered. We provide illustrative statistical code as supplemental material for investigators to use in their own studies, to promote use of these estimators in practice and improve compatibility between studies that are investigating these time-to-event outcomes. Interested readers are also referred to the many excellent articles in the literature [

The following is a list of the abbreviations used throughout the manuscript:

The authors declare that they have no competing interests.

GNB wrote the manuscript, performed the data analysis, and created the supplemental material. CB performed the simulation studies. JAR provided the CAPO data and clinical insight to the problem. JM originally conceived of the problem and supervised the research. All authors helped with drafting the manuscript, and read and approved the final draft.

The complement of the KM estimator based on the subdistribution hazard has been previously shown to be equivalent to the cumulative incidence estimator [

for all times

The proof goes by induction. Consider the first jump (i.e., the first 0 → 1 transition), at time _{i}, then for time _{i+1 }we have for the RHS of (8):

which completes the proof.

The pre-publication history for this paper can be accessed here:

Click here for file

Supplement describing use of R for calculation of the cumulative incidence estimator.

Click here for file

Click here for file

Click here for file

Click here for file

This research was supported in part by the National Institutes of Health [P30-ES014443 and

P20-RR/DE177702 for GNB], [1R03OH008957-01A2, PO2 728 0800015499, RSGHP-09-099-01-CPHP for JM]. The authors gratefully acknowledge the work of the CAPO investigators in building and maintaining the CAPO data set (see Additional File