NASS-GES is a weighted stratified sample of police-reported traffic crashes resulting in personal injury or property damage (NHTSA, 2005). Data are obtained from 60 Primary Sampling Units (PSU) within the United States, identified as a city, county, or group of counties (together including a total of 77 out of the 3142 US counties), selected to represent 14 predetermined types of geographic areas (PSU Strata). Next, a sample of police jurisdictions (PJ) is selected from each PSU. Finally, within the selected PJs, cases are selected from each of six types (Case Strata). The overall probability that a case will be selected is the product of the probabilities of selection at each level. From the inverse of this overall probability, NASS-GES derives a weight, which will be designated here as W_GES, to allow estimation of the actual totals and proportions in the US population. Further details about the sample design are given in a NASS-GES Technical Note (NHTSA, 1991).

Data for 2002–2008 were downloaded from the NASS-GES internet site. Analysis for this study included only injured subjects, namely those who had been described in a police crash report as having “possible injury”, “injury of unknown severity”, “non-incapacitating injury”, “incapacitating injury” or “fatal injury”. Characteristics of subjects involved in a crash from which at least one person died (“fatal crash”) were compared to subjects involved in nonfatal crashes. Characteristics of the persons, vehicles, and events included age, sex, pedestrian/occupant, safety belt use, ejection, vehicle speed (high/low), vehicle damage (severe/other), head-on collision, number of vehicles involved, number of persons injured, time of crash, and individual outcome (lived/died). Missing values were replaced with imputed values provided by NASS-GES. Vehicle speeds were considered high if the police estimate of the speed was at least 50 miles per hour; if there was no police estimate, vehicle speed was considered high if the posted speed limit was at least 50 miles per hour (Clark, 2003).

The NASS-GES Technical Note also lists the cities or counties in each of the 60 PSUs (NHTSA, 1991). For each severely injured person, a crash county was assigned as: 1) The PSU county if the PSU contained a single county; 2) The driver’s residence county if the PSU contained more than one county and the driver’s ZIP code matched one of the included counties; or 3) The most frequent residence county for the PSU if it contained more than one county and the driver’s ZIP code did not match any included county. This approach resulted in 77 crash counties, which could be linked using the Federal Information Processing Standard (FIPS) county codes (US Census, 2011) to county-level data in the Area Resource File (HRSA, 2011). In particular, variables were created indicating the county Rural-Urban Continuum Code (RUCC), which classifies US counties into nine groups from most urban (1) to most rural (9).

The complex sampling design of NASS-GES was accommodated using the “svy” commands available in Stata, with the first stage consisting of PSU grouped within PSU Strata, a second stage consisting of PJ, and a third stage consisting of individual cases grouped within Case Strata. The “subpopulation” option was used to restrict analysis to injured persons. The probability of death for injured persons was modeled using the command “svy:logit” and the independent variables described above. The analysis was repeated for the smaller subsample of injured persons involved in crashes where at least one person died (a fatal crash sample).

For each crash in which one or more subjects had been injured, one injured subject was randomly selected as an “index person”. Logistic regression with the independent variables described above, including variables relating to the index person, was then used to estimate the probability that one or more subjects in the crash had been fatally injured. For each crash, this produced a “propensity score” (P_FC) estimating the probability that the crash would be included in a subsample limited to fatal crashes.

An “inverse propensity” weighting factor (W_FC) was then calculated as the inverse of the propensity score (1/P_FC), rounded to the nearest integer. This weight was applied to all injured persons in the fatal crash sample, whether or not they were the randomly-selected index person. Using the Stata “expand” command, each subject in the fatal crash subsample was replicated once, and a weight of W_GES*(W_FC -1) was assigned to each of the replicated cases. Identical results could be obtained by creating (W_FC -1) replications of each subject and retaining the original weight of W_GES for each case. Replicated cases (which will also be referred to as “pseudosubjects”) retained the characteristics of the original cases from which they had been replicated, except that pseudosubjects were all recorded as survivors, whether or not the original subject had survived. The pseudosubjects were appended to the original cases in the fatal crash sample, creating a “pseudopopulation”, theoretically having the same characteristics as the population of injured persons who would have been included in the fatal crash sample if their crash resulted in one or more deaths.

Figure 1 depicts graphically the rationale for obtaining an “inverse propensity weight”. For each possible covariate pattern, subjects have a propensity (P_FC) to be involved in a fatal crash (and therefore included in a sample limited to fatal crashes). Subjects involved in a fatal crash include a number D who died and a number L who lived. Subjects not involved in a fatal crash consist of a number X, all of whom by definition survived. Since P_FC can be estimated as (D+L)/(D+L+X), simple algebra gives X!|1PFC!1|!D!L!.

Figure 2 depicts the steps in creating a “pseudopopulation”. “Pseudosubjects” are created by replicating each original subject (those categorized D or L in Figure 1) in the original fatal crash sample; characteristics of each original subject (indexed by the letter i) are retained by the pseudosubjects it generates, except that all of the pseudosubjects are considered survivors. The inverse propensity weight is defined as W_FCi = 1/P_FCi. The pseudopopulation consists of the pseudosubjects (with weight W_FCi -1) appended to the original subjects (with weight 1).

Characteristics of the pseudopopulation resulting from appending pseudosubjects to the NASS-GES fatal crash subsample were compared to the characteristics of the original NASS-GES stratified random sample. This pseudopopulation was analyzed as a one-stage unstratified sample with the weights adjusted as described above. The probability of death for injured subjects in the pseudopopulation was modeled using the command “svy:logit” and the independent variables described above, and compared to the results obtained from the original NASS-GES sample.

Data from 2007 were then downloaded from the FARS internet site (NHTSA, 2008) and variables required for the weighting equations were modified to be as similar as possible to those in NASS-GES. FARS is essentially a census (attempted 100% sample) of all fatal crashes in the United States, so weights and sampling design do not need to be considered. County crash locations identified in FARS were classified by RUCC (HRSA, 2011). Frequencies and regression results (using ordinary logistic regression) were compared to those obtained from the fatal crash subsample that had been created from NASS-GES. FARS cases were then replicated using the inverse-propensity weights that had been derived from the NASS-GES sample: Weights of 1 were assigned to original FARS cases and (W_FCi - 1) to replicated cases. Subjects in this FARS-based pseudopopulation were analyzed using the Stata “svy” commands, as if they had been selected in a single stage with probability P_FCi from a population of injured subjects. The probability of death for injured persons in the pseudopopulation was modeled using the command “svy:logit” and the independent variables described above. Results were compared to those obtained from the original NASS-GES sample.

Finally, the total of the weights applied to the FARS data (pseudosubjects in addition to original subjects) was used to estimate the number of persons who had been injured in each county during 2007. These estimates were compared to the 2007 totals reported for US counties on available state websites, which are provided as an addendum to the report of Goldstein and colleagues (2011). For comparison, weighted estimates were also derived from NASS-GES data and a propensity score that did not include RUCC codes. For each RUCC category, the observed case fatality rate was compared to the case fatality rate that would be predicted from a model that did not include RUCC variables in calculating the propensity score.

NASS-GES for years 2002–2008 contained records on over a million persons, of whom 65% (weighted proportion 82%) did not suffer any injury. There were 354,557 injured subjects, corresponding (after weighting) to an injured population of 18,578,787. These subjects were injured in 227,014 crashes, corresponding to 12,608,943 injury crashes after weighting. In the NASS-GES sample, 17,737 subjects were injured in one of 8,595 crashes in which at least one person died (“fatal crashes”), corresponding (after weighting) to a population of 374,026 injured in one of 193,322 fatal crashes. Thus, the probability that an injured person was in a fatal crash was approximately 374,026/18,578,787 = 2.01%, while the probability that an injury crash was a fatal crash was approximately 193,322/12,608,943 = 1.53%.

Table 1 demonstrates that injured persons in fatal crashes were more likely to be older, male, and either pedestrians, motorcyclists, or unbelted (and sometimes ejected) occupants of vehicles that were travelling at high speeds, rolled over, or suffered severe vehicle damage. Fatal crashes were also more frequent with head-on collisions, after midnight, and in rural areas. The more persons who were injured, the more likely one of them died. The relative importance of these factors, controlling for each of the others, is reflected in the logistic regression equations predicting the probability that a subject would be in a fatal crash (Table 2, “Model 1”).

Table 2 can be used to calculate the probability that a given subject with an injury in the NASS-GES sample would be included in a fatal crash subsample (i.e., the propensity score). For example, a 50-year-old male driver wearing a belt during the daytime, who crashed into a tree in the most rural county at high speed with severe vehicle damage, but injuring nobody else, would have an odds of being in a fatal crash (in this case, with no other person injured, the same as dying himself) equal to 0.000700*1.74*1.70*2.68*2.92*1.91=0.0309. Like any probability, the corresponding propensity score can be obtained as odds/(odds+1), giving 0.03. The weighting factor determining the number of cases in the pseudopopulation would be the nearest whole number to 1/(0.03), namely 33. Therefore the pseudopopulation would contain the original subject from the fatal crash subsample plus 32 replicated cases; whether or not the original subject survived, all the replicated cases (pseudosubjects) would be analyzed as survivors.

An alternative propensity score was also obtained without including county RUCC information (Table 2, “Model 2”). The other coefficients were virtually the same as in Model 1. A potential use for the weights derived from Model 2 will be described below.

Creation of a pseudopopulation from the NASS-GES fatal crash subsample using Model 1 allowed characteristics of the full NASS-GES sample of injured persons to be closely approximated. Logistic regression models predicting mortality demonstrate the biased results obtained from a non-random fatal crash sample, the results after adjustment of the bias using the replication method, and the results obtained from the original population-based sample (Table 3). Odds ratios in Table 3 approximate the risk ratio with respect to the referent category of females aged 15–39, who were belted non-ejected occupants of vehicles that were not traveling at high speed, did not roll over, and did not suffer severe damage, in crashes that did not involve head-on collision, occurred between 6AM and midnight, and were in urban or suburban areas (RUCC 1–3). Without reweighting, the effects of high speed, vehicle rollover, and rural location are not seen, and there is even an apparent “protective effect” for head-on collisions. After reweighting, covariate effects were similar to those seen in the original population-based sample.

Alternative specifications of the sample design did not affect point estimates of regression coefficients (and therefore did not affect weights). However, specification of the number of sampling levels and strata made an important difference in the calculation of standard errors. Standard errors for the completely specified GES sample were most closely approximated by considering the replicated sample as a simple unstratified random sample.

The characteristics of FARS subjects were similar to those in the NASS-GES fatal crash subsample (Table 1). A model predicting mortality using unweighted FARS data gave biased results similar to those obtained from the NASS-GES fatal crash subsample, but a model after replication and weighting gave results similar to those obtained from the original NASS-GES population-based sample (Table 3).

Table 4 demonstrates some other potential uses of weighted FARS data, using data from counties that independently reported their numbers of traffic injuries.(Goldstein et al., 2011) Predictions using weights derived from Model 1 or Model 2 both underestimated the number of injuries in urban counties. However, predictions from Model 1 were much closer than those from Model 2 for rural counties. Comparison of the observed case fatality rate to the case fatality rate predicted from Model 2 (not including RUCC) demonstrates the increased mortality in rural counties.

Persons injured in fatal crashes are clearly different from those in nonfatal crashes. However, if the most important sampling biases can be identified and quantified, the inverse of the propensity for selection may be used to estimate the characteristics of the original population even when only a non-random sample is available (Haneuse et al., 2009; Kang & Schafer, 2007; Lu, Jin, Chen, & Gluer, 2006). Similar methods have been used to adjust for non-random treatment classification or survey nonresponse (Hernan & Robins, 2006; Rao, Sigurdson, Doody, & Graubard, 2005), and resulting weights have been applied with some success for regression analyses (Howe, Cole, Chmiel, & Munoz; Pan & Schaubel, 2009). Other applications of propensity scores in traffic safety research have been recently described by Sasidharan and Donnell (2012).

The inverse probability weighting method depends upon identifying enough characteristics of the subjects in the population and in the biased subsample so that, after controlling for these characteristics, the selection of the subject into the subsample is approximately random. In our example, when a person with given age, sex, belt use, and crash characteristics is injured, there is a certain probability that someone will die as a result of the crash. Additional randomness is introduced because the person who dies may or may not be the person whose characteristics are part of the model.

A simpler approach to adjusting for sampling bias might stratify by one or more characteristics and give a weight to each of these. However, in order to allow for interactions between different factors, this would require a number of categories rising exponentially with the number of factors and would quickly become cumbersome. In essence, the propensity score method extends the categorical method, because it creates a number of categories corresponding to the number of covariate patterns in the subsample that are possible in the given regression equation.

The methods described here assume that the characteristics of persons with injuries are similar (after controlling for identifiable factors) whether or not they die. For some applications, it might be useful to limit the subpopulation to persons with “incapacitating” or “fatal” injuries. Another alternative might be to estimate the probability that a person with nonfatal injury would be included as a survivor in the fatal crash sample. However, since a large proportion of the injured persons in fatal crashes die, this alternative would ignore much of the information in the fatal crash sample, including all of the information from crashes where the only injured person also dies.

It would be possible to calculate a propensity score for each individual person without regard to their being in the same vehicle or crash with others, although this would ignore the correlations within these clusters. Some method other than the random identification of an “index person” (e.g., multilevel modeling) could also be used to account for person-level effects on the probability of inclusion in a fatal crash sample. However, it would become rather complex to include information on more than one individual person and then to devise an appropriate system of adjusting the weights based upon these results.

When using survey data, such as NASS-GES, it is relatively easy to obtain point estimates using sample weights. However, it is not easy to account properly for the sample design and other adjustments when calculating standard errors, and this is an area of ongoing theoretical research (Gelman, 2007). The general principles of error estimation for complex survey data have been described elsewhere (Li & Levy, 2009; Roberts, Rao, & Kumar, 1987). In practice, considering the pseudopopulation as a one-stage unstratified random sample in this study provided reasonable approximations to the regression standard errors obtained using the original NASS-GES sample and design specifications.

If a database for an entire population is available, then it is obviously unnecessary to analyze a pseudopopulation, and we created a NASS-GES pseudopopulation from its fatal crash subsample only for the purpose of validation. However, if a database for a population involved in non-fatal as well as fatal traffic crashes is not available, then a pseudopopulation based on a fatal crash sample may be the best way to estimate the characteristics of the larger population. For example, if policy makers in some region of the United States wanted to estimate the increased mortality among their citizens attributable to not wearing safety belts, a reasonable estimate could be made using a pseudopopulation based upon FARS data from their own region and weights obtained from Table 2.

Weighting FARS data may have also have value for estimating the number of persons injured due to traffic crashes, as demonstrated in Table 4. The Federal Highway Administration formerly collected nonfatal injury data from the various states, but stopped doing this after 1996 because “these data have been erroneous and can be misleading”(OHIM, 1999). It is understandable that thousands of police jurisdictions may have considerable variability in the identification of crashes and injury severities. However, if it is assumed that fatal crashes are identified similarly throughout the United States, and that NASS-GES is a reasonably accurate representation of the country, then an estimate using the methods described here may actually be more accurate than relying on state reports of varying quality and completeness.

Conversely, if the reported data from a state, county, or other geographic unit are considered reasonably accurate, Table 4 also shows that they can be used to formulate an observed-to-expected ratio for that geographic area, in this case using weights derived from a propensity score that does not include any geographic data. A similar approach could be undertaken to investigate the possibility of residual confounding due to causes other than geographic diversity.

Previous studies using NASS-GES have demonstrated the predictive effect of person-level, event-level, and geographic characteristics on the probability of death given injury(Clark, 2003; NHTSA, 2008). In particular, the increased risk of mortality in rural areas has been demonstrated, even controlling for other determinants of injury severity that might be more common in rural areas. The methods described here provide further evidence of this relationship.

The methods we describe depend upon the assumption that, conditional upon the covariates used in the estimation of propensity, inclusion of injured persons in a fatal crash sample is essentially random. This assumption is more likely to be satisfied if the population for which the propensity is calculated has been in relatively severe crashes, which is why we restricted it to crashes involving personal injury; if the population were restricted only to crashes with incapacitating or fatal injury, the propensity should be even more accurate. However, this might restrict the applicability of the pseudopopulation that would be generated.

The degree to which other outcomes or characteristics of the pseudopopulation resemble those of the original population also depends upon the randomness of the propensity for being in the fatal crash sample. This argues in favor of using as many variables as possible to determine the propensity score. However, there are limitations in the similarity and completeness of variables used both in NASS-GES and in FARS, and probably in any other combination of databases to which these methods could be applied.

Weighting methods can also be unstable if other assumptions are violated, for example when a propensity score is very low (perhaps due to a small number of anomalous cases) and therefore results in an excessive weight or number of replications. One proposed ad hoc approach to this problem would truncate excessive weights or replications. Certainly, any inferential method based on sampling assumptions should be verified by population data whenever possible.

As a general concept, the inverse propensity weighting method simply adds another sampling level to the inverse probability of selection weighting method of NASS-GES, and adjusts the weights accordingly. If the procedures proposed here can be validated using other sources that combine fatal and nonfatal crash data, they may be a useful basis for making inferences from samples that include only fatal crashes, in particular the Fatality Analysis Reporting System.