In this research, we developed a lane change statistical model and demonstrated its use for accelerated evaluation of a frontal collision avoidance algorithm. The data used is from the Safety Pilot Model Deployment database [44]. The SPMD program aims to demonstrate connected vehicle technologies in a real-world environment. It recorded naturalistic driving of 2842 equipped vehicles in Ann Arbor, Michigan for more than two years. As of April 2015, 34.9 million miles were logged, making SPMD one of the largest public N-FOT databases ever.

As shown in Fig. 3, a lane change was detected and recorded by an SPMD vehicle when the Lane Change Vehicle (LCV) crosses the lane markers. In the SPMD program, 98 sedans are equipped with Data Acquisition System and MobilEye [45], which provides: a) relative position to the lane change vehicle (range), and b) lane tracking measures pertaining to the lane delineation both from the painted boundary lines and road edge characteristics. The error of range measurement is around 10% at 90 m and 5% at 45 m [46].

To ensure consistency of the used dataset, the following criteria were applied: •υ(tLC)∈(2m/s,40m/s) •υL(tLC)∈(2m/s,40m/s) (1)•RL(tLC)∈(0.1m,75m) where t_LC is the time when the center line of the LCV crosses the lane markers; υ_L and υ are the velocities of the LCV and the SPMD vehicle; R_L is the range defined as the distance between the rear edge of the LCV and the front edge of the SPMD vehicle. 403 581 lane changes were detected in total. Fig. 4 shows the locations of the identified lane changes.

A lane change can be divided into three phases: the decision to initiate a lane change, gap (range) acceptance, and lane change execution [42]. In this research, we focus on the effects of gap acceptance, which is mainly captured by three variables: υ_L(t_LC), R_L(t_LC) and Time To Collision (TTC) of AVs, defined as (2)TTCL=−RLṘL where Ṙ_L is the derivative of R_L. In the following, unless mentioned specifically, υ_L, R_L and TTC_L are the variables at t_LC.

The distribution of υ_L is shown in Fig. 5. The division of highways and local roads is embodied by the bimodal shape of the histogram. υ_L is assumed to remain constant during the lane change. Only the events with a negative range rate are used to build the lane change model. Out of 403 581 lane change events, 173 692 are with negative range rate.

Larger R_L and TTC_L indicate the scenario is safer which are the majority cases in naturalistic driving, while Smaller R_L and TTC_L indicate the scenario is less safe and rarer. Therefore, we define the variables of interest as reciprocal of R_L and TTC_L to put the rare events in the tail of the distribution to naturally fit the naturalistic driving statistics. To capture the influence of vehicle speed on range and TTC, we divided lane change events into low, medium. and high velocity conditions. Fig. 6 shows that υ_L has little influence on the distribution of RL−1. We use a standard Matlab package [47] to search for a proper distribution to fit RL−1, which examines 17 different types of distributions and examine goodness-of-fit by using Bayesian Information Criterion [48]. Fig. 7 illustrates the fitting of RL−1 using a Pareto distribution defined as (3)fRL−1(x|kRL−1,σRL−1,θRL−1)=1σRL−1(1+kRL−1x−θRL−1sRL−1)−1−1/kRL−1 where the shape parameter kRL−1, the scale parameter σRL−1, and the threshold parameter θRL−1 are all positive. Note that, due to the physical limitations mentioned in (1), the Pareto distribution in (3) is in fact truncated at 1/0.1 m⁻¹ and 1/75 m⁻¹. For the sake of conciseness, we show the untruncated version throughout this paper. The same holds for all other fitted distributions in this paper.

The histograms of TTCL−1 for different velocity intervals are shown in Fig. 8. As the vehicle speed increases, the mean of TTCL−1 decreases. Based on the analysis using MATLAB fitting package [47], TTCL−1 can be approximated by both Pareto distribution and exponential distribution with 0.23% relative difference in BIC. We used the exponential distribution (4)fTTCL−1(x|λTTCL−1)=1λTTCL−1e−x/λTTCL−1 for simplicity, where the scaling factor λTTCL−1 varies with the speed of the LCV. Here we define λTTCL−1 as the mean value instead of the rate of the exponential distribution, because mean value has more intuitive physical meaning.

The dependence of λTTCL−1 on vehicle speed is shown in Fig. 9. As the vehicle speed increases, λTTCL−1 decreases. The blue circles represent λTTCL−1 at the center points of υ_L intervals. We use linear interpolation and extrapolation to create smooth λTTCL−1 for all vehicle speeds.

The effect of range on TTC is very limited, as can be seen in Fig. 10. This indicates that R_L and TTC_L can be modeled independently given the same υ_L. Ṙ_L can then be calculated from (5) (5)ṘL=−TTCL−1RL−1. Finally, the velocity of the host vehicle υ can be calculated from (6)υ=υL−ṘL.

In summary, the lane change events are generated in the following order: a) generate υ_L based on the empirical distributions shown in Fig. 5; b) generate RL−1 using Fig. 7; c) generate TTCL−1 using the Exponential distribution with parameters shown in Fig. 9; and finally d) calculate υ using (5) and (6).

Monte Carlo method [49] typically aims to generate unbiased statistical samples to estimate the expected value of a stochastic process. Let Ω be the sample space for all possible events, and ℇ ⊂ Ω be the rare events of interest, e.g., the occurrence of a crash. Let x be a random vector describing the motions of the lane change vehicle. The indicator function of the event ℇ is defined as (7)Iℰ(x)={1,if x∈ℰ0,otherwise.

Our goal is to estimate the probability of ℇ happening, i.e., (8)γ=P(ℰ)=E(Iℰ(x)). The CMC approach generates independent and identically distributed samples x₁, x₂, …, x_n of x, and then calculate the sample average (9)γ^n=1n∑i=0nIℰ(xi).

We state some statistical properties of CMC. First, under mild conditions, the Strong Law of Large Numbers [49] holds, i.e., (10)P(limn→8γ^n=γ)=1.

Moreover, the Central Limit Theorem [49] implies that, when n is large, γ̂_n follows approximately the normal distribution 𝒩(E(γ̂_n), σ²(γ̂_n)) with the mean (11)E(γ^n)=E(1n∑i=0nIℰ(xi))=γ and variance (12)σ2(γ^n)=Var(γ^n)=Var(1n∑i=0nIℰ(xi))=1n2∑i=0nVar(Iℰ(xi))=γ(1−γ)n.

The accuracy of the estimation is represented by the relative half-width, which is the half-width of the confidence interval relative to the probability to be estimated. With the Confidence Level at 100 (1 − α)%, the relative half-width of γ̂_n is defined as (13)lr=lαγ where l_α is the half-width given by (14)lα=zασ(γ^n) and z_α is defined as (15)za=Φ−1(1−α/2) where Φ⁻¹ is the inverse cumulative distribution function of 𝒩(0,1). To ensure l_r is smaller than a constant β, we need (16)lαγ=zασ(γ^n)γ=zαγγ(1−γ)n=za1−γγn≤β. In other words (17)n≥zα2β2·1−γγ.

Equation (17) reveals that when ℇ is rare, i.e., γ0, the required test number n goes to infinity. This means that a huge test number is required to maintain a satisfactory half-width relative to the magnitude of a rare event probability γ. This is the reason why CMC is slow.

IS is a so-called variance reduction technique that is effective in handling rare events. IS has been successfully applied to evaluate critical events in reliability [50], finance [51], insurance [52], and telecommunication networks [53]. General overviews about IS can be found in [54]–[56].

To explain the concept of IS, we denote f(x) as the original joint density function of the random vector x. The core idea of IS is to replace f(x) with a new density f*(x) that has a higher likelihood for the rare events to happen. Using a different distribution, however, leads to biased samples, and the key of IS is to provide a mechanism to compensate for this bias and compute correct crash rate at the end.

We describe this mechanism as follows. First, we define the likelihood ratio L (Radon-Nikodym derivative [57]) as (18)L(x)=f(x)f*(x).

The probability of E satisfies (19)P(ℰ)=Ef(Iℰ(x))=∫Iℰ(x)f(x)dx=∫[Iℰ(x)L(x)]f*(x)dx=Ef*(Iℰ(x)L(x)). One required condition for (19) to hold is that f*(x) must be absolutely continuous with respect to f(x) within ℇ, i.e., (20)∀x∈ℰ:f*(x)=0⇒f(x)=0 which guarantees the validity of L in (18). The IS sample is I_ℇ(x_i)L(x_i) where x_i is generated under f*(x), which is an unbiased estimator for γ. The overall IS estimator for test number n is then (21)γ^n=1n∑i=0nIℰ(xi)L(xi). Note that although a continuous model is used in this paper, similar approaches can be applied to the discrete model as well.

Now consider the relative half-width of CI constructed by IS (22)lr*=lαγ=(zασs(γ^n)γ=za(Ef*(γ^n2)−Ef*2(γ^n)γn=zαEf*(Iℰ2(x)L2(x))−γ2γn=zαnEf*(Iℰ2(x)L2(x))γ2−1≤β. The required minimum test number is then (23)n≥zα2β2(Ef*(Iℰ2(x)L2(x))γ2−1).

When f*(x) is properly chosen, Ef*(Iℰ2(x)L2(x)) can be close to γ², resulting in a smaller number of tests (i.e., the evaluation is accelerated).

When a slower lane changing vehicle cut in front of the AV, the events of interest are defined as (24)ℰ={min(RL(t))<Rℰ|tLC<t≤tLC+TLC} where T_LC represents duration of the lane change event; R_ℇ is the critical range. Equation (24) means that if the minimum range is smaller than R_ℇ anytime during the lane change event, this lane change belongs to the ℇ set.

The random vector x consists of three random variables [υL,TTCL−1,RL−1]. υ_L is generated using the empirical distributions shown in Fig. 5. The IS approach considers the modified probability density functions of TTCL−1 and RL−1 denoted by fTTCL−1*(x) and fRL−1*(x). The likelihood ratio is then (25)L(RL−1=x,TTCL−1=y)=fRL−1(x)fTTCL−1(y)fRL−1*(x)fTTCL−1*(y). From (19), the probability of ℇ can be estimated as (26)P(ℰ)=Ef(Iℰ(x))=Ef*(Iℰ(x)L(x)).

The only task left is then to construct proper fTTCL−1*(x) and fRL−1*(x) to accelerate the evaluation procedure.

The choice of IS distribution is critical to the success of the IS method. The Cross Entropy (CE) method, first proposed by Rubinstein [58], is an iterative search procedure to find good IS distribution within a prescribed parametric family.

To understand how CE works, we first point out an important observation: the theoretical optimal IS distribution is always the conditional distribution given that the rare event of interest happens, namely (27)fzv*(x)={f(x)γ,Iℰ(x)=10,Iℰ(x)=0.

With fzv*(x), any sampled x leads to a rare event so that the indicator function I_ℇ (x) constantly equals to one. The likelihood ratio (28)Lzv(x)=f(x)fzv*(x)=γ. The probability of the rare events is calculated by (29)γ^n=1n∑i=0NIℰ(xn)L(xn)=1n∑i=0nγ=γ.

In other words, γ̂_n equals to γ for all n. The distribution fzv*(x) is optimal in the sense that any sample generated from it has zero variance, and hence the required test number to construct confidence level to any precision is 1; thus it is also known as the zero variance IS distribution [56]. Unfortunately, this distribution cannot be implemented directly because it requires the knowledge of γ, which is exactly what we want to estimate. However, it provides a benchmark to get good IS distributions: A good IS distribution should be close to the zero-variance distribution.

To describe how CE works, we define the Kullback-Leibler (KL) divergence (30)fKL(f˜ϑ(x),fzv*(x))=∫log [fzυ*(x)f˜ϑ(x)]fzv*(x)dx as a measure of the difference between f̃_ϑ (x) and fzv*(x). The idea of CE is to find an IS distribution over the family of distributions f̃_ϑ(x) (controlled by ϑ) that has the minimum KL divergence with fzv*(x), i.e., (31)ϑ*=arg min ϑfKL(f˜ϑ(x),fzv*(x)). Substituting (30) into (31), we have (32)ϑ*=arg minϑ∫log [fzv*(x))f˜ϑ(x)]fzv*(x)dx=arg minϑ∫{log[fzv*(x)]fzv*(x)−log[f˜ϑ(x)]fzv*(x)}dx. Note the first term inside the integration is independent of ϑ (32) can be simplified to (33)ϑ*=arg maxϑ∫log[f˜ϑ(x)]fzv*(x)dx. Substituting (27) into (33), we have (34)ϑ*=arg maxϑ∫log[f˜ϑ(x)]f(x)P(ℰ)Iℰ(x)dx=arg maxϑ∫log [f˜ϑ(x)]f(x)Iℰ(x)dx. The CE method is an iterative scheme to sequentially improve the IS distribution and optimize ϑ* using (33). At the ith iteration, we use f̃_ϑi(x) as the IS distribution to run the Monte Carlo. Then, letting L̃_ϑi(x) = f(x)/ (f̃_ϑi (x), from (34), ϑ_i+1 can be derived as (35)ϑi+1=arg maxϑ∫log[f˜ϑ(x)]L˜ϑiIℰ(x)f˜ϑi(x)dx≈arg maxϑÊf˜ϑi[log (f˜ϑ(x))L˜ϑi(x)Iℰ(x)] where I_ℇ(x) are samples in the previous iteration and Ê_f̃ϑi [·] denotes the empirical average.

There are many possible choices for the family of f̃_ϑ(x). Here we use a popular class named the Exponential Change of Measure (ECM) for TTCL−1.

Recall that TTCL−1~exp(λTTCL−1(υL)). ECM considers the family (36)f˜TTCL−1(x)=exp(ϑTTCL−1ECMx−Ψ(ϑTTCL−1ECM))fTTCL−1(x) parametrized by ϑTTCL−1ECM, where Ψ(ϑTTCL−1) is the logarithmic moment generation function of TTCL−1, i.e., (37)Ψ(ϑTTCL−1ECM)=log E(exp(ϑTTCL−1ECMTTCL−1)). It can be derived that (38)f˜TTCL−1(x)=(1λTTCL−1−ϑTTCL−1ECM) exp (−(1λTTCL−1−ϑTTCL−1ECM)x) where ϑTTCL−1ECM<1/λTTCL−1 and λTTCL−1>0. To make ϑTTCL−1ECM have the same scale as λTTCL−1, we apply a nonlinear mapping by letting (39)ϑTTCL−1ECM=ϑTTCL−1ϑTTCL−1λTTCL−1−λTTCL−12 with ϑTTCL−1<λTTCL−1. Substitute (39) into (38), we have (40)f˜TTCL−1(x|ϑTTCL−1)=(1λTTCL−1−λTTCL−1) exp (xλTTCL−1−ϑTTCL−1). RL−1 follows a (truncated) Pareto distribution, i.e., (41)fRL−1(x)=Pareto (x|kRL−1,σRL−1,θRL−1). We apply an ECM of the exponential distribution as our family of IS distributions, where we first construct an exponential distribution (42)f˜RL−1(x)=1λRL−1exp (−1λRL−1x) with λRL−1, which gives (42) the smallest least square error to (41). With similar procedure, we have (43)f˜RL−1(x|ϑRL−1)=(1λRL−1−ϑRL−1) exp (−x(λRL−1−ϑRL−1)). Using this approximate ECM instead of an ECM applied to a truncated Pareto reduces the computation complexity in the optimization step since a closed form can be obtained in each Cross Entropy iteration.

The overall likelihood ratio is (44)L˜(RL−1=x,TTCL−1=y)=f(x)f˜ϑi(x)=fRL−1(x)fTTCL−1(y)f˜RL−1(x)f˜TTCL−1(y). For low-velocity conditions, i.e., υL∈(5m/s,15m/s) we simulate N tests with initial condition x=[υL,TTCL−1,RL−1] where υ_L follows the low velocity portion (5 m/s~15 m/s) of the empirical distribution shown in Fig. 5. RL−1 and TTCL−1 follows f˜RL−1(x) and f˜TTCL−1(x). Apply (38) and (43) to (35). The optimal parameter ϑRL−1 and ϑTTCL−1 can be derived analytically (45)ϑRL−1=∑j=1NL˜(xj)Iℰ(xj)(λRL−1−RL,j−1)∑j=1NL˜(xj)Iℰ(xj) (46)ϑTTCL−1=∑j=1NL˜(xj)Iℰ(xj)(λTTCL−1−TTCL−1)∑1NL˜(xj)Iℰ(xj) where j is the index for each simulation. The newly obtained ϑRL−1 and ϑTTCL−1 can be used in the next iteration.

The same procedure can be used to obtain optimal parameters in medium and high-velocity conditions.

The AV is designed to be equipped with both Adaptive Cruise Control (ACC) [59] and Autonomous Emergency Braking (AEB). When the driving is perceived to be safe (TTC_L ≥ TTC_AEB), it is controlled by the ACC. The ACC is approximated by a discrete Proportional-Integral (PI) controller [59] to achieve a desired time headway THWdACC. Use the time headway error tHWErr as the controller input (47)tHWErr=tHW−THWdACC where t_HW is the current time headway, defined as (48)tHW=RLυ.

The discrete PI controller can be described in the discrete-time domain as (49)Ad(z)THWErr(z)=KpACC+KiACCTs2z+1z−1 where A_d and THWErr are the Z transformation of the command acceleration a_d and tHWErr; T_s is the sampling time; gains KpACC and KiACC are calculated using the Matlab Control Toolbox using the following requirements: a) Loop bandwidth = 10 rad/s, and b) Phase margin = 60 degree. The control power of ACC system is saturated to a constant acceleration aACCMax, i.e., |ad|≤aACCMax. To implement the PI controller in the time domain, taking the inverse Z transformation of (49), we get (50)ad(k+1)=ad(k)+KpACC(tHWErr(k)−tHWErr(k−1))+KiACC(tHWErr(k)+tHWErr(k−1))Ts/2.

The AEB model was extracted from a 2011 Volvo V60, based on a test conducted by ADAC (Allgemeiner Deutscher Automobil-Club e.V.) [60] (see Fig. 11). It is analyzed using test track data, owner’s manuals, European New Car Assessment Program (Euro NCAP) information, and videos during vehicle operation [61]. The AEB algorithm becomes active when TTC_L < TTC_AEB, where TTC_AEB depends on the vehicle speed as shown in Fig. 12. Once triggered, AEB aims to achieve acceleration a_AEB. In [61] a_AEB was assumed to be −10 m/s² on high friction roads. The build-up of deceleration is subject to a rate limit r_AEB as shown in Fig. 13. It should be noted that the AEB modeled here is an approximation but not necessarily a good representation of the actual AEB system on production vehicles.

A first order lag with a time constant τ_AV is used to model the transfer function from the commanded acceleration to the actual acceleration for simplicity. The proposed accelerated evaluation process can be applied on other vehicle models such as CarSim [62], if more accurate simulations are desired.

The simulation parameters are listed in Table I.

Three kinds of events were analyzed in this study:

Conflict.

A conflict event happens when an AV appears in the proximity zone of the LCV between time t_LC and t_LC + T_LC. As shown in Fig. 14, the proximity zone is the area in the adjacent lane from 4 feet in front of the bumper of the LCV to 30 feet behind the rear bumper of the LCV [42, p. ix]. This area generally includes the blind spot and the area beside and behind the vehicle in which another vehicle is likely to travel.

The Cross Entropy is used to find optimal ϑTTCL−1 and ϑRL−1. The values of ϑTTCL−1 and ϑRL−1 in the tenth iteration are used in the simulations to estimate the probability of conflicts (conflict rate) in a lane change scenario. 100 lane changes are simulated in each iteration. As shown in Fig. 15, three sets of ϑTTCL−1 and ϑRL−1 are obtained with low, medium and high velocities. All ϑRL−1 converge to about −0.12, whereas values of ϑTTCL−1 float around zero. As the conflict events are defined based on R_L, R_L has a direct impact on the occurrence of the event. Therefore f̃_ϑ (x) is largely affected by fRL−1(x), and ϑTTCL−1 converges to zero (f˜TTCL−1(x)f˜TTCL−1(x)).

Both accelerated evaluation and the non-accelerated simulations (based on CMC) were conducted to demonstrate the performance and credibility of the proposed approach. Fig. 16 shows that the accelerated test is unbiased as the conflict rate converges to the one estimated in the non-accelerated simulation.

The convergence is reached when the relative half-width l_r is below β = 0.2 with 80% confidence. Fig. 17 shows that the accelerated evaluation achieves this confidence level after N_acc = 364 simulations, while the naturalistic simulations take N_nature = 5.90e3 simulations.

In the SPMD database, during 1 325 964 miles naturalistic driving, 173 592 lane changes were identified with negative range rates. The frequency of lane change can be estimated as (51)rlc=1 325 964173 592=7.64[mile/lane change]. The driving distance needed in naturalistic test is thus (52)Dnature=rlc·Nnature. The test distance in accelerated evaluation (53)DACC=∑n=1NACC∫t=tLCtLC+𝒯LCυ(n)(t)dt where υ⁽ⁿ⁾(t) represents the velocity of AV at time t in the n_th test and the termination time (54)𝒯LC=min {min (t|RL(t)<Rℰ),TLC}. The accelerated rate is defined as (55)racc=DnatureDacc. The acceleration is achieved from both the modeling of lane change scenarios and the application of Importance Sampling and Cross Entropy techniques.

A crash happens when the range R_L becomes negative, i.e., R_ℇ = 0 in (24). Similar to the conflict events analysis, another Cross Entropy analysis is conducted to find optimal ϑTTCL−1 and ϑRL−1 for crash events. Because crashes are rarer than the conflict events, 500 lane changes are simulated in each iteration. As shown in Fig. 18, three different values of ϑTTCL−1 were obtained from the iterative search for different velocity intervals, whereas ϑRL−1 converges to values close to zero. It can be explained that in the crash analysis, the safety critical function (AEB) on AV is mainly affected by TTC. Therefore ϑTTCL−1 has a larger impact on the occurrence of the crash. The estimation of the crash rate under accelerated and naturalistic conditions are shown in Fig. 19. The convergence is reached with 80% confidence level and β = 0.2 as shown in Fig. 20.

Injuries are also important indicators of the performance of AVs. Here we focus on injuries with the Maximum Abbreviated Injury Score equal or larger than 2 (MAIS2+), representing moderate-to-fatal injuries. The probability of injury is related to the relative velocity at the crash time t_crash (56)Δυ=−ṘL(tcrash)>0. The probability of moderate-to-fatal injuries for the AV passengers is estimated by a nonlinear model (57)Pinj(Δυ)={11+e−(β0+β1Δυ+β2)crash0no crash which was proposed by Kusano and Gabler [63] shown in Fig.21 with parameters β₀ = −6.068, β₁ = 0.1, and β₂ = −0.6234. The injury rate E(P_inj(Δυ)) is calculated as (58)E(Pinj(Δυ))=Êf*(Pinj(Δυ))≈1n∑i=0NaccPinj(Δυ(xn))L(xn) where L is the likelihood and x_n represents the random variables ([υL,TTCL−1,RL−1]) in the nth simulation. The modified statistics used in crash events (shown in Fig. 18) are used to calculate the injury rate. The estimation results and convergence are shown in Figs. 22 and 23.

The accelerated rates of conflict, crash and injury events are summarized in Table II. The accelerated rates of crashes and injuries are higher than that of conflicts. This is because crashes and injuries occur with much lower probabilities than conflicts. The IS techniques generally have better performance when target events are rarer.