This study used a commercially available IVMS from one vendor, and this IVMS vendor supplied all equipment, coding of videos for observed driving behaviors (Table 1), and training on the use of the technology. For privacy purposes, NIOSH researchers did not code videos. However, the entire database of videos used in the study was accessible both to the industry partners and to the IVMS vendor. The IVMS vendor is not named here so as not to imply endorsement of any one IVMS vendor, as multiple commercial vendors exist to provide similar types of technology and services.

Each IVMS unit included two camera views: a forward-facing exterior camera view and driver-facing interior camera. The IVMS unit captured two types of video events: (a) regular-threshold triggered video events and (b) constant-threshold triggered video events, with the constant-threshold triggered video events being used for the analysis in this study. Regular-threshold triggered video events represent video events that are typically captured within the IVMS vendor's commercial services program, whereas constant-threshold triggered video events were captured for the specific purposes of this study as explained further below. For both types of events, the IVMS unit was configured to capture 30-s of video, audio and other event data (15 s before and 15 s after the trigger). The recorded video events were given a time and date stamp, and the unit of measurement was video events per vehicle per 24-h day.

Both regular-threshold and constant-threshold triggered video events relied on the same set of triggering algorithms that primarily use a three-axial accelerometer. These triggering algorithms detect vehicle maneuvers such as hard braking, acceleration, cornering, swerving, and sudden forces.

Different methods were used to manage the triggering sensitivity levels for the two sets of video events. Sensitivity levels for regular-threshold triggered events were set to initial levels based on the IVMS vendor's guidance and experience with driving events, as has been done in other published research (e.g., Hickman & Hanowski, 2011; McGehee et al., 2007), with the goal of maximizing the capture of safety-related events, and minimizing the capture of videos that were recorded but don't show any risky driving behaviors by the driver. Then, triggering sensitivity levels were periodically changed, if needed, to ensure consistent inflow of video events as drivers learned to “drive softer” and avoid triggering cameras. The changes were consistently applied to all vehicles of a given type to ensure that drivers were always evaluated based on a consistent set of criteria. From the vendor's perspective, this method ensures a consistent sampling rate for driver behaviors in order to avoid overstating the improvement of driver behaviors. For example, if one simply counts the number of video events that involve mobile phone use of a given driver but leaves the triggering sensitivity levels constant for the period of the study, any measured reduction in the number of video events with risky distracted behavior would have two components: (a) the actual reduction in risky behavior (mobile phone usage), and (b) the reduction in the number of triggered video events as the driver learned to “drive softer” (but may have continued the risky driving behavior such as use of the mobile phone). Regular-threshold triggered events were not used as an outcome measure in the current study, they were just used as a constant pool of videos for driver coaching.

In addition to the regular-threshold triggered video events, the IVMS vendor also captured constant-threshold triggered video events explicitly for the purposes of this study and based on a specific request from the study researchers. Triggering sensitivity levels for the constant-threshold triggered video events were set to values more sensitive (meaning it was easier to trigger and record videos) that what the IVMS vendor typically uses for commercial clients (regular-threshold), and were then left unchanged for the duration of the study (such as 0.236 g for hard braking for a medium duty truck). Unlike the regular-threshold triggered video events, the constant-threshold triggered video events were not immediately selected for offload (wirelessly transmitted from the IVMS unit to the central database). Rather, a process was put in place to randomly sample and select for offload one constant-threshold triggered video events per vehicle per day. If a regular-threshold triggered video was selected in the constant-threshold triggered sampling scheme (possible because the constant-threshold triggering sensitivity level was lower than the regular-threshold triggering sensitivity level), the vendor did not have to review the video again. The remainder of the constant-threshold triggered video events (after the one was sampled and downloaded) was not retained for use in the study. Constant-threshold triggered video events (the one randomly selected video event per vehicle per day) were the main unit of analysis used to evaluate the interventions. The constant-threshold triggered video events allowed researchers to evaluate the effectiveness of feedback from the technology and avoid any potential bias, if it existed, due to the method that IVMS vendor typically used to capture the regular-threshold triggered video events for commercial clients. Additionally, the procedure of havinga triggering threshold held constant for the entire duration of a study (as it was for the constant-threshold video events), as opposed to the slightly variable triggering threshold (as it was done for the capture of regular-threshold video events), is more comparable to methods of other previously published work evaluating IVMS technology (e.g., Hickman & Hanowski, 2011; Lisk et al., 2013; McGehee et al., 2007).

The IVMS unit included a small box-like device to provide immediate feedback to the driver with a series of lights that indicated when a risky driving maneuver had been executed; the device was located inside the vehicle cab near the windshield-mounted rearview mirror. The light remained green when ‘safe’ driving was occurring but flashed red or yellow to denote potentially risky driving maneuvers, with the red light indicating more severe risky driving behaviors. The thresholds between colors were determined by algorithms set by the IVMS vendor. Triggered video events (both regular-threshold and constant-threshold triggered videos) were reviewed by the vendor's trained observers for approximately 60 individual risky driving behaviors such as driving unbelted, smoking, hand-held device use while driving, unsafe stopping, and speeding (see Table 1 for a list of all possible coded risky driving behaviors used in the study). An overall severity category from 0 to 4 was then calculated for each video based on the sum of scores assigned to the individual risky driving behaviors. A severity score of 0 indicated the video was reviewed and no risky driving behaviors seen, with 1 through 4 denoting an increasing level of safety concern. The methods of coding individual behaviors seen within a video, and overall severity score assigned to the video, were held constant over the course of the study, such that a video showing a driver unbelted for example would be given the same severity score in all periods of the study. Collisions were not given a severity score by the vendor due to liability concerns, and are not included in this analysis.

The main outcome measure determined a priori in the study protocol was any constant-threshold triggered video that was vendor-scored as a severity level of 3 or 4. These may be a single severe event seen by trained coders in the video, such as aggressive driving, texting on hand-held phone, hands off the wheel, or driving the wrong way, or multiple lesser risky behaviors (such as driving unbelted and moderate speeding) seen together such that the total score of the multiple lesser behaviors elevated it to severity 3 or 4 (hereafter “overall risky driving”). The scoring methods for the behaviors seen in each video were held constant over the course of the study. As learned from earlier published research using IVMS data, many individual risky driving behaviors may be too rare on their own to be used as an outcome measure, such as single instances of running a red light, running a stop sign, driving on wrong side of road, etc. as reported in the findings of McGehee et al. (2007). Therefore the total count of overall risky driving video events was the key outcome of interest in this study, as was similarly done for the earlier research of Hickman and Hanowski (2011). Driving unbelted as an individual behavior was not considered by the IVMS vendor to be a severity 3 or 4 event, but was included as an outcome of interest in this study due to its public health significance (e.g. Cameron, Crandall, Olson, & Sklar, 2001; Chen et al., 2015; Evans, 1986; Huang & Lai, 2011; National Highway Traffic Safety Administration, 1999), as well as its use as an outcome in earlier IVMS research (Levick & Swanson, 2005).

The IVMS vendor provided training to supervisors on all feedback mechanisms in a train-the-trainer format to industry partners prior to the installation of IVMS equipment. The goal of the training was to provide supervisors of drivers with a complete overview of the IVMS system and to equip supervisors with the necessary information to present and describe the IVMS program to their drivers, as well as to access and view videos, and coach their own drivers. The training also helped the supervisors address any drivers' questions and concerns in order to familiarize drivers with the program prior to implementation. The orientation also helped familiarize supervisors with IVMS equipment.

This study evaluated two types of feedback from the IVMS to the driver:

Instant driver feedback (referred to as IDF-only) from the feedback light on the IVMS unit inside the vehicle cab. A green light indicated safe driving, and a flashing yellow or red light informed the driver about risky driving behaviors. There were no sounds associated with the lights. Drivers were trained by supervisors on how to interpret the IDF feedback. Supervisors did not receive any summary information about the IDF feedback that went to the drivers.

In addition to these two main types of feedback that went only to the intervention groups, graphic feedback was given to all sites participating in the study, both intervention and control. The graphic feedback was posted weekly at each site and showed aggregated data on safe driving for all drivers at each site (no individual drivers identified or specific behaviors mentioned). This third form of feedback was used so that all drivers would be receiving at least some meaningful information about their driving performance from the IVMS that were installed on the vehicles. The group feedback came in the form of a vendor-generated graph showing miles driven without a severity 3 or 4 event for all drivers at the site, and if drivers were improving as a whole, the trend line should go up as miles of safe driving increased. These weekly graphs were distributed to supervisors by the IVMS vendor, and were presented to drivers via display in common areas and at staff meetings to establish the general goal of improving driving behavior with positive supervisor support.

This research study protocol was reviewed and approved by the National Institute for Occupational Safety and Health (NIOSH)'s Institutional Review Board. NIOSH had no direct contact with drivers and did not obtain any personally identifying information for this analysis. The study population came from two companies from two industries, the “Support activities for oil and gas operations” and “General freight trucking, local” industrial classifications of the North American Industry Classification System (NAICS) codes (NAICS codes 213,112 and 48,411, respectively) (Office of Management and Budget, 2007). These two companies implemented IVMS technology in a subset of their fleets as a pilot effort. The employees were technicians driving pick-up trucks to oil and gas operations where they would perform maintenance and support activities, and drivers of refrigerated box trucks in the 26,000–33,000 lb range doing short-haul deliveries of goods to convenience stores. The industry partners selected a total of 20 sites (7 from truck transportation and 13 from oil and gas) located in 12 states (CA, CO, LA, MA, MD, NJ, OK, PA, TX, UT, VA, WA) within the United States. All vehicles at each of 20 sites had IVMS installed so that all drivers at each site would have the same equipment and working conditions. The 20 sites were apportioned into intervention and control groups (described in more detail in Section 2.5 Study Design). The study population was a dynamic cohort where the amount of person-time contributing to the study could vary for each individual depending on how long they were employed at the site. Data from any employee at the sites operating a vehicle equipped with IVMS during the course of the study were included (i.e., if an employee terminated employment during the course of the study, a newly hired employee could use the same vehicle equipped with IVMS). All participants in the study were anonymous, and any data analyzed at the driver level (i.e., coaching data) identified drivers by only a unique anonymous number created by the IVMS vendor. No demographic data (such as gender, age, or race) were available for the study population, however it is likely that most of the drivers in this study were male as gender distribution data from the truck transportation and oil and gas industries show that the majority of workers in these two industries are male (Bureau of Labor Statistics, 2015; Chen, Fang, Guo, & Hanowski, 2016; Sieber et al., 2014).

In the oil and gas industry partner, drivers could use the vehicles for both work and personal purposes, and it was not possible to differentiate between events logged during work or personal use time in this study because data were not available for workers' daily on and off work hours. Workers' family members were not permitted to drive the vehicles unless it was an emergency situation. Generally one driver was assigned to one vehicle during the course of the study. In the truck transportation industry partner, multiple drivers at the site could drive a vehicle within the same 24-h period, due to shift changes. For both industry partners, in general, both vehicles and drivers remained at the same site throughout the course of the study.

The study was conducted from April 2012 through July 2014 and entailed a 3-group, 4–period cross-over design for each company (Fig. 1). Groups 1 and 2 were intervention groups, and Group 3 was a control group. Video events were triggered and reviewed in all 3 groups during all 4 periods of the study. In period 1, no groups received any IDF-only or Coaching + IDF feedback, and period 1 was considered the baseline period. In period 2, Group 1 received IDF-only feedback, Group 2 received Coaching + IDF feedback, and Group 3 received no feedback. In period 3, Group 1 received Coaching + IDF feedback, Group 2 received IDF-only feedback, and Group 3, the control group, received no feedback. In period 4, all 3 groups entered an end-baseline period identical to the initial baseline period. The graphic feedback chart data was given to Groups 1, 2 and 3 during all 4 periods of the study, including the beginning and end baseline periods. Supervisors had online access to the triggered video events only for Group 1 and 2, and only during the Coaching + IDF intervention periods.

In the truck transportation company, non-random methods were used to assign the 7 sites to the 3 groups (3 sites in Group 1, 2 sites in Group 2, 2 sites in Group 3). In the oil and gas support operations company, the 13 sites were randomized to the 3 groups (5 sites to Group 1, 5 sites to Group 2, 3 sites to Group 3). Because of the mix of methods used to assign sites to groups in the two companies, the overall study design was considered to be quasi-experimental (Harris et al., 2004; Rothman & Greenland, 1998). All trucks at each of the 20 study sites were equipped with IVMS; a total of 315 IVMS were installed at the start of the study. There were a total of 163 IVMS-equipped trucks in the oil and gas operations company (64 in Group 1, 35 in Group 2, and 64 in Group 3) and 152 IVMS-equipped trucks in the truck transportation company (53 in Group 1, 50 in Group 2, and 49 in Group 3). Collection and review of regular-threshold video events began immediately after vehicles were equipped with IVMS, but drivers drove the vehicles performing regular work duties for at least 10 weeks before collection and review of constant video events began.

Logistic regression was the statistical method used in this study for analyzing dichotomous response data. Each individual constant-threshold video was a unit of analysis. A response of one indicated that the behavior of interest occurred within the constant-threshold video event, and a response of zero indicated that the behavior of interest did not occur within the constant-threshold video event, with factors suspected to affect the response incorporated into the model for analysis of relationships (Hosmer & Lemeshow, 2000; Walker, 1997). In this study, logistic regression was used to model the probability of risky driving behavior (a dichotomous outcome: risky driving behavior present, or risky driving behavior absent) in constant video events and to test for significant differences in the probability of risky driving behavior between all groups of interest; a generalized estimating equation (GEE) approach was used to account for repeated measurements on the same vehicles over time (Stokes, Davis, & Koch, 1995). The analysis was performed using the GENMOD procedure in SAS v. 9.3 (SAS Institute Inc., 2011). The prediction variables were group (3 levels), period (4 levels), and the interaction between group and period (12 levels), and contrasts were constructed using parameter estimates from the model to test specific hypotheses of interest. Data from intervention Groups 1 and 2 were combined to test some of the hypotheses of interest, and are hereafter referred to as the “intervention” group, while Group 3 remains the “control” group. The following were the five questions of interest to be addressed (repeated for both outcome measures, overall risky driving behaviors, and driving unbelted, so 10 overall) in the contrast estimate analysis:

Was there a significant difference in the decline of the probability of risky driving behavior from baseline between the intervention group's (Group 1 and Group 2 combined) Coaching + IDF period and the control group (Group 3)?

Data from both companies were used in aggregate in the analysis because comparing the companies to one another was not an objective of the study, and adding a company variable to the model resulted in only a small decrease in quasi-likelihood under the independence model criterion (Pan, 2001). Due to the multiple hypotheses to be tested (n = 10), contrasts were determined to be significant when p < 0.05, after performing Holm's Bonferroni correction (Aicken & Gensler, 1996).

During the study period, there were a total of 73,099 constant-threshold triggered video events randomly sampled and downloaded for review and inclusion in the study from all 625 drivers in the study (as identified by unique anonymous driver identification numbers). Of those constant-threshold triggered video events (hereafter just “video events”), 1670 (2% of the total) had missing values and could not be used, and an additional 11,711 video events (16% of the total) had an obstructed camera view, either partially or fully obstructing the view of the driver and/or the external camera view. Because the obstructed-view video events could not be coded properly, they were omitted from the analysis, leaving 59,718 (82% of the total) fully visible video events which were used in this analysis. Of the fully visible video events, 55% did not show any risky driving behaviors and the remaining 45% showed some level of driving behavior of safety concern (anything with severity 1 and above).

For the entire study period, frequency counts were done for risky driving behaviors coded Severity 1–4 in triggered video events (Table 2). More than one driving behavior could be coded from each video event (e.g., a driver could be driving unbelted, and eating while driving). Of the 34,899 coded behaviors, driving unbelted was the most commonly seen risky driving behavior, coded 14,185 times, or 40.6% of total video events. This was followed by distractions at 31.9% of total video events, and then by unsafe stopping, speeding, and hand-held mobile device use. Within the subset of all triggered video events that were coded as more severe (severity 3 or 4) by the IVMS vendor (Table 3), the top five most common behaviors were the same as in the overall video events; however, the rank order differed within the top five.

During the Coaching + IDF feedback period, supervisors had the goal of meeting with every driver that had severity 3 or 4 driving video events during that week for a coaching session. From these data, an ever-coached metric was calculated, meaning any driver that had at least one severity 3 or 4 regular-threshold video event at any time during the coaching intervention period was coached at least once during that period. Of the 324 drivers who drove during Coaching + IDF feedback period, 292 drivers triggered at least one severity 3 or 4 regular-threshold video event during that time. Of those, 258 (88%) had a coaching session logged by their supervisor in the vendor's online video response center. Of the 13 intervention sites that received Coaching + IDF intervention, 6 sites showed 100% of their drivers being coached during coaching sessions, 5 sites showed 90–99% of drivers coached, 3 sites showed 80–88% of drivers coached, and 1 site showed only 52% of drivers coached.

The five questions of interest outlined in the statistical analysis section (Section 2.5) were examined using overall risky driving (severity 3 or 4 video events) as an outcome measure. Data were combined as shown in Table 4 and Fig. 2 to test these questions of interest.

The first question of interest was to determine if there was a significant difference in the decline of the probability of risky driving behavior from baseline between the intervention group's (Group 1 and Group 2 combined) Coaching + IDF period and the control group (Group 3). While the control group's odds of risky driving behaviors declined from baseline to the treatment periods, the intervention group had a significantly greater reduction in odds of risky driving behaviors during Coaching + IDF feedback periods in comparison to the control group (ORadj = 0.52 95% CI 0.33–0.82; Holm-adjusted p = 0.032).

The second question of interest was whether there was a significant difference in the decline of the probability of risky driving behavior from baseline between IDF-only in its “pure” form (from Group 1 where IDF-only feedback came first and was not influenced by preceding Coaching + IDF feedback) and the control group (Group 3). The findings showed that IDF-only in its “pure” form did not show a significantly greater reduction in odds of risky driving from baseline than the control group (ORadj = 0.86, 95% CI 0.51–1.43; Holm-adjusted p > 0.05).

The third question was whether there a significant difference in the decline of the probability of risky driving behavior from baseline between intervention group's IDF-only feedback period (Group 1 only) and intervention group's (Group 1 and Group 2 combined) Coaching + IDF period. In comparison to IDF-only feedback, Coaching + IDF showed a significantly larger reduction in odds of risky driving from baseline (ORadj =0.61 95% CI 0.43–0.86; Holm-adjusted p = 0.035).

The fourth question was whether there was a significant difference in the decline of the probability of risky driving behavior from baseline between intervention Group 1 and Group 2 during the treatment periods, where Group 1 had IDF-only followed by Coaching + IDF feedback, and Group 2 had Coaching + IDF followed by IDF-only feedback. To examine the temporal effect of order of presentation of IDF-only feedback, Group 2's average (Coaching + IDF and IDF-only) departure from baseline for periods 2 and 3 was compared to Group 1's average (IDF-only and Coaching + IDF) departure from baseline for periods 2 and 3. It was found that the two were not significantly different from one another (ORadj =1.10, 95% CI 0.66–1.86; Holm-adjusted p > 0.05).

The fifth question was to see if there was a significant difference in the decline of the probability of risky driving behavior from baseline to the end baseline period between the intervention group (Group 1 and Group 2 combined) and control group (Group 3). To examine the sustained effect of prior exposure to Coaching + IDF feedback, the combined intervention group's end baseline decline compared to their beginning baseline was compared to the control group's end decline compared to its beginning baseline. A significant difference between the intervention and control groups was found (ORadj =0.27, 95% CI 0.12–0.60; Holm-adjusted p = 0.012).

The IVMS vendor did not consider driving unbelted to be a severity 3 or 4 event, so while driving unbelted was a component of approximately 20% of the severity 3 or 4 video events (Table 3), it was the most commonly seen risky driving behavior of any severity level at 41% (Table 2). The five questions of interest outlined in the statistical analysis section (Section 2.5) were also examined using driving unbelted as an outcome. Data were combined as shown in Table 4 and Fig. 3 to test specific questions of interest.

The first question of interest was to determine if there was a significant difference in the decline of the probability of risky driving behavior from baseline between the intervention group's (Group 1 and Group 2 combined) Coaching + IDF period and the control group (Group 3). The intervention group had a significant reduction in odds of driving unbelted during Coaching + IDF feedback periods in comparison to the control group, which showed an increase (ORadj = 0.18 95% CI 0.08–0.41; Holm-adjusted p < 0.001).

The second question of interest was whether there was a significant difference in the decline of the probability of driving unbelted from baseline between IDF-only in its “pure” form (from Group 1 where IDF-only feedback came first and was not influenced by preceding Coaching + IDF feedback) and the control group (Group 3). IDF-only in its “pure” form did not show a reduction in odds of driving unbelted from baseline in comparison to the control group (ORadj = 0.66 95% CI 0.30–1.45; Holm-adjusted p > 0.05).

The third question was whether there a significant difference in the decline of the probability of driving unbelted from baseline between intervention group's IDF-only feedback period (Group 1 only) and intervention group's (Group 1 and Group 2) Coaching + IDF period. In comparison to “pure” IDF-only feedback, Coaching + IDF showed a significantly larger reduction in odds of driving unbelted from baseline than the IDF-only group (ORadj = 0.27 95% CI 0.15–0.48; Holm-adjusted p = 0.035).

The fourth question was whether there was a significant difference in the decline of the probability of driving unbelted from baseline between intervention Group 1 and Group 2 during the treatment periods, where Group 1 had IDF-only followed by Coaching + IDF feedback, and Group 2 had Coaching + IDF followed by IDF-only feedback. To examine the temporal effect of order of presentation of IDF-only feedback, Group 2's average (Coaching + IDF and IDF-only) departure from baseline for periods 2 and 3 was compared to the Group 1's average departure from baseline for periods 2 and 3 (IDF-only and Coaching+IDF). It was found that the two were not significantly different from one another (ORadj = 1.24, 95% CI 0.53–2.89; Holm-adjusted p N 0.05).

The fifth question was to see if there was a significant difference in the decline of the probability of driving unbelted from baseline to the end baseline period between the intervention group and control group. To examine the sustained effect of prior exposure to Coaching + IDF feedback, the intervention group's end baseline decline compared to its beginning baseline was compared to the control group's end decline compared to its beginning baseline. No significant difference in the odds of driving unbelted was found between the intervention and control group in this comparison (ORadj = 0.57, 95% CI 0.16–2.00; Holm-adjusted p > 0.05).