Near-Miss Narratives from the Fire Service: A Bayesian Analysis

Details

• Personal Author:
  Lehto M ; Smith GS ; Taylor JA
• Description:
  Background: In occupational safety research, narrative text analysis has been combined with coded surveillance data to improve identification and understanding of injuries and their circumstances. Injury data give information about incidence and the direct cause of an injury, while near-miss data enable the identification of various hazards within an organization or industry. Further, near-miss data provide an opportunity for surveillance and risk reduction. The National Firefighter Near-Miss Reporting System (NFFNMRS) is a voluntary reporting system that collects narrative text data on near-miss and injurious events within the fire and emergency services industry. In recent research, autocoding techniques using Bayesian models have been used to categorize/code injury narratives with up to 90% accuracy, thereby reducing the amount of human effort required to manually code large datasets. Autocoding techniques have not yet been applied to near-miss narrative data. Furthermore, the data collected in the Contributing Factors (CF) and Loss Potential (LP) fields of the NFFNMRS have not been analyzed. We sought to examine the utility of these quantitative variables, particularly in relation to injuries and near-misses. Methods: We manually assigned mechanism of injury codes to previously un-coded narratives from the NFFNMRS and used this as a training set to develop two Bayesian autocoding models, Fuzzy and Naïve. We calculated sensitivity, specificity, PPV, ROC curves, confusion matrices, and top predictor word lists. We also evaluated the effect of training set size on prediction sensitivity and compared the models' predictive ability as related to injury outcome. We cross-validated a subset of the prediction set for accuracy of the models when coding de novo, and cross-validated the training set to assess variation in the wordlists and probabilities as a function of the training set. To examine CFs in relation to injury outcome, we performed descriptive analyses between the variables and completed a 3-model latent class analysis to identify whether the 21 CFs could be reduced to fewer categories. For LP, we performed descriptive analyses as well. Results: Overall, the Fuzzy model performed better than Naïve, with a sensitivity of 0.74 compared to 0.678. Cross-validation of the prediction set showed sensitivity reached 0.602, where Fuzzy and Naïve had the same prediction. As the number of records in the training set increased, the models performed at a higher sensitivity, suggesting that both the Fuzzy and Naïve models were essentially "learning." Injury records were predicted with greater sensitivity than near-miss records. There was no evidence of a relationship between CFs or LP and injury outcome. Furthermore, the CF categories did not reduce into fewer categories, as the categories showed an overall poor fit to the model. Conclusion: The application of Bayesian autocoding methods can successfully code both near-misses and injuries in longer-than-average narratives with non-specific prompts regarding injury. Such coding allowed for the creation of two new quantitative data elements for injury outcome and injury mechanism. With the CFs and LP as currently captured in the data system, we conclude that these variables and answer selections lack definition and are poorly understood by the individuals reporting to the system and add little value to the system as a result. [Description provided by NIOSH]
• Subjects:
  Accidents, Occupational Firefighters Hazardous Substances Occupational Health Safety
• Keywords:
  Accident Analysis Fire Fighters Hazard Identification Injuries Models Statistical Analysis Surveillance
• Series:
  Grant Final Reports
• Publisher:
  National Institute for Occupational Safety and Health
• Document Type:
  Text
• Funding:
  Grant
• Genre:
  Final Grant Report
• Place as Subject:
  Indiana ; Maryland ; OSHA Region 3 ; OSHA Region 5 ; Pennsylvania
• CIO:
  National Institute for Occupational Safety and Health (NIOSH)
• Division:
  OEP - Office of Extramural Programs
• Topic:
  Workplace Safety & Health
• Location:
  North America
• Pages in Document:
  1-39
• NIOSHTIC Number:
  nn:20056927
• NTIS Accession Number:
  PB2019-101420
• Citation:
  Atlanta, GA: U.S. Department of Health and Human Services, Public Health Service, Centers for Disease Control and Prevention, National Institute for Occupational Safety and Health, R03-OH-009984, 2013 Sep; :1-39
• Contact Point Address:
  Jennifer A. Taylor, PhD, MPH, Drexel University School of Public Health, 1505 Race St. MS 1034, Philadelphia, PA 19102-1192
• Email:
  Jat65@drexel.edu
• Federal Fiscal Year:
  2013
• NORA Priority Area:
  Public Safety
• Performing Organization:
  Drexel University, Philadelphia, Pennsylvania
• Peer Reviewed:
  False
• Start Date:
  20110901
• Source Full Name:
  National Institute for Occupational Safety and Health
• End Date:
  20130831
• Collection(s):
  National Institute for Occupational Safety and Health
• Main Document Checksum:
  urn:sha-512:84720bf3856a0b1599def5c39972ed7fc16f5185532e53c8ee4d4511cd4c7b4715639cbbcf7c33416a202a999153c9d26d13ce5804bc075de642a422f8e58bf0
• Download URL:
  https://stacks.cdc.gov/view/cdc/218918/cdc_218918_DS1.pdf
• File Type:
  [PDF - 928.83 KB ]