Blending Qualitative and Computational Linguistics Methods for Fidelity Assessment: Experience with the Familias Unidas Preventive Intervention

Details

• Alternative Title:
  Adm Policy Ment Health
• Personal Author:
  Gallo, Carlos ; Pantin, Hilda ; Villamar, Juan ; Prado, Guillermo ; Tapia, Maria ; Ogihara, Mitsunori ; Cruden, Gracelyn ; Brown, C Hendricks
• Description:
  Careful fidelity monitoring and feedback are critical to implementing effective interventions. A wide range of procedures exist to assess fidelity; most are derived from observational assessments (Schoenwald and Garland, Psycholog Assess 25:146-156, 2013). However, these fidelity measures are resource intensive for research teams in efficacy/effectiveness trials, and are often unattainable or unmanageable for the host organization to rate when the program is implemented on a large scale. We present a first step towards automated processing of linguistic patterns in fidelity monitoring of a behavioral intervention using an innovative mixed methods approach to fidelity assessment that uses rule-based, computational linguistics to overcome major resource burdens. Data come from an effectiveness trial of the Familias Unidas intervention, an evidence-based, family-centered preventive intervention found to be efficacious in reducing conduct problems, substance use and HIV sexual risk behaviors among Hispanic youth. This computational approach focuses on "joining," which measures the quality of the working alliance of the facilitator with the family. Quantitative assessments of reliability are provided. Kappa scores between a human rater and a machine rater for the new method for measuring joining reached 0.83. Early findings suggest that this approach can reduce the high cost of fidelity measurement and the time delay between fidelity assessment and feedback to facilitators; it also has the potential for improving the quality of intervention fidelity ratings.
• Keywords:
  Adolescent Conduct Disorder Evidence-Based Practice Family Health Female Health Services Research Hispanic Or Latino HIV Infections Humans Linguistics Machine Learning Male Preventive Medicine Process Assessment, Health Care Professional-Family Relations Qualitative Research Randomized Controlled Trials As Topic Reproducibility Of Results Risk-Taking School Health Services Sexual Behavior Statistics As Topic Unsafe Sex Video Recording

  More +
• Source:
  Adm Policy Ment Health. 42(5):574-585
• Pubmed ID:
  24500022
• Pubmed Central ID:
  PMC4125546
• Document Type:
  Journal Article
• Funding:
  P30AI073961/AI/NIAID NIH HHSUnited States/ ; P30DA027828/DA/NIDA NIH HHSUnited States/ ; R01DA025192/DA/NIDA NIH HHSUnited States/ ; U01PS0000671/PS/NCHHSTP CDC HHSUnited States/ ; R01 DA025192/DA/NIDA NIH HHSUnited States/ ; P30 AI073961/AI/NIAID NIH HHSUnited States/ ; U01 PS000671/PS/NCHHSTP CDC HHSUnited States/ ; P30 DA027828/DA/NIDA NIH HHSUnited States/
• Volume:
  42
• Issue:
  5
• Collection(s):
  CDC Public Access
• Main Document Checksum:
  urn:sha256:903466ba08e90e1e3b54fc08955d018141d281fa76c96d08c0c1ea98bdaf2d73
• Download URL:
  https://stacks.cdc.gov/view/cdc/35516/cdc_35516_DS1.pdf
• File Type:
  [PDF - 811.31 KB ]