A Multimodal Approach For Deep-Learning Classification of Vocal Fold Pathologies in Stroboscopy.

Details

• Alternative Title:
  The Laryngoscope, June 2026: A Multimodal Approach For Deep-Learning Classification of Vocal Fold Pathologies in Stroboscopy.
• Journal Article:
  The Laryngoscope
• Personal Author:
  Surapaneni, Sruthi ; Kutler, Rachel B. ; Setzen, Sean A. ; Kim, Yeo Eun ; Yao, Peter ; Siddiqui, Sana H. ; Pitman, Michael J. ; Sulica, Lucian ; Elemento, Olivier ; Khosravi, Pegah ; Rameau, Anaïs
• Description:
  To develop and validate a multimodal deep-learning classifier trained on stroboscopic image, voice, and clinicodemographic data, differentiating between three different vocal fold (VF) states: healthy (HVF), unilateral paralysis (UVFP), and VF lesions, including benign and malignant pathologies.
  
  Patients with UVFP (n = 54), VF lesions (n = 42), and HVF (n = 41) were retrospectively identified. Image frames and voice samples were extracted from stroboscopic videos. Clinicodemographic variables were collected from the electronic health record. Patient-level data was independently divided into training (80%) and testing (20%). Visual features were extracted using a transformer DINOv2 and acoustic features were extracted using Librosa. All three feature modalities were combined using a custom multilayer perceptron. Unimodality models using only image or only voice data were trained for comparison. Accuracy and F1 scores were used to validate the models.
  
  On a hold-out test set, the multimodal classifier demonstrated stronger performance (76.9% accuracy) compared to the image classifier (61.5% accuracy) and audio classifier (65.4% accuracy). On an external dataset, the multimodal classifier accuracy dropped to 45%, though still an improvement compared to accuracies of 42% and 31% for the video-only and audio-only modalities, respectively.
  
  In this proof-of-concept study, we successfully developed a multimodal dataset and classifier for VF pathology, demonstrating the potential of combining stroboscopic frames, voice and text data. The multimodal classifier achieved higher accuracy than the image-only model and audio-only models. Future models should validate these findings on larger datasets.
• Content Notes:
  Author manuscript
• Subjects:
  • Humans
  • Deep Learning
  • Vocal Cords
  • Stroboscopy
  • Retrospective Studies
  • Male
  • Female
  • Middle Aged
  • Adult
  • Aged
  More +
• Keywords:
  • artificial intelligence
  • transformer
  • multimodal
• Source:
  The Laryngoscope, June 2026
• DOI:
  https://doi.org/10.1002/lary.70355
• ISSN:
  0023-852X ; 1531-4995
• Pubmed ID:
  41489089
• Pubmed Central ID:
  PMC12980015
• Document Type:
  Text
• Funding:
  K76 AG079040/AG/NIA NIH HHSUnited States/ ; OT2 OD032720/CD/ODCDC CDC HHSUnited States/ ; K76 AG079040/AG/NIA NIH HHSUnited States/ ; OT2 OD032720/CD/ODCDC CDC HHSUnited States/
• Genre:
  Journal Article
• Pages in Document:
  14 pdf pages
• Download URL:
  https://stacks.cdc.gov/view/cdc/260868/cdc_260868_DS1.pdf
• File Type:
  [PDF - 282.81 KB]
• Collection(s):
  CDC Public Access
• Main Document Checksum:
  urn:sha-512:04c1deeafef6902e161108edfe7fc00cfecdd300a08c6e80319d83e32fefcc3e6655cf240931fee2bac57de6fd4c984304a2d8c424c7a99c319813f07433c6b3