We identified 25 US-based and international OHNS journals and downloaded all publications from these journals between November 2022 and July 2024. This list included the top 10 most cited OHNS journals, as identified by Google Scholar, and 15 additional prominent subspecialty journals.²⁰ We included both fully Open Access journals and Hybrid Access journals which publish both Open Access and Subscription only papers. The author guidelines for all 25 journals were reviewed to determine whether the use of generative AI is formally permitted. Impact factors for each journal were also collected in September 2024. See Table 1 for a list of journals and their publication information.

We then reviewed all publications to exclude Letters to the Editor, Corrections, Comments, Replies, and Consensus Statements. All other publication types, including Original Investigations, Review Articles, Case Reports, Perspectives, etc. were included. Then, using ChatGPT-4, we developed a Python program in Visual Studio Code (Version 1.90.0 (Universal)) to scan the downloaded articles and generate a Microsoft Excel spreadsheet (Microsoft Office 2024, Microsoft, Redmond, United States) reporting the frequency of the 100 adjectives disproportionately used by LLMs in each publication.¹⁹ Then, 5% of articles from each journal were randomly selected and manually verified to ensure the script accurately calculated word counts. Furthermore, articles in which a certain adjective was particularly overexpressed (>10 occurrences) were excluded, as it was determined that this was an indication that the adjective was part of the study’s main themes. We manually recorded the country of origin, and the highest degree obtained by the first author for each publication included in the study. After data collection, we excluded articles from countries where ChatGPT is not available, as indicated by the OpenAI website (see Supplementary Table 1).

We used the Mann-Kendall test and logistic regression to assess the trend in the use of GPT-associated adjectives over the study period. The month of the journal issue served as a proxy for publication date; articles from journal issues spanning multiple months (e.g., “March/April 2024”) were excluded from the time analysis. Mean word count per article pre-March 2023 and post-March 2023 were analyzed using a paired t-test. Logistic regression was employed to examine the association between journal impact factor, as of September 2024, and the frequency of GPT-associated adjective use. For the country analysis, a Mann-Whitney U Test was conducted to compare differences between English-speaking and non-English-speaking countries, while ANOVA analyzed differences among articles from various geographic regions. Analysis of Covariance (ANCOVA) testing was used to adjust for possible confounders. In the author degree analysis, an unpaired t-test was performed to evaluate differences between articles written by first authors with a doctoral degree and those without.

Applying our primary inclusion and exclusion criteria, we analyzed 25 OHNS journals, identifying 7,134 original articles published between March 2023 and July 2024 (Table 1). After excluding articles from countries where ChatGPT is unavailable, 6,325 articles met the criteria for inclusion in our study. Of these, the majority (66.9%) were published in general ENT journals, defined as those covering two or more subspecialties. The most frequently represented subspecialty journals were Otology (n = 781; 12.3%), Head and Neck (n = 364; 5.8%), Pediatrics (n = 329; 5.20%), Laryngology (n = 307; 4.85%), Rhinology (n = 184; 2.90%), and Facial Plastic and Reconstructive Surgery (n = 125; 1.98%).

Additionally, using the same 25 journals, we collected 1,617 articles published between November 2022 and February 2023. After excluding articles from countries where ChatGPT is unavailable, 1,377 articles from this period met inclusion criteria. Similarly, a majority were published between general ENT journals (n = 810; 58.8%), followed by Otology (n = 220; 16.0%), Pediatrics (n = 102; 7.40%), Rhinology (n = 87; 6.32%), Head and Neck (n = 80; 5.81%), Laryngology (n = 61: 4.42%), and Facial Plastic and Reconstructive Surgery (n = 17; 1.23%).

Impact factors, updated as of September 2024, were obtained from the Observatory of International Research (OOIR).²¹ Of the 25 journals, 18 (72%) had documented impact factors and were included in our analysis. We observed a weak but statistically significant negative correlation between impact factor and GPT-associated adjective use (correlation coefficient = −0.15; p < 0.001). In comparisons between fully Open Access journals and Hybrid Access or Subscription-based journals, no significant differences in GPT-associated adjective use were identified.

Articles that were written prior to March 2023 were included to establish a baseline for adjective usage prior to the public release of ChatGPT-4. When comparing articles pre-March 2023 to post-March 2023, there was significant difference in mean adjective usage per article (5.62 versus 6.01; p = 0.0031).

Articles post-March 2023 were initially categorized into three groups based on their publication dates: the first half of 2023 (March to June), the second half of 2023 (July to December), and the first half of 2024 (January to July). A significant difference was observed in the frequency of GPT-associated adjectives across these three periods (5.56, 5.66, and 6.09 words per article, respectively; p < 0.001).

Articles were then sorted by publication date, using journal issue month as a proxy. For this analysis, articles from journal issues covering multiple months (e.g., “March/April 2024”) were excluded. The Mann-Kendall test identified a significant upward trend in GPT-generated adjective use over time (p < 0.001). An Ordinary Least Squares (OLS) regression analysis also demonstrated a significant increase in GPT-associated adjective use over time (p = 0.0141; Figure 1).

Of the 6,325 articles published after March 2023 that met inclusion criteria, 3,526 (55.7%) had first authors from countries where English is the primary language, while 2,799 (44.3%) had first authors from countries where English is not the primary language. Across individual journals and in the overall dataset, authors from English-speaking countries used a significantly higher number of GPT-associated adjectives compared to their counterparts from non-English-speaking countries (median: 4.0 vs. 3.0, p < 0.001). This relationship remained significant when controlling for journal impact factor and first author education level (p < 0.001). Detailed data on GPT-associated adjective use, stratified by the authors’ country affiliations, is provided in Table 2.

Additionally, articles were categorized based on the geographic region of the country of authorship: Africa, Asia, Europe, Middle East, North America, South America, Oceania, and Multiple. North America was the most represented region with 3,074 articles (48.62%), followed by Europe with 1,458 articles (23.06%) and Asia with 1,244 articles (19.67%). Comprehensive data stratified by geographic region is presented in Table 3. A one-way ANOVA test revealed a statistically significant effect of geographic region on GPT-associated adjective use (F-value: 17.543, p = 0.000). To further investigate this effect, a Tukey’s HSD post-hoc test was conducted. Oceania was found to have a significantly higher GPT-associated adjective use compared to both Africa (p = 0.0087) and North America (p = 0.0036). Conversely, Asia exhibited consistently lower GPT-associated adjective use compared to Europe (p < 0.001), the Middle East (p < 0.001), North America (p < 0.001), and Oceania (p < 0.001). When controlling for impact factor, geographic region remained a significant predictor of GPT-associated adjective use overall, but the individual region differences were not significant.

Of the 6,325 articles published post-March 2023 written in countries where ChatGPT is available, only 3,057 articles (48.3%) included information on the educational degree(s) attained by the first author. Degrees were classified into three levels based on the highest degree attained, doctoral level degree (76.48%), bachelor’s level degree (19.79%), and master’s level degree (3.73%). Articles authored by individuals with doctoral degrees used slightly fewer GPT-associated adjectives compared to those with non-doctoral degrees (5.02 words per article vs. 5.30), though this difference was not statistically significant (p = 0.13). An ANOVA test indicated a statistically significant difference among the degree groups (F-statistic: 6.407, p = 0.0017). Post-hoc analysis using Tukey’s HSD test identified significant differences between bachelor’s and master’s level authors (5.07 vs. 6.51, respectively) and between the doctoral and master’s level authors (5.02 vs. 6.51, respectively). No significant difference was observed between the bachelor’s and doctoral level degree groups.

While our study employs a validated model to evaluate AI-generated text at scale, this model only considers adjective usage and does not account for other factors. AI-generated text often has more subtle features such as sentence structure, syntax, punctuation usage, and overall text fluency.

We included articles published from November 2022 through July 2024. However, many articles are conceptualized and written before their print publication. The inability to precisely determine when an article was written represents another limitation of our study.

In analyzing whether the highest degree attained by the first author is significantly associated with GPT-generated text, we categorized author degrees into two groups: doctoral versus non-doctoral degrees. This binary classification simplifies the hierarchy seen particularly in US medicine, where different levels of training, such as a resident physician, fellow, and attending physician, all hold the same M.D. degree.

It should also be noted that this study only focused on ChatGPT-4, which was released in March 2023. OpenAI released ChatGPT-4o in May 2024, which replaced GPT-4 as the default model in July 2024. Our results may not apply to this newer model as we do not have data indicating the most commonly used adjectives with this newer version.