Assessing ChatGPT's Responses to Otolaryngology Patient Questions

OBJECTIVE

This study aims to evaluate ChatGPT's performance in addressing real-world otolaryngology patient questions, focusing on accuracy, comprehensiveness, and patient safety, to assess its suitability for integration into healthcare.

METHODS

A cross-sectional study was conducted using patient questions from the public online forum Reddit's r/AskDocs, where medical advice is sought from healthcare professionals. Patient questions were input into ChatGPT (GPT-3.5), and responses were reviewed by 5 board-certified otolaryngologists. The evaluation criteria included difficulty, accuracy, comprehensiveness, and bedside manner/empathy. Statistical analysis explored the relationship between patient question characteristics and ChatGPT response scores. Potentially dangerous responses were also identified.

RESULTS

Patient questions averaged 224.93 words, while ChatGPT responses were longer at 414.93 words. The accuracy scores for ChatGPT responses were 3.76/5, comprehensiveness scores were 3.59/5, and bedside manner/empathy scores were 4.28/5. Longer patient questions did not correlate with higher response ratings. However, longer ChatGPT responses scored higher in bedside manner/empathy. Higher question difficulty correlated with lower comprehensiveness. Five responses were flagged as potentially dangerous.

CONCLUSION

While ChatGPT exhibits promise in addressing otolaryngology patient questions, this study demonstrates its limitations, particularly in accuracy and comprehensiveness. The identification of potentially dangerous responses underscores the need for a cautious approach to AI in medical advice. Responsible integration of AI into healthcare necessitates thorough assessments of model performance and ethical considerations for patient safety.

A New Patient-reported Outcomes Measure for Surgically Treated Epiphora: Tearing Assessment and Rating Scale-12 (TEARS-12)

BACKGROUND

No widely accepted, validated instrument currently exists to measure clinical outcomes in patients who undergo dacryocystorhinostomy (DCR) for treatment of epiphora.

OBJECTIVE

To develop a patient-reported outcome measure applicable to this population.

METHODS

Psychometric evaluations, consultation with experts, and review of the literature informed item generation of a 12-question questionnaire to incorporate the most relevant symptoms experienced by patients with nasolacrimal duct obstruction. This questionnaire, known as the Tearing Assessment and Rating Scale-12 (TEARS-12), was administered to 32 patients before and after intervention, in the form of endoscopic DCR. Statistical analysis was performed to measure internal consistency, responsiveness, and test-retest reliability.

RESULTS

Pre-operative and post-operative TEARS-12 scores (28.2 [standard error (SE) 3.19] vs 11.8 [SE 3.25], respectively, P = 0.001) demonstrated improved patient outcome within 6 weeks following endoscopic DCR. Cronbach's alpha for the questionnaire was 0.90, indicating high overall reliability. Additionally, each question demonstrated internal reliability, with a corrected item-total correlation greater than 0.30. The intraclass correlation between the two pre-operative scores was 0.858 (P < 0.001), indicating high test-retest reliability.

CONCLUSION

TEARS-12 is a statistically valid, easy-to-administer instrument to measure clinical outcomes in patients who undergo endoscopic DCR.

Invasive fungal sinusitis: A comparison of pediatric versus adult cases

BACKGROUND

Invasive fungal sinusitis (IFS) is a rare infection with high mortality, mainly impacting immunocompromised patients. Given its significant mortality, timely recognition and treatment is crucial. This study aims to highlight the differences in presentation of IFS between pediatric and adult patients to aid in prompt diagnosis and treatment of this condition.

METHODS

A comprehensive literature search of PubMed, EMBASE, Web of Science, Global Index Medicus, Global Health (EBSCO) and Cochrane Database of Systematic Reviews was conducted to identify articles relating to IFS. Patient demographics, comorbidities, presentation, disease characteristics, treatments and outcomes were extracted from the studies, and statistical analyses were conducted to compare these variables between pediatric and adult patients.

RESULTS

111 studies identified 22 pediatric and 132 adult patients worldwide. Children were more likely to have hematologic malignancies compared to adults (59.1 % vs. 15.2 %, p < 0.001). Facial symptoms such as pain, edema, and numbness were the most common symptoms for both age groups. In the pediatric population, fever and nasal or oral mucosal lesions were more common presenting symptoms (both p < 0.001). Pediatric patients were more likely to present without disease extension beyond the sinuses (p < 0.001). There was no significant difference in either medication treatment or mortality between the two cohorts.

CONCLUSION

IFS often presents with non-specific symptoms and a unique presentation in pediatric and adult populations. Clinical awareness of the varying presentations in both populations is important to treat in a timely manner given the rapid progression and high mortality rates of IFS.

Evaluation of narrow-band imaging in the diagnosis of sinonasal inverted papilloma

KEY POINTS

Narrow-band imaging (NBI) can be used to differentiate benign sinonasal lesions NBI can be used in the preoperative identification of sinonasal inverted papilloma Future studies can focus on NBI for recurrent inverted papilloma and surgical margin guidance.

Endoscopic sinus surgery for chronic rhinosinusitis: 22-item Sino-Nasal Outcome Test 5-year results

BACKGROUND

Currently, limited data exist as to long-term disease-specific outcomes after endoscopic sinus surgery (ESS) for chronic rhinosinusitis (CRS). In this report, we present outcomes data collected over 5 years in a large prospective cohort of patients who underwent surgical treatment for CRS.

METHODS

Patients who completed the 22-item Sino-Nasal Outcome Test (SNOT-22) before ESS were asked to complete the same survey at yearly intervals for 5 years thereafter. The impact of patients' demographics and clinical characteristics on clinical outcome was measured using mixed effects modeling for univariate and multivariable analysis. Multivariable logistic regression threshold analyses were also performed.

RESULTS

Among the 925 patients enrolled in the study, the SNOT-22 was completed after ESS by 641 patients (69.3%) at 1 year and 338 patients (36.5%) at 5 years. The mean preoperative SNOT-22 score of 48.5 (standard error [SE], 0.69) improved at all subsequent yearly time-points, including 5 years (mean, 25.8; SE, 1.08; p < 0.0001). SNOT-22 scores over time were impacted by nasal polyps (p < 0.001), environmental allergies (p = 0.010), diabetes (p = 0.032), and preoperative Lund-Mackay score (p < 0.001).

CONCLUSION

Endoscopic sinus surgery appears to be effective at improving the quality of life of patients with CRS for both the short and long term. The successful outcome of such surgery, however, is influenced by a variety of clinical factors, which should be taken into account by clinicians who are considering surgical treatment of CRS for select patients.

Acute and Chronic Sinusitis

Based on a review of the most current medical literature, this article outlines the basic concepts and classifications of rhinosinusitis, and delineates best practices for clinical diagnoses and the most up-to-date management strategies. Learning to recognize and differentiate these conditions helps facilitate appropriate and timely diagnoses as well as helping practitioners provide their patients with better counseling and care.