Evaluation and Comparison of Ophthalmic Scientific Abstracts and References by Current Artificial Intelligence Chatbots

Importance

Language-learning model-based artificial intelligence (AI) chatbots are growing in popularity and have significant implications for both patient education and academia. Drawbacks of using AI chatbots in generating scientific abstracts and reference lists, including inaccurate content coming from hallucinations (ie, AI-generated output that deviates from its training data), have not been fully explored.

Objective

To evaluate and compare the quality of ophthalmic scientific abstracts and references generated by earlier and updated versions of a popular AI chatbot.

Design, Setting, and Participants

This cross-sectional comparative study used 2 versions of an AI chatbot to generate scientific abstracts and 10 references for clinical research questions across 7 ophthalmology subspecialties. The abstracts were graded by 2 authors using modified DISCERN criteria and performance evaluation scores.

Main Outcome and Measures

Scores for the chatbot-generated abstracts were compared using the t test. Abstracts were also evaluated by 2 AI output detectors. A hallucination rate for unverifiable references generated by the earlier and updated versions of the chatbot was calculated and compared.

Results

The mean modified AI-DISCERN scores for the chatbot-generated abstracts were 35.9 and 38.1 (maximum of 50) for the earlier and updated versions, respectively (P = .30). Using the 2 AI output detectors, the mean fake scores (with a score of 100% meaning generated by AI) for the earlier and updated chatbot-generated abstracts were 65.4% and 10.8%, respectively (P = .01), for one detector and were 69.5% and 42.7% (P = .17) for the second detector. The mean hallucination rates for nonverifiable references generated by the earlier and updated versions were 33% and 29% (P = .74).

Conclusions and Relevance

Both versions of the chatbot generated average-quality abstracts. There was a high hallucination rate of generating fake references, and caution should be used when using these AI resources for health education or academic purposes.

Implementation of a pilot teleretinal screening protocol for hydroxychloroquine retinopathy in a Los Angeles County safety net clinic

INTRODUCTION

This study aimed to determine whether teleretinal screening for hydroxychloroquine retinopathy (HCQR) improves clinical efficiency and adherence to recommended screening guidelines compared to face-to-face screening among patients in a large safety net medical system.

METHODS

In this retrospective cohort study of a consecutive sample of 590 adult patients with active HCQ prescriptions seen in the outpatient ophthalmology clinic at Los Angeles County + University of Southern California Medical Center from 1 September 2018 to 25 November 2019, 203 patients underwent technician-only tele-HCQR screening (THRS), and 387 patients underwent screening with traditional face-to-face visits (F2FV) with an eye-care provider. Data on clinic efficiency measures (appointment wait time and encounter duration) and adherence to recommended screening guidelines were collected and compared between the two cohorts.

RESULTS

Compared to F2FV, the THRS cohort experienced significantly shorter median (interquartile range) time to appointment (2.5 (1.5-4.6) vs. 5.1 (2.9-8.4) months; p < 0.0001), shorter median encounter duration (1 (0.8-1.4) vs. 3.7 (2.5-5.2) hours; p < 0.0001) and higher proportion of complete baseline screening (102/104 (98.1%) vs. 68/141 (48.2%); p < 0.001) and complete chronic screening (98/99 (99%) vs. 144/246 (58.5%); p < 0.001).

DISCUSSION

A pilot THRS protocol was successfully implemented at a major safety net eye clinic in Los Angeles County, resulting in a 50.9% reduction in wait times for screening, 72.9% reduction in encounter duration and 49.9% and 40.5% increases in proportions of complete baseline and chronic screening, respectively. Tele-HCQ retinal screening protocols may improve timeliness to care and screening adherence for HCQR in the safety net setting.

Representation of Race and Ethnicity in Randomized Clinical Trials of Diabetic Macular Edema and Retinal Vein Occlusion Compared to 2010 US Census Data

Importance

Diverse enrollment and adequate representation of racial and ethnic minority groups in randomized clinical trials (RCTs) are valuable to ensure external validity and applicability of results.

Objective

To compare the distribution of race and ethnicity in RCTs of diabetic macular edema (DME) and macular edema from retinal vein occlusion (RVO) to that of US Census data.

Design, Setting, and Participants

This was a cross-sectional retrospective analysis comparing racial and ethnic demographic characteristics of US-based RCTs of DME and RVO between 2004 and 2020 with 2010 US Census data. PubMed and ClinicalTrials.gov were searched to screen for completed phase 3 RCTs with published results. Of 169 trials screened, 146 were excluded because they were incomplete, did not report race and ethnicity, or were not based in the US, and 23 trials were included (15 DME and 8 RVO). The number and percentage of American Indian or Alaska Native, Asian, Black, Hispanic, Native Hawaiian or Other Pacific Islander, and White participants was recorded in each RCT. The demographic distribution and proportion was compared to the reported distribution and proportion in the 2010 US Census using the χ2 test.

Main Outcomes and Measures

Overrepresentation, underrepresentation, or representation commensurate with 2010 US Census data in the racial and ethnic populations of RCTs of retinal vascular disease.

Results

In 23 included RCTs of DME and RVO, there were a total of 38 participants (0.4%) who identified as American Indian or Alaska Native and Native Hawaiian or Other Pacific Islander (groups combined owing to small numbers), 415 Asian participants (4.4%), 904 Black participants (9.6%), 954 Hispanic participants (10.1%), and 7613 White participants (80.4%). By comparison, the 2010 US Census data indicated that 1.1% of the US population self-reported as American Indian or Alaska Native and Native Hawaiian or Other Pacific Islander (groups combined for comparison in this study), 4.8% self-reported as Asian, 12.6% as Black or African American, 16.3% as Hispanic, and 63.7% as White. American Indian or Alaska Native and Hawaiian or Other Pacific Islander participants were underrepresented in 2 trials, neither overrepresented nor underrepresented in 20, and not overrepresented in any of the included trials. Asian participants were underrepresented in 10 trials, overrepresented in 4, and neither overrepresented nor underrepresented in 8. Black participants were underrepresented in 9 trials, overrepresented in 2, and neither overrepresented nor underrepresented in 11. Hispanic participants were underrepresented in 15 trials, overrepresented in 2, and neither overrepresented nor underrepresented in 5. White participants were underrepresented in 2 trials, overrepresented in 14, and neither overrepresented nor underrepresented in 7. The χ2 values comparing RCT demographic distribution to US 2010 Census data were significantly different in 22 of 23 included RCTs.

Conclusions and Relevance

The findings in this study indicated a discrepancy between racial and ethnic demographic data in RCTs of DME and RVO and the US population according to the 2010 Census. White study participants were most frequently overrepresented, and Hispanic study participants were most frequently underrepresented. These findings support the need for more efforts to recruit underrepresented racial and ethnic minorities to improve external validity in trial findings.

Performance of a Machine-Learning Computational Image Analysis Algorithm in Retinal Fluid Quantification for Patients With Diabetic Macular Edema and Retinal Vein Occlusions

BACKGROUND AND OBJECTIVE

The objective is to validate an automated artificial intelligence model in detecting and quantifying fluid in diabetic macular edema (DME) and retinal vein occlusion (RVO) optical coherence tomography images.

PATIENTS AND METHODS

DME (n = 100) and RVO (n = 100) images of adult patients were reviewed. The performance of machine-learning (ML) computational image analysis algorithm was evaluated against consensus manual grading. Main outcomes were accuracy and sensitivity for detection and Pearson's correlation coefficients for quantification.

RESULTS

The ML algorithm had a high accuracy and sensitivity in both DME (intraretinal fluid [IRF]: 0.92, 0.97; subretinal fluid [SRF]: 0.93, 1.00) and RVO (IRF: 0.94, 0.99; SRF: 0.93, 1.00). It had moderate-high correlation in quantifying fluid in DME (total retinal fluid: 0.88; IRF: 0.88; SRF: 0.97) and RVO (total retinal fluid: 0.83; IRF: 0.76; SRF: 0.64).

CONCLUSION

The ML algorithm is highly accurate and sensitive in detecting fluid in DME and RVO optical coherence tomography images and effectively quantifies IRF and SRF in both disease states, particularly in images with low to moderate fluid burden. [Ophthalmic Surg Lasers Imaging Retina. 2022;53:123-131.].

Screenplays and Screenwriting as an Innovative Teaching Tool in Medical Ethics Education

Innovation in ethics pedagogy has continued to evolve and incorporate other forms of storytelling aimed at improving student engagement and learning. The use of bioethics narratives in feature-length films, medical television shows, or short clips in the classroom has a well-established history. In parallel, screenplays present an opportunity for an active approach to ethical engagement. We argue that screenplays and screenwriting provide a rich supplement to current medical ethics teaching and serve as a strong form of reflective learning.

Macro- and microscale variables regulate stent haemodynamics, fibrin deposition and thrombomodulin expression

Drug eluting stents are associated with late stent thrombosis (LST), delayed healing and prolonged exposure of stent struts to blood flow. Using macroscale disturbed and undisturbed fluid flow waveforms, we numerically and experimentally determined the effects of microscale model strut geometries upon the generation of prothrombotic conditions that are mediated by flow perturbations. Rectangular cross-sectional stent strut geometries of varying heights and corresponding streamlined versions were studied in the presence of disturbed and undisturbed bulk fluid flow. Numerical simulations and particle flow visualization experiments demonstrated that the interaction of bulk fluid flow and stent struts regulated the generation, size and dynamics of the peristrut flow recirculation zones. In the absence of endothelial cells, deposition of thrombin-generated fibrin occurred primarily in the recirculation zones. When endothelium was present, peristrut expression of anticoagulant thrombomodulin (TM) was dependent on strut height and geometry. Thinner and streamlined strut geometries reduced peristrut flow recirculation zones decreasing prothrombotic fibrin deposition and increasing endothelial anticoagulant TM expression. The studies define physical and functional consequences of macro- and microscale variables that relate to thrombogenicity associated with the most current stent designs, and particularly to LST.