ChatGPT as a tool for conducting literature review for dry eye disease

Generating Informed Consent Documents Related to Blepharoplasty Using ChatGPT

PURPOSE

This study aimed to demonstrate the performance of the popular artificial intelligence (AI) language model, Chat Generative Pre-trained Transformer (ChatGPT) (OpenAI, San Francisco, CA, U.S.A.), in generating the informed consent (IC) document of blepharoplasty.

METHODS

A total of 2 prompts were provided to ChatGPT to generate IC documents. Four board-certified plastic surgeons and 4 nonmedical staff members evaluated the AI-generated IC documents and the original IC document currently used in the clinical setting. They assessed these documents in terms of accuracy, informativeness, and accessibility.

RESULTS

Among board-certified plastic surgeons, the initial AI-generated IC document scored significantly lower than the original IC document in accuracy ( p < 0.001), informativeness ( p = 0.005), and accessibility ( p = 0.021), while the revised AI-generated IC document scored lower compared with the original document in accuracy ( p = 0.03) and accessibility ( p = 0.021). Among nonmedical staff members, no statistical significance of 2 AI-generated IC documents was observed compared with the original document in terms of accuracy, informativeness, and accessibility.

CONCLUSIONS

The results showed that current ChatGPT cannot be used as a distinct patient education resource. However, it has the potential to make better IC documents when improving the professional terminology. This AI technology will eventually transform ophthalmic plastic surgery healthcare systematics by enhancing patient education and decision-making via IC documents.

Utility of artificial intelligence-based large language models in ophthalmic care

PURPOSE

With the introduction of ChatGPT, artificial intelligence (AI)-based large language models (LLMs) are rapidly becoming popular within the scientific community. They use natural language processing to generate human-like responses to queries. However, the application of LLMs and comparison of the abilities among different LLMs with their human counterparts in ophthalmic care remain under-reported.

RECENT FINDINGS

Hitherto, studies in eye care have demonstrated the utility of ChatGPT in generating patient information, clinical diagnosis and passing ophthalmology question-based examinations, among others. LLMs' performance (median accuracy, %) is influenced by factors such as the iteration, prompts utilised and the domain. Human expert (86%) demonstrated the highest proficiency in disease diagnosis, while ChatGPT-4 outperformed others in ophthalmology examinations (75.9%), symptom triaging (98%) and providing information and answering questions (84.6%). LLMs exhibited superior performance in general ophthalmology but reduced accuracy in ophthalmic subspecialties. Although AI-based LLMs like ChatGPT are deemed more efficient than their human counterparts, these AIs are constrained by their nonspecific and outdated training, no access to current knowledge, generation of plausible-sounding 'fake' responses or hallucinations, inability to process images, lack of critical literature analysis and ethical and copyright issues. A comprehensive evaluation of recently published studies is crucial to deepen understanding of LLMs and the potential of these AI-based LLMs.

SUMMARY

Ophthalmic care professionals should undertake a conservative approach when using AI, as human judgement remains essential for clinical decision-making and monitoring the accuracy of information. This review identified the ophthalmic applications and potential usages which need further exploration. With the advancement of LLMs, setting standards for benchmarking and promoting best practices is crucial. Potential clinical deployment requires the evaluation of these LLMs to move away from artificial settings, delve into clinical trials and determine their usefulness in the real world.

Leveraging ChatGPT for ophthalmic education: A critical appraisal

In recent years, the advent of artificial intelligence (AI) has transformed many sectors, including medical education. This editorial critically appraises the integration of ChatGPT, a state-of-the-art AI language model, into ophthalmic education, focusing on its potential, limitations, and ethical considerations. The application of ChatGPT in teaching and training ophthalmologists presents an innovative method to offer real-time, customized learning experiences. Through a systematic analysis of both experimental and clinical data, this editorial examines how ChatGPT enhances engagement, understanding, and retention of complex ophthalmological concepts. The study also evaluates the efficacy of ChatGPT in simulating patient interactions and clinical scenarios, which can foster improved diagnostic and interpersonal skills. Despite the promising advantages, concerns regarding reliability, lack of personal touch, and potential biases in the AI-generated content are scrutinized. Ethical considerations concerning data privacy and potential misuse are also explored. The findings underline the need for carefully designed integration, continuous evaluation, and adherence to ethical guidelines to maximize benefits while mitigating risks. By shedding light on these multifaceted aspects, this paper contributes to the ongoing discourse on the incorporation of AI in medical education, offering valuable insights and guidance for educators, practitioners, and policymakers aiming to leverage modern technology for enhancing ophthalmic education.

Review of emerging trends and projection of future developments in large language models research in ophthalmology

BACKGROUND

Large language models (LLMs) are fast emerging as potent tools in healthcare, including ophthalmology. This systematic review offers a twofold contribution: it summarises current trends in ophthalmology-related LLM research and projects future directions for this burgeoning field.

METHODS

We systematically searched across various databases (PubMed, Europe PMC, Scopus and Web of Science) for articles related to LLM use in ophthalmology, published between 1 January 2022 and 31 July 2023. Selected articles were summarised, and categorised by type (editorial, commentary, original research, etc) and their research focus (eg, evaluating ChatGPT's performance in ophthalmology examinations or clinical tasks).

FINDINGS

We identified 32 articles meeting our criteria, published between January and July 2023, with a peak in June (n=12). Most were original research evaluating LLMs' proficiency in clinically related tasks (n=9). Studies demonstrated that ChatGPT-4.0 outperformed its predecessor, ChatGPT-3.5, in ophthalmology exams. Furthermore, ChatGPT excelled in constructing discharge notes (n=2), evaluating diagnoses (n=2) and answering general medical queries (n=6). However, it struggled with generating scientific articles or abstracts (n=3) and answering specific subdomain questions, especially those regarding specific treatment options (n=2). ChatGPT's performance relative to other LLMs (Google's Bard, Microsoft's Bing) varied by study design. Ethical concerns such as data hallucination (n=27), authorship (n=5) and data privacy (n=2) were frequently cited.

INTERPRETATION

While LLMs hold transformative potential for healthcare and ophthalmology, concerns over accountability, accuracy and data security remain. Future research should focus on application programming interface integration, comparative assessments of popular LLMs, their ability to interpret image-based data and the establishment of standardised evaluation frameworks.

Updates in Diagnostic Imaging for Infectious Keratitis: A Review

Infectious keratitis (IK) is among the top five leading causes of blindness globally. Early diagnosis is needed to guide appropriate therapy to avoid complications such as vision impairment and blindness. Slit lamp microscopy and culture of corneal scrapes are key to diagnosing IK. Slit lamp photography was transformed when digital cameras and smartphones were invented. The digital camera or smartphone camera sensor's resolution, the resolution of the slit lamp and the focal length of the smartphone camera system are key to a high-quality slit lamp image. Alternative diagnostic tools include imaging, such as optical coherence tomography (OCT) and in vivo confocal microscopy (IVCM). OCT's advantage is its ability to accurately determine the depth and extent of the corneal ulceration, infiltrates and haze, therefore characterizing the severity and progression of the infection. However, OCT is not a preferred choice in the diagnostic tool package for infectious keratitis. Rather, IVCM is a great aid in the diagnosis of fungal and Acanthamoeba keratitis with overall sensitivities of 66-74% and 80-100% and specificity of 78-100% and 84-100%, respectively. Recently, deep learning (DL) models have been shown to be promising aids for the diagnosis of IK via image recognition. Most of the studies that have developed DL models to diagnose the different types of IK have utilised slit lamp photographs. Some studies have used extremely efficient single convolutional neural network algorithms to train their models, and others used ensemble approaches with variable results. Limitations of DL models include the need for large image datasets to train the models, the difficulty in finding special features of the different types of IK, the imbalance of training models, the lack of image protocols and misclassification bias, which need to be overcome to apply these models into real-world settings. Newer artificial intelligence technology that generates synthetic data, such as generative adversarial networks, may assist in overcoming some of these limitations of CNN models.

Benchmarking large language models' performances for myopia care: a comparative analysis of ChatGPT-3.5, ChatGPT-4.0, and Google Bard

BACKGROUND

Large language models (LLMs) are garnering wide interest due to their human-like and contextually relevant responses. However, LLMs' accuracy across specific medical domains has yet been thoroughly evaluated. Myopia is a frequent topic which patients and parents commonly seek information online. Our study evaluated the performance of three LLMs namely ChatGPT-3.5, ChatGPT-4.0, and Google Bard, in delivering accurate responses to common myopia-related queries.

METHODS

We curated thirty-one commonly asked myopia care-related questions, which were categorised into six domains-pathogenesis, risk factors, clinical presentation, diagnosis, treatment and prevention, and prognosis. Each question was posed to the LLMs, and their responses were independently graded by three consultant-level paediatric ophthalmologists on a three-point accuracy scale (poor, borderline, good). A majority consensus approach was used to determine the final rating for each response. 'Good' rated responses were further evaluated for comprehensiveness on a five-point scale. Conversely, 'poor' rated responses were further prompted for self-correction and then re-evaluated for accuracy.

FINDINGS

ChatGPT-4.0 demonstrated superior accuracy, with 80.6% of responses rated as 'good', compared to 61.3% in ChatGPT-3.5 and 54.8% in Google Bard (Pearson's chi-squared test, all p ≤ 0.009). All three LLM-Chatbots showed high mean comprehensiveness scores (Google Bard: 4.35; ChatGPT-4.0: 4.23; ChatGPT-3.5: 4.11, out of a maximum score of 5). All LLM-Chatbots also demonstrated substantial self-correction capabilities: 66.7% (2 in 3) of ChatGPT-4.0's, 40% (2 in 5) of ChatGPT-3.5's, and 60% (3 in 5) of Google Bard's responses improved after self-correction. The LLM-Chatbots performed consistently across domains, except for 'treatment and prevention'. However, ChatGPT-4.0 still performed superiorly in this domain, receiving 70% 'good' ratings, compared to 40% in ChatGPT-3.5 and 45% in Google Bard (Pearson's chi-squared test, all p ≤ 0.001).

INTERPRETATION

Our findings underscore the potential of LLMs, particularly ChatGPT-4.0, for delivering accurate and comprehensive responses to myopia-related queries. Continuous strategies and evaluations to improve LLMs' accuracy remain crucial.

FUNDING

Dr Yih-Chung Tham was supported by the National Medical Research Council of Singapore (NMRC/MOH/HCSAINV21nov-0001).