1
|
Wang H, Masselos K, Tong J, Connor HRM, Scully J, Zhang S, Rafla D, Posarelli M, Tan JCK, Agar A, Kalloniatis M, Phu J. ChatGPT for addressing patient-centred frequently asked questions in glaucoma clinical practice. Ophthalmol Glaucoma 2024:S2589-4196(24)00183-2. [PMID: 39424063 DOI: 10.1016/j.ogla.2024.10.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/16/2024] [Revised: 09/17/2024] [Accepted: 10/11/2024] [Indexed: 10/21/2024]
Abstract
PURPOSE Large language models such as ChatGPT-3.5 are often used by the public to answer questions related to daily life, including health advice. This study evaluated the responses of ChatGPT-3.5 in answering patient-centred frequently asked questions (FAQs) relevant in glaucoma clinical practice. DESIGN Prospective cross-sectional survey. METHODS Twelve experts across a range of clinical, education and research practices in optometry and ophthalmology. Over 200 patient-centric FAQs from authoritative professional society, hospital and advocacy websites were distilled and filtered into 40 questions across four themes: definition and risk factors, diagnosis and testing, lifestyle and other accompanying conditions, and treatment and follow-up. The questions were individually input into ChatGPT-3.5 to generate responses. The responses were graded by the twelve experts individually. MAIN OUTCOME MEASURES A 5-point Likert scale (1 = strongly disagree; 5 = strongly agree) was used to grade ChatGPT-3.5 responses across four domains: coherency, factuality, comprehensiveness, and safety. RESULTS Across all themes and domains, median scores were all 4 ("agree"). Comprehensiveness had the lowest scores across domains (mean 3.7±0.9), followed by factuality (mean 3.9±0.9), and coherency and safety (mean 4.1±0.8 for both). Examination of the individual 40 questions showed that 8 (20%), 17 (42.5%), 24 (60%) and 8 (20%) of the questions had average scores below 4 (i.e. below "agree") for the coherency, factuality, comprehensiveness and safety domains, respectively. Free-text comments by the experts highlighted omissions of facts and comprehensiveness (e.g. secondary glaucoma) and remarked on the vagueness of some responses (i.e. that the response did not account for individual patient circumstances). CONCLUSIONS ChatGPT-3.5 responses to FAQs in glaucoma were generally agreeable in terms of coherency, factuality, comprehensiveness, and safety. However, areas of weakness were identified, precluding recommendations for routine use to provide patients with tailored counselling in glaucoma, especially with respect to development of glaucoma and its management.
Collapse
Affiliation(s)
- Henrietta Wang
- School of Optometry and Vision Science, University of New South Wales, Kensington, New South Wales, Australia; Centre for Eye Health, University of New South Wales, Kensington, New South Wales, Australia
| | - Katherine Masselos
- Centre for Eye Health, University of New South Wales, Kensington, New South Wales, Australia; Department of Ophthalmology, Prince of Wales Hospital, Randwick, New South Wales, Australia
| | - Janelle Tong
- School of Optometry and Vision Science, University of New South Wales, Kensington, New South Wales, Australia; Centre for Eye Health, University of New South Wales, Kensington, New South Wales, Australia
| | - Heather R M Connor
- School of Medicine (Optometry), Deakin University, Waurn Ponds, Victoria, Australia; The Royal Victorian Eye and Ear Hospital, East Melbourne, Victoria, Australia
| | - Janelle Scully
- Australian College of Optometry, Carlton, Victoria, Australia
| | - Sophia Zhang
- School of Optometry and Vision Science, University of New South Wales, Kensington, New South Wales, Australia; Centre for Eye Health, University of New South Wales, Kensington, New South Wales, Australia
| | - Daniel Rafla
- School of Optometry and Vision Science, University of New South Wales, Kensington, New South Wales, Australia
| | - Matteo Posarelli
- Department of Ophthalmology, Liverpool University Hospitals, Liverpool, UK
| | - Jeremy C K Tan
- Department of Ophthalmology, Prince of Wales Hospital, Randwick, New South Wales, Australia; Faculty of Medicine and Health, University of New South Wales, Kensington, NSW, Australia
| | - Ashish Agar
- Centre for Eye Health, University of New South Wales, Kensington, New South Wales, Australia; Department of Ophthalmology, Prince of Wales Hospital, Randwick, New South Wales, Australia; Faculty of Medicine and Health, University of New South Wales, Kensington, NSW, Australia
| | - Michael Kalloniatis
- School of Optometry and Vision Science, University of New South Wales, Kensington, New South Wales, Australia; School of Medicine (Optometry), Deakin University, Waurn Ponds, Victoria, Australia; University of Houston College of Optometry, Houston, Texas, USA
| | - Jack Phu
- School of Optometry and Vision Science, University of New South Wales, Kensington, New South Wales, Australia; Centre for Eye Health, University of New South Wales, Kensington, New South Wales, Australia; Faculty of Medicine and Health, University of Sydney, Sydney, NSW; Concord Clinical School, Concord Repatriation General Hospital, Concord, NSW.
| |
Collapse
|
2
|
Elson MJ, Giangiacomo A, Maa AY, Branson SV, Maika E, Lin A, Gill K, Machuk RWA, Behn D, Kanjee R, Dotchin SA, Strungaru H, Lee T, Ramstead C, Gan K. Early Experience With Full-scope Shared-care Teleglaucoma in Canada. J Glaucoma 2022; 31:79-83. [PMID: 34172632 DOI: 10.1097/ijg.0000000000001905] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2020] [Accepted: 06/04/2021] [Indexed: 12/29/2022]
Abstract
PRCIS Large Canadian full-scope, shared-care teleglaucoma facilitates efficient management and diversion of medically stable patients away from overburdened subspecialty clinics while allowing patients the convenience of shorter travel, shorter wait time, and continuity with one provider. This report shares Care1 protocol, early patient characteristics, and quality data. PURPOSE This paper describes early experience with Care1, a large full-scope, shared-care teleglaucoma program. Optometrists located in high-demand locations saw patients in-person, acquired clinical history, performed a physical examination, organized diagnostic testing, then uploaded data to a proprietary online platform where they were able to collaborate with participating ophthalmologists to make plans for patient care. MATERIALS AND METHODS The Care1 database was queried for all patients with a diagnosis of glaucoma or glaucoma suspect seen between February 2016 and March 2017. Clinical characteristics like diagnosis, ocular medication history, best-corrected visual acuity, intraocular pressure, cup-to-disc ratios, optical coherence tomography imaging results, and central corneal thickness were collected. Quality metrics studied included rates of referral to an in-person ophthalmologist and consistency of cup-to-disc assessments between in-person optometrists and remote ophthalmologists. RESULTS A total of 4070 patients received care at a Care1 teleophthalmology site in 2 provinces for glaucoma assessment from February 2016 to March 2017. The population was 55.1% female, and the average age was 57.8 years. Overall, 97.3% of patients had a best-corrected visual acuity between 20/20 and 20/40 and 3.3% had an intraocular pressure >26. An in-person consultation with an ophthalmologist was recommended for 1.9% of patients. CONCLUSION Early experience with this full-scope, shared-care teleglaucoma program in Canada indicates it is a convenient, collaborative model of care for glaucoma suspects, and medically stable glaucoma patients.
Collapse
Affiliation(s)
- Molly J Elson
- Department of Ophthalmology, Emory University School of Medicine
| | - Annette Giangiacomo
- VISN 7, Regional Telehealth Services, Atlanta Veterans Affairs Medical Center, Atlanta, GA
- Medical College of Wisconsin, Milwaukee, WI
| | - A Y Maa
- Department of Ophthalmology, Emory University School of Medicine
- VISN 7, Regional Telehealth Services, Atlanta Veterans Affairs Medical Center, Atlanta, GA
| | - Sara V Branson
- Department of Ophthalmology, Emory University School of Medicine
| | | | | | - Kulbir Gill
- Faculty of Medicine, University of British Columbia, Vancouver, BC
| | | | - Darren Behn
- Faculty of Medicine and Dentistry, University of Alberta, Edmonton
| | - Raageen Kanjee
- Max Rady College of Medicine, University of Manitoba, Winnipeg, MB
| | | | | | - Thomas Lee
- Faculty of Medicine, University of Ottawa, Ottawa, ON, Canada
| | | | - Kenman Gan
- Care1 Telemedicine
- Faculty of Medicine, University of British Columbia, Vancouver, BC
- Faculty of Medicine and Dentistry, University of Alberta, Edmonton
| |
Collapse
|
3
|
Choudhari NS, Mundhe G, Khanna R, Rathi V, Garudadri CS. Toward Better Health Outcomes in Rural and Under-served Areas: L. V. Prasad Eye Institute’s Diagonal Model of Glaucoma Care. Ophthalmic Epidemiol 2019; 26:420-429. [DOI: 10.1080/09286586.2019.1646292] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/26/2022]
Affiliation(s)
- Nikhil S. Choudhari
- VST Glaucoma Center, Dr. Kallam Anji Reddy campus, L. V. Prasad Eye Institute, Hyderabad, India
- Bhosle Gopal Rao Patil Eye Center, L. V. Prasad Eye Institute, Nirmal, Telangana, India
| | - Gajendra Mundhe
- Bhosle Gopal Rao Patil Eye Center, L. V. Prasad Eye Institute, Nirmal, Telangana, India
| | - Rohit Khanna
- Gullapalli Pratibha Rao International Center for Advancement of Rural Eye Care (GPR-ICARE), L. V. Prasad Eye Institute, Hyderabad, India
| | - Varsha Rathi
- Gullapalli Pratibha Rao International Center for Advancement of Rural Eye Care (GPR-ICARE), L. V. Prasad Eye Institute, Hyderabad, India
| | | |
Collapse
|
4
|
Braithwaite T, Calvert M, Gray A, Pesudovs K, Denniston AK. The use of patient-reported outcome research in modern ophthalmology: impact on clinical trials and routine clinical practice. PATIENT-RELATED OUTCOME MEASURES 2019; 10:9-24. [PMID: 30774489 PMCID: PMC6352858 DOI: 10.2147/prom.s162802] [Citation(s) in RCA: 80] [Impact Index Per Article: 16.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 12/31/2022]
Abstract
This review article considers the rising demand for patient-reported outcome measures (PROMs) in modern ophthalmic research and clinical practice. We review what PROMs are, how they are developed and chosen for use, and how their quality can be critically appraised. We outline the progress made to develop PROMs in each clinical subspecialty. We highlight recent examples of the use of PROMs as secondary outcome measures in randomized controlled clinical trials and consider the impact they have had. With increasing interest in using PROMs as primary outcome measures, particularly where interventions have been found to be of equivalent efficacy by traditional outcome metrics, we highlight the importance of instrument precision in permitting smaller sample sizes to be recruited. Our review finds that while there has been considerable progress in PROM development, particularly in cataract, glaucoma, medical retina, and low vision, there is a paucity of useful tools for less common ophthalmic conditions. Development and validation of item banks, administered using computer adaptive testing, has been proposed as a solution to overcome many of the traditional limitations of PROMs, but further work will be needed to examine their acceptability to patients, clinicians, and investigators.
Collapse
Affiliation(s)
- Tasanee Braithwaite
- Centre for Patient Reported Outcomes Research and NIHR Birmingham Biomedical Research Centre, University of Birmingham, Edgbaston, Birmingham, UK, .,Moorfields Eye Hospital, London, UK,
| | - Melanie Calvert
- Centre for Patient Reported Outcomes Research and NIHR Birmingham Biomedical Research Centre, University of Birmingham, Edgbaston, Birmingham, UK, .,Institute of Applied Health Research, University of Birmingham, Edgbaston, Birmingham, UK
| | - Alastair Gray
- Health Economics Research Centre, Nuffield Department of Population Health, University of Oxford, Oxford, UK
| | | | - Alastair K Denniston
- Centre for Patient Reported Outcomes Research and NIHR Birmingham Biomedical Research Centre, University of Birmingham, Edgbaston, Birmingham, UK, .,Department of Ophthalmology, Queen Elizabeth Hospital Birmingham, University Hospitals Birmingham NHSFT, Birmingham, UK.,Institute of Inflammation and Ageing, University of Birmingham, Birmingham, UK.,NIHR Biomedical Research Centre (Moorfields Eye Hospital/UCL), London, UK
| |
Collapse
|