1
|
Dang TH, Rieu-Werden ML, Kobrin SC, Tiro JA, Werner C, Lykken JM, Chubak J, Atlas SJ, Higashi RT, Lee SC, Haas JS, Skinner CS, Silver MI, Feldman S. Association Between Clinician Confidence and Making Guideline-Recommended Decisions in the Management of Abnormal Cervical Cancer Screening Results. J Gen Intern Med 2024:10.1007/s11606-024-08943-z. [PMID: 39060782 DOI: 10.1007/s11606-024-08943-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/10/2024] [Accepted: 07/05/2024] [Indexed: 07/28/2024]
Abstract
BACKGROUND Guidelines for managing abnormal cervical cancer screening results are complex and adherence is challenging for clinicians. Previous studies have identified gaps in knowledge as a possible cause; few have explored the confidence clinicians have in their management decisions. Confidence in decision-making may influence management practices, particularly when guidelines are complex and evolving. OBJECTIVE Assess whether confidence in decision-making is associated with making guideline-concordant recommendations for abnormal cervical cancer screening results. DESIGN A clinician survey used vignettes to ask clinicians to make a management recommendation for different abnormal results and rate their level of confidence in their response. PARTICIPANTS Physicians and advanced practice providers (APPs) at three diverse health systems in Washington, Texas, and Massachusetts. MAIN MEASURES Correct response to each vignette based on either the 2012 or 2019 American Colposcopy and Cervical Pathology (ASCCP) management guidelines. KEY RESULTS In total, 501 clinicians completed the survey between October and December 2020 (response rate 53.7%). Overall, most clinicians made guideline-recommended management decisions for two vignettes (73.2 and 73.7%), but fewer were confident in their selection (48.3% and 46.6%, respectively). Clinicians who reported high levels of confidence were more often correct than those who reported lower levels of confidence (85.8% vs. 62.2% and 87.5% vs. 60.7%, both p<0.001). After adjusting for clinician and practice characteristics, confidence remained significantly associated with selecting the correct answer. CONCLUSIONS Clinician confidence in management decisions for abnormal cervical cancer screening results was significantly associated with knowing guideline-concordant recommendations. Given the complexity of cervical cancer management guidelines, solutions to improve clinician confidence in decision-making are needed.
Collapse
Affiliation(s)
- Tin H Dang
- Division of General Internal Medicine, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | - Meghan L Rieu-Werden
- Division of General Internal Medicine, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | - Sarah C Kobrin
- Division of Cancer Control and Population Sciences, National Cancer Institute, Bethesda, MD, USA
| | - Jasmin A Tiro
- Department of Public Health Sciences, University of Chicago Biological Sciences Division, Chicago, IL, USA
| | - Claudia Werner
- Department of Obstetrics and Gynecology, University of Texas Southwestern Medical Center, Dallas, TX, USA
- Parkland Health and Hospital System, Dallas, TX, USA
| | - Jacquelyn M Lykken
- Peter O'Donnell Jr. School of Public Health, University of Texas Southwestern Medical Center, Dallas, TX, USA
| | - Jessica Chubak
- Kaiser Permanente Washington Health Research Institute, Seattle, WA, USA
| | - Steven J Atlas
- Division of General Internal Medicine, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | - Robin T Higashi
- Peter O'Donnell Jr. School of Public Health, University of Texas Southwestern Medical Center, Dallas, TX, USA
| | - Simon C Lee
- Department of Population Health, Medical Center, University of Kansas, Kansas City, KS, USA
| | - Jennifer S Haas
- Division of General Internal Medicine, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | - Celette Sugg Skinner
- Peter O'Donnell Jr. School of Public Health, University of Texas Southwestern Medical Center, Dallas, TX, USA
- Harold C Simmons Comprehensive Cancer Center, University of Texas Southwestern Medical Center, Dallas, TX, USA
| | - Michelle I Silver
- Department of Surgery, Washington University School of Medicine in St. Louis, St. Louis, MO, USA
| | - Sarah Feldman
- Division of Gynecologic Oncology, Brigham and Women's Hospital, Harvard Medical School, 75 Francis Street, Boston, MA, 02115, USA.
| |
Collapse
|
2
|
Pereyra L, Schlottmann F, Steinberg L, Lasa J. Colorectal Cancer Prevention: Is Chat Generative Pretrained Transformer (Chat GPT) ready to Assist Physicians in Determining Appropriate Screening and Surveillance Recommendations? J Clin Gastroenterol 2024:00004836-990000000-00265. [PMID: 38319619 DOI: 10.1097/mcg.0000000000001979] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/11/2023] [Accepted: 01/12/2024] [Indexed: 02/07/2024]
Abstract
OBJECTIVE To determine whether a publicly available advanced language model could help determine appropriate colorectal cancer (CRC) screening and surveillance recommendations. BACKGROUND Poor physician knowledge or inability to accurately recall recommendations might affect adherence to CRC screening guidelines. Adoption of newer technologies can help improve the delivery of such preventive care services. METHODS An assessment with 10 multiple choice questions, including 5 CRC screening and 5 CRC surveillance clinical vignettes, was inputted into chat generative pretrained transformer (ChatGPT) 3.5 in 4 separate sessions. Responses were recorded and screened for accuracy to determine the reliability of this tool. The mean number of correct answers was then compared against a control group of gastroenterologists and colorectal surgeons answering the same questions with and without the help of a previously validated CRC screening mobile app. RESULTS The average overall performance of ChatGPT was 45%. The mean number of correct answers was 2.75 (95% CI: 2.26-3.24), 1.75 (95% CI: 1.26-2.24), and 4.5 (95% CI: 3.93-5.07) for screening, surveillance, and total questions, respectively. ChatGPT showed inconsistency and gave a different answer in 4 questions among the different sessions. A total of 238 physicians also responded to the assessment; 123 (51.7%) without and 115 (48.3%) with the mobile app. The mean number of total correct answers of ChatGPT was significantly lower than those of physicians without [5.62 (95% CI: 5.32-5.92)] and with the mobile app [7.71 (95% CI: 7.39-8.03); P < 0.001]. CONCLUSIONS Large language models developed with artificial intelligence require further refinements to serve as reliable assistants in clinical practice.
Collapse
Affiliation(s)
- Lisandro Pereyra
- Department of Gastroenterology
- Endoscopy Unit, Department of Surgery
| | - Francisco Schlottmann
- Endoscopy Unit, Department of Surgery
- Department of Surgery, Hospital Alemán of Buenos Aires
| | - Leandro Steinberg
- Department of Gastroenterology, Fundacion Favaloro, Buenos Aires, Argentina
| | - Juan Lasa
- Department of Gastroenterology, CEMIC, Buenos Aires, Argentina
| |
Collapse
|
3
|
Clare CA, Liu C, Greenberg A, Liberatos P, Channen L, Ram K, Fernandez S, Harley J. Colorectal Cancer Screening Knowledge and Practices Among Practicing Obstetrician-Gynecologists and Residents. WOMEN'S HEALTH REPORTS (NEW ROCHELLE, N.Y.) 2023; 4:1-10. [PMID: 36727095 PMCID: PMC9883669 DOI: 10.1089/whr.2022.0065] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 11/16/2022] [Indexed: 06/18/2023]
Abstract
BACKGROUND Colorectal cancer (CRC) is the second leading cause of cancer death in the US, the third most diagnosed cancer in women, and the second leading cause of cancer death in women. The aims of our study are to (1) investigate knowledge of and adherence to CRC screening guidelines by obstetrician-gynecologists (Ob/Gyns) and (2) assess whether this knowledge/adherence vary by demographic and practice characteristics. METHODS An anonymous cross-sectional survey was distributed to a convenience sample of 142 practicing Obs/Gyns drawn from National Medical Association section members/conference attendees and hospital Ob/Gyn department members. RESULTS Most respondents (80.3%) viewed colorectal screening within the scope of Ob/Gyn practice, and 71.8% used the American College of Obstetricians and Gynecologists guidelines for screening. Most respondents were knowledgeable regarding CRC screening but not in all areas. On average they only identified half of the 10 risk factors listed and only one-quarter correctly identified the age when screening can stop. Residents were somewhat more knowledgeable about screening guidelines and risk factors than attendings. More than half of respondents (57.8%) reported always initiating CRC screening for the appropriate age and risk factors. Respondents identified education and awareness (56.3%) and patients' unwillingness to undergo an invasive procedure (75.4%) as barriers to screening. CONCLUSIONS Knowledge regarding CRC screening was less than optimal and differed by attending/resident status. Greater emphasis should be placed on CRC screening and guidelines training for primary care providers like Ob/Gyns. Some of this could be accomplished through maintenance of certification and continued integration into residency education.
Collapse
Affiliation(s)
- Camille A. Clare
- Department of Obstetrics and Gynecology, New York Medical College, Valhalla, New York, USA
| | - Clive Liu
- Department of Obstetrics and Gynecology, New York Medical College, Valhalla, New York, USA
| | - Arielle Greenberg
- New York Medical College, School of Medicine, Valhalla, New York, USA
| | - Penny Liberatos
- Department of Public Health, New York Medical College, Valhalla, New York, USA
| | - Lindsey Channen
- Department of Medicine, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Kavitha Ram
- Department of Obstetrics and Gynecology, Jamaica Hospital Medical Center, Jamaica, New York, USA
| | - Silvia Fernandez
- Department of Gastroenterology, New York City Health + Hospitals/Metropolitan, New York, New York, USA
| | - Jennifer Harley
- Department of Gastroenterology, New York City Health + Hospitals/Metropolitan, New York, New York, USA
| |
Collapse
|