Trainees' knowledge and application of guideline recommendations for colorectal cancer screening and surveillance

Association Between Clinician Confidence and Making Guideline-Recommended Decisions in the Management of Abnormal Cervical Cancer Screening Results

BACKGROUND

Guidelines for managing abnormal cervical cancer screening results are complex and adherence is challenging for clinicians. Previous studies have identified gaps in knowledge as a possible cause; few have explored the confidence clinicians have in their management decisions. Confidence in decision-making may influence management practices, particularly when guidelines are complex and evolving.

OBJECTIVE

Assess whether confidence in decision-making is associated with making guideline-concordant recommendations for abnormal cervical cancer screening results.

DESIGN

A clinician survey used vignettes to ask clinicians to make a management recommendation for different abnormal results and rate their level of confidence in their response.

PARTICIPANTS

Physicians and advanced practice providers (APPs) at three diverse health systems in Washington, Texas, and Massachusetts.

MAIN MEASURES

Correct response to each vignette based on either the 2012 or 2019 American Colposcopy and Cervical Pathology (ASCCP) management guidelines.

KEY RESULTS

In total, 501 clinicians completed the survey between October and December 2020 (response rate 53.7%). Overall, most clinicians made guideline-recommended management decisions for two vignettes (73.2 and 73.7%), but fewer were confident in their selection (48.3% and 46.6%, respectively). Clinicians who reported high levels of confidence were more often correct than those who reported lower levels of confidence (85.8% vs. 62.2% and 87.5% vs. 60.7%, both p<0.001). After adjusting for clinician and practice characteristics, confidence remained significantly associated with selecting the correct answer.

CONCLUSIONS

Clinician confidence in management decisions for abnormal cervical cancer screening results was significantly associated with knowing guideline-concordant recommendations. Given the complexity of cervical cancer management guidelines, solutions to improve clinician confidence in decision-making are needed.

Colorectal Cancer Prevention: Is Chat Generative Pretrained Transformer (Chat GPT) ready to Assist Physicians in Determining Appropriate Screening and Surveillance Recommendations?

OBJECTIVE

To determine whether a publicly available advanced language model could help determine appropriate colorectal cancer (CRC) screening and surveillance recommendations.

BACKGROUND

Poor physician knowledge or inability to accurately recall recommendations might affect adherence to CRC screening guidelines. Adoption of newer technologies can help improve the delivery of such preventive care services.

METHODS

An assessment with 10 multiple choice questions, including 5 CRC screening and 5 CRC surveillance clinical vignettes, was inputted into chat generative pretrained transformer (ChatGPT) 3.5 in 4 separate sessions. Responses were recorded and screened for accuracy to determine the reliability of this tool. The mean number of correct answers was then compared against a control group of gastroenterologists and colorectal surgeons answering the same questions with and without the help of a previously validated CRC screening mobile app.

RESULTS

The average overall performance of ChatGPT was 45%. The mean number of correct answers was 2.75 (95% CI: 2.26-3.24), 1.75 (95% CI: 1.26-2.24), and 4.5 (95% CI: 3.93-5.07) for screening, surveillance, and total questions, respectively. ChatGPT showed inconsistency and gave a different answer in 4 questions among the different sessions. A total of 238 physicians also responded to the assessment; 123 (51.7%) without and 115 (48.3%) with the mobile app. The mean number of total correct answers of ChatGPT was significantly lower than those of physicians without [5.62 (95% CI: 5.32-5.92)] and with the mobile app [7.71 (95% CI: 7.39-8.03); P < 0.001].

CONCLUSIONS

Large language models developed with artificial intelligence require further refinements to serve as reliable assistants in clinical practice.

Colorectal Cancer Screening Knowledge and Practices Among Practicing Obstetrician-Gynecologists and Residents

BACKGROUND

Colorectal cancer (CRC) is the second leading cause of cancer death in the US, the third most diagnosed cancer in women, and the second leading cause of cancer death in women. The aims of our study are to (1) investigate knowledge of and adherence to CRC screening guidelines by obstetrician-gynecologists (Ob/Gyns) and (2) assess whether this knowledge/adherence vary by demographic and practice characteristics.

METHODS

An anonymous cross-sectional survey was distributed to a convenience sample of 142 practicing Obs/Gyns drawn from National Medical Association section members/conference attendees and hospital Ob/Gyn department members.

RESULTS

Most respondents (80.3%) viewed colorectal screening within the scope of Ob/Gyn practice, and 71.8% used the American College of Obstetricians and Gynecologists guidelines for screening. Most respondents were knowledgeable regarding CRC screening but not in all areas. On average they only identified half of the 10 risk factors listed and only one-quarter correctly identified the age when screening can stop. Residents were somewhat more knowledgeable about screening guidelines and risk factors than attendings. More than half of respondents (57.8%) reported always initiating CRC screening for the appropriate age and risk factors. Respondents identified education and awareness (56.3%) and patients' unwillingness to undergo an invasive procedure (75.4%) as barriers to screening.

CONCLUSIONS

Knowledge regarding CRC screening was less than optimal and differed by attending/resident status. Greater emphasis should be placed on CRC screening and guidelines training for primary care providers like Ob/Gyns. Some of this could be accomplished through maintenance of certification and continued integration into residency education.