Communicating With Diverse Patients About Participating in a Biobank: A Randomized Multisite Study Comparing Electronic and Face-to-Face Informed Consent Processes

Large Language Model-Based Chatbot vs Surgeon-Generated Informed Consent Documentation for Common Procedures

Importance

Informed consent is a critical component of patient care before invasive procedures, yet it is frequently inadequate. Electronic consent forms have the potential to facilitate patient comprehension if they provide information that is readable, accurate, and complete; it is not known if large language model (LLM)-based chatbots may improve informed consent documentation by generating accurate and complete information that is easily understood by patients.

Objective

To compare the readability, accuracy, and completeness of LLM-based chatbot- vs surgeon-generated information on the risks, benefits, and alternatives (RBAs) of common surgical procedures.

Design, Setting, and Participants

This cross-sectional study compared randomly selected surgeon-generated RBAs used in signed electronic consent forms at an academic referral center in San Francisco with LLM-based chatbot-generated (ChatGPT-3.5, OpenAI) RBAs for 6 surgical procedures (colectomy, coronary artery bypass graft, laparoscopic cholecystectomy, inguinal hernia repair, knee arthroplasty, and spinal fusion).

Main Outcomes and Measures

Readability was measured using previously validated scales (Flesh-Kincaid grade level, Gunning Fog index, the Simple Measure of Gobbledygook, and the Coleman-Liau index). Scores range from 0 to greater than 20 to indicate the years of education required to understand a text. Accuracy and completeness were assessed using a rubric developed with recommendations from LeapFrog, the Joint Commission, and the American College of Surgeons. Both composite and RBA subgroup scores were compared.

Results

The total sample consisted of 36 RBAs, with 1 RBA generated by the LLM-based chatbot and 5 RBAs generated by a surgeon for each of the 6 surgical procedures. The mean (SD) readability score for the LLM-based chatbot RBAs was 12.9 (2.0) vs 15.7 (4.0) for surgeon-generated RBAs (P = .10). The mean (SD) composite completeness and accuracy score was lower for surgeons' RBAs at 1.6 (0.5) than for LLM-based chatbot RBAs at 2.2 (0.4) (P < .001). The LLM-based chatbot scores were higher than the surgeon-generated scores for descriptions of the benefits of surgery (2.3 [0.7] vs 1.4 [0.7]; P < .001) and alternatives to surgery (2.7 [0.5] vs 1.4 [0.7]; P < .001). There was no significant difference in chatbot vs surgeon RBA scores for risks of surgery (1.7 [0.5] vs 1.7 [0.4]; P = .38).

Conclusions and Relevance

The findings of this cross-sectional study suggest that despite not being perfect, LLM-based chatbots have the potential to enhance informed consent documentation. If an LLM were embedded in electronic health records in a manner compliant with the Health Insurance Portability and Accountability Act, it could be used to provide personalized risk information while easing documentation burden for physicians.

Improving the Practice of Obtaining Informed Consent for Biobanking in Clinical Settings

Background: Biobanks form key research support infrastructures that ensure the highest sample quality for scientific research. Their activity must align closely and proportionally to the interests of researchers, donors, and society. Informed consent (IC) is a central tool to guarantee the protection of donors' rights and interests. Aim: This study aimed to analyze the challenges of obtaining IC for biobanking in clinical settings and ways to improve this process. Methods: Biobank Bellvitge University Hospital HUB-ICO-IDIBELL in Barcelona received 8671 IC forms between 2017 and 2020. The mistakes that caused IC forms to be rejected by the Biobank were analyzed. In addition, interventions aimed at physicians to improve the IC process were evaluated through a calculation of the relative risk (RR). Finally, physicians who submitted samples to the Biobank, most of whom are involved in research activities, were surveyed about the barriers to collecting IC and how to improve this process. Results: During 2017-2020, 19.6% of IC forms were rejected. The most relevant cause of rejection was the use of outdated IC forms, followed by missing patient information or mistakes having been made by the physician. Evaluation of the rejection rates before and after interventions to improve the IC process suggests significant improvement (27.7% before interventions (January 2017-May 2018) compared to 9.6% after interventions (February-December 2020), RR 0.4 95% CI 0.34-0.47; p < 0.0001). According to the physicians, the most important barrier to collecting IC is the time constraint, and they consider digitalization as a viable solution. Conclusions: Our research offers a view of the less well-understood practical challenges that physicians and biobanks face when collecting IC in clinical settings. It suggests that, despite multiple challenges, continuous monitoring, training, and information programs for physicians are key to optimizing the IC process in clinical settings.

Comparative Effectiveness of eConsent: Systematic Review

BACKGROUND

Providing informed consent means agreeing to participate in a clinical trial and having understood what is involved. Flawed informed consent processes, including missing dates and signatures, are common regulatory audit findings. Electronic consent (eConsent) uses digital technologies to enable the consenting process. It aims to improve participant comprehension and engagement with study information and to address data quality concerns.

OBJECTIVE

This systematic literature review aimed to assess the effectiveness of eConsent in terms of patient comprehension, acceptability, usability, and study enrollment and retention rates, as well as the effects of eConsent on the time patients took to perform the consenting process ("cycle time") and on-site workload in comparison with traditional paper-based consenting.

METHODS

The systematic review was conducted and reported in accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. Ovid Embase and Ovid MEDLINE were systematically searched for publications reporting original, comparative data on the effectiveness of eConsent in terms of patient comprehension, acceptability, usability, enrollment and retention rates, cycle time, and site workload. The methodological validity of the studies that compared outcomes for comprehension, acceptability, and usability across paper consent and eConsent was assessed. Study methodologies were categorized as having "high" validity if comprehensive assessments were performed using established instruments.

RESULTS

Overall, 37 publications describing 35 studies (13,281 participants) were included. All studies comparing eConsenting and paper-based consenting for comprehension (20/35, 57% of the studies; 10 with "high" validity), acceptability (8/35, 23% of the studies; 1 with "high" validity), and usability (5/35, 14% of the studies; 1 with "high" validity) reported significantly better results with eConsent, better results but without significance testing, or no significant differences in overall results. None of the studies reported better results with paper than with eConsent. Among the "high" validity studies, 6 studies on comprehension reported significantly better understanding of at least some concepts, the study on acceptability reported statistically significant higher satisfaction scores, and the study on usability reported statistically significant higher usability scores with eConsent than with paper (P<.05 for all). Cycle times were increased with eConsent, potentially reflecting greater patient engagement with the content. Data on enrollment and retention were limited. Comparative data from site staff and other study researchers indicated the potential for reduced workload and lower administrative burden with eConsent.

CONCLUSIONS

This systematic review showed that compared with patients using paper-based consenting, patients using eConsent had a better understanding of the clinical trial information, showed greater engagement with content, and rated the consenting process as more acceptable and usable. eConsent solutions thus have the potential to enhance understanding, acceptability, and usability of the consenting process while inherently being able to address data quality concerns, including those related to flawed consenting processes.