How readable are orthognathic surgery consent forms?

Clear aligner therapy informed consent forms: A quality and readability evaluation

OBJECTIVE

The aim of the present study was to evaluate the quality and readability of content contained within clear aligner therapy (CAT) informed consent forms.

METHODS

CAT informed consent forms were identified via an online search. The presence of details related to CAT-related processes, risks, benefits and alternatives in each form was recorded. A 4-point Likert type scale was used to determine the quality of content (QOC). The readability of content was evaluated with the Simple Measure of Gobbledegook (SMOG) and Flesch Reading Ease Score (FRES).

RESULTS

A total of 42 forms satisfied selection criteria. Nineteen (45.2%) were authored by companies who provided aligners to patients via clinicians. The QOC regarding CAT-related treatment processes [median 2.0; IQR 0, 2] and benefits [median 2.0; IQR 1, 2] was adequate. The QOC scores regarding treatment alternatives, consequences of no treatment and relapse were poor. There was no difference (P=0.59) in the median (IQR) QOC of the informed consent forms provided by direct-to-consumer (DTC) aligner providers [10 (8.25, 16.25)] and non-DTC aligner providers [12 (10, 14)]. The median (IQR) SMOG score was 12.1 (10.9, 12.7) and FRES was 39.0 (36.0, 44.25).

CONCLUSIONS

The QOC of the evaluated forms was incomplete and poor. The content was difficult to read and failed to reach recommended readability standards. Consent is unlikely to be valid if it is based solely on the content of the forms. Clinicians need to be aware of the limitations of informed consent forms for CAT particularly in relation to alternatives, prognosis, risks, and the need for long-term maintenance of results.

Examination of the reliability and readability of Chatbot Generative Pretrained Transformer's (ChatGPT) responses to questions about orthodontics and the evolution of these responses in an updated version

INTRODUCTION

This study aimed to assess the reliability and readability of Chatbot Generative Pretrained Transformer (ChatGPT) responses to questions about orthodontics and the evolution of these responses in an updated version.

METHODS

Frequently asked questions about orthodontics by laypeople on Web sites were determined using the Google Search Tool. These questions were asked to both ChatGPT's March 23 version and May 24 version on April 20, 2023, and July 12, 2023, respectively. Responses were assessed for readability and reliability using the Flesch-Kincaid and DISCERN tests.

RESULTS

The mean DISCERN value for general questions was 2.96 ± 0.05, 3.04 ± 0.06, 2.38 ± 0.27, and 2.82 ± 0.31 for treatment-related questions; the mean Flesch-Kincaid Reading Ease score for general questions was 29.28 ± 8.22, 25.12 ± 7.39, 47.67 ± 10.77, and 41.60 ± 9.54 for treatment-related questions; mean Flesch-Kincaid Grade Level for general questions was 14.52 ± 1.48 and 14.04 ± 1.25 and 11.90 ± 2.08 and 11.41 ± 1.88 for treatment-related questions; in first and second evaluations respectively (P = 0.001).

CONCLUSIONS

In the second evaluation, the reliability of the answers given to general questions and treatment-related questions increased. However, in both evaluations, the reliability of the answers was found to be moderate according to the DISCERN tool. On the second evaluation, Flesch Reading Ease Scores for both general questions and treatment-related questions decreased, meaning that the readability of the new response texts became more difficult. Flesch-Kincaid Grade Level results were found at the college graduate level in the first and second evaluations for general questions and at the high school level in the first and second evaluations for treatment-related questions.

What Is Health Literacy Among Orthognathic Surgery Patients?

BACKGROUND

Health literacy of orthognathic surgery patients has not been thoroughly evaluated.

PURPOSE

The purpose of this study was to estimate health literacy and identify risk factors associated with inadequate health literacy in orthognathic surgery patients.

STUDY DESIGN, SETTING, SAMPLE

A cross-sectional study was implemented utilizing patients ages 14-80 years who presented for orthognathic surgery evaluation between September 2021 and December 2022. Subjects were excluded from the study if they did not complete the orthognathic surgery evaluation, were not between the ages of 14-80 years old, or did not complete the Brief Health Literacy Screening Tool (BRIEF) questionnaire during intake. Subjects who have not undergone orthognathic surgery but completed the initial evaluation for orthognathic surgery were included in the study.

PREDICTOR VARIABLES

The predictor variables were a set of risk factors for inadequate health literacy: age, sex, primary language, race, estimated household income, and diagnosis.

MAIN OUTCOME VARIABLE

The main outcome variable was health literacy assessed using the BRIEF questionnaire. During intake, subjects completed the BRIEF questionnaire consisting of four questions scored on an ordinal scale of 1-5. Inadequate health literacy was defined as a BRIEF score ≤16.

COVARIATES

Not applicable.

ANALYSES

Bivariate and multivariate analyses were performed. P < .05 was considered statistically significant.

RESULTS

Of 150 patients presenting for orthognathic surgery, fifteen percent of patients had inadequate health literacy via the BRIEF test. The mean age of those with adequate health literacy was 27.9 years (standard deviation, ±12.5) compared to 18.5 years (standard deviation, ±5.7) for those with inadequate health literacy (P = <.001). After adjusting for sex, language, race, estimated household income, and diagnosis via multivariate analysis, increasing age was associated with decreased odds of inadequate health literacy (adjusted odds ratio = 0.81; confidence interval, 0.72-0.92; P = <.001).

CONCLUSION AND RELEVANCE

In the complex process of orthognathic surgery, it is essential to identify patients with inadequate health literacy that may require additional health literacy interventions. Ultimately, 15% of orthognathic surgery subjects had inadequate health literacy, and younger patients were the most susceptible as the odds of inadequate health literacy decreased with increasing age.

Nursing Roles in the Quality of Information in Informed Consent Forms of a Spanish County Hospital

(1) Background: Because of their direct and continuous contact with the patient, nurses play a relevant role in ensuring that informed consent forms are complete and easy to read and comprehend. The objective of this study was to analyze the legibility and formal quality of informed consent forms for non-surgical procedures in a county hospital. (2) Methods: The readability of these forms was analyzed using the INFLESZ scale and the information they provided according to the formal quality criteria established for these forms. (3) Results: Readability was difficult in 78.08% of the forms analyzed. No form fulfilled all the criteria, the most non-compliant being the non-appearance of the verification of delivery of a copy to the patient (100%), the contraindications (94.59%), and the alternatives (83.78%) of the procedure. Statistically significant differences were observed between disciplines with respect to the INFLESZ readability score and the formal quality score, but no statistically significant correlation was found between the two scores. (4) Conclusions: The informed consent forms for non-surgical procedures analyzed presented mostly difficult readability and poor formal quality, making it difficult for patients to have understandable and complete information. Nursing professionals should be actively involved in their improvement to facilitate patient decision making.

Large Language Model-Based Chatbot vs Surgeon-Generated Informed Consent Documentation for Common Procedures

Importance

Informed consent is a critical component of patient care before invasive procedures, yet it is frequently inadequate. Electronic consent forms have the potential to facilitate patient comprehension if they provide information that is readable, accurate, and complete; it is not known if large language model (LLM)-based chatbots may improve informed consent documentation by generating accurate and complete information that is easily understood by patients.

Objective

To compare the readability, accuracy, and completeness of LLM-based chatbot- vs surgeon-generated information on the risks, benefits, and alternatives (RBAs) of common surgical procedures.

Design, Setting, and Participants

This cross-sectional study compared randomly selected surgeon-generated RBAs used in signed electronic consent forms at an academic referral center in San Francisco with LLM-based chatbot-generated (ChatGPT-3.5, OpenAI) RBAs for 6 surgical procedures (colectomy, coronary artery bypass graft, laparoscopic cholecystectomy, inguinal hernia repair, knee arthroplasty, and spinal fusion).

Main Outcomes and Measures

Readability was measured using previously validated scales (Flesh-Kincaid grade level, Gunning Fog index, the Simple Measure of Gobbledygook, and the Coleman-Liau index). Scores range from 0 to greater than 20 to indicate the years of education required to understand a text. Accuracy and completeness were assessed using a rubric developed with recommendations from LeapFrog, the Joint Commission, and the American College of Surgeons. Both composite and RBA subgroup scores were compared.

Results

The total sample consisted of 36 RBAs, with 1 RBA generated by the LLM-based chatbot and 5 RBAs generated by a surgeon for each of the 6 surgical procedures. The mean (SD) readability score for the LLM-based chatbot RBAs was 12.9 (2.0) vs 15.7 (4.0) for surgeon-generated RBAs (P = .10). The mean (SD) composite completeness and accuracy score was lower for surgeons' RBAs at 1.6 (0.5) than for LLM-based chatbot RBAs at 2.2 (0.4) (P < .001). The LLM-based chatbot scores were higher than the surgeon-generated scores for descriptions of the benefits of surgery (2.3 [0.7] vs 1.4 [0.7]; P < .001) and alternatives to surgery (2.7 [0.5] vs 1.4 [0.7]; P < .001). There was no significant difference in chatbot vs surgeon RBA scores for risks of surgery (1.7 [0.5] vs 1.7 [0.4]; P = .38).

Conclusions and Relevance

The findings of this cross-sectional study suggest that despite not being perfect, LLM-based chatbots have the potential to enhance informed consent documentation. If an LLM were embedded in electronic health records in a manner compliant with the Health Insurance Portability and Accountability Act, it could be used to provide personalized risk information while easing documentation burden for physicians.