Lists of potential diagnoses that final-year medical students need to consider: a modified Delphi study

Diagnosis in Bytes: Comparing the Diagnostic Accuracy of Google and ChatGPT 3.5 as an Educational Support Tool

BACKGROUND

Adopting advanced digital technologies as diagnostic support tools in healthcare is an unquestionable trend accelerated by the COVID-19 pandemic. However, their accuracy in suggesting diagnoses remains controversial and needs to be explored. We aimed to evaluate and compare the diagnostic accuracy of two free accessible internet search tools: Google and ChatGPT 3.5.

METHODS

To assess the effectiveness of both medical platforms, we conducted evaluations using a sample of 60 clinical cases related to urological pathologies. We organized the urological cases into two distinct categories for our analysis: (i) prevalent conditions, which were compiled using the most common symptoms, as outlined by EAU and UpToDate guidelines, and (ii) unusual disorders, identified through case reports published in the 'Urology Case Reports' journal from 2022 to 2023. The outcomes were meticulously classified into three categories to determine the accuracy of each platform: "correct diagnosis", "likely differential diagnosis", and "incorrect diagnosis". A group of experts evaluated the responses blindly and randomly.

RESULTS

For commonly encountered urological conditions, Google's accuracy was 53.3%, with an additional 23.3% of its results falling within a plausible range of differential diagnoses, and the remaining outcomes were incorrect. ChatGPT 3.5 outperformed Google with an accuracy of 86.6%, provided a likely differential diagnosis in 13.3% of cases, and made no unsuitable diagnosis. In evaluating unusual disorders, Google failed to deliver any correct diagnoses but proposed a likely differential diagnosis in 20% of cases. ChatGPT 3.5 identified the proper diagnosis in 16.6% of rare cases and offered a reasonable differential diagnosis in half of the cases.

CONCLUSION

ChatGPT 3.5 demonstrated higher diagnostic accuracy than Google in both contexts. The platform showed satisfactory accuracy when diagnosing common cases, yet its performance in identifying rare conditions remains limited.

Core Procedural Skills to be Taught for III-Year Undergraduate Medical Students: A Modified Delphi Study

Introduction

While medical students are expected to learn procedural skills during their training, there is no consensus on their level of learning. Further, the most essential procedural skills across medical curricula which need to be taught during their III-year clinical posting are often not considered. The purpose of this study was to identify the core procedural skills needed to be taught during the III-year undergraduate medical students clinical posting.

Methods

A three-round, online Modified Delphi method was used to identify consensus on selecting the most essential procedural skills prescribed in National Medical Commission (NMC) curriculum 2019. In Round 1, a list of 54 procedural skills from the National Medical Commission's (NMC) Graduate Medical Education Regulations (GMER) 2019 curriculum was distributed to 22 experts in pre-clinical medical education and multidisciplinary clinicians.They rated the skills in terms of importance. In Round 2, the skills that received consensus in Round 1 were presented, resulting in 13 skills for evaluation. Round 3 further narrowed down the skills to a final consensus of 6. An interclass correlation coefficient of 0.767 among experts indicates a substantial level of reliability.

Results

Consensus was achieved for six procedural skills, each demonstrating over 80% agreement among the experts. These skills include basic life support, intravenous cannulation, urinary catheterization for both male and female patients, nasogastric tube insertion, oxygen administration, and basic suturing. Notably, all these skills received the highest level of agreement, surpassing 90% consensus.

Conclusions

The results of the modified Delphi study offer crucial insights into the procedural skills that should be included in the curriculum for third-year undergraduate medical students during their clinical rotations in a tertiary care teaching hospital. Faculty members at these institutions differ in their opinions regarding the importance of teaching specific procedural skills, influenced by their teaching background and the student cohorts they instruct. From a comprehensive list, six skills have been pinpointed as the most vital through the modified Delphi technique. Moreover, the Delphi technique is acknowledged as a valuable method for achieving a consensus on prioritizing the training of certifiable skills.

The influence of Gamification on medical students' diagnostic decision making and awareness of medical cost: a mixed-method study

BACKGROUND

The gamification of learning increases student enjoyment, and motivation and engagement in learning tasks. This study investigated the effects of gamification using decision-making cards (DMCs) on diagnostic decision-making and cost using case scenarios.

METHOD

Thirty medical students in clinical clerkship participated and were randomly assigned to 14 small groups of 2-3 medical students each. Decision-making was gamified using DMCs with a clinical information heading and medical cost on the front, and clinical information details on the back. First, each team was provided with brief clinical information on case scenarios. Subsequently, DMCs depending on the case were distributed to each team, and team members chose cards one at a time until they reached a diagnosis of the case. The total medical cost was then scored based on the number and contents of cards drawn. Four case scenarios were conducted. The quantitative outcomes including confidence in effective clinical decision-making, motivation to learn diagnostic decision-making, and awareness of medical costs were measured before and after our gamification by self-evaluation using a 7-point Likert scale. The qualitative component consisted of a content analysis on the benefits of learning clinical reasoning using DMCs.

RESULT

Confidence in effective clinical decision-making, motivation to learn diagnostic decision-making, and awareness of medical cost were significantly higher after the gamification. Furthermore, comparing the clinical case scenario tackled last with the one tackled first, the average medical cost of all cards drawn by students decreased significantly from 11,921 to 8,895 Japanese yen. In the content analysis, seven advantage categories of DMCs corresponding to clinical reasoning components were extracted (information gathering, hypothesis generation, problem representation, differential diagnosis, leading or working diagnosis, diagnostic justification, and management and treatment).

CONCLUSION

Teaching medical students clinical reasoning using DMCs can improve clinical decision-making confidence and learning motivation, and reduces medical cost in clinical case scenarios. In addition, it can help students to acquire practical knowledge, deepens their understanding of clinical reasoning, and identifies several important clinical reasoning skills including diagnostic decision-making and awareness of medical costs. Gamification using DMCs can be an effective teaching method for improving medical students' diagnostic decision-making and reducing costs.

Identifying relevant topics and their competency levels for dental materials science: a fuzzy Delphi study

BACKGROUND

Dental materials science is an important core course in undergraduate dental programs which integrates foundational concepts of chemical engineering and materials science into clinical dentistry. The present study aimed to identify relevant dental materials science topics for Malaysian undergraduate dental curricula and to determine their appropriate competency levels in terms of cognitive and psychomotor taxonomies.

METHODS

Potential dental materials science topics were drafted in alignment with the revised national competency statement. The list of topics was further amended after comparing it with those recommended topics in the literature. Fuzzy Delphi method was applied. Experts were selected based on the different inclusion criteria. They ranked the topics using a five-point Likert scale and recommended the appropriate cognitive and psychomotor levels. Next, fuzzy evaluation was performed. Consensus was deemed for a topic to be included if (a) the average expert agreement was ≥ 75%, (b) the d-construct threshold value for each topic was ≤ 0.2 and (c) the average fuzzy number was ≥ 0.5.

RESULTS

Sixty-two experts participated in the study. They accepted 33 out of 36 potential dental materials science topics. The average Likert score and fuzzy number ranged from 3.63 to 4.92 and 0.526 to 0.784, respectively. Furthermore, "Endodontic materials" was ranked as the most significant topic. Meanwhile, many topics required dental students to demonstrate a cognitive level of "Apply" and a psychomotor level of "Guided response". Based on mean scores, "Impression materials" was rated as the most cognitively demanding topic, whilst "Temporary restorative materials" was the most demanding topic for psychomotor taxonomy.

CONCLUSION

The present study has identified relevant dental materials science topics and their appropriate cognitive and psychomotor levels using the Fuzzy Delphi approach. The findings of the present study form the basis for future studies to develop measurable learning outcomes, design corresponding innovative pedagogy and propose assessment criteria for each topic.

Clinical Clerkships in General Medicine Enable Students to Acquire Basic Medical Competencies and Experience in Community-Based Integrated Care: A Descriptive Questionnaire-Based Study

Background No previous research has targeted educators regarding educational practice and the achievements of students in terms of the learning objectives of clinical clerkships in university general medicine departments of Japan. We aimed to clarify the characteristics of clinical clerkships in Japanese general medicine departments using a questionnaire administered to chairpersons of university general medicine departments. Methods This was a descriptive questionnaire-based study using Google Forms (Google, Inc., Mountain View, CA, USA). We asked the chairpersons of general medicine departments in Japanese universities the following questions, with responses given on a 5-point Likert scale: Question 1: How well are primary symptoms in the national model core curriculum for undergraduate medical education taught in clinical clerkships in university general medicine departments? Question 2: How successfully can students achieve the learning objectives of the national model core curriculum for undergraduate medical education through clinical clerkships in general medicine departments of university hospitals? Question 3: How successfully can students achieve the learning objectives of the national model core curriculum for undergraduate medical education through clinical clerkships in other community clinics or hospitals? The results of the questionnaire responses are described as mean±standard deviation. Results Of the 71 Japanese universities with general medicine departments, 43 were included in the analysis. For Question 1, the symptoms and pathophysiologies with a mean score of 4 points or higher were fever, general malaise, anorexia, weight loss or gain, edema, abdominal pain, lymphadenopathy, and headache. All those symptoms require basic medical competencies. For Questions 2 and 3, the intramural clinical clerkship of general medicine departments had a higher mean score than the extramural clinical clerkship for diagnostic reasoning that emphasizes medical history and physical examination and a comprehensive approach to patients with multiple health problems. In contrast, the extramural clinical clerkship, in which medical students can build experience with community-integrated care, had a mean score of 3 points or higher for all items. Conclusions The clinical clerkship in general medicine departments of Japanese universities provides students with chances to acquire clinical competencies regarding primary symptoms and pathophysiologies. Additionally, the extramural clinical clerkship provides experience in community-based integrated care, including home medical care, collaboration, health and welfare, and long-term care.

Hybrid PBL and Pure PBL: Which one is more effective in developing clinical reasoning skills for general medicine clerkship?-A mixed-method study

This study aims to compare the effectiveness of Hybrid and Pure problem-based learning (PBL) in teaching clinical reasoning skills to medical students. The study sample consisted of 99 medical students participating in a clerkship rotation at the Department of General Medicine, Chiba University Hospital. They were randomly assigned to Hybrid PBL (intervention group, n = 52) or Pure PBL group (control group, n = 47). The quantitative outcomes were measured with the students' perceived competence in PBL, satisfaction with sessions, and self-evaluation of competency in clinical reasoning. The qualitative component consisted of a content analysis on the benefits of learning clinical reasoning using Hybrid PBL. There was no significant difference between intervention and control groups in the five students' perceived competence and satisfaction with sessions. In two-way repeated measure analysis of variance, self-evaluation of competency in clinical reasoning was significantly improved in the intervention group in "recalling appropriate differential diagnosis from patient's chief complaint" (F(1,97) = 5.295, p = 0.024) and "practicing the appropriate clinical reasoning process" (F(1,97) = 4.016, p = 0.038). According to multiple comparisons, the scores of "recalling appropriate history, physical examination, and tests on clinical hypothesis generation" (F(1,97) = 6.796, p = 0.011), "verbalizing and reflecting appropriately on own mistakes," (F(1,97) = 4.352, p = 0.040) "selecting keywords from the whole aspect of the patient," (F(1,97) = 5.607, p = 0.020) and "examining the patient while visualizing his/her daily life" (F(1,97) = 7.120, p = 0.009) were significantly higher in the control group. In the content analysis, 13 advantage categories of Hybrid PBL were extracted. In the subcategories, "acquisition of knowledge" was the most frequent subcategory, followed by "leading the discussion," "smooth discussion," "getting feedback," "timely feedback," and "supporting the clinical reasoning process." Hybrid PBL can help acquire practical knowledge and deepen understanding of clinical reasoning, whereas Pure PBL can improve several important skills such as verbalizing and reflecting on one's own errors and selecting appropriate keywords from the whole aspect of the patient.

Defining the observable processes of patient care related to social determinants of health

INTRODUCTION

An understanding of social determinants of health (SDH) and patients' social circumstances is recommended to deliver contextualised care. However, the processes of patient care related to SDH in clinical settings have not been described in detail. Observable practice activities (OPAs) are a collection of learning objectives and activities that must be observed in daily practice and can be used to describe the precise processes for professionals to follow in specific situations (process OPA.) METHODS: We used a modified Delphi technique to generate expert consensus about the process OPA for patient care related to SDH in primary care settings. To reflect the opinions of various stakeholders, the expert panel comprised clinical professionals (physicians, nurses, public health nurses, social workers, pharmacists and medical clerks), residents, medical students, researchers (medical education, health care, sociology of marginalised people), support members for marginalised people and patients. The Delphi rounds were conducted online. In Round 1, a list of potentially important steps in the processes of care was distributed to panellists. The list was modified, and one new step was added. In Round 2, all steps were acknowledged with few modifications.

RESULTS

Of 63 experts recruited, 61 participated, and all participants completed the Delphi rounds. A total of 14 observable steps were identified, which were divided into four components: communication, practice, maintenance and advocacy. The importance of ongoing patient-physician relationships and collaboration with professionals and stakeholders was emphasised for the whole process of care.

DISCUSSION

This study presents the consensus of a variety of experts on the process OPA for patient care related to SDHs. Further research is warranted to investigate how this Communication-Practice-Maintenance-Advocacy framework could affect medical education, quality of patient care, and patient outcomes.

Planning a Collection of Virtual Patients to Train Clinical Reasoning: A Blueprint Representative of the European Population

Background: Virtual patients (VPs) are a suitable method for students to train their clinical reasoning abilities. We describe a process of developing a blueprint for a diverse and realistic VP collection (prior to VP creation) that facilitates deliberate practice of clinical reasoning and meets educational requirements of medical schools. Methods: An international and interdisciplinary partnership of five European countries developed a blueprint for a collection of 200 VPs in four steps: (1) Defining the criteria (e.g., key symptoms, age, sex) and categorizing them into disease-, patient-, encounter- and learner-related, (2) Identifying data sources for assessing the representativeness of the collection, (3) Populating the blueprint, and (4) Refining and reaching consensus. Results: The blueprint is publicly available and covers 29 key symptoms and 176 final diagnoses including the most prevalent medical conditions in Europe. Moreover, our analyses showed that the blueprint appears to be representative of the European population. Conclusions: The development of the blueprint required a stepwise approach, which can be replicated for the creation of other VP or case collections. We consider the blueprint an appropriate starting point for the actual creation of the VPs, but constant updating and refining is needed.