Case records of the Massachusetts General Hospital--continuing to learn from the patient

Evaluation of large language models as a diagnostic aid for complex medical cases

Background

The use of large language models (LLM) has recently gained popularity in diverse areas, including answering questions posted by patients as well as medical professionals.

Objective

To evaluate the performance and limitations of LLMs in providing the correct diagnosis for a complex clinical case.

Design

Seventy-five consecutive clinical cases were selected from the Massachusetts General Hospital Case Records, and differential diagnoses were generated by OpenAI's GPT3.5 and 4 models.

Results

The mean number of diagnoses provided by the Massachusetts General Hospital case discussants was 16.77, by GPT3.5 30 and by GPT4 15.45 (p < 0.0001). GPT4 was more frequently able to list the correct diagnosis as first (22% versus 20% with GPT3.5, p = 0.86), provide the correct diagnosis among the top three generated diagnoses (42% versus 24%, p = 0.075). GPT4 was better at providing the correct diagnosis, when the different diagnoses were classified into groups according to the medical specialty and include the correct diagnosis at any point in the differential list (68% versus 48%, p = 0.0063). GPT4 provided a differential list that was more similar to the list provided by the case discussants than GPT3.5 (Jaccard Similarity Index 0.22 versus 0.12, p = 0.001). Inclusion of the correct diagnosis in the generated differential was correlated with PubMed articles matching the diagnosis (OR 1.40, 95% CI 1.25-1.56 for GPT3.5, OR 1.25, 95% CI 1.13-1.40 for GPT4), but not with disease incidence.

Conclusions and relevance

The GPT4 model was able to generate a differential diagnosis list with the correct diagnosis in approximately two thirds of cases, but the most likely diagnosis was often incorrect for both models. In its current state, this tool can at most be used as an aid to expand on potential diagnostic considerations for a case, and future LLMs should be trained which account for the discrepancy between disease incidence and availability in the literature.

Can clinical case discussions foster clinical reasoning skills in undergraduate medical education? A randomised controlled trial

OBJECTIVE

Fostering clinical reasoning is a mainstay of medical education. Based on the clinicopathological conferences, we propose a case-based peer teaching approach called clinical case discussions (CCDs) to promote the respective skills in medical students. This study compares the effectiveness of different CCD formats with varying degrees of social interaction in fostering clinical reasoning.

DESIGN, SETTING, PARTICIPANTS

A single-centre randomised controlled trial with a parallel design was conducted at a German university. Study participants (N=106) were stratified and tested regarding their clinical reasoning skills right after CCD participation and 2 weeks later.

INTERVENTION

Participants worked within a live discussion group (Live-CCD), a group watching recordings of the live discussions (Video-CCD) or a group working with printed cases (Paper-Cases). The presentation of case information followed an admission-discussion-summary sequence.

PRIMARY AND SECONDARY OUTCOME MEASURES

Clinical reasoning skills were measured with a knowledge application test addressing the students' conceptual, strategic and conditional knowledge. Additionally, subjective learning outcomes were assessed.

RESULTS

With respect to learning outcomes, the Live-CCD group displayed the best results, followed by Video-CCD and Paper-Cases, F(2,87)=27.07, p<0.001, partial η²=0.384. No difference was found between Live-CCD and Video-CCD groups in the delayed post-test; however, both outperformed the Paper-Cases group, F(2,87)=30.91, p<0.001, partial η²=0.415. Regarding subjective learning outcomes, the Live-CCD received significantly better ratings than the other formats, F(2,85)=13.16, p<0.001, partial η²=0.236.

CONCLUSIONS

This study demonstrates that the CCD approach is an effective and sustainable clinical reasoning teaching resource for medical students. Subjective learning outcomes underline the importance of learner (inter)activity in the acquisition of clinical reasoning skills in the context of case-based learning. Higher efficacy of more interactive formats can be attributed to positive effects of collaborative learning. Future research should investigate how the Live-CCD format can further be improved and how video-based CCDs can be enhanced through instructional support.

Clinical presentation, staging and long-term evolution of parathyroid cancer

BACKGROUND

Parathyroid cancer is rare and often fatal. This review provides an in-depth analysis of 330 clinical cases reported in detail. These data are used to inform a proposal for a hitherto lacking TNM staging system.

MATERIALS AND METHODS

All case reports or series with sufficient case details of parathyroid cancer were identified from PubMed, and data were analyzed using SPSS.

RESULTS

Of 330 patients, 117 (35%) died of disease and 207 (63%) experienced recurrence in a total of 2007 follow-up years and a mean length of follow-up of 6.1 years. Histopathology findings rather than biochemical or clinical features predict outcome. In univariate analysis, survival and recurrence rates are significantly influenced by gender (male relative risk [RR] 1.7, 95% confidence interval [95% CI] 1.0-2.7, P < .01), and presence of vascular invasion (RR 4.3, 95% CI 1.1-17.7, P < .01), or lymph node metastases (RR 6.2, 95 %CI 0.9-42.9, P < .001). Failure to perform oncological surgery carries a high risk for recurrence and death (local versus en bloc resection RR 2.0, CI 1.2-3.2, P < .01) as for redo surgery. Staging by a novel anatomy-based TNM system identifies significant outcome variation as to recurrence and death. Separation of patients into low and high risk identifies a 3.5-7.0 fold higher risk of recurrence and death (P < .01) for the high-risk group. Distant metastases predominantly target mediastinum and lung.

CONCLUSION

Understaging and undertreatment are shown to contribute to high recurrence rates and death toll. To improve outcome, en bloc resection including central lymph node dissection should be the minimal surgical approach in any patient with suspected parathyroid cancer.

Viewpoint: leaving the "empty glass" of problem-based learning behind: new assumptions and a revised model for case study in preclinical medical education

The popularity of problem-based learning (PBL) reflects medical educators' recognition that case study can enhance the preclinical medical school curriculum. However, the PBL method itself has features, particularly its reliance on small-group work with tutor-facilitators, that are expensive to implement and that limit the potential educational value of case study. The author systematically analyzes specific aspects of the PBL methodology and concludes that the PBL approach misuses the faculty, tends to compromise the authenticity of cases, and results in an unnecessarily varied and impoverished educational experience for students. Approaches to case study with different assumptions need to be devised. A model is proposed that shifts the goal of case study from development of problem-solving skills to development of ideas that allow meaningful engagement in sophisticated discussions of medicine. In this model, the method shifts from self-directed learning to independent study guided by the expertise of the faculty. One possible approach to case study based on this model is briefly described. It consists of reading published cases from the medical literature, with analysis and discussion of the cases led by faculty experts in large-group format. The approach immerses students in an authentic, state-of-the-art discussion of medicine and is easily incorporated into any curriculum structure at limited cost. The author argues that, contrary to the claims of proponents, the glass is "mostly empty" for PBL and that we can generate the higher-level discussion that case study merits only by moving away from PBL's extraneous assumptions.

Doctors – A species on the verge of extinction?

Medicine is undergoing profound change, but the basic format of the medical encounter has remained unchanged. Nevertheless, medicine in the 22nd century may be fully computerized, and a possible model is shortly depicted in this paper. Computer applications are constantly increasing their share in medical diagnosis, and may ultimately replace physicians. Treatment decisions have been submitted to standardized treatment guidelines, which may be applied more efficiently by computer applications. Although hundreds of studies have evaluated computerized tools in diagnosis and treatment, the possibility that computer applications may replace human physicians in the future is rarely raised. The effects of this process on doctors and medicine may be tremendous and will probably be felt even in early stages, and therefore, this process should be a subject of open discussion.