Assessment of a Simulated Case-Based Measurement of Physician Diagnostic Performance

Automating hybrid collective intelligence in open-ended medical diagnostics

Collective intelligence has emerged as a powerful mechanism to boost decision accuracy across many domains, such as geopolitical forecasting, investment, and medical diagnostics. However, collective intelligence has been mostly applied to relatively simple decision tasks (e.g., binary classifications). Applications in more open-ended tasks with a much larger problem space, such as emergency management or general medical diagnostics, are largely lacking, due to the challenge of integrating unstandardized inputs from different crowd members. Here, we present a fully automated approach for harnessing collective intelligence in the domain of general medical diagnostics. Our approach leverages semantic knowledge graphs, natural language processing, and the SNOMED CT medical ontology to overcome a major hurdle to collective intelligence in open-ended medical diagnostics, namely to identify the intended diagnosis from unstructured text. We tested our method on 1,333 medical cases diagnosed on a medical crowdsourcing platform: The Human Diagnosis Project. Each case was independently rated by ten diagnosticians. Comparing the diagnostic accuracy of single diagnosticians with the collective diagnosis of differently sized groups, we find that our method substantially increases diagnostic accuracy: While single diagnosticians achieved 46% accuracy, pooling the decisions of ten diagnosticians increased this to 76%. Improvements occurred across medical specialties, chief complaints, and diagnosticians' tenure levels. Our results show the life-saving potential of tapping into the collective intelligence of the global medical community to reduce diagnostic errors and increase patient safety.

A Pilot Study on the Feasibility of Developing and Implementing a Mobile App for the Acquisition of Clinical Knowledge and Competencies by Medical Students Transitioning from Preclinical to Clinical Years

Due to the COVID-19 pandemic and the consequent restrictions, universities have had to adapt their curricula substantially to new schemes in which remote learning is of the essence. In this study, we assess the feasibility of developing a mobile app supplementary to the distant teaching paradigm for the “Cardiology” module of the “General Pathology” subject in undergraduate Medical Education, and we evaluate its impact and acceptability. A cohort of volunteer second-year medical students (n = 44) had access to the app, and their opinions on its utility (1−10) were collected. Additionally, the students were invited to refer their expected satisfaction (1−10) with a blended learning methodology overlapping this new tool with the traditional resources. The average expected satisfaction had been compared to the average satisfaction obtained by just the traditional methodology in other modules from the same subject. Through a qualitative approach, we defined the strengths and weaknesses of the tool. Seventy-seven percent of the participants rated at 8/10 or more the potential learning value of the application and, if used as a supplement to traditional teaching, it would also statistically improve the satisfaction of students (6.52 vs. 8.70, p < 0.001). Similarly, the qualitative data corroborated the benefits of such innovation. Multidisciplinary collaborations are encouraged to develop teaching innovations, although further research should aim to better define the effectiveness of learning with these resources.

Improving diagnosis by feedback and deliberate practice: one-on-one coaching for diagnostic maturation

Reflective practice is essential for the ongoing maturation of clinicians and requires regular self-evaluation in association with ongoing mentoring and feedback. Currently, most resident physicians do not have access to educational experiences that fulfill these needs. We present a novel model for structured one-on-one longitudinal coaching using the principles of deliberate practice to improve diagnostic skills. This is an easily implementable educational model that can be replicated in residencies across the country to improve clinical reasoning. Skills learned through this program have the potential not only to bolster the academic approach to patients but to also directly improve the clinical assessment and care of patients under the trainee's care.

Twelve tips for creating a medical education podcast

Podcasts and other asynchronous remote learning pedagogies have become increasingly utilized in undergraduate and graduate medical education. They offer a convenient and accessible form of education for personalized learning and have been developed across multiple disciplines of medicine. This paper discusses the current literature on medical education podcasts, offers practical examples of existing free podcast content, and offers guidance on planning, designing, and executing the creation of a new educational podcast. It discusses best practices for the engagement of learners and logistical considerations for recording and hosting the material. These Twelve Tips hope to empower educators to utilize resources across the world to offer the highest quality opportunities for learners of all levels.

Impact of take-home messages written into slide presentations delivered during lectures on the retention of messages and the residents' knowledge: a randomized controlled study

BACKGROUND

Lectures with slide presentations are widely used to teach evidence-based medicine to large groups. Take-home messages (THMs) are poorly identified and recollected by students. We investigated whether an instruction to list THMs in written form on slides would improve the retention thereof by residents, and the residents' level of knowledge, 1 month after lectures.

METHODS

Prospective blinded randomized controlled study was conducted. Twelve lectures (6 control and 6 intervention lectures) were delivered to 73 residents. For the intervention lectures, the lecturers were instructed to incorporate clear written THMs into their slide presentations. The outcomes were ability of resident to recollect THMs delivered during a lecture (as assessed by accordance rate between the lecturers' and residents' THMs) and knowledge (as assessed by multiple choice questions (MCQs)).

RESULTS

Data for 3738 residents' THMs and 3410 MCQs were analyzed. The intervention did not significantly increase the number of THMs written on slides (77% (n = 20/26), 95% CI 56-91 vs 64% (n = 18/28), 95% CI 44-81, p = 0.31) nor THMs retention (13% (n = 238/1791), 95% CI 12-15 vs 17% (n = 326/1947), 95% 15-18, p = 0.40) nor knowledge (63.8 ± 26.2 vs 61.1 ± 31.4 /100 points, p = 0.75). In multivariable analyses performed with all THMs written on slides from the two groups, a superior knowledge was associated with notetaking during lectures (OR 1.88, 95% CI 1.41-2.51) and THMs retention (OR 2.17, 95% CI 1.54-3.04); and THMs retention was associated with written THMs (OR 2.94, 95% CI 2.20-3.93).

CONCLUSIONS

In lectures delivered to residents, a third of the THMs were not in written form. An intervention based on an explicit instruction to lecturers to provide THMs in written form in their slide presentations did not result in increased use of written THMs into the slide presentation or improvement of the THMs retention or level of knowledge. However, we showed that there was a strong positive association between writing THMs on a slide, retention of THMs and residents' knowledge. Further researches are needed to assess interventions to increase written THMs in lectures by faculty.

TRIAL REGISTRATION

ClinicalTrials.gov NCT01795651 (Fev 21, 2013).

The genealogy of teaching clinical reasoning and diagnostic skill: the GEL Study

The genealogy of graduate medical education in America begins at the bedside. However, today's graduate medical trainees work in a training environment that is vastly different from medical training a century ago. The goal of the Graduate Medical Education Laboratory (GEL) Study, supported by the American Medical Association's (AMA) "Reimagining Residency" initiative, is to determine the factors in the training environment that most contribute to resident well-being and developing diagnostic skills. We believe that increasing time at the bedside will improve clinical skill, increase professional fulfillment, and reduce workplace burnout. Our graduate medical education laboratory will test these ideas to understand which interventions can be shared among all training programs. Through the GEL Study, we aim to ensure resident readiness for practice as we understand, then optimize, the learning environment for trainees and staff.

Diagnostic errors in the neonatal intensive care unit: State of the science and new directions

Diagnostic errors remain understudied in neonatology. The limited available evidence, however, suggests that diagnostic errors in the neonatal intensive care unit (NICU) result in significant and long-term consequences. In this narrative review, we discuss how the concept of diagnostic errors framed as missed opportunities can be applied to the non-linear nature of diagnosis in a critical care environment such as the NICU. We then explore how the etiology of an error in diagnosis can be related to both individual cognitive factors as well as organizational and systemic factors - all of which often contribute to the error. This multifactorial causation has limited the development of methodology to measure diagnostic errors as well as strategies to mitigate and prevent their adverse effects. We recommend research focused on the frequency and etiology of diagnostic error in the NICU as well as potential mitigation strategies to advance this important field in neonatal intensive care.