A survey comparison of educational interventions for teaching pneumatic otoscopy to medical students

How to design effective educational videos for teaching evidence-based medicine to undergraduate learners - systematic review with complementing qualitative research to develop a practicable guide

BACKGROUND

eLearning can be an effective tool to achieve learning objectives. It facilitates asynchronous distance learning, increasing flexibility for learners and instructors. In this context, the high educational value of videos provides an invaluable primary component for longitudinal digital curricula, especially for maintaining knowledge on otherwise rarely taught subjects. Although literature concerning eLearning evaluation exists, research comprehensively describing how to design effective educational videos is lacking. In particular, studies on the requirements and design goals of educational videos need to be complemented by qualitative research using grounded theory methodology.

METHODS

Due to the paucity of randomized controlled trials in this area, there is an urgent need to generate recommendations based on a broader fundament than a literature search alone. Thus, the authors have employed grounded theory as a guiding framework, augmented by Mayring's qualitative content analysis and commonly used standards. An adaptive approach was conducted based on a literature search and qualitative semi-structured interviews. Drawing on these results, the authors elaborated a guide for creating effective educational videos.

RESULTS

The authors identified 40 effective or presumedly effective factors fostering the success of video-based eLearning in teaching evidence-based medicine, providing a ready-to-use checklist. The information collected via the interviews supported and enriched much of the advice found in the literature.

DISCUSSION

To the authors' knowledge, this type of comprehensive guide for video-based eLearning needs has not previously been published. The interviews considerably contributed to the results. Due to the grounded theory-based approach, in particular, consensus was achieved without the presence of a formal expert panel. Although the guide was created with a focus on teaching evidence-based medicine, due to the general study selection process and research approach, the recommendations are applicable to a wide range of subjects in medical education where the teaching aim is to impart conceptual knowledge.

Artificial intelligence as an auxiliary tool in pediatric otitis media diagnosis

OBJECTIVES

In order to promote the use of AI technology as the auxiliary tool in pediatric otitis media diagnosis, we use the convolutional neural networks and deep learning for image classification and disease diagnosis. We also designed a Pediatric Otitis Media Classifier to analyze and classify the images for physicians.

METHODS

A pediatric otitis media classifier was designed for junior physicians (doctors who have been engaged in clinical practice for a short time) as an auxiliary diagnostic tool. To design this classifier for children with otitis media, we used a large number of images of acute otitis media (AOM), secretory otitis media (OME), and normal otoscope images to obtain the optimal convolutional neural network model.

RESULTS

The average recognition accuracies of the ZFNet and the TSL16 for classification were 97.87 % and 97.62 %, far exceeding the accuracy of human diagnosis. The results of using the Pediatric Otitis Media Classifier show that we can use the classifier to correctly identify the image types of child middle ear infections.

CONCLUSIONS

We developed the Pediatric Otitis Media Classifier for the successful automated classification of AOM and OME in children using otoscopic images. In contrast to the traditional diagnosis of pediatric otitis media, which relies heavily on the experience of doctors, the diagnostic accuracy of even experienced physicians is only approximately 80 %. With AI technology, we can improve the accuracy rate to over 98 %, which can effectively assist doctors in auxiliary diagnosis. It also reduces delayed treatment, antibiotic misuse, and unnecessary surgery caused by misdiagnosis.

Utilization of Video Otoscopes for Otoscopy Skills Training of Third Year Medical Students

Purpose

Effective teaching and assessment of otologic examinations are challenging. Current methods of teaching otoscopy using traditional otoscopes have significant limitations. We hypothesized that use of all-in-one video otoscopes provides students with an opportunity for real-time faculty feedback and re-practicing of skills, increasing self-reported confidence.

Methods

An otoscopy microskills competency checklist was provided to third-year medical students during their pediatric clerkship to self-assess otoscopy technique during patient examinations, and to clinical preceptors to assess and provide feedback during exams. Over the course of two years, we collected data from students randomly assigned to train on a video otoscope or a traditional otoscope during the clerkship. Pre- and post-clerkship surveys measured confidence in performing otoscopy microskills, making a diagnosis and documentation of findings. For those students who trained on the video otoscope, we solicited post-clerkship feedback on the experience of using a video otoscope.

Results

Pre-clerkship confidence did not differ between the groups, but the video otoscope trained group had significantly higher scores than the traditional otoscope trained group on all self-reported technical and diagnostic microskills confidence questions items post-clerkship. Students trained on video otoscopes had a significant increase in confidence with all microskills items (p-values<0.001), however confidence in the traditional otoscope trained group did not change over time (p-values>0.10). Qualitative feedback from the video otoscope trained group reflected positive experiences regarding "technique/positioning" and "feedback from preceptors.".

Conclusion

Teaching otoscopy skills to pediatric clerkship medical students using a video otoscope significantly enhanced confidence compared to those training on a traditional otoscope by 1. enabling preceptors and students to simultaneously visualize otoscopy findings 2. allowing preceptors to provide real-time feedback and 3. providing opportunity for deliberate practice of microskills. We encourage the use of video otoscopes to augment student confidence and self-efficacy when training in otoscopy.

Comparative validity of three simulation platforms for objective assessment of otoscopy skills

OBJECTIVES

To assess the face, construct and content validity of three different platforms for otoscopy skills assessment, using a traditional otoscope with manikin, digital otoscope (Tympahealth) with manikin, and traditional otoscope with a low-cost model ear (SimEar).

DESIGN

Prospective mixed methods study.

SETTING

Tertiary hospital.

PARTICIPANTS

Postgraduate trainees and expert assessors.

MAIN OUTCOME MEASURES

Face and Content validity based on expert assessor ranking on each model and their feedback from semi-structured interviews. Construct validity based on Objective Structured Clinical Examination scores.

RESULTS

Each platform differed in face, construct and content validity scores, with no one platform consistently outperforming others. Three main themes were identified during thematic analysis of expert assessor interviews: ability to assess what is seen, anatomical reality, and ease of use. The low-cost model showed greatest potential, where modification to include a silicone ear could lead to high validity with marginal increase in cost.

CONCLUSION

Several modalities for assessing otoscopy skills exist, each with advantages and disadvantages. Modifications to a low-cost model, for use with either a traditional or digital otoscope, could prove to be the best model.

"Human vs Machine" Validation of a Deep Learning Algorithm for Pediatric Middle Ear Infection Diagnosis

OBJECTIVE

We compared the diagnostic performance of human clinicians with that of a neural network algorithm developed using a library of tympanic membrane images derived from children taken to the operating room with the intent of performing myringotomy and possible tube placement for recurrent acute otitis media (AOM) or otitis media with effusion (OME).

STUDY DESIGN

Retrospective cohort study.

SETTING

Tertiary academic medical center from 2018 to 2021.

METHODS

A training set of 639 images of tympanic membranes representing normal, OME, and AOM was used to train a neural network as well as a proprietary commercial image classifier from Google. Model diagnostic prediction performance in differentiating normal vs nonpurulent vs purulent effusion was scored based on classification accuracy. A web-based survey was developed to test human clinicians' diagnostic accuracy on a novel image set, and this was compared head to head against our model.

RESULTS

Our model achieved a mean prediction accuracy of 80.8% (95% CI, 77.0%-84.6%). The Google model achieved a prediction accuracy of 85.4%. In a validation survey of 39 clinicians analyzing a sample of 22 endoscopic ear images, the average diagnostic accuracy was 65.0%. On the same data set, our model achieved an accuracy of 95.5%.

CONCLUSION

Our model outperformed certain groups of human clinicians in assessing images of tympanic membranes for effusions in children. Reduced diagnostic error rates using machine learning models may have implications in reducing rates of misdiagnosis, potentially leading to fewer missed diagnoses, unnecessary antibiotic prescriptions, and surgical procedures.

Artificial intelligence to diagnose ear disease using otoscopic image analysis: a review

AI relates broadly to the science of developing computer systems to imitate human intelligence, thus allowing for the automation of tasks that would otherwise necessitate human cognition. Such technology has increasingly demonstrated capacity to outperform humans for functions relating to image recognition. Given the current lack of cost-effective confirmatory testing, accurate diagnosis and subsequent management depend on visual detection of characteristic findings during otoscope examination. The aim of this manuscript is to perform a comprehensive literature review and evaluate the potential application of artificial intelligence for the diagnosis of ear disease from otoscopic image analysis.

The Development, Implementation, and Evaluation of an Acute Otitis Media Education Website

OBJECTIVE

To develop, implement, and evaluate an acute otitis media (AOM) education website for clinician-educators.

METHODS

We developed an education website following Kern's curriculum model.

RESULTS

The website contained peer-reviewed content, educational objectives, library search pages to identify evidence-based resources, and a faculty toolbox with instructional and evaluation instruments. Pediatric clinician-educators were purposefully sampled from different clinic sites to evaluate the website. Semistructured interviews explored key website components for content and usability in clinical teaching. In grounded theory tradition, investigators used the constant comparative method with qualitative analysis software to identify themes and representative quotations. Eleven faculty members (9 females and 2 males with teaching experience from 6 to 26 years) participated in the study. Identified themes were: 1) value of visual impact for learning specific topics, 2) promotion of efficiency in teaching clinical topics, 3) varying approaches for using website, and 4) faculty's self-report of knowledge and self-efficacy needs.

CONCLUSIONS

An education website may enhance the teaching of AOM, accommodate different teaching preferences, promote efficiency in teaching, and advance clinician-educator knowledge and skills. Next steps include evaluation of learners' perspectives, generalizability in varied teaching settings, and assessment of higher learning outcomes including impact on knowledge, skills, and patient outcomes.

Machine Learning for Accurate Intraoperative Pediatric Middle Ear Effusion Diagnosis

OBJECTIVES

Misdiagnosis of acute and chronic otitis media in children can result in significant consequences from either undertreatment or overtreatment. Our objective was to develop and train an artificial intelligence algorithm to accurately predict the presence of middle ear effusion in pediatric patients presenting to the operating room for myringotomy and tube placement.

METHODS

We trained a neural network to classify images as " normal" (no effusion) or "abnormal" (effusion present) using tympanic membrane images from children taken to the operating room with the intent of performing myringotomy and possible tube placement for recurrent acute otitis media or otitis media with effusion. Model performance was tested on held-out cases and fivefold cross-validation.

RESULTS

The mean training time for the neural network model was 76.0 (SD ± 0.01) seconds. Our model approach achieved a mean image classification accuracy of 83.8% (95% confidence interval [CI]: 82.7-84.8). In support of this classification accuracy, the model produced an area under the receiver operating characteristic curve performance of 0.93 (95% CI: 0.91-0.94) and F1-score of 0.80 (95% CI: 0.77-0.82).

CONCLUSIONS

Artificial intelligence-assisted diagnosis of acute or chronic otitis media in children may generate value for patients, families, and the health care system by improving point-of-care diagnostic accuracy. With a small training data set composed of intraoperative images obtained at time of tympanostomy tube insertion, our neural network was accurate in predicting the presence of a middle ear effusion in pediatric ear cases. This diagnostic accuracy performance is considerably higher than human-expert otoscopy-based diagnostic performance reported in previous studies.

Current Status of Handheld Otoscopy Training: A Systematic Review

OBJECTIVE

Otoscopy is a frequently performed procedure and competency in this skill is important across many specialties. We aim to systematically review current medical educational evidence for training of handheld otoscopy skills.

METHODS

Following the PRISMA guideline, studies reporting on training and/or assessment of handheld otoscopy were identified searching the following databases: PubMed, Embase, OVID, the Cochrane Library, PloS Medicine, Directory of Open Access Journal (DOAJ), and Web of Science. Two reviewers extracted data on study design, training intervention, educational outcomes, and results. Quality of educational evidence was assessed along with classification according to Kirkpatrick's model of educational outcomes.

RESULTS

The searches yielded a total of 6064 studies with a final inclusion of 33 studies for the qualitative synthesis. Handheld otoscopy training could be divided into workshops, physical simulators, web-based training/e-learning, and smartphone-enabled otoscopy. Workshops were the most commonly described educational intervention and typically consisted of lectures, hands-on demonstrations, and training on peers. Almost all studies reported a favorable effect on either learner attitude, knowledge, or skills. The educational quality of the studies was reasonable but the educational outcomes were mostly evaluated on the lower Kirkpatrick levels with only a single study determining the effects of training on actual change in the learner behavior.

CONCLUSION

Overall, it seems that any systematic approach to training of handheld otoscopy is beneficial in training regardless of learner level, but the heterogeneity of the studies makes comparisons between studies difficult and the relative effect sizes of the interventions could not be determined.

Teaching pediatric otoscopy skills to the medical student in the clinical setting: preceptor perspectives and practice

BACKGROUND

Acute otitis media (AOM) is the most frequent indication for antibiotic treatment of children in the United States. Its diagnosis relies on visualization of the tympanic membrane, a clinical skill acquired through a deliberate approach. Instruction in pediatric otoscopy begins in medical school. Medical students receive their primary experience with pediatric otoscopy during the required pediatric clerkship, traditionally relying on an immersion, apprentice-type learning model. A better understanding of their preceptors' clinical and teaching practices could lead to improved skill acquisition. This study investigates how pediatric preceptors (PP) and members of the Council on Medical Student Education in Pediatrics (COMSEP) perceive teaching otoscopy.

METHODS

A 30-item online survey was administered to a purposeful sample of PP at six institutions in 2017. A comparable 23-item survey was administered to members through the 2018 COMSEP Annual Survey. Only COMSEP members who identified themselves as teaching otoscopy to medical students were asked to complete the otoscopy-related questions on the survey.

RESULTS

Survey respondents included 58% of PP (180/310) and 44% (152/348) of COMSEP members. Forty-one percent (62/152) of COMSEP member respondents identified themselves as teaching otoscopy and completed the otoscopy-related questions. The majority agreed that standardized curricula are needed (PP 78%, COMSEP members 97%) and that all graduating medical students should be able to perform pediatric otoscopy (PP 95%, COMSEP members 79%). Most respondents reported usefulness of the American Academy of Pediatrics (AAP) AOM guidelines (PP 95%, COMSEP members 100%). More COMSEP members than PP adhered to the AAP's diagnostic criteria (pediatric preceptors 42%, COMSEP members 93%). The most common barriers to teaching otoscopy were a lack of assistive technology (PP 77%, COMSEP members 56%), presence of cerumen (PP 58%, COMSEP members 60%), time to teach in direct patient care (PP 46%, COMSEP members 48%), and parent anxiety (PP 62%, COMSEP members 54%).

CONCLUSIONS

Our study identified systemic and individual practice patterns and barriers to teaching pediatric otoscopy. These results can inform education leaders in supporting and enabling preceptors in their clinical teaching. This approach can be adapted to ensure graduating medical students obtain intended core clinical skills.

Low-fidelity otoscopy simulation and anatomy training: A randomized controlled trial

OBJECTIVES

To evaluate whether the use of low-fidelity otoscopy simulation improved medical students' theoretical knowledge of middle ear anatomy and pathologies compared to traditional teaching methods.

METHODS

This was a randomized controlled trial. Simulation workshops were conducted in April 2019 in the Lyon Sud University medical faculty, France. Students were randomly assigned to the simulation group (n=105) or to the control group (n=95). The students in the control group answered a questionnaire evaluating theoretical knowledge (25 true-false questions) before the simulation tutorial, while the students in the simulation group answered the same questions after the tutorial. Both groups also filled out a satisfaction questionnaire for feedback.

RESULTS

196 of the 200 students who participated in the study completed the knowledge assessment questionnaire. Scores were 32.0% higher in the simulation group than in the control group (mean scores, 12.0/20 vs. 9.1/20; P<0.0001). 184 of the 191 students who completed the satisfaction questionnaire (96.3%) were satisfied or very satisfied with the workshop, and all but one (99.5%) recommended keeping it in the curriculum. In the free comments fields, students highlighted the educational value of learning without the stress of patient discomfort.

CONCLUSION

Otoscopy simulation is an effective training method, improving theoretical knowledge compared with conventional theoretical training.