Content validity evidence for a simulation-based test of handheld otoscopy skills

Comparative validity of three simulation platforms for objective assessment of otoscopy skills

OBJECTIVES

To assess the face, construct and content validity of three different platforms for otoscopy skills assessment, using a traditional otoscope with manikin, digital otoscope (Tympahealth) with manikin, and traditional otoscope with a low-cost model ear (SimEar).

DESIGN

Prospective mixed methods study.

SETTING

Tertiary hospital.

PARTICIPANTS

Postgraduate trainees and expert assessors.

MAIN OUTCOME MEASURES

Face and Content validity based on expert assessor ranking on each model and their feedback from semi-structured interviews. Construct validity based on Objective Structured Clinical Examination scores.

RESULTS

Each platform differed in face, construct and content validity scores, with no one platform consistently outperforming others. Three main themes were identified during thematic analysis of expert assessor interviews: ability to assess what is seen, anatomical reality, and ease of use. The low-cost model showed greatest potential, where modification to include a silicone ear could lead to high validity with marginal increase in cost.

CONCLUSION

Several modalities for assessing otoscopy skills exist, each with advantages and disadvantages. Modifications to a low-cost model, for use with either a traditional or digital otoscope, could prove to be the best model.

Smartphone-Enabled versus Conventional Otoscopy in Detecting Middle Ear Disease: A Meta-Analysis

Traditional otoscopy has some limitations, including poor visualization and inadequate time for evaluation in suboptimal environments. Smartphone-enabled otoscopy may improve examination quality and serve as a potential diagnostic tool for middle ear diseases using a telemedicine approach. The main objectives are to compare the correctness of smartphone-enabled otoscopy and traditional otoscopy and to evaluate the diagnostic confidence of the examiner via meta-analysis. From inception through 20 January 2022, the Cochrane Library, PubMed, EMBASE, Web of Science, and Scopus databases were searched. Studies comparing smartphone-enabled otoscopy with traditional otoscopy regarding the outcome of interest were eligible. The relative risk (RR) for the rate of correctness in diagnosing ear conditions and the standardized mean difference (SMD) in diagnostic confidence were extracted. Sensitivity analysis and trial sequential analyses (TSAs) were conducted to further examine the pooled results. Study quality was evaluated by using the revised Cochrane risk of bias tool 2. Consequently, a total of 1840 examinees were divided into the smartphone-enabled otoscopy group and the traditional otoscopy group. Overall, the pooled result showed that smartphone-enabled otoscopy was associated with higher correctness than traditional otoscopy (RR, 1.26; 95% CI, 1.06 to 1.51; p = 0.01; I² = 70.0%). Consistently significant associations were also observed in the analysis after excluding the simulation study (RR, 1.10; 95% CI, 1.00 to 1.21; p = 0.04; I² = 0%) and normal ear conditions (RR, 1.18; 95% CI, 1.01 to 1.40; p = 0.04; I² = 65.0%). For the confidence of examiners using both otoscopy methods, the pooled result was nonsignificant between the smartphone-enabled otoscopy and traditional otoscopy groups (SMD, 0.08; 95% CI, -0.24 to 0.40; p = 0.61; I² = 16.3%). In conclusion, smartphone-enabled otoscopy was associated with a higher rate of correctness in the detection of middle ear diseases, and in patients with otologic complaints, the use of smartphone-enabled otoscopy may be considered. More large-scale studies should be performed to consolidate the results.

Undergraduate audiology students' perceived competence and confidence in conducting otoscopic examination following video otoscopic training

BACKGROUND

Emerging research indicates that video otoscopy can be used as a teaching tool to enhance students' ability to identify outer and middle ear pathologies. However, there is little research on the perceptions of audiology students regarding their competence and confidence following video otoscopic training, and how they view the use of video otoscopy as a teaching tool. Therefore, this study aimed to determine undergraduate (UG) audiology students' perceived competence and confidence in conducting otoscopy following training by video otoscopic examination.

METHODS

A survey methodology with a cross sectional design was employed. An electronic questionnaire was distributed to all third and fourth year (senior) (N = 79) UG audiology students using Survey Monkey. Ethical approval and permission from relevant stakeholders were obtained. Data were analysed using both descriptive and inferential statistics.

RESULTS

60 % of the students felt competent in performing otoscopy, while 63.3 % felt less competent in interpreting otoscopic examination findings. 43.3 % felt they can confidently and competently identify outer ear pathologies. There was no association between the number of video otoscopic examinations performed and perceived competence or/and confidence. There was also no statistically significant relationship between year of study (e.g., third year versus fourth year) and perceived competence or/and confident (p = 0.7131). Almost all (97 %) students felt that video otoscopic training should continue to be part of the clinical training as it helped them enhance their skills in performing otoscopy.

CONCLUSIONS

Current findings highlight the need to improve students' practical training, incorporating pathologic ears into the curriculum. These findings also highlight the importance of supplementing practical training methodologies with changing technological advancements, particularly where tele-audiology opportunities may exist.

Face, Content, and Construct Validity of a Virtual Reality Otoscopy Simulator and Applicability to Medical Training

Objective

Otologic diseases are common in all age groups and can significantly impair the function of this important sensory organ. To make a correct diagnosis, the correct handling of the otoscope and a correctly performed examination are essential. A virtual reality simulator could make it easier to teach this difficult-to-teach skill. The aim of this study was to assess the face, content, and construct validity of the novel virtual reality otoscopy simulator and the applicability to otologic training.

Study Design

Face and content validity was assessed with a questionnaire. Construct validity was assessed in a prospectively designed controlled trial.

Setting

Training for medical students at a tertiary referral center.

Method

The questionnaire used a 6-point Likert scale. The otoscopy was rated with a modified Objective Structured Assessment of Technical Skills. Time to complete the task and the percentage of the assessed eardrum surface were recorded.

Results

The realism of the simulator and the applicability to medical training were assessed across several items. The ratings suggested good face and content validity as well as usefulness and functionality of the simulator. The otolaryngologists significantly outperformed the student group in all categories measured (P < .0001), suggesting construct validity of the simulator.

Conclusion

In this study, we could demonstrate face, content, and construct validity for a novel high-fidelity virtual reality otoscopy simulator. The results encourage the use of the otoscopy simulator as a complementary tool to traditional teaching methods in a curriculum for medical students.