Validating the virtual: a deep dive into ultrasound simulator metrics in otorhinolaryngology

PURPOSE

This study aimed to assess the validity of simulation-based assessment of ultrasound skills for thyroid ultrasound.

METHODS

The study collected validity evidence for simulation-based ultrasound assessment of thyroid ultrasound skills. Experts (n = 8) and novices (n = 21) completed a test containing two tasks and four cases on a virtual reality ultrasound simulator (U/S Mentor's Neck Ultrasound Module). Validity evidence was collected and structured according to Messick's validity framework. The assessments being evaluated included built-in simulator metrics and expert-based evaluations using the Objective Structured Assessment of Ultrasound Skills (OSAUS) scale.

RESULTS

Out of 64 built-in simulator metrics, 9 (14.1%) exhibited validity evidence. The internal consistency of these metrics was strong (Cronbach's α = 0.805) with high test-retest reliability (intraclass correlation coefficient = 0.911). Novices achieved an average score of 41.9% (SD = 24.3) of the maximum, contrasting with experts at 81.9% (SD = 16.7). Time comparisons indicated minor differences between experts (median: 359 s) and novices (median: 376.5 s). All OSAUS items differed significantly between the two groups. The correlation between correctly entered clinical findings and the OSAUS scores was 0.748 (p < 0.001). The correlation between correctly entered clinical findings and the metric scores was 0.801 (p < 0.001).

CONCLUSION

While simulation-based training is promising, only 14% of built-in simulator metrics could discriminate between novices and ultrasound experts. Already-established competency frameworks such as OSAUS provided strong validity evidence for the assessment of otorhinolaryngology ultrasound competence.

Is the prediction of one or two ipsilateral positive lymph nodes by computerized tomography and ultrasound reliable enough to restrict therapeutic neck dissection in oral squamous cell carcinoma (OSCC) patients?

INTRODUCTION

Proper management of the clinically involved neck in OSCC patients continues to be a matter of debate. Our aim was to analyze the accuracy of computerized tomography (CT) and ultrasound (US) in anticipating the exact location of lymph node (LN) metastases of OSCC patients across the AAO-HNS (American Academy of Otolaryngology-Head and Neck Surgery) levels ipsi- and contralaterally. Furthermore, we wanted to assess the suitability of therapeutic selective neck dissection (SND) in patients with one or two ipsilateral positive nodes upon clinical staging (cN1/cN2a and cN2b(2/x) patients).

METHODS

We prospectively analyzed the LN status of patients with primary OSCC using CT and US from 2007 to 2013. LNs were individually assigned to a map containing the AAO-HNS levels; patients bearing a single or just two ipsilateral positive nodes (designated cN1/cN2a or cN2b(2/x) patients either by CT (CT group) or US alone (US group) or in a group combining findings of CT and US (CTUS group)) received an ipsi-ND (I-V) and a contra-ND (I-IV). 78% of the LNs were sent individually for routine histopathological examination; the remaining were dissected and analyzed per neck level.

RESULTS

Upon the analysis of 1.670 LNs of 57 patients, the exact location of pathology proven LN metastases in cN1 patients was more precisely predicted by US compared to CT with confirmed findings only in levels IA, IB und IIA. Clearly decreasing the number of missed lesions, the findings in the CTUS group nearly kept the spatial reliability of the US group. The same analysis for patients with exactly two supposed ipsilateral lesions (cN2b(2/x)) yielded confirmed metastases from levels I to V for both methods individually and in combination and, therefore, render SND insufficient for these cases.

CONCLUSION

Our findings stress the importance of conducting both, CT and US, in patients with primary OSCC. Only the combination of their findings warrants the application of therapeutic SND in patients with a single ipsilateral LN metastasis (cN1/cN2a patients) but not in patients with more than one lesion upon clinical staging (≥ cN2b).

In-Office Ultrasonographic Evaluation of Neck Masses/Thyroid Nodules

Office-based ultrasonography is increasingly becoming an integral part of an otolaryngology-head and neck surgery practice. A thorough knowledge of the ultrasonic appearance of normal and abnormal pathology are key for performing/interpreting office-based head and neck ultrasonography. A focused but systematic approach allows for efficient and effective office-based head and neck ultrasonography. Office-based ultrasonography also allows for imaging procedures expanding the otolaryngologist's armamentarium. Ultrasound-guided fine needle aspiration (USgFNA) is an integral part of clinician-performed ultrasonography because it allows cytologic diagnosis of suspicious lesions. Understanding the successful techniques and pitfalls in this procedure are critical for the otolaryngologist performing USgFNA.