Validation of hand motion analysis as an objective assessment tool for the Focused Assessment with Sonography for Trauma examination

Development of a Sensor Technology to Objectively Measure Dexterity for Cardiac Surgical Proficiency

BACKGROUND

Technical skill is essential for good outcomes in cardiac surgery. However, no objective methods exist to measure dexterity while performing surgery. The purpose of this study was to validate sensor-based hand motion analysis (HMA) of technical dexterity while performing a graft anastomosis within a validated simulator.

METHODS

Surgeons at various training levels performed an anastomosis while wearing flexible sensors (BioStamp nPoint, MC10 Inc) with integrated accelerometers and gyroscopes on each hand to quantify HMA kinematics. Groups were stratified as experts (n = 8) or novices (n = 18). The quality of the completed anastomosis was scored using the 10 Point Microsurgical Anastomosis Rating Scale (MARS10). HMA parameters were compared between groups and correlated with quality. Logistic regression was used to develop a predictive model from HMA parameters to distinguish experts from novices.

RESULTS

Experts were faster (11 ± 6 minutes vs 21 ± 9 minutes; P = .012) and used fewer movements in both dominant (340 ± 166 moves vs 699 ± 284 moves; P = .003) and nondominant (359 ± 188 moves vs 567 ± 201 moves; P = .02) hands compared with novices. Experts' anastomoses were of higher quality compared with novices (9.0 ± 1.2 MARS10 vs 4.9 ± 3.2 MARS10; P = .002). Higher anastomosis quality correlated with 9 of 10 HMA parameters, including fewer and shorter movements of both hands (dominant, r = -0.65, r = -0.46; nondominant, r = -0.58, r = -0.39, respectively).

CONCLUSIONS

Sensor-based HMA can distinguish technical dexterity differences between experts and novices, and correlates with quality. Objective quantification of hand dexterity may be a valuable adjunct to training and education in cardiac surgery training programs.

FAST skill assessment from kinematics data using convolutional neural networks

PURPOSE

FAST is a point of care ultrasound study that evaluates for the presence of free fluid, typically hemoperitoneum in trauma patients. FAST is an essential skill for Emergency Physicians. Thus, it requires objective evaluation tools that can reduce the necessity of direct observation for proficiency assessment. In this work, we use deep neural networks to automatically assess operators' FAST skills.

METHODS

We propose a deep convolutional neural network for FAST proficiency assessment based on motion data. Prior work has shown that operators demonstrate different domain-specific dexterity metrics that can distinguish novices, intermediates, and experts. Therefore, we augment our dataset with this domain knowledge and employ fine-tuning to improve the model's classification capabilities. Our model, however, does not require specific points of interest (POIs) to be defined for scanning.

RESULTS

The results show that the proposed deep convolutional neural network can classify FAST proficiency with 87.5% accuracy and 0.884, 0.886, 0.247 sensitivity for novices, intermediates, and experts, respectively. It demonstrates the potential of using kinematics data as an input in FAST skill assessment tasks. We also show that the proposed domain-specific features and region fine-tuning increase the model's classification accuracy and sensitivity.

CONCLUSIONS

Variations in probe motion at different learning stages can be derived from kinematics data. These variations can be used for automatic and objective skill assessment without prior identification of clinical POIs. The proposed approach can improve the quality and objectivity of FAST proficiency evaluation. Furthermore, skill assessment combining ultrasound images and kinematics data can provide a more rigorous and diversified evaluation than using ultrasound images alone.

Assessment of changes in gaze patterns during training in point-of-care ultrasound

BACKGROUND

Point-of-care ultrasound (POCUS) is a core skill in emergency medicine (EM), however, there is a lack of objective competency measures. Eye-tracking technology is a potentially useful assessment tool, as gaze patterns can reliably discriminate between experts and novices across medical specialties. We aim to determine if gaze metrics change in an independent and predictable manner during ultrasound training.

METHODS

A convenience sample of first-year residents from a single academic emergency department was recruited. Participants interpreted 16 ultrasound videos of the focused assessment with sonography for trauma (FAST) scan while their gaze patterns were recorded using a commercially available eye-tracking device. The intervention group then completed an introductory ultrasound course whereas the control group received no additional education. The gaze assessment was subsequently repeated. The primary outcome was total gaze duration on the area of interest (AOI). Secondary outcomes included time to fixation, mean duration of first fixation and mean number of fixations on the AOI.

RESULTS

10 EM residents in the intervention group and 10 non-EM residents in the control group completed the study. After training, there was an 8.8 s increase in the total gaze time on the AOI in the intervention group compared to a 4.0 s decrease in the control group (p = .03). EM residents were also 3.8 s quicker to fixate on the AOI whereas the control group became 2.5 s slower (p = .04). There were no significant interactions on the number of fixations (0.43 vs. 0.18, p = .65) or duration of first fixation on the AOI (0.02 s vs. 0.06 s, p = .63).

CONCLUSIONS

There are significant and quantifiable changes in gaze metrics, which occur with incremental learning after an ultrasound course. Further research is needed to validate the serial use of eye-tracking technology in following a learner's progress toward competency in point-of-care ultrasound image interpretation.

Contextual interference for skills development and transfer in laparoscopic surgery: a randomized controlled trial

INTRODUCTION

Past education literature has shown benefits for random practice schedules (termed contextual interference) for skills retention and transfer to novel tasks. The purpose of fundamentals of laparoscopic surgery (FLS) training is to develop skills in simulation and transfer to new in vivo intraoperative experiences. The study objective was to assess whether individuals trained over a fixed number of trials in the FLS tasks would outperform untrained controls on an unpracticed previously validated bile duct cannulation task and scoring system and to determine whether random training schedules conferred any relative advantage.

METHODS

44 trainees with no laparoscopic experience were recruited to participate. 35 were randomized to practice the FLS tasks using either a blocked or random training schedule. Nine were randomized to no additional training (controls). Participant performance was measured throughout training to monitor skills acquisition and were then tested on an unpracticed bile duct cannulation simulation task 4 to 6 weeks later. Outcomes included previously validated FLS scores and hand-motion analyses.

RESULTS

All 44 participants completed the study. Trained individuals in both groups showed significant improvements in all FLS tasks after training. There were no differences between groups in performance on the cannulation task median scores (Blocked: 89.8 [IQR:37.6]; Random: 83.2 [32.3]; Control: 83.6 [19.1]; p = 0.955), number of hand motions (Blocked: 42.5 [IQR:130.3]; Random: 75.3 [111.3]; Control: 63.0 [71.8]; p = 0.912), or distance traveled by participants hands (Blocked: 2.0 m [IQR:5.8]; Random: 3.8 [8.9]; Control: 2.6 [2.5]; p = 0.816). Cannulation task performance had no correlation with total FLS performance, R2 linear = 0.014, p = 0.445.

CONCLUSIONS

Skills acquired from conventional FLS tasks did not effectively transfer to a laparoscopic bile duct cannulation task. Neither blocked nor random practice schedules conferred a relative advantage. These findings provide evidence that cannulation is a distinct skill from what is taught and assessed in FLS.

Development and Validation of a Deep Learning Strategy for Automated View Classification of Pediatric Focused Assessment With Sonography for Trauma

OBJECTIVE

Pediatric focused assessment with sonography for trauma (FAST) is a sequence of ultrasound views rapidly performed by clinicians to diagnose hemorrhage. A technical limitation of FAST is the lack of expertise to consistently acquire all required views. We sought to develop an accurate deep learning view classifier using a large heterogeneous dataset of clinician-performed pediatric FAST.

METHODS

We developed and conducted a retrospective cohort analysis of a deep learning view classifier on real-world FAST studies performed on injured children less than 18 years old in two pediatric emergency departments by 30 different clinicians. FAST was randomly distributed to training, validation, and test datasets, 70:20:10; each child was represented in only one dataset. The primary outcome was view classifier accuracy for video clips and still frames.

RESULTS

There were 699 FAST studies, representing 4925 video clips and 1,062,612 still frames, performed by 30 different clinicians. The overall classification accuracy was 97.8% (95% confidence interval [CI]: 96.0-99.0) for video clips and 93.4% (95% CI: 93.3-93.6) for still frames. Per view still frames were classified with an accuracy: 96.0% (95% CI: 95.9-96.1) cardiac, 99.8% (95% CI: 99.8-99.8) pleural, 95.2% (95% CI: 95.0-95.3) abdominal upper quadrants, and 95.9% (95% CI: 95.8-96.0) suprapubic.

CONCLUSION

A deep learning classifier can accurately predict pediatric FAST views. Accurate view classification is important for quality assurance and feasibility of a multi-stage deep learning FAST model to enhance the evaluation of injured children.

Task model-specific operator skill assessment in routine fetal ultrasound scanning

Purpose

For highly operator-dependent ultrasound scanning, skill assessment approaches evaluate operator competence given available data, such as acquired images and tracked probe movement. Operator skill level can be quantified by the completeness, speed, and precision of performing a clinical task, such as biometry. Such clinical tasks are increasingly becoming assisted or even replaced by automated machine learning models. In addition to measurement, operators need to be competent at the upstream task of acquiring images of sufficient quality. To provide computer assistance for this task requires a new definition of skill.

Methods

This paper focuses on the task of selecting ultrasound frames for biometry, for which operator skill is assessed by quantifying how well the tasks are performed with neural network-based frame classifiers. We first develop a frame classification model for each biometry task, using a novel label-efficient training strategy. Once these task models are trained, we propose a second task model-specific network to predict two skill assessment scores, based on the probability of identifying positive frames and accuracy of model classification.

Results

We present comprehensive results to demonstrate the efficacy of both the frame-classification and skill-assessment networks, using clinically acquired data from two biometry tasks for a total of 139 subjects, and compare the proposed skill assessment with metrics of operator experience.

Conclusion

Task model-specific skill assessment is feasible and can be predicted by the proposed neural networks, which provide objective assessment that is a stronger indicator of task model performance, compared to existing skill assessment methods.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11548-022-02642-y.

Characterizing the biomechanical differences between novice and expert point-of-care ultrasound practitioners using a low-cost gyroscope and accelerometer integrated sensor: A pilot study

INTRODUCTION

Point-of-care ultrasound (POCUS) has become an important diagnostic tool in acute care medicine; however, little is known about the biomechanical differences between novice and expert practitioners.

METHODS

A low-cost ($50 CAD) gyroscope and accelerometer integrated sensor was assembled and affixed to an ultrasound probe. Seventeen participants, nine novices and eight experts, were recruited to perform three abdominal and four cardiac scans on a standardized patient. Participant demographics, time per scan, average acceleration, average angular velocity, decay in acceleration and angular velocity over time, and frequency of probe movements were analyzed. Video capture with blinded video review was scored.

RESULTS

On video review, experts had higher image optimization and acquisition scores for both abdominal and cardiac scans. Experts had shorter scan times for abdominal (7 s vs. 26 s, p = 0.003) and cardiac (11 s vs. 26 s, p < 0.001) scans. There was no difference in average acceleration (g) between novices and experts performing abdominal (1.02 vs. 1.01, p = 0.50) and cardiac (1.01 vs. 1.01, p = 0.45) scans. Experts had lower angular velocity (°/s) for abdominal scans (10.00 vs. 18.73, p < 0.001) and cardiac scans (15.61 vs. 20.33, p = 0.02) There was a greater decay in acceleration over time for experts performing cardiac scans compared to novices (-0.194 vs. -0.050, p = 0.03) but not for abdominal scans or when measuring angular velocity. The frequency of movements (Hz) was higher for novices compared to experts for abdominal (16.68 vs. 13.79, p < 0.001) and cardiac (17.60 vs. 13.63, p = 0.002) scans.

DISCUSSION

This study supports the feasibility of a low-cost gyroscope and accelerometer integrated sensor to quantify the biomechanics of POCUS. It may also support the concept of "window shopping" as a method by which experts obtain abdominal and cardiac views, where sliding is used to find an acoustic window, then smaller rocking and tilting probe movements are used to refine the image.

Hand Motion Analysis Illustrates Differences When Drilling Cadaveric and Printed Temporal Bone

OBJECTIVE

Temporal bone simulation is now commonly used to augment cadaveric education. Assessment of these tools is ongoing, with haptic modeling illustrating dissimilar motion patterns compared to cadaveric opportunities. This has the potential to result in maladaptive skill development. It is hypothesized that trainee drill motion patterns during printed model dissection may likewise demonstrate dissimilar hand motion patterns.

METHODS

Resident surgeons dissected 3D-printed temporal bones generated from microCT data and cadaveric simulations. A magnetic position tracking system (TrakSTAR Ascension, Yarraville, Australia) captured drill position and orientation. Skill assessment included cortical mastoidectomy, thinning procedures (sigmoid sinus, dural plate, posterior canal wall) and facial recess development. Dissection was performed by 8 trainees (n = 5 < PGY3 > n = 3) using k-cos metrics to analyze drill strokes within position recordings. K-cos metrics define strokes by change in direction, providing metrics for stroke duration, curvature, and length.

RESULTS

T-tests between models showed no significant difference in drill stroke frequency (cadaveric = 1.36/s, printed = 1.50/s, P < .40) but demonstrate significantly shorter duration (cadaveric = 0.37 s, printed = 0.16 s, P < .01) and a higher percentage of curved strokes (cadaveric = 31, printed = 67, P < .01) employed in printed bone dissection. Junior staff used a higher number of short strokes (junior = 0.54, senior = 0.38, P < .01) and higher percentage of curved strokes (junior = 35%, senior = 21%, P < .01).

CONCLUSIONS

Significant differences in hand motions were present between simulations, however the significance is unclear. This may indicate that printed bone is not best positioned to be the principal training schema.

Design and Validity Evidence for a Unique Endoscopy Simulator Using a Commercial Video Game

Background Procedural simulation enhances early endoscopy training. Multiple commercial simulators are available; however, their application is limited by cost and poor user compliance. First-person "shooter" (FPS) video games are popular. In this study, we aimed to show that a novel in-house designed colonoscope controller used to play an FPS video game shares similar constructs with real-life endoscopy. Methodology Participants completed the first three levels on an FPS video game, Portal (Valve Corporation, Bellevue, WA), first using a conventional controller and then the modified endoscope controller. A total of 12 expert endoscopists and 12 surgical residents with minimal endoscopy experience were evaluated based on completion time, button presses, and hand motion analyses. Results Experts outperformed novices for completion time (expert: 944 seconds; novice: 1,515 seconds; p = 0.006) and hand movements (expert: 1,263.1; novice: 2,052.6; p = 0.004) in using the novel colonoscope controller. There was no difference in button presses or total path length traveled. Furthermore, performance did not differ using conventional game controls. Conclusions Experts outperformed novices using the endoscope but not the conventional controller with respect to the economy of movement and completion time. This result confirms that our endoscope-controlled video game shares similar paradigms with real-life endoscopy and serves as a first step toward creating a more enjoyable and cheaper alternative to commercially available endoscopy simulators.

Skills Classification in Cardiac Ultrasound with Temporal Convolution and Domain Knowledge Using a Low-Cost Probe Tracker

As point-of-care ultrasound (POCUS) becomes more integrated into clinical practice, it is essential to address all aspects of ultrasound operator proficiency. Ultrasound proficiency requires the ability to acquire, interpret and integrate bedside ultrasound images. The difference in image acquisition psychomotor skills between novice (trainee) and expert (instructor) ultrasonographer has not been described. We created an inexpensive system, called Probe Watch, to record probe motion and assess image acquisition in cardiac POCUS using an inertial measurement device and software for data recording based on open-source components. We designed a temporal convolutional network for skills classification from probe motion that integrates clinical domain knowledge. We further designed data augmentation methods to improve its generalization. Subsequently, we validated the setup and assessment method on a set of novice and expert sonographers performing cardiac ultrasound in a simulation-based training environment. The proposed methods classified participants as novice or expert with areas under the receiver operating characteristic curve of 0.931 and 0.761 for snippets and trials, respectively. Integrating domain knowledge into the neural network had added value. Furthermore, we identified the most discriminative features for assessment. Probe Watch quantifies motion during cardiac ultrasound and provides insight into probe motion behavior. It may be deployed during cardiac ultrasound training to monitor learning curves objectively and automatically.

Use of Hand-motion Analysis to Assess Competence and Skill Decay for Cardiac and Lung Point-of-care Ultrasound

OBJECTIVES

Assessment of competence in technical skills, including point-of-care ultrasound (POCUS), is required before a novice can safely perform the skill independently. Ongoing assessment of competence is also required because technical skills degrade over time, especially when they are infrequently performed or complex. Hand-motion analysis (HMA) is an objective assessment tool that has been used to evaluate competency in many technical skills. The purpose of this study was to demonstrate the feasibility and validity of HMA as an assessment tool for competence in both simple and complex technical skills as well as skill degradation over time.

METHODS

This prospective cohort study included 36 paramedics with no POCUS experience and six physicians who were fellowship trained in POCUS. The novices completed a 4-hour didactic and hands-on training program for cardiac and lung POCUS. HMA measurements, objective structured clinical examinations (OSCE), and written examinations were collected for novices immediately before and after training as well as 2 and 4 months after training. Expert HMA metrics were also recorded.

RESULTS

Expert HMA metrics for cardiac and lung POCUS were significantly better than those of novices. After completion of the training program, the novices improved significantly in all HMA metrics, knowledge test scores, and OSCE scores. Novices showed skill degradation in cardiac POCUS based on HMA metrics and OSCE scores while lung POCUS image acquisition skills were preserved. Novices deemed competent by OSCE score performed significantly better in HMA metrics than those not deemed competent.

CONCLUSION

We have demonstrated that HMA is a feasible and valid tool for assessment of competence in technical skills and can also evaluate skill degradation over time. Skill degradation appears more apparent in complex skills, such as cardiac POCUS. HMA may provide a more efficient and reliable assessment of technical skills, including POCUS, when compared to traditional assessment tools.

Utilization of Flexible-Wearable Sensors to Describe the Kinematics of Surgical Proficiency

BACKGROUND

Traditional assessment (e.g., checklists, videotaping) for surgical proficiency may lead to subjectivity and does not predict performance in the clinical setting. Hand motion analysis is evolving as an objective tool for grading technical dexterity; however, most devices accompany with technical limitations or discomfort. We purpose the use of flexible wearable sensors to evaluate the kinematics of surgical proficiency.

METHODS

Surgeons were recruited and performed a vascular anastomosis task in a single institution. A modified objective structured assessment of technical skills (mOSATS) was used for technical qualification. Flexible wearable sensors (BioStamp RCTM, mc10 Inc., Lexington, MA) were placed on the dorsum of the dominant hand (DH) and nondominant hand (nDH) to measure kinematic parameters: path length (T_path), mean (V_mean) and peak (V_peak) velocity, number of hand movements (N_move), ratio of DH to nDH movements (rMov), and time of task (tTask) and further compared with the mOSATS score.

RESULTS

Participants were categorized as experts (n = 12) and novices (n = 8) based on a cutoff mean mOSATS score. Significant differences for tTask (P = 0.02), rMov (P = 0.07), DH T_path (P = 0.04), V_mean (P = 0.07), V_peak (P = 0.04), and nDH N_move (P = 0.02) were in favor of the experts. Overall, mOSATS had significant correlation with tTask (r = -0.69, P = 0.001), N_move of DH (r = -0.44, P = 0.047) and nDH (r = -0.66, P = 0.001), and rMov (r = 0.52, P = 0.017).

CONCLUSIONS

Hand motion analysis evaluated by flexible wearable sensors is feasible and informative. Experts utilize coordinated two-handed motion, whereas novices perform one-handed tasks in a hastily jerky manner. These tendencies create opportunity for improvement in surgical proficiency among trainees.

Objective and automated assessment of surgical technical skills with IoT systems: A systematic literature review

The assessment of surgical technical skills to be acquired by novice surgeons has been traditionally done by an expert surgeon and is therefore of a subjective nature. Nevertheless, the recent advances on IoT (Internet of Things), the possibility of incorporating sensors into objects and environments in order to collect large amounts of data, and the progress on machine learning are facilitating a more objective and automated assessment of surgical technical skills. This paper presents a systematic literature review of papers published after 2013 discussing the objective and automated assessment of surgical technical skills. 101 out of an initial list of 537 papers were analyzed to identify: 1) the sensors used; 2) the data collected by these sensors and the relationship between these data, surgical technical skills and surgeons' levels of expertise; 3) the statistical methods and algorithms used to process these data; and 4) the feedback provided based on the outputs of these statistical methods and algorithms. Particularly, 1) mechanical and electromagnetic sensors are widely used for tool tracking, while inertial measurement units are widely used for body tracking; 2) path length, number of sub-movements, smoothness, fixation, saccade and total time are the main indicators obtained from raw data and serve to assess surgical technical skills such as economy, efficiency, hand tremor, or mind control, and distinguish between two or three levels of expertise (novice/intermediate/advanced surgeons); 3) SVM (Support Vector Machines) and Neural Networks are the preferred statistical methods and algorithms for processing the data collected, while new opportunities are opened up to combine various algorithms and use deep learning; and 4) feedback is provided by matching performance indicators and a lexicon of words and visualizations, although there is considerable room for research in the context of feedback and visualizations, taking, for example, ideas from learning analytics.

Differences in Gaze Fixation Location and Duration Between Resident and Fellowship Sonographers Interpreting a Focused Assessment With Sonography in Trauma

OBJECTIVES

We quantified the gaze fixation duration of resident and fellowship sonographers interpreting a prerecorded focused assessment with sonography in trauma (FAST). We hypothesized that all sonographers would fixate on each relevant anatomic relationship but that the duration of fixation would differ.

METHODS

We conducted a cross-sectional study collecting and analyzing the gaze fixations of a convenience sample of current resident and fellowship sonographers. All sonographers viewed a standardized FAST video, and their gaze fixations were recorded using a Tobii X3-120 eye-tracking bar. Gaze fixations over nine anatomic regions of interest (ROIs) were identified. These were assessed for normality and analyzed using the Wilcoxon rank sum test at an alpha of 0.05 and Bonferroni correction p value of <0.0034. The chi-square test and Pearson's correlation were performed to assess statistical association.

RESULTS

The gaze fixation recordings of 24 resident and eight fellowship sonographers were suitable for analysis. Fourteen of the 24 resident sonographers viewed all ROIs in the FAST, whereas all eight fellowship sonographers viewed each of the nine relevant ROIs. Five ROIs were identified over which at least one resident sonographer did not have a gaze fixation. No statistically significant difference was identified between groups. Resident sonographers gaze fixated over the left upper quadrant (LUQ) splenorenal interface for a median (interquartile range) of 10.64 (9.73-11.60) seconds. The fellowship group viewed the same ROI for 8.43 (6.64-8.95) seconds (p < 0.003). All participants viewed this ROI. No other ROIs had a statistical difference.

CONCLUSION

Five ROIs were identified that were not visually interrogated by all resident sonographers. Only 14 of 24 resident sonographers visually interrogated every area in the FAST, whereas all fellowship sonographers interrogated every ROI. A statistically significant difference was found in gaze fixation duration between resident and fellowship sonographers in one ROI. Further study is required for gaze fixation assessment to become a tool for the interpretation component of point-of-care ultrasound.

Differentiating Operator Skill during Routine Fetal Ultrasound Scanning using Probe Motion Tracking

In this paper, we consider differentiating operator skill during fetal ultrasound scanning using probe motion tracking. We present a novel convolutional neural network-based deep learning framework to model ultrasound probe motion in order to classify operator skill levels, that is invariant to operators' personal scanning styles. In this study, probe motion data during routine second-trimester fetal ultrasound scanning was acquired by operators of known experience levels (2 newly-qualified operators and 10 expert operators). The results demonstrate that the proposed model can successfully learn underlying probe motion features that distinguish operator skill levels during routine fetal ultrasound with 95% accuracy.

Consensus-Based Expert Development of Critical Items for Direct Observation of Point-of-Care Ultrasound Skills

BACKGROUND

Point-of-care ultrasound (POCUS) is increasingly used in a number of medical specialties. To support competency-based POCUS education, workplace-based assessments are essential.

OBJECTIVE

We developed a consensus-based assessment tool for POCUS skills and determined which items are critical for competence. We then performed standards setting to set cut scores for the tool.

METHODS

Using a modified Delphi technique, 25 experts voted on 32 items over 3 rounds between August and December 2016. Consensus was defined as agreement by at least 80% of the experts. Twelve experts then performed 3 rounds of a standards setting procedure in March 2017 to establish cut scores.

RESULTS

Experts reached consensus for 31 items to include in the tool. Experts reached consensus that 16 of those items were critically important. A final cut score for the tool was established at 65.2% (SD 17.0%). Cut scores for critical items are significantly higher than those for noncritical items (76.5% ± SD 12.4% versus 53.1% ± SD 12.2%, P < .0001).

CONCLUSIONS

We reached consensus on a 31-item workplace-based assessment tool for identifying competence in POCUS. Of those items, 16 were considered critically important. Their importance is further supported by higher cut scores compared with noncritical items.

Needle tip tracking for ultrasound-guided peripheral nerve block procedures-An observer blinded, randomised, controlled, crossover study on a phantom model

BACKGROUND

The Onvision needle tip tracking (NTT) is a new technology consisting of a needle with an ultrasound sensor close to the needle tip and a console for computerised signal processing. The aim of the study was to evaluate NTT technology during ultrasound-guided simulated peripheral nerve block procedures in a porcine phantom model.

METHODS

Forty anaesthesiologists performed in-plane and out-of-plane simulated nerve blocks with and without NTT guidance. The primary outcome measure was procedure time. Secondary outcomes were hand movements and the path length travelled by the hands measured by motion analysis, precision of the needle tip related to the target structure, success rates and violations of the target structure, and the participants confidence whether their procedure would be successful or not.

RESULTS

Procedure time was reduced from 66.7 (SD = 47.5) seconds to 43.8 (SD = 29.2) seconds when NTT was used for out-of-plane procedures (P = 0.002). The number of hand movements of the probe hand was 13.9 (SD = 30.2) with NTT and 22.8 (SD = 30.0) without NTT (P = 0.019). No significant differences were registered during the performance of in-plane procedures. The participants confidence in a presumed block success was increased with both in-plane procedures (8.50 (SD = 1.18) with NTT vs 7.65 (SD = 1.96), P = 0.004) and out-of-plane procedures (8.50 (SD = 1.09) vs 7.10 (SD = 1.89), P = 0.0001).

CONCLUSIONS

The new NTT technology significantly reduced the procedure time and the number of hand movements for ultrasound-guided out-of-plane PNB procedures. No significant differences were found for the in-plane procedures.

Development and evaluation of rhinoplasty spreader graft suture simulator for novice surgeons

OBJECTIVE

Surgical simulators aimed at mimicking elements of rhinoplasty surgery, specifically those aimed at improving cartilage suturing, are not available. Here, we present a surgical simulator for spreader graft placement that uses cartilage rather than synthetic materials and gauge improvement using objective measures for suture placement accuracy, speed, and efficiency of hand motion.

METHODS

Twenty-two otolaryngologists in two groups (residents [10] and experts [12]) were instructed to secure the two spreader graft specimen into position with three mattress sutures on a nose model that used porcine septal cartilage as a proxy for the human counterpart. Hand motion was tracked using an electromagnetic position sensing device. The time required to complete the suture task, total hand displacement, cumulative number of hand motion direction changes, and accuracy of suture insertion were measured. These measurements were compared between the two cohort groups for construct validity. The subjects completed a survey to evaluate realism and value of the model.

RESULTS

The expert group had a lower mean time required to complete the task (P < 0.05), total hand displacement (P < 0.01), and number of hand motion direction changes (P < 0.001). No significant difference was observed between the two groups in suture precision measurement. The subjects agreed on the face validity and usefulness of the trainer.

CONCLUSIONS

Our study suggests that the simulator may be a useful tool to objectively gauge suturing efficiency. Devices such as this may be useful for developing skill with suturing cartilage tissue and potentially be used to assess resident acquisition of surgical skill.

LEVEL OF EVIDENCE

NA Laryngoscope, 129:344-350, 2019.

Basic ultrasound training assessment in the initial abdominal trauma screening

OBJECTIVE

to verify the efficiency and usefulness of basic ultrasound training in trauma (FAST - Focused Assessment with Sonography in Trauma) for emergency physicians in the primary evaluation of abdominal trauma.

METHODS

a longitudinal and observational study was carried out from 2015 to 2017, with 11 emergency physicians from Hospital Universitário do Oeste do Paraná, submitted to ultrasound training in emergency and trauma (USET® - SBAIT). FAST results started to be collected two months after the course. These were compared with a composite score of complementary exams and surgical findings. Information was stored in a Microsoft Excel program database and submitted to statistical analysis.

RESULTS

FAST was performed in 120 patients. In the study, 38.4% of the assessed patients had a shock index ≥0.9. The composite score detected 40 patients with free peritoneal fluid, whereas FAST detected 27 cases. The method sensitivity was 67.5%, specificity was 98.7%, the positive predictive value was 96.4%, the negative predictive value was 85.39% and accuracy was 88%. All those with a positive FAST had a shock index ≥0.9. Fifteen patients with positive FAST and signs of instability were immediately submitted to surgery.

CONCLUSIONS

the basic training of emergency physicians in FAST showed efficiency and usefulness in abdominal trauma assessment. Due to its low cost and easy implementation, this modality should be considered as a screening strategy for patients with abdominal trauma in health systems.

The Effects of Music on Microsurgical Technique and Performance: A Motion Analysis Study

BACKGROUND

Music is commonly played in operating rooms (ORs) throughout the country. If a preferred genre of music is played, surgeons have been shown to perform surgical tasks quicker and with greater accuracy. However, there are currently no studies investigating the effects of music on microsurgical technique. Motion analysis technology has recently been validated in the objective assessment of plastic surgery trainees' performance of microanastomoses. Here, we aimed to examine the effects of music on microsurgical skills using motion analysis technology as a primary objective assessment tool.

METHODS

Residents and fellows in the Plastic and Reconstructive Surgery program were recruited to complete a demographic survey and participate in microsurgical tasks. Each participant completed 2 arterial microanastomoses on a chicken foot model, one with music playing, and the other without music playing. Participants were blinded to the study objectives and encouraged to perform their best. The order of music and no music was randomized. Microanastomoses were video recorded using a digitalized S-video system and deidentified. Video segments were analyzed using ProAnalyst motion analysis software for automatic noncontact markerless video tracking of the needle driver tip.

RESULTS

Nine residents and 3 plastic surgery fellows were tested. Reported microsurgical experience ranged from 1 to 10 arterial anastomoses performed (n = 2), 11 to 100 anastomoses (n = 9), and 101 to 500 anastomoses (n = 1). Mean age was 33 years (range, 29-36 years), with 11 participants right-handed and 1 ambidextrous. Of the 12 subjects tested, 11 (92%) preferred music in the OR. Composite instrument motion analysis scores significantly improved with playing preferred music during testing versus no music (paired t test, P <0.001). Improvement with music was significant even after stratifying scores by order in which variables were tested (music first vs no music first), postgraduate year, and number of anastomoses (analysis of variance, P < 0.01).

CONCLUSIONS

Preferred music in the OR may have a positive effect on trainees' microsurgical performance; as such, trainees should be encouraged to participate in setting the conditions of the OR to optimize their comfort and, possibly, performance. Moreover, motion analysis technology is a useful tool with a wide range of applications for surgical education and outcomes optimization.

Sonographic Accuracy as a Novel Tool for Point-of-care Ultrasound Competency Assessment

OBJECTIVES

The Focused Assessment with Sonography in Trauma (FAST) is a point-of-care ultrasound (PoCUS) study that is routine in trauma patient assessment. Many organizations have published training guidelines, which grant competency through the completion of a fixed number of observed scans. This approach is incongruent with current trends in competency-based medical education. We aim to objectively quantify probe motion and user accuracy to differentiate groups of PoCUS operators.

METHODS

Emergency medicine residents were recruited in two groups. The novice group (n = 15) had limited PoCUS experience, whereas the intermediate group (n = 14) had completed at least 50 supervised FAST examinations. Both groups underwent assessment on a live human model. Residents from the novice group returned (n = 9) after completing a curriculum and repeated the assessment using the identical experimental construct.

RESULTS

Significant differences (p < 0.05) were found between the novice and both the intermediate and the novice returned groups in time, path length, and points of interest (POIs) scanned. Novices required more time to complete the full examination (290.82 seconds vs. 197.41 seconds vs. 271.79 seconds), utilized more motion (9392.07 mm vs. 4052.73 mm vs. 4985.05 mm), and imaged fewer POIs (48.13% vs. 95.00% vs. 100.00%) when compared to intermediates and returning novices, respectively. No difference was found between the intermediate and novice returned groups for the complete examination. Spearman's correlation was calculated between variables within each group. Correlations between time and path length were statistically significant (p < 0.05) with novice, intermediate, and novice returned values of 0.67, 0.65, and 0.90. Interestingly, neither time nor path length consistently correlated with POIs scanned in any group.

CONCLUSION

Differences in probe motion efficiency and POIs scanned between novices and intermediate or returning novice users show promise for use as a quantitative objective assessment tool. Unlike in surgical literature, accuracy did not correlate with path length or time to examination completion.