Motion Smoothness Metrics for Cannulation Skill Assessment: What Factors Matter?

Objective and Automated Quantification of Instrument Handling for Open Surgical Suturing Skill Assessment: A Simulation-Based Study

Goal: Vascular surgical procedures are challenging and require proficient suturing skills. To develop these skills, medical training simulators with objective feedback for formative assessment are gaining popularity. As hardware advancements offer more complex, unique sensors, determining effective task performance measures becomes imperative for efficient suturing training. Methods: 97 subjects of varying clinical expertise completed four trials on a suturing skills measurement and feedback platform (SutureCoach). Instrument handling metrics were calculated from electromagnetic motion trackers affixed to the needle driver. Results: The results of the study showed that all metrics significantly differentiated between novices (no medical experience) from both experts (attending surgeons/fellows) and intermediates (residents). Rotational motion metrics were more consistent in differentiating experts and intermediates over traditionally used tooltip motion metrics. Conclusions: Our work emphasizes the importance of tool motion metrics for open suturing skills assessment and establishes groundwork to explore rotational motion for quantifying a critical facet of surgical performance.

Measuring hand movement for suturing skill assessment: A simulation-based study

BACKGROUND

To maximize patient safety, surgical skills education is increasingly adopting simulation-based curricula for formative skills assessment and training. However, many standardized assessment tools rely on human raters for performance assessment, which is resource-intensive and subjective. Simulators that provide automated and objective metrics from sensor data can address this limitation. We present an instrumented bench suturing simulator, patterned after the clock face radial suturing model from the Fundamentals of Vascular Surgery, for automated and objective assessment of open suturing skills.

METHODS

For this study, 97 participants (35 attending surgeons, 32 residents, and 30 novices) were recruited at national vascular conferences. Automated hand motion metrics, especially focusing on rotational motion analysis, were developed from the inertial measurement unit attached to participants' hands, and the proposed suite of metrics was used to differentiate between the skill levels of the 3 groups.

RESULTS

Attendings' and residents' performances were found to be significantly different from novices for all metrics. Moreover, most of our novel metrics could successfully distinguish between finer skill differences between attending and resident groups. In contrast, traditional operative skill metrics, such as time and path length, were unable to distinguish attendings from residents.

CONCLUSION

This study provides evidence for the effectiveness of rotational motion analysis in assessing suturing skills. The suite of inertial measurement unit-based hand motion metrics introduced in this study allows for the incorporation of hand movement data for suturing skill assessment.

Needle Angle Matters: An Investigation of the Effect of Needle Angle on Hemodialysis Cannulation Skill

Key Points

Three metrics that quantify cannulation skill on the basis of needle angle are introduced. All three needle angle metrics were demonstrated to be useful in predicting cannulation outcomes on the simulator.

Background

Cannulation is critical for maintaining a functional vascular access for patients on hemodialysis. However, relatively little is known about the quantitative aspects of needle insertion dynamics required for skilled cannulation. In this article, we introduce three kinds of metrics that quantify needle insertion angle—recognized as important for safe and effective cannulation—during cannulation on an instrumented simulator for skill assessment. Three questions were examined in this study: (1 ) Are simulator-based needle angle metrics related to cannulation success? (2 ) Are needle angle metrics related to simulated blood flashback quality? and (3 ) Can needle angle metrics be used to distinguish between high and low skill levels?

Methods

Fifty-one cannulators with varying degrees of clinical experience performed cannulation on the instrumented simulator. Each participant cannulated 16 times on different fistulas with varying geometries. During each trial, needle angle along with other sensor data was obtained through a motion sensor placed inside the needle. Data analysis was conducted by relating needle angle over time with our previously validated simulator-based cannulation outcome metrics.

Results

The results revealed that all three types of needle angle metrics were useful in predicting the probability of cannulation success. In addition, they were also correlated with flashback quality metrics. Furthermore, these metrics successfully distinguished between high and low performers regardless of whether they were classified using subjective ratings or objective scores. These results indicate that needle insertion angle is an important component of cannulation skill.

Conclusions

The simulator-based metrics for needle insertion angle presented in this work measure a key aspect of skilled cannulation. As such, if implemented in a structured way, these metrics could lead to competency-based skill assessment and training for cannulation in the future. Raising the bar of cannulation skill of our clinicians can have a tangible effect on patient outcomes.

Relating process and outcome metrics for meaningful and interpretable cannulation skill assessment: A machine learning paradigm

BACKGROUND AND OBJECTIVES

The quality of healthcare delivery depends directly on the skills of clinicians. For patients on hemodialysis, medical errors or injuries caused during cannulation can lead to adverse outcomes, including potential death. To promote objective skill assessment and effective training, we present a machine learning approach, which utilizes a highly-sensorized cannulation simulator and a set of objective process and outcome metrics.

METHODS

In this study, 52 clinicians were recruited to perform a set of pre-defined cannulation tasks on the simulator. Based on data collected by sensors during their task performance, the feature space was then constructed based on force, motion, and infrared sensor data. Following this, three machine learning models- support vector machine (SVM), support vector regression (SVR), and elastic net (EN)- were constructed to relate the feature space to objective outcome metrics. Our models utilize classification based on the conventional skill classification labels as well as a new method that represents skill on a continuum.

RESULTS

With less than 5% of trials misplaced by two classes, the SVM model was effective in predicting skill based on the feature space. In addition, the SVR model effectively places both skill and outcome on a fine-grained continuum (versus discrete divisions) that is representative of reality. As importantly, the elastic net model enabled the identification of a set of process metrics that highly impact outcomes of the cannulation task, including smoothness of motion, needle angles, and pinch forces.

CONCLUSIONS

The proposed cannulation simulator, paired with machine learning assessment, demonstrates definite advantages over current cannulation training practices. The methods presented here can be adopted to drastically increase the effectiveness of skill assessment and training, thereby potentially improving clinical outcomes of hemodialysis treatment.

Evaluation of Gait Smoothness in Patients with Stroke Undergoing Rehabilitation: Comparison between Two Metrics

The use of quantitative methods to analyze the loss in gait smoothness, an increase in movement intermittency which is a distinguishing hallmark of motor deficits in stroke patients, has gained considerable attention in recent years. In the literature, the spectral arc length (SPARC), as well as metrics based on the measurement of the jerk, such as the log dimensionless jerk (LDLJ), are currently employed to assess smoothness. However, the optimal measure for evaluating the smoothness of walking in stroke patients remains unknown. Here, we investigated the smoothness of the body's center of mass (BCoM) trajectory during gait, using an optoelectronic system, in twenty-two subacute and eight chronic patients before and after a two-month rehabilitation program. The two measures were evaluated for their discriminant validity (ability to differentiate the smoothness of the BCoM trajectory calculated on the cycle of the affected and unaffected limb, and between subacute and chronic patients), validity (correlation with clinical scales), and responsiveness to the intervention. According to our findings, the LDLJ outperformed the SPARC in terms of the examined qualities. Based on data gathered using an optoelectronic system, we recommend using the LDLJ rather than the SPARC to investigate the gait smoothness of stroke patients.

Simulator-Based Metrics for Quantifying Vascular Palpation Skill for Cannulation

Palpation is essential for accurate diagnosis and treatment in many clinical examinations and procedures. Specifically, vascular palpation is used to diagnose cardiovascular health issues and identify anatomical landmarks in the peripheral vascular system. However, little attention has been given to quantifying what comprises skilled vascular palpation; therefore, this study aims to objectively quantify the differences between high performer (HP), mid performer (MP), and low performer (LP) behavior towards understanding vascular palpation skills. Eleven HPs, twenty-five MPs, and ten LPs completed sixteen trials on our simulator under various conditions. There were four fistulas, two skin thicknesses, and two motor vibration intensities. Finger force and location data were recorded for each trial on the simulator. We examined three types of palpation metrics: time, force, and location. All three types of metrics demonstrated statistically significant differences between HP and LP palpation behavior. Therefore, these metrics could be used for structured and standardized palpation skills training in the future, potentially improving patient outcomes.

Smoothness of movement in idiopathic cervical dystonia

Smoothness (i.e. non-intermittency) of movement is a clinically important property of the voluntary movement with accuracy and proper speed. Resting head position and head voluntary movements are impaired in cervical dystonia. The current work aims to evaluate if the smoothness of voluntary head rotations is reduced in this disease. Twenty-six cervical dystonia patients and 26 controls completed rightward and leftward head rotations. Patients’ movements were differentiated into “towards-dystonia” (rotation accentuated the torticollis) and “away-dystonia”. Smoothness was quantified by the angular jerk and arc length of the spectrum of angular speed (i.e. SPARC, arbitrary units). Movement amplitude (mean, 95% CI) on the horizontal plane was larger in controls (63.8°, 58.3°–69.2°) than patients when moving towards-dystonia (52.8°, 46.3°–59.4°; P = 0.006). Controls’ movements (49.4°/s, 41.9–56.9°/s) were faster than movements towards-dystonia (31.6°/s, 25.2–37.9°/s; P < 0.001) and away-dystonia (29.2°/s, 22.9–35.5°/s; P < 0.001). After taking into account the different amplitude and speed, SPARC-derived (but not jerk-derived) indices showed reduced smoothness in patients rotating away-dystonia (1.48, 1.35–1.61) compared to controls (1.88, 1.72–2.03; P < 0.001). Poor smoothness is a motor disturbance independent of movement amplitude and speed in cervical dystonia. Therefore, it should be assessed when evaluating this disease, its progression, and treatments.