Validity and reliability of metrics for translation of regional anaesthesia performance from cadavers to patients

Attaining expertise in regional anaesthesia training using a multifactorial approach incorporating deliberate practice

The most effective way of delivering regional anaesthesia training and the best means of demonstrating competency have not been established. Clinical competency, based on the Dreyfus and Dreyfus lexicon, appears unachievable using current training approaches. Lessons should be taken from the worlds of music, chess, and sports. Modern skills training programmes should be built on an explicit and detailed understanding with measurement of a variety of factors such as perception, attention, psychomotor and visuospatial function, and kinesthetics, coupled with quantitative, accurate, and reliable measurement of performance.

Using Virtual Reality to teach ultrasound-guided needling skills for regional anaesthesia: A randomised controlled trial

STUDY OBJECTIVE

We previously designed and validated a virtual reality-based simulator to help train novices in ultrasound-guided needling skills necessary for safe and competent ultrasound-guided regional anaesthesia. This study was designed to compare the performance and error rates of novices trained by a human faculty aided with the assistance of this virtual reality simulator (virtual reality-assisted training), versus novices trained wholly by humans (conventional training).

DESIGN, SETTING, AND PARTICIPANTS

In this single centre, randomised controlled study, we used a standardised teaching protocol, rigorous blinding, iterative training of assessors, and validated global rating scale and composite error score checklists to assess skills learning of novice participants.

MAIN RESULTS

We recruited 45 novices and scored 270 assessments of performance and error rates. Inter-rater correlation coefficient of reliability of scoring between assessors for the global rating scale was 0.84 (95%CI 0.68-0.92) and for the composite error score checklist was 0.87 (95%CI 0.73-0.93). After adjustment for age, sex, Depression, Anxiety and Stress-21, and baseline score, there was no statistical difference for virtual reality-assisted training compared to conventional training in final global rating score (average treatment effect -3.30 (95%CI-13.07-6.48), p = 0.51) or in the final composite error score (average treatment effect 1.14 (95%CI -0.60-2.88), p = 0.20). Realism in the virtual reality simulator was similar to real-life when measured by the Presence Questionnaire, all components p > 0.79; and task workload assessed by the NASA-Task Load Index was not statistically different between groups, average treatment effect 5.02 (95%CI -3.51-13.54), p = 0.25. Results were achieved in the virtual reality-assisted group with half the human faculty involvement.

CONCLUSION

Novices trained using a hybrid, virtual reality-assisted teaching program showed no superiority to novices trained using a conventional teaching program, but with less burden on teaching resources.

Anatomical Studies Evaluating Pediatric Regional Anesthesia: A Scoping Review

BACKGROUND

Pediatric regional anesthesia has been driven by the gradual rise in the adoption of opioid-sparing strategies and the growing concern over the possible adverse effects of general anesthetics on neurodevelopment. Nonetheless, performing regional anesthesia studies in a pediatric population is challenging and accounts for the scarce evidence. This study aimed to review the scientific foundation of studies in cadavers to assess regional anesthesia techniques in children.

METHODS

We searched the following databases MEDLINE, EMBASE, and Web of Science. We included anatomical cadaver studies assessing peripheral nerve blocks in children. The core data collected from studies were included in tables and comprised block type, block evaluation, results, and conclusion.

RESULTS

The search identified 2409 studies, of which, 16 were anatomical studies on the pediatric population. The techniques evaluated were the erector spinae plane block, ilioinguinal/iliohypogastric nerve block, sciatic nerve block, maxillary nerve block, paravertebral block, femoral nerve block, radial nerve block, greater occipital nerve block, infraclavicular brachial plexus block, and infraorbital nerve block.

CONCLUSION

Regional anesthesia techniques are commonly performed in children, but the lack of anatomical studies may result in reservations regarding the dispersion and absorption of local anesthetics. Further anatomical research on pediatric regional anesthesia may guide the practice.

Simulation-Based Education in Acute and Chronic Pain Training

PURPOSE OF REVIEW

The use of simulation-based education (SBE) in medical training has expanded greatly and has grown to include high fidelity and task simulation along with hybrid models using patient actors to enhance education and training of critical events as well as technical skills.

RECENT FINDINGS

In the field of anesthesiology, SBE has been particularly useful for crisis resource management and rare critical scenarios and new research into the use of SBE using task simulation for procedural skill development has been done highlighting the benefits to subspecialty procedural training. Medical simulation has become a common practice in medical training and research. SBE has demonstrated positive outcomes in improving technical skills, knowledge, comfort, and clinical performance. The widespread implementation of SBE in regional anesthesia and chronic pain training varies, with cost and availability being factors. Nonetheless, SBE has shown great potential in enhancing education and preparing physicians in subspecialties of anesthesia.

Implementing ultrasound-guided nerve blocks in the emergency department: A low-cost, low-fidelity training approach

Background

Managing acute pain is a common challenge in the emergency department (ED). Though widely used in perioperative settings, ED-based ultrasound-guided nerve blocks (UGNBs) have been slow to gain traction. Here, we develop a low-cost, low-fidelity, simulation-based training curriculum in UGNBs for emergency physicians to improve procedural competence and confidence.

Methods

In this pre-/postintervention study, ED physicians were enrolled to participate in a 2-h, in-person simulation training session composed of a didactic session followed by rotation through stations using handmade pork-based UGNB models. Learner confidence with performing and supervising UGNBs as well as knowledge and procedural-based competence were assessed pre- and posttraining via electronic survey quizzes. One-way repeated-measures ANOVAs and pairwise comparisons were conducted. The numbers of nerve blocks performed clinically in the department pre- and postintervention were compared.

Results

In total, 36 participants enrolled in training sessions, eight participants completed surveys at all three data collection time points. Of enrolled participants, 56% were trainees, 39% were faculty, 56% were female, and 53% self-identified as White. Knowledge and competency scores increased immediately postintervention (mean ± SD t0 score 66.9 ± 8.9 vs. t1 score 90.4 ± 11.7; p < 0.001), and decreased 3 months postintervention but remained elevated above baseline (t2 scores 77.2 ± 11.5, compared to t0; p = 0.03). Self-reported confidence in performing UGNBs increased posttraining (t0 5.0 ± 2.3 compared to t1 score 7.1 ± 1.5; p = 0.002) but decreased to baseline levels 3 months postintervention (t2 = 6.0 ± 1.9, compared to t0; p = 0.30).

Conclusions

A low-cost, low-fidelity simulation curriculum can improve ED provider procedural-based competence and confidence in performing UGNBs in the short term, with a trend toward sustained improvement in knowledge and confidence. Curriculum adjustments to achieve sustained improvement in confidence performing and supervising UGNBs long term are key to increased ED-based UGNB use.

Augmented Reality in Ultrasound-Guided Regional Anaesthesia: An Exploratory Study on Models With Potential Implications for Training

Introduction Needle tip visualisation is a key skill required for the safe practice of ultrasound-guided regional anaesthesia (UGRA). This exploratory study assesses the utility of a novel augmented reality device, NeedleTrainer™, to differentiate between anaesthetists with varying levels of UGRA experience in a simulated environment. Methods Four groups of five participants were recruited (n = 20): novice, early career, experienced anaesthetists, and UGRA experts. Each participant performed three simulated UGRA blocks using NeedleTrainer™ on healthy volunteers (n = 60). The primary aim was to determine whether there was a difference in needle tip visibility, as calculated by the device, between groups of anaesthetists with differing levels of UGRA experience. Secondary aims included the assessment of simulated block conduct by an expert assessor and subjective participant self-assessment. Results The percentage of time the simulated needle tip was maintained in view was higher in the UGRA expert group (57.1%) versus the other three groups (novice 41.8%, early career 44.5%, and experienced anaesthetists 43.6%), but did not reach statistical significance (p = 0.05). An expert assessor was able to differentiate between participants of different UGRA experience when assessing needle tip visibility (novice 3.3 out of 10, early career 5.1, experienced anaesthetists 5.9, UGRA expert group 8.7; p < 0.01) and final needle tip placement (novice 4.2 out of 10, early career 5.6, experienced anaesthetists 6.8, UGRA expert group 8.9; p < 0.01). Subjective self-assessment by participants did not differentiate UGRA experience when assessing needle tip visibility (p = 0.07) or final needle tip placement (p = 0.07). Discussion An expert assessor was able to differentiate between participants with different levels of UGRA experience in this simulated environment. Objective NeedleTrainer™ and subjective participant assessments did not reach statistical significance. The findings are novel as simulated needling using live human subjects has not been assessed before, and no previous studies have attempted to objectively quantify needle tip visibility during simulated UGRA techniques. Future research should include larger sample sizes to further assess the potential use of such technology.

Patterns of Skills Acquisition in Anesthesiologists During Simulated Interscalene Block Training on a Soft Embalmed Thiel Cadaver: Cohort Study

BACKGROUND

The demand for regional anesthesia for major surgery has increased considerably, but only a small number of anesthesiologists can provide such care. Simulations may improve clinical performance. However, opportunities to rehearse procedures are limited, and the clinical educational outcomes prescribed by the Royal College of Anesthesiologists training curriculum 2021 are difficult to attain. Educational paradigms, such as mastery learning and dedicated practice, are increasingly being used to teach technical skills to enhance skills acquisition. Moreover, high-fidelity, resilient cadaver simulators are now available: the soft embalmed Thiel cadaver shows physical characteristics and functional alignment similar to those of patients. Tissue elasticity allows tissues to expand and relax, fluid to drain away, and hundreds of repeated injections to be tolerated without causing damage. Learning curves and their intra- and interindividual dynamics have not hitherto been measured on the Thiel cadaver simulator using the mastery learning and dedicated practice educational paradigm coupled with validated, quantitative metrics, such as checklists, eye tracking metrics, and self-rating scores.

OBJECTIVE

Our primary objective was to measure the learning slopes of the scanning and needling phases of an interscalene block conducted repeatedly on a soft embalmed Thiel cadaver over a 3-hour period of training.

METHODS

A total of 30 anesthesiologists, with a wide range of experience, conducted up to 60 ultrasound-guided interscalene blocks over 3 hours on the left side of 2 soft embalmed Thiel cadavers. The duration of the scanning and needling phases was defined as the time taken to perform all the steps correctly. The primary outcome was the best-fit linear slope of the log-log transformed time to complete each phase. Our secondary objectives were to measure preprocedural psychometrics, describe deviations from the learning slope, correlate scanning and needling phase data, characterize skills according to clinical grade, measure learning curves using objective eye gaze tracking and subjective self-rating measures, and use cluster analysis to categorize performance irrespective of grade.

RESULTS

The median (IQR; range) log-log learning slopes were -0.47 (-0.62 to -0.32; -0.96 to 0.30) and -0.23 (-0.34 to -0.19; -0.71 to 0.27) during the scanning and needling phases, respectively. Locally Weighted Scatterplot Smoother curves showed wide variability in within-participant performance. The learning slopes of the scanning and needling phases correlated: ρ=0.55 (0.23-0.76), P<.001, and ρ=-0.72 (-0.46 to -0.87), P<.001, respectively. Eye gaze fixation count and glance count during the scanning and needling phases best reflected block duration. Using clustering techniques, fixation count and glance were used to identify 4 distinct patterns of learning behavior.

CONCLUSIONS

We quantified learning slopes by log-log transformation of the time taken to complete the scanning and needling phases of interscalene blocks and identified intraindividual and interindividual patterns of variability.

Eye Tracking Use in Surgical Research: A Systematic Review

INTRODUCTION

Eye tracking (ET) is a popular tool to study what factors affect the visual behaviour of surgical team members. To our knowledge, there have been no reviews to date that evaluate the broad use of ET in surgical research. This review aims to identify and assess the quality of this evidence, to synthesize how ET can be used to inform surgical practice, and to provide recommendations to improve future ET surgical studies.

METHODS

In line with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, a systematic literature review was conducted. An electronic search was performed in MEDLINE, Cochrane Central, Embase, and Web of Science databases up to September 2020. Included studies used ET to measure the visual behaviour of members of the surgical team during surgery or surgical tasks. The included studies were assessed by two independent reviewers.

RESULTS

A total of 7614 studies were identified, and 111 were included for data extraction. Eleven applications were identified; the four most common were skill assessment (41%), visual attention assessment (22%), workload measurement (17%), and skills training (10%). A summary was provided of the various ways ET could be used to inform surgical practice, and three areas were identified for the improvement of future ET studies in surgery.

CONCLUSIONS

This review provided a comprehensive summary of the various applications of ET in surgery and how ET could be used to inform surgical practice, including how to use ET to improve surgical education. The information provided in this review can also aid in the design and conduct of future ET surgical studies.

A paired comparison of nerve dimensions using B-Mode ultrasound and shear wave elastography during regional anaesthesia

Introduction:

Shear wave elastography (SWE) presents nerves in colour, but the dimensions of its colour maps have not been validated with paired B-Mode nerve images. Our primary objective was to define the bias and limits of agreement of SWE with B-Mode nerve diameter. Our secondary objectives were to compare nerve area and shape, and provide a clinical standard for future application of new colour imaging technologies such as artificial intelligence.

Materials and Methods:

Eleven combined ultrasound-guided regional nerve blocks were conducted using a dual-mode transducer. Two raters outlined nerve margins on 110 paired B-Mode and SWE images every second for 20 s before and during injection. Bias and limits of agreement were plotted on Bland-Altman plots. We hypothesized that the bias of nerve diameter would be <2.5% and that the percent limits of agreement would lie ±0.67% (2 SD) of the bias.

Results:

There was no difference in the bias (95% confidence interval (CI) limits of agreement) of nerve diameter measurement, 0.01 (−0.14 to 0.16) cm, P = 0.85, equivalent to a 1.4% (−56.6% to 59.5) % difference. The bias and limits of agreement were 0.03 (−0.08 to 0.15) cm2, P = 0.54 for cross-sectional nerve area; and 0.02 (−0.03 to 0.07), P = 0.45 for shape. Reliability (ICC) between raters was 0.96 (0.94–0.98) for B-Mode nerve area and 0.91 (0.83–0.95) for SWE nerve area.

Conclusions:

Nerve diameter measurement from B-Mode and SWE images fell within a priori measures of bias and limits of agreement.

Use of Generalizability Theory for Exploring Reliability of and Sources of Variance in Assessment of Technical Skills: A Systematic Review and Meta-Analysis

PURPOSE

Competency-based education relies on the validity and reliability of assessment scores. Generalizability (G) theory is well suited to explore the reliability of assessment tools in medical education but has only been applied to a limited extent. This study aimed to systematically review the literature using G-theory to explore the reliability of structured assessment of medical and surgical technical skills and to assess the relative contributions of different factors to variance.

METHOD

In June 2020, 11 databases, including PubMed, were searched from inception through May 31, 2020. Eligible studies included the use of G-theory to explore reliability in the context of assessment of medical and surgical technical skills. Descriptive information on study, assessment context, assessment protocol, participants being assessed, and G-analyses was extracted. Data were used to map G-theory and explore variance components analyses. A meta-analysis was conducted to synthesize the extracted data on the sources of variance and reliability.

RESULTS

Forty-four studies were included; of these, 39 had sufficient data for meta-analysis. The total pool included 35,284 unique assessments of 31,496 unique performances of 4,154 participants. Person variance had a pooled effect of 44.2% (95% confidence interval [CI], 36.8%-51.5%). Only assessment tool type (Objective Structured Assessment of Technical Skills-type vs task-based checklist-type) had a significant effect on person variance. The pooled reliability (G-coefficient) was 0.65 (95% CI, .59-.70). Most studies included decision studies (39, 88.6%) and generally seemed to have higher ratios of performances to assessors to achieve a sufficiently reliable assessment.

CONCLUSIONS

G-theory is increasingly being used to examine reliability of technical skills assessment in medical education, but more rigor in reporting is warranted. Contextual factors can potentially affect variance components and thereby reliability estimates and should be considered, especially in high-stakes assessment. Reliability analysis should be a best practice when developing assessment of technical skills.

Validation of the soft-embalmed Thiel cadaver as a high-fidelity simulator of pressure during targeted nerve injection

INTRODUCTION

Although administration of regional anesthesia nerve blocks has increased during the COVID-19 pandemic, training opportunities in regional anesthesia have reduced. Simulation training may enhance skills, but simulators must be accurate enough for trainees to engage in a realistic way-for example, detection of excessive injection pressure. The soft-embalmed Thiel cadaver is a life-like, durable simulator that is used for dedicated practice and mastery learning training in regional anesthesia. We hypothesized that injection opening pressure in perineural tissue, at epineurium and in subepineurium were similar to opening pressures measured in experimental animals, fresh frozen cadavers, glycol soft-fix cadavers and patients.

METHODS

We systematically reviewed historical data, then conducted three validation studies delivering a 0.5 mL hydrolocation bolus of embalming fluid and recording injection pressure. First, we delivered the bolus at 12 mL/min at epimysium, perineural tissue, epineurium and in subepineurium at 48 peripheral nerve sites on three cadavers. Second, we delivered the bolus at using three infusion rates: 1 mL/min, 6 mL/min and 12 mL/min on epineurium at 70 peripheral nerve sites on five cadavers. Third, we repeated three injections (12 mL/min) at 24 epineural sites over the median and sciatic nerves of three cadavers.

RESULTS

Mean (95%) injection pressure was greater at epineurium compared with subepineurium (geometric ratio 1.2 (95% CI: 0.9 to 1.6)), p=0.04, and perineural tissue (geometric ratio 5.1 (95% CI: 3.7 to 7.0)), p<0.0001. Mean (95%) injection pressure was greater at 12 mL/min compared with 1 mL/min (geometric ratio 1.6 (95% CI: 1.2 to 2.1), p=0.005). Pressure measurements were similar in study 3 (p>0.05 for all comparisons).

DISCUSSION

We conclude that the soft-embalmed Thiel cadaver is a realistic simulator of injection opening pressure.

Needle insertion forces and fluid injection pressures during targeting of nerves in a soft embalmed cadaver model

PURPOSE

Forceful needle-nerve contact and high subepineural pressures and are recognised causes of nerve damage. Pressure and force measurements are necessary to inform the mechanisms of nerve injury, build virtual simulator environments and provide operator feedback during simulation training. However, the range of pressures and forces encountered at tissue layers during targeted needle insertion and fluid injection are not known.

METHODS

We built a needle that recorded in-line pressure during fluid injection and continuously measured force at the needle tip. Two anaesthetists were randomised to insert a 21 g block needle at 48 nerve sites on both sides of 3 soft embalmed Thiel cadavers. Our objective was to measure pressure and force during the course of targeted nerve injection at epimysium, in perineural tissue, on epineurium and during subepineural injection. At each interface, we infused a 0.5 ml bolus of embalming solution at a rate of 12 ml min^-1 and recorded the pressure response. Force was measured continuously in the background throughout the procedure.

RESULTS

Pressure was greater at epineurium and within subepineurium than perineural tissue, geometric ratio (95% CI) 4.7 (3.0-7.3) kPa and 3.8 (2.5-5.7) kPa, respectively, both P < 0.0001. Force on nerve contact and on nerve penetration was greater than force in perineural tissue, geometric ratios (95% CI) 3.0 (1.9-4.7) N and 3.6 (2.2-7.5) N, respectively, both P < 0.0001. On nerve contact, 1 in 6 insertions were ≥ 5 N CONCLUSIONS: Despite valid infusion pressures, anaesthetists exerted excessive force on nerves.

Novel approaches to needle tracking and visualisation

The accuracy and reliability of ultrasound are still insufficient to guarantee complete and safe nerve block for all patients. Injection of local anaesthetic close to, but not touching, the nerve is key to outcomes, but the exact relationship between the needle tip and nerve epineurium is difficult to evaluate, even with ultrasound. Ultrasound has insufficient resolution, tissues are difficult to discern due to acoustic impedance and needles are more difficult to see with increased angulation. The limitations of ultrasound have shifted the focus of innovation towards bio-markers that help detect needle tip position by utilising the physical properties of tissues, (e.g. pressure, electrical, optics, acoustic and elastic). Although most are at the laboratory stage and results are as yet only available from phantom or cadaver studies, clinical trials are imminent. For example, fine optical fibres placed within the lumen of block needles can measure needle tip pressure. Electrical impedance differentiates between intraneural and perineural needle tip placement. A new tip tracker needle has a piezo element embedded at its distal end that tracks the needle tip in-plane and out-of-plane as a blue/red or green circle depending on its relative location within the beam. Micro-ultrasound at the tip of the needle is in development. Early images using 40MHz in anaesthetised pigs reveal muscle striation, distinct epineurium and 30-40 fascicles > 75 micron in diameter. The next few years will see a technological revolution in tip-tracking technology that has the potential to improve patient safety and, in doing so, change practice.

Education and training in ultrasound-guided regional anaesthesia and pain medicine

PURPOSE OF REVIEW

Effective and safe regional anaesthesia and pain medicine procedures require clinicians to learn and master complex theoretical knowledge and motor skills. This review aims to summarize articles relevant to education and training in these skill sets in the previous 2 years.

RECENT FINDINGS

Twenty-two articles were identified, investigating nine out of the 13 top-ranked research topics in education and training in regional anaesthesia. Research topics addressed by these articles included prerotation simulation, deliberate practice combined with formative assessment tools, validation of assessment tools, three-dimensional-printed models, and knowledge translation from simulation to clinical practice. Emerging concepts investigated for their applications in regional anaesthesia included eye-tracking as a surrogate metric when evaluating proficiency, and elastography aiding visual salience to distinguish appropriate perineural and inappropriate intraneural injections.

SUMMARY

Research into education and training in regional anaesthesia covered multiple and diverse topics. Methodological limitations were noted in several articles, reflecting the difficulties in designing and conducting medical education studies. Nonetheless, the evidence-base continues to mature and innovations provide exciting future possibilities.

The effect of an ultrasound-activated needle tip tracker needle on the performance of sciatic nerve block on a soft embalmed Thiel cadaver

In this study, we measured the performance of medical students and anaesthetists using a new tracker needle during simulated sciatic nerve block on soft embalmed cadavers. The tracker needle incorporates a piezo element near its tip that generates an electrical signal in response to insonation. A circle, superimposed on the ultrasound image surrounding the needle tip, changes size and colour according to the position of the piezo element within the ultrasound beam. Our primary objective was to compare sciatic block performance with the tracker switched on and off. Our secondary objectives were to record psychometrics, procedure efficiency, participant self-regulation and focused attention using eye-tracking technology. Our primary outcome measures were the number of steps successfully performed and the number of errors committed during each block. Videos were scored by trained experts using validated checklists. Sequential tracker activation and deactivation was randomised equally within subjects. With needle activation, steps improved in 10 (25%) subjects and errors reduced in six (15%) subjects. The most important steps were: needle tip identification before injection, OR (95%CI) 2.12 (1.61-2.80; p < 0.001); and needle tip identification before advance of the needle, 1.80 (1.36-2.39; p < 0.001). The most important errors were: failure to identify the needle tip before injection, 2.40 (1.78-3.24; p < 0.001); and failure to quickly regain needle tip position when tip visibility was lost, 2.03 (1.5-2.75; p < 0.001). In conclusion, needle-tracking technology improved performance in a quarter of subjects.