Leading in the development, standardised evaluation, and adoption of artificial intelligence in clinical practice: regional anaesthesia as an example

A recent study by Suissa and colleagues explored the clinical relevance of a medical image segmentation metric (Dice metric) commonly used in the field of artificial intelligence (AI). They showed that pixel-wise agreement for physician identification of structures on ultrasound images is variable, and a relatively low Dice metric (0.34) correlated to a substantial agreement on subjective clinical assessment. We highlight the need to bring structure and clinical perspective to the evaluation of medical AI, which clinicians are best placed to direct.

Research priorities in regional anaesthesia: an international Delphi study

BACKGROUND

Regional anaesthesia use is growing worldwide, and there is an increasing emphasis on research in regional anaesthesia to improve patient outcomes. However, priorities for future study remain unclear. We therefore conducted an international research prioritisation exercise, setting the agenda for future investigators and funding bodies.

METHODS

We invited members of specialist regional anaesthesia societies from six continents to propose research questions that they felt were unanswered. These were consolidated into representative indicative questions, and a literature review was undertaken to determine if any indicative questions were already answered by published work. Unanswered indicative questions entered a three-round modified Delphi process, whereby 29 experts in regional anaesthesia (representing all participating specialist societies) rated each indicative question for inclusion on a final high priority shortlist. If ≥75% of participants rated an indicative question as 'definitely' include in any round, it was accepted. Indicative questions rated as 'definitely' or 'probably' by <50% of participants in any round were excluded. Retained indicative questions were further ranked based on the rating score in the final Delphi round. The final research priorities were ratified by the Delphi expert group.

RESULTS

There were 1318 responses from 516 people in the initial survey, from which 71 indicative questions were formed, of which 68 entered the modified Delphi process. Eleven 'highest priority' research questions were short listed, covering themes of pain management; training and assessment; clinical practice and efficacy; technology and equipment.

CONCLUSIONS

We prioritised unanswered research questions in regional anaesthesia. These will inform a coordinated global research strategy for regional anaesthesia and direct investigators to address high-priority areas.

Artificial intelligence for ultrasound scanning in regional anaesthesia: a scoping review of the evidence from multiple disciplines

BACKGROUND

Artificial intelligence (AI) for ultrasound scanning in regional anaesthesia is a rapidly developing interdisciplinary field. There is a risk that work could be undertaken in parallel by different elements of the community but with a lack of knowledge transfer between disciplines, leading to repetition and diverging methodologies. This scoping review aimed to identify and map the available literature on the accuracy and utility of AI systems for ultrasound scanning in regional anaesthesia.

METHODS

A literature search was conducted using Medline, Embase, CINAHL, IEEE Xplore, and ACM Digital Library. Clinical trial registries, a registry of doctoral theses, regulatory authority databases, and websites of learned societies in the field were searched. Online commercial sources were also reviewed.

RESULTS

In total, 13,014 sources were identified; 116 were included for full-text review. A marked change in AI techniques was noted in 2016-17, from which point on the predominant technique used was deep learning. Methods of evaluating accuracy are variable, meaning it is impossible to compare the performance of one model with another. Evaluations of utility are more comparable, but predominantly gained from the simulation setting with limited clinical data on efficacy or safety. Study methodology and reporting lack standardisation.

CONCLUSIONS

There is a lack of structure to the evaluation of accuracy and utility of AI for ultrasound scanning in regional anaesthesia, which hinders rigorous appraisal and clinical uptake. A framework for consistent evaluation is needed to inform model evaluation, allow comparison between approaches/models, and facilitate appropriate clinical adoption.

Augmented Reality in Ultrasound-Guided Regional Anaesthesia: An Exploratory Study on Models With Potential Implications for Training

Introduction Needle tip visualisation is a key skill required for the safe practice of ultrasound-guided regional anaesthesia (UGRA). This exploratory study assesses the utility of a novel augmented reality device, NeedleTrainer™, to differentiate between anaesthetists with varying levels of UGRA experience in a simulated environment. Methods Four groups of five participants were recruited (n = 20): novice, early career, experienced anaesthetists, and UGRA experts. Each participant performed three simulated UGRA blocks using NeedleTrainer™ on healthy volunteers (n = 60). The primary aim was to determine whether there was a difference in needle tip visibility, as calculated by the device, between groups of anaesthetists with differing levels of UGRA experience. Secondary aims included the assessment of simulated block conduct by an expert assessor and subjective participant self-assessment. Results The percentage of time the simulated needle tip was maintained in view was higher in the UGRA expert group (57.1%) versus the other three groups (novice 41.8%, early career 44.5%, and experienced anaesthetists 43.6%), but did not reach statistical significance (p = 0.05). An expert assessor was able to differentiate between participants of different UGRA experience when assessing needle tip visibility (novice 3.3 out of 10, early career 5.1, experienced anaesthetists 5.9, UGRA expert group 8.7; p < 0.01) and final needle tip placement (novice 4.2 out of 10, early career 5.6, experienced anaesthetists 6.8, UGRA expert group 8.9; p < 0.01). Subjective self-assessment by participants did not differentiate UGRA experience when assessing needle tip visibility (p = 0.07) or final needle tip placement (p = 0.07). Discussion An expert assessor was able to differentiate between participants with different levels of UGRA experience in this simulated environment. Objective NeedleTrainer™ and subjective participant assessments did not reach statistical significance. The findings are novel as simulated needling using live human subjects has not been assessed before, and no previous studies have attempted to objectively quantify needle tip visibility during simulated UGRA techniques. Future research should include larger sample sizes to further assess the potential use of such technology.

Assistive artificial intelligence for ultrasound image interpretation in regional anaesthesia: an external validation study

BACKGROUND

Ultrasonound is used to identify anatomical structures during regional anaesthesia and to guide needle insertion and injection of local anaesthetic. ScanNav Anatomy Peripheral Nerve Block (Intelligent Ultrasound, Cardiff, UK) is an artificial intelligence-based device that produces a colour overlay on real-time B-mode ultrasound to highlight anatomical structures of interest. We evaluated the accuracy of the artificial-intelligence colour overlay and its perceived influence on risk of adverse events or block failure.

METHODS

Ultrasound-guided regional anaesthesia experts acquired 720 videos from 40 volunteers (across nine anatomical regions) without using the device. The artificial-intelligence colour overlay was subsequently applied. Three more experts independently reviewed each video (with the original unmodified video) to assess accuracy of the colour overlay in relation to key anatomical structures (true positive/negative and false positive/negative) and the potential for highlighting to modify perceived risk of adverse events (needle trauma to nerves, arteries, pleura, and peritoneum) or block failure.

RESULTS

The artificial-intelligence models identified the structure of interest in 93.5% of cases (1519/1624), with a false-negative rate of 3.0% (48/1624) and a false-positive rate of 3.5% (57/1624). Highlighting was judged to reduce the risk of unwanted needle trauma to nerves, arteries, pleura, and peritoneum in 62.9-86.4% of cases (302/480 to 345/400), and to increase the risk in 0.0-1.7% (0/160 to 8/480). Risk of block failure was reported to be reduced in 81.3% of scans (585/720) and to be increased in 1.8% (13/720).

CONCLUSIONS

Artificial intelligence-based devices can potentially aid image acquisition and interpretation in ultrasound-guided regional anaesthesia. Further studies are necessary to demonstrate their effectiveness in supporting training and clinical practice.

CLINICAL TRIAL REGISTRATION

NCT04906018.

Evaluation of the impact of assistive artificial intelligence on ultrasound scanning for regional anaesthesia

BACKGROUND

Ultrasound-guided regional anaesthesia relies on the visualisation of key landmark, target, and safety structures on ultrasound. However, this can be challenging, particularly for inexperienced practitioners. Artificial intelligence (AI) is increasingly being applied to medical image interpretation, including ultrasound. In this exploratory study, we evaluated ultrasound scanning performance by non-experts in ultrasound-guided regional anaesthesia, with and without the use of an assistive AI device.

METHODS

Twenty-one anaesthetists, all non-experts in ultrasound-guided regional anaesthesia, underwent a standardised teaching session in ultrasound scanning for six peripheral nerve blocks. All then performed a scan for each block; half of the scans were performed with AI assistance and half without. Experts assessed acquisition of the correct block view and correct identification of sono-anatomical structures on each view. Participants reported scan confidence, experts provided a global rating score of scan performance, and scans were timed.

RESULTS

Experts assessed 126 ultrasound scans. Participants acquired the correct block view in 56/62 (90.3%) scans with the device compared with 47/62 (75.1%) without (P=0.031, two data points lost). Correct identification of sono-anatomical structures on the view was 188/212 (88.8%) with the device compared with 161/208 (77.4%) without (P=0.002). There was no significant overall difference in participant confidence, expert global performance score, or scan time.

CONCLUSIONS

Use of an assistive AI device was associated with improved ultrasound image acquisition and interpretation. Such technology holds potential to augment performance of ultrasound scanning for regional anaesthesia by non-experts, potentially expanding patient access to these techniques.

CLINICAL TRIAL REGISTRATION

NCT05156099.

Finding the fifth intercostal space for chest drain insertion: guidelines and ultrasound

OBJECTIVES

International guidelines exist for chest drain insertion and recommend identifying the fifth intercostal space or above, around the midaxillary line. In a recent study, applying these guidelines in cadavers risked insertion in the 6th intercostal space or below in 80% of cases. However, there are limitations of cadaveric studies and this investigation uses ultrasound to determine the intercostal space identified when applying these guidelines in healthy adult volunteers.

METHODS

On each side of the chest wall in 31 volunteers, the position for drain insertion was identified using the European Trauma Course method, Advanced Trauma Life Support (ATLS) method, British Thoracic Society's 'safe triangle' and the 'traditional' method of palpation. Ultrasound imaging was used to determine the relationship of the skin marks with the underlying intercostal spaces.

RESULTS

Five methods were assessed on 60 sides. In contrast to the cadaveric study, 94% of skin marks lay over a safe intercostal space. However, the range of intercostal spaces found spanned the second to the seventh space. In 44% of women, the inferior boundary of the 'safe triangle' and the ATLS guidelines located the sixth intercostal space or below.

CONCLUSIONS

Current guidelines often identify a safe site for chest drain insertion, although the same site is not reproducibly found. In addition, women appear to be at risk of subdiaphragmatic drain insertion when the nipple is used to identify the fifth intercostal space. Real-time ultrasonography can be used to confirm the intercostal space during this procedure, although a safe guideline is still needed for circumstances in which ultrasound is not possible.