Variability between human experts and artificial intelligence in identification of anatomical structures by ultrasound in regional anaesthesia: a framework for evaluation of assistive artificial intelligence

Regional anaesthesia research priorities: a Regional Anaesthesia UK (RA-UK) priority setting partnership involving patients, carers and healthcare professionals

INTRODUCTION

Regional anaesthesia provides important clinical benefits to patients but is underutilised. A barrier to widespread adoption may be the focus of regional anaesthesia research on novel techniques rather than evaluating and optimising existing approaches. Research priorities in regional anaesthesia identified by anaesthetists have been published, but the views of patients, carers and other healthcare professionals have not been considered previously. Therefore, we launched a multidisciplinary research priority setting partnership that aimed to establish key regional anaesthesia research priorities for the UK.

METHODS

Research suggestions from key stakeholders (defined by their interaction with regional anaesthesia) were gathered using an online survey. These suggestions were analysed to identify common themes and then combined to formulate indicative research questions. After an extensive literature review, unanswered and partially answered questions were prioritised via an interim online survey and then ranked as a top 10 list during a final live virtual multidisciplinary prioritisation workshop.

RESULTS

In total, 210 individuals completed the initial survey and suggested 518 research questions. Fifty-seven indicative questions were formed, of which three were considered fully answered after literature review and one not feasible. The interim online survey received 335 responses, which identified the 24 highest priority questions from the 53 presented. At the final live prioritisation workshop, through a nominal group process, we identified the top 10 regional anaesthesia research priorities. These aligned with three broad thematic areas: pain management (two questions); patient safety (six questions); and recovery from surgery (two questions).

DISCUSSION

This initiative has resulted in a list of research questions prioritised by patients, carers and a multidisciplinary group of healthcare professionals that should be used to inform and support future regional anaesthesia research in the UK.

Prospective randomized evaluation of the sustained impact of assistive artificial intelligence on anesthetists' ultrasound scanning for regional anesthesia

Objectives

Ultrasound-guided regional anesthesia (UGRA) relies on acquiring and interpreting an appropriate view of sonoanatomy. Artificial intelligence (AI) has the potential to aid this by applying a color overlay to key sonoanatomical structures.The primary aim was to determine whether an AI-generated color overlay was associated with a difference in participants' ability to identify an appropriate block view over a 2-month period after a standardized teaching session (as judged by a blinded assessor). Secondary outcomes included the ability to identify an appropriate block view (unblinded assessor), global rating score and participant confidence scores.

Design

Randomized, partially blinded, prospective cross-over study.

Setting

Simulation scans on healthy volunteers. Initial assessments on 29 November 2022 and 30 November 2022, with follow-up on 25 January 2023 - 27 January 2023.

Participants

57 junior anesthetists undertook initial assessments and 51 (89.47%) returned at 2 months.

Intervention

Participants performed ultrasound scans for six peripheral nerve blocks, with AI assistance randomized to half of the blocks. Cross-over assignment was employed for 2 months.

Main outcome measures

Blinded experts assessed whether the block view acquired was acceptable (yes/no). Unblinded experts also assessed this parameter and provided a global performance rating (0-100). Participants reported scan confidence (0-100).

Results

AI assistance was associated with a higher rate of appropriate block view acquisition in both blinded and unblinded assessments (p=0.02 and <0.01, respectively). Participant confidence and expert rating scores were superior throughout (all p<0.01).

Conclusions

Assistive AI was associated with superior ultrasound scanning performance 2 months after formal teaching. It may aid application of sonoanatomical knowledge and skills gained in teaching, to support delivery of UGRA beyond the immediate post-teaching period.

Trial registration number

www.clinicaltrials.govNCT05583032.

Regional Anaesthesia, Contemporary Techniques, and Associated Advancements: A Narrative Review

In particular, the application of regional anaesthesia techniques in existing medicine can be characterized as experiencing regular changes in recent decades. It is useful for obtaining accurate and efficient pain management solutions, from the basic spinal and epidural blocks to the novel ultrasound nerve blocks and constant catheter procedures. These advancements do enhance not only the value of the perioperative period but also the patient's rated optimization as enhancing satisfaction, better precision, and the safety of nerve block placement. The use of ultrasound technology makes it even easier to determine the proper positioning of the needle and to monitor nerve block placement. Moreover, the duration and efficiency of regional anaesthesia are being enhanced by state-of-the-art approaches, which come in the form of liposomal bupivacaine, and better recovery plans and protocols, which shorten recovery time and decrease the number of hospital days. As these methods develop further, more improvements in the safety, efficacy, and applicability of regional anaesthesia in contemporary medicine are anticipated through continued research and innovation.

Leading in the development, standardised evaluation, and adoption of artificial intelligence in clinical practice: regional anaesthesia as an example

A recent study by Suissa and colleagues explored the clinical relevance of a medical image segmentation metric (Dice metric) commonly used in the field of artificial intelligence (AI). They showed that pixel-wise agreement for physician identification of structures on ultrasound images is variable, and a relatively low Dice metric (0.34) correlated to a substantial agreement on subjective clinical assessment. We highlight the need to bring structure and clinical perspective to the evaluation of medical AI, which clinicians are best placed to direct.

Celebrating the state of the art and innovations in regional anaesthesia in the British Journal of Anaesthesia

To coincide with the annual scientific meeting of Regional Anaesthesia UK in London 2024, where there is a joint scientific session with the British Journal of Anaesthesia, a special regional anaesthesia edition of the journal has been produced. This editorial offers some highlights from the manuscripts contained within the special edition.

Research priorities in regional anaesthesia: an international Delphi study

BACKGROUND

Regional anaesthesia use is growing worldwide, and there is an increasing emphasis on research in regional anaesthesia to improve patient outcomes. However, priorities for future study remain unclear. We therefore conducted an international research prioritisation exercise, setting the agenda for future investigators and funding bodies.

METHODS

We invited members of specialist regional anaesthesia societies from six continents to propose research questions that they felt were unanswered. These were consolidated into representative indicative questions, and a literature review was undertaken to determine if any indicative questions were already answered by published work. Unanswered indicative questions entered a three-round modified Delphi process, whereby 29 experts in regional anaesthesia (representing all participating specialist societies) rated each indicative question for inclusion on a final high priority shortlist. If ≥75% of participants rated an indicative question as 'definitely' include in any round, it was accepted. Indicative questions rated as 'definitely' or 'probably' by <50% of participants in any round were excluded. Retained indicative questions were further ranked based on the rating score in the final Delphi round. The final research priorities were ratified by the Delphi expert group.

RESULTS

There were 1318 responses from 516 people in the initial survey, from which 71 indicative questions were formed, of which 68 entered the modified Delphi process. Eleven 'highest priority' research questions were short listed, covering themes of pain management; training and assessment; clinical practice and efficacy; technology and equipment.

CONCLUSIONS

We prioritised unanswered research questions in regional anaesthesia. These will inform a coordinated global research strategy for regional anaesthesia and direct investigators to address high-priority areas.

Artificial intelligence for ultrasound scanning in regional anaesthesia: a scoping review of the evidence from multiple disciplines

BACKGROUND

Artificial intelligence (AI) for ultrasound scanning in regional anaesthesia is a rapidly developing interdisciplinary field. There is a risk that work could be undertaken in parallel by different elements of the community but with a lack of knowledge transfer between disciplines, leading to repetition and diverging methodologies. This scoping review aimed to identify and map the available literature on the accuracy and utility of AI systems for ultrasound scanning in regional anaesthesia.

METHODS

A literature search was conducted using Medline, Embase, CINAHL, IEEE Xplore, and ACM Digital Library. Clinical trial registries, a registry of doctoral theses, regulatory authority databases, and websites of learned societies in the field were searched. Online commercial sources were also reviewed.

RESULTS

In total, 13,014 sources were identified; 116 were included for full-text review. A marked change in AI techniques was noted in 2016-17, from which point on the predominant technique used was deep learning. Methods of evaluating accuracy are variable, meaning it is impossible to compare the performance of one model with another. Evaluations of utility are more comparable, but predominantly gained from the simulation setting with limited clinical data on efficacy or safety. Study methodology and reporting lack standardisation.

CONCLUSIONS

There is a lack of structure to the evaluation of accuracy and utility of AI for ultrasound scanning in regional anaesthesia, which hinders rigorous appraisal and clinical uptake. A framework for consistent evaluation is needed to inform model evaluation, allow comparison between approaches/models, and facilitate appropriate clinical adoption.