Robotic and artificial intelligence for keyhole neurosurgery: the ROBOCAST project, a multi-modal autonomous path planner

Intraoperative Applications of Artificial Intelligence in Robotic Surgery: A Scoping Review of Current Development Stages and Levels of Autonomy

OBJECTIVE

A scoping review of the literature was conducted to identify intraoperative artificial intelligence (AI) applications for robotic surgery under development and categorize them by (1) purpose of the applications, (2) level of autonomy, (3) stage of development, and (4) type of measured outcome.

BACKGROUND

In robotic surgery, AI-based applications have the potential to disrupt a field so far based on a master-slave paradigm. However, there is no available overview about this technology's current stage of development and level of autonomy.

METHODS

MEDLINE and EMBASE were searched between January 1, 2010 and May 21, 2022. Abstract screening, full-text review, and data extraction were performed independently by 2 reviewers. The level of autonomy was defined according to the Yang and colleagues' classification and stage of development according to the Idea, Development, Evaluation, Assessment, and Long-term follow-up framework.

RESULTS

One hundred twenty-nine studies were included in the review. Ninety-seven studies (75%) described applications providing Robot Assistance (autonomy level 1), 30 studies (23%) application enabling Task Autonomy (autonomy level 2), and 2 studies (2%) application achieving Conditional autonomy (autonomy level 3). All studies were at Idea, Development, Evaluation, Assessment, and Long-term follow-up stage 0 and no clinical investigations on humans were found. One hundred sixteen (90%) conducted in silico or ex vivo experiments on inorganic material, 9 (7%) ex vivo experiments on organic material, and 4 (3%) performed in vivo experiments in porcine models.

CONCLUSIONS

Clinical evaluation of intraoperative AI applications for robotic surgery is still in its infancy and most applications have a low level of autonomy. With increasing levels of autonomy, the evaluation focus seems to shift from AI-specific metrics to process outcomes, although common standards are needed to allow comparison between systems.

The role of robotic surgery in neurological cases: A systematic review on brain and spine applications

The application of robotic surgery technologies in neurological surgeries resulted in some advantages compared to traditional surgeries, including higher accuracy and dexterity enhancement. Its success in various surgical fields, especially in urology, cardiology, and gynecology surgeries was reported in previous studies, and similar advantages in neurological surgeries are expected. Surgeries in the central nervous system with the pathology of millimeters through small working channels around vital tissue need especially high precision. Applying robotic surgery is therefore an interesting dilemma for these situations. This article reviews various studies published on the application of brain and spine robotic surgery and discusses the current application of robotic technology in neurological cases.

Artificial Intelligence in Brain Tumour Surgery-An Emerging Paradigm

Artificial intelligence (AI) platforms have the potential to cause a paradigm shift in brain tumour surgery. Brain tumour surgery augmented with AI can result in safer and more effective treatment. In this review article, we explore the current and future role of AI in patients undergoing brain tumour surgery, including aiding diagnosis, optimising the surgical plan, providing support during the operation, and better predicting the prognosis. Finally, we discuss barriers to the successful clinical implementation, the ethical concerns, and we provide our perspective on how the field could be advanced.

Artificial intelligence-enhanced intraoperative neurosurgical workflow: state of the art and future perspectives

BACKGROUND

Artificial Intelligence (AI) and Machine Learning (ML) augment decision-making processes and productivity by supporting surgeons over a range of clinical activities: from diagnosis and preoperative planning to intraoperative surgical assistance. We reviewed the literature to identify current AI platforms applied to neurosurgical perioperative and intraoperative settings and describe their role in multiple subspecialties.

METHODS

A systematic review of the literature was conducted following the PRISMA guidelines. PubMed, EMBASE, and Scopus databases were searched from inception to December 31, 2020. Original articles were included if they: presented AI platforms implemented in perioperative, intraoperative settings and reported ML models' performance metrics. Due to the heterogeneity in neurosurgical applications, a qualitative synthesis was deemed appropriate. The risk of bias and applicability of predicted outcomes were assessed using the PROBAST tool.

RESULTS

41 articles were included. All studies evaluated a supervised learning algorithm. A total of 10 ML models were described; the most frequent were neural networks (n = 15) and tree-based models (n = 13). Overall, the risk of bias was medium-high, but applicability was considered positive for all studies. Articles were grouped into 4 categories according to the subspecialty of interest: neuro-oncology, spine, functional and other. For each category, different prediction tasks were identified.

CONCLUSIONS

In this review, we summarize the state-of-art applications of AI for the intraoperative augmentation of neurosurgical workflows across multiple subspecialties. ML models may boost surgical team performances by reducing human errors and providing patient-tailored surgical plans, but further and higher-quality studies need to be conducted.

Cranial neurosurgical robotics

OBJECT

The purpose of this review is to highlight the major factors limiting the progress of robotics development in the field of cranial neurosurgery.

METHODS

A literature search was performed focused on published reports of any Neurosurgical technology developed for use in cranial neurosurgery. Technology was reviewed and assessed for strengths and weaknesses, use in patients and whether or not the project was active or closed.

RESULTS

Published reports of 24 robots are discussed going back to 1985. In total, there were 9 robots used in patients (PUMA, Robot Hand, EXPERT, Neuromate, Evolution 1, ROSA, iSYS1, NeuroArm and NeuRobot) and only 2 active today (ROSA, NeuroArm). Of all clinically active systems, only three were used in more than 30 patients (ROSA, iSYS1 & NeuroArm). Projects were limited by cost, technology adoption, and clinical utility to actually improve workflow. The most common use of developed robots is for Stereotaxis.

CONCLUSIONS

There is a clear void in the area of cranial neurosurgery regarding robotics technology despite success in other fields of surgery. Significant factors such as cost, technology limitations, market size and regulatory pathway all contribute to a steep gradient for success.

Automated neurosurgical stereotactic planning for intraoperative use: a comprehensive review of the literature and perspectives

The creation of intracranial stereotactic trajectories, from entry point to target point, is still mostly done manually by the neurosurgeon. The development of automated stereotactic planning tools has been described in the literature. This systematic review aims to assess the effectiveness of stereotactic planning procedure automation and develop tools for patients undergoing neurosurgical stereotactic procedures. PubMed/MEDLINE, EMBASE, Google Scholar, CINAHL, PsycINFO, and Cochrane Register of Controlled Trials databases were searched from inception to September 1, 2019, at the exception of Google Scholar (from 1 January 2010 to September 1, 2019) in French and English. Eligible studies included all studies proposing automated stereotactic planning. A total of 1543 studies were screened. Forty-two studies were included in the systematic review, including 18 (42.9%) conference papers. The surgical procedures planned automatically were mainly deep brain stimulation (n = 14, 33.3%), stereoelectroencephalography (n = 12, 28.6%), and not specified (n = 10, 23.8%). The most frequently used surgical constraints to plan the trajectory were blood vessels (n = 32, 76.2%), cerebral sulci (n = 27, 64.3%), and cerebral ventricles (n = 23, 54.8%). The distance from blood vessels ranged from 1.96 to 4.78 mm for manual trajectories and from 2.47 to 7.0 mm for automated trajectories. At least one neurosurgeon was involved in 36 studies (85.7%). The automated stereotactic trajectory was preferred in 75.4% of the studied cases (range 30-92.9). Only 3 (7.1%) studies were multicentric. No study reported prospective use of the planning software. Stereotactic planning automation is a promising tool to provide valuable stereotactic trajectories for clinical applications.

Implementing Precision Medicine and Artificial Intelligence in Plastic Surgery: Concepts and Future Prospects

Precision medicine, or the individualization of evidence-based medicine, is forthcoming. As surgeons, we must be prepared for the integration of patient and system factors. Plastic surgeons regard themselves as innovators and early adopters. As such, we need our adaptability now more than ever to implement digital advancements and precision medicine into our practices. The integration of artificial intelligence (AI) technology and the capture of big data techniques should foster the next great leaps in medicine and surgery, allowing us to capture the detailed minutiae of precision medicine. The algorithmic process of artificial neural networks will guide large-scale analysis of data, including features such as pattern recognition and rapid quantification, to organize and distribute data to surgeons seamlessly. This vast digital collection of information, commonly termed “big data,” is only one potential application of AI. By incorporating big data, the cognitive abilities of a surgeon can be complemented by the computer to improve patient-centered care. Furthermore, the use of AI will provide individual patients with increased access to the broadening world of precision medicine. Therefore, plastic surgeons must learn how to use AI within the contexts of our practices to keep up with an evolving field in medicine. Although rudimentary in its practice, we present a glimpse of the potential applications of AI in plastic surgery to incorporate the practice of precision medicine into the care that we deliver.

An introduction and overview of machine learning in neurosurgical care

BACKGROUND

Machine learning (ML) is a branch of artificial intelligence that allows computers to learn from large complex datasets without being explicitly programmed. Although ML is already widely manifest in our daily lives in various forms, the considerable potential of ML has yet to find its way into mainstream medical research and day-to-day clinical care. The complex diagnostic and therapeutic modalities used in neurosurgery provide a vast amount of data that is ideally suited for ML models. This systematic review explores ML's potential to assist and improve neurosurgical care.

METHOD

A systematic literature search was performed in the PubMed and Embase databases to identify all potentially relevant studies up to January 1, 2017. All studies were included that evaluated ML models assisting neurosurgical treatment.

RESULTS

Of the 6,402 citations identified, 221 studies were selected after subsequent title/abstract and full-text screening. In these studies, ML was used to assist surgical treatment of patients with epilepsy, brain tumors, spinal lesions, neurovascular pathology, Parkinson's disease, traumatic brain injury, and hydrocephalus. Across multiple paradigms, ML was found to be a valuable tool for presurgical planning, intraoperative guidance, neurophysiological monitoring, and neurosurgical outcome prediction.

CONCLUSIONS

ML has started to find applications aimed at improving neurosurgical care by increasing the efficiency and precision of perioperative decision-making. A thorough validation of specific ML models is essential before implementation in clinical neurosurgical care. To bridge the gap between research and clinical care, practical and ethical issues should be considered parallel to the development of these techniques.

Prevalence of haptic feedback in robot-mediated surgery: a systematic review of literature

With the successful uptake and inclusion of robotic systems in minimally invasive surgery and with the increasing application of robotic surgery (RS) in numerous surgical specialities worldwide, there is now a need to develop and enhance the technology further. One such improvement is the implementation and amalgamation of haptic feedback technology into RS which will permit the operating surgeon on the console to receive haptic information on the type of tissue being operated on. The main advantage of using this is to allow the operating surgeon to feel and control the amount of force applied to different tissues during surgery thus minimising the risk of tissue damage due to both the direct and indirect effects of excessive tissue force or tension being applied during RS. We performed a two-rater systematic review to identify the latest developments and potential avenues of improving technology in the application and implementation of haptic feedback technology to the operating surgeon on the console during RS. This review provides a summary of technological enhancements in RS, considering different stages of work, from proof of concept to cadaver tissue testing, surgery in animals, and finally real implementation in surgical practice. We identify that at the time of this review, while there is a unanimous agreement regarding need for haptic and tactile feedback, there are no solutions or products available that address this need. There is a scope and need for new developments in haptic augmentation for robot-mediated surgery with the aim of improving patient care and robotic surgical technology further.

Multimodal connectivity based eloquence score computation and visualisation for computer-aided neurosurgical path planning

Non-invasive assessment of cognitive importance has been a major challenge for planning of neurosurgical procedures. In the past decade, in vivo brain imaging modalities have been considered for estimating the ‘eloquence’ of brain areas. In order to estimate the impact of damage caused by an access path towards a target region inside of the skull, multi-modal metrics are introduced in this paper. Accordingly, this estimated damage is obtained by combining multi-modal metrics. In other words, this damage is an aggregate of intervened grey matter volume and axonal fibre numbers, weighted by their importance within the assigned anatomical and functional networks. To validate these metrics, an exhaustive search algorithm is implemented for characterising the solution space and visually representing connectional cost associated with a path initiated from underlying points. In this presentation, brain networks are built from resting state functional magnetic resonance imaging (fMRI) and deterministic tractography. their results demonstrate that the proposed approach is capable of refining traditional heuristics, such as choosing the minimal distance from the lesion, by supplementing connectional importance of the resected tissue. This provides complementary information to help the surgeon in avoiding important functional hubs and their anatomical linkages; which are derived from neuroimaging modalities and incorporated to the related anatomical landmarks.

Toward a Knowledge-Driven Context-Aware System for Surgical Assistance

Complex surgeries complications are increasing, thus making an efficient surgical assistance is a real need. In this work, an ontology-based context-aware system was developed for surgical training/assistance during Thoracentesis by using image processing and semantic technologies. We evaluated the Thoracentesis ontology and implemented a paradigmatic test scenario to check the efficacy of the system by recognizing contextual information, e.g. the presence of surgical instruments on the table. The framework was able to retrieve contextual information about current surgical activity along with information on the need or presence of a surgical instrument.

Virtual wall-based haptic-guided teleoperated surgical robotic system for single-port brain tumor removal surgery

This article presents a haptic-guided teleoperation for a tumor removal surgical robotic system, so-called a SIROMAN system. The system was developed in our previous work to make it possible to access tumor tissue, even those that seat deeply inside the brain, and to remove the tissue with full maneuverability. For a safe and accurate operation to remove only tumor tissue completely while minimizing damage to the normal tissue, a virtual wall-based haptic guidance together with a medical image-guided control is proposed and developed. The virtual wall is extracted from preoperative medical images, and the robot is controlled to restrict its motion within the virtual wall using haptic feedback. Coordinate transformation between sub-systems, a collision detection algorithm, and a haptic-guided teleoperation using a virtual wall are described in the context of using SIROMAN. A series of experiments using a simplified virtual wall are performed to evaluate the performance of virtual wall-based haptic-guided teleoperation. With haptic guidance, the accuracy of the robotic manipulator's trajectory is improved by 57% compared to one without. The tissue removal performance is also improved by 21% ( p < 0.05). The experiments show that virtual wall-based haptic guidance provides safer and more accurate tissue removal for single-port brain surgery.

Treatment of Glioma Using neuroArm Surgical System

The use of robotic technology in the surgical treatment of brain tumour promises increased precision and accuracy in the performance of surgery. Robotic manipulators may allow superior access to narrow surgical corridors compared to freehand or conventional neurosurgery. This paper reports values and ranges of tool-tissue interaction forces during the performance of glioma surgery using an MR compatible, image-guided neurosurgical robot called neuroArm. The system, capable of microsurgery and stereotaxy, was used in the surgical resection of glioma in seven cases. neuroArm is equipped with force sensors at the end-effector allowing quantification of tool-tissue interaction forces and transmits force of dissection to the surgeon sited at a remote workstation that includes a haptic interface. Interaction forces between the tool tips and the brain tissue were measured for each procedure, and the peak forces were quantified. Results showed maximum and minimum peak force values of 2.89 N (anaplastic astrocytoma, WHO grade III) and 0.50 N (anaplastic oligodendroglioma, WHO grade III), respectively, with the mean of peak forces varying from case to case, depending on type of the glioma. Mean values of the peak forces varied in range of 1.27 N (anaplastic astrocytoma, WHO grade III) to 1.89 N (glioblastoma with oligodendroglial component, WHO grade IV). In some cases, ANOVA test failed to reject the null hypothesis of equality in means of the peak forces measured. However, we could not find a relationship between forces exerted to the pathological tissue and its size, type, or location.

Surgical robotics beyond enhanced dexterity instrumentation: a survey of machine learning techniques and their role in intelligent and autonomous surgical actions

PURPOSE

Advances in technology and computing play an increasingly important role in the evolution of modern surgical techniques and paradigms. This article reviews the current role of machine learning (ML) techniques in the context of surgery with a focus on surgical robotics (SR). Also, we provide a perspective on the future possibilities for enhancing the effectiveness of procedures by integrating ML in the operating room.

METHODS

The review is focused on ML techniques directly applied to surgery, surgical robotics, surgical training and assessment. The widespread use of ML methods in diagnosis and medical image computing is beyond the scope of the review. Searches were performed on PubMed and IEEE Explore using combinations of keywords: ML, surgery, robotics, surgical and medical robotics, skill learning, skill analysis and learning to perceive.

RESULTS

Studies making use of ML methods in the context of surgery are increasingly being reported. In particular, there is an increasing interest in using ML for developing tools to understand and model surgical skill and competence or to extract surgical workflow. Many researchers begin to integrate this understanding into the control of recent surgical robots and devices.

CONCLUSION

ML is an expanding field. It is popular as it allows efficient processing of vast amounts of data for interpreting and real-time decision making. Already widely used in imaging and diagnosis, it is believed that ML will also play an important role in surgery and interventional treatments. In particular, ML could become a game changer into the conception of cognitive surgical robots. Such robots endowed with cognitive skills would assist the surgical team also on a cognitive level, such as possibly lowering the mental load of the team. For example, ML could help extracting surgical skill, learned through demonstration by human experts, and could transfer this to robotic skills. Such intelligent surgical assistance would significantly surpass the state of the art in surgical robotics. Current devices possess no intelligence whatsoever and are merely advanced and expensive instruments.

Robotic Skull Base Surgery via Supraorbital Keyhole Approach

BACKGROUND:

The supraorbital keyhole approach has been used in anterior skull base tumor and aneurysm surgery. However, there are debates regarding the safety and limitations of this kind of approach.

OBJECTIVE:

To determine the feasibility and potential benefits of surgical robotic technology in minimally invasive neurosurgery.

METHODS:

Two fresh cadaver heads were studied with the da Vinci Surgical System with 0° and 30° stereoscopic endoscopes to visualize neuroanatomy. The ability of the system to suture and place clips under the keyhole approach was tested.

RESULTS:

The da Vinci Surgical System was used throughout the supraorbital transeyebrow keyhole approach. With the use of standard microdissection techniques, the optic nerve, optic chiasm, carotid artery, and third cranial nerve were visualized. The sylvian fissure was then exposed from the proximal sylvian membrane to the distal M1 segment. With the EndoWrist microforceps, suturing can be achieved smoothly to close a defect created on the M2 artery. Although the benefits in adjusting clips during aneurysm surgery could be provided by an articulating applier, a proper robotic applier is not currently available.

CONCLUSION:

The minimally invasive supraorbital keyhole surgery can be achieved with the da Vinci Surgical System in cadaver models. This system provides neurosurgeons with broader vision and articulable instruments, which standard microsurgical systems do not provide. Further studies are necessary to evaluate the safety and benefits of using the da Vinci Surgical System in minimally invasive neurosurgery.

Automatic trajectory planner for StereoElectroEncephaloGraphy procedures: a retrospective study

In StereoElectroEncephaloGraphy (SEEG) procedures, intracerebral electrodes are implanted in order to identify the epileptogenic zone in drug-resistant epileptic patients. This paper presents an automatic multitrajectory planner that computes the best trajectory in terms of distance from vessels and guiding screws angle, once the candidate entry and target regions are quickly and roughly defined. The planning process is designed also to spare some brain structures, such as cella media and trigone of the lateral ventricles and brain stem. The planner was retrospectively evaluated on 15 patients who had previously undergone SEEG investigation. Quantitative comparison was performed computing for each patient and for each electrode trajectory 1) the Euclidean distance from the closest vessel; 2) the trajectory incidence angle (guiding screws angle); and 3) the sulcality value. The automatic planner proved to satisfy the clinical requirements, planning safe trajectories in a clinical-compatible timeframe. Qualitative evaluation performed by three neurosurgeons showed that the automatically computed trajectories would have been accepted by them.

Accurate multi-robot targeting for keyhole neurosurgery based on external sensor monitoring

Robotics has recently been introduced in surgery to improve intervention accuracy, to reduce invasiveness and to allow new surgical procedures. In this framework, the ROBOCAST system is an optically surveyed multi-robot chain aimed at enhancing the accuracy of surgical probe insertion during keyhole neurosurgery procedures. The system encompasses three robots, connected as a multiple kinematic chain (serial and parallel), totalling 13 degrees of freedom, and it is used to automatically align the probe onto a desired planned trajectory. The probe is then inserted in the brain, towards the planned target, by means of a haptic interface. This paper presents a new iterative targeting approach to be used in surgical robotic navigation, where the multi-robot chain is used to align the surgical probe to the planned pose, and an external sensor is used to decrease the alignment errors. The iterative targeting was tested in an operating room environment using a skull phantom, and the targets were selected on magnetic resonance images. The proposed targeting procedure allows about 0.3 mm to be obtained as the residual median Euclidean distance between the planned and the desired targets, thus satisfying the surgical accuracy requirements (1 mm), due to the resolution of the diffused medical images. The performances proved to be independent of the robot optical sensor calibration accuracy.

Modular multiple sensors information management for computer-integrated surgery

BACKGROUND

In the past 20 years, technological advancements have modified the concept of modern operating rooms (ORs) with the introduction of computer-integrated surgery (CIS) systems, which promise to enhance the outcomes, safety and standardization of surgical procedures. With CIS, different types of sensor (mainly position-sensing devices, force sensors and intra-operative imaging devices) are widely used. Recently, the need for a combined use of different sensors raised issues related to synchronization and spatial consistency of data from different sources of information.

METHODS

In this study, we propose a centralized, multi-sensor management software architecture for a distributed CIS system, which addresses sensor information consistency in both space and time. The software was developed as a data server module in a client-server architecture, using two open-source software libraries: Image-Guided Surgery Toolkit (IGSTK) and OpenCV. The ROBOCAST project (FP7 ICT 215190), which aims at integrating robotic and navigation devices and technologies in order to improve the outcome of the surgical intervention, was used as the benchmark. An experimental protocol was designed in order to prove the feasibility of a centralized module for data acquisition and to test the application latency when dealing with optical and electromagnetic tracking systems and ultrasound (US) imaging devices.

RESULTS

Our results show that a centralized approach is suitable for minimizing synchronization errors; latency in the client-server communication was estimated to be 2 ms (median value) for tracking systems and 40 ms (median value) for US images.

CONCLUSION

The proposed centralized approach proved to be adequate for neurosurgery requirements. Latency introduced by the proposed architecture does not affect tracking system performance in terms of frame rate and limits US images frame rate at 25 fps, which is acceptable for providing visual feedback to the surgeon in the OR.

Force feedback in a piezoelectric linear actuator for neurosurgery

BACKGROUND

Force feedback in robotic minimally invasive surgery allows the human operator to manipulate tissues as if his/her hands were in contact with the patient organs. A force sensor mounted on the probe raises problems with sterilization of the overall surgical tool. Also, the use of off-axis gauges introduces a moment that increases the friction force on the bearing, which can easily mask off the signal, given the small force to be measured.

METHODS

This work aims at designing and testing two methods for estimating the resistance to the advancement (force) experienced by a standard probe for brain biopsies within a brain-like material. The further goal is to provide a neurosurgeon using a master-slave tele-operated driver with direct feedback on the tissue mechanical characteristics. Two possible sensing methods, in-axis strain gauge force sensor and position-position error (control-based method), were implemented and tested, both aimed at device miniaturization. The analysis carried out was aimed at fulfilment of the psychophysics requirements for force detection and delay tolerance, also taking into account safety, which is directly related to the last two issues. Controller parameters definition is addressed and consideration is given to development of the device with integration of a haptic interface.

RESULTS

Results show better performance of the control-based method (RMSE < 0.1 N), which is also best for reliability, sterilizability, and material dimensions for the application addressed.

CONCLUSIONS

The control-based method developed for force estimation is compatible with the neurosurgical application and is also capable of measuring tissue resistance without any additional sensors. Force feedback in minimally invasive surgery allows the human operator to manipulate tissues as if his/her hands were in contact with the patient organs.