Brain tumor classification on intraoperative contrast-enhanced ultrasound

Machine Learning in Neurosurgery: Toward Complex Inputs, Actionable Predictions, and Generalizable Translations

Machine learning can predict neurosurgical diagnosis and outcomes, power imaging analysis, and perform robotic navigation and tumor labeling. State-of-the-art models can reconstruct and generate images, predict surgical events from video, and assist in intraoperative decision-making. In this review, we will detail the neurosurgical applications of machine learning, ranging from simple to advanced models, and their potential to transform patient care. As machine learning techniques, outputs, and methods become increasingly complex, their performance is often more impactful yet increasingly difficult to evaluate. We aim to introduce these advancements to the neurosurgical audience while suggesting major potential roadblocks to their safe and effective translation. Unlike the previous generation of machine learning in neurosurgery, the safe translation of recent advancements will be contingent on neurosurgeons' involvement in model development and validation.

Ultrasonic Aspirator for Tissue Contact Detection: An Online Classification on Time-Series

The goal of neurosurgical tumor surgery is to remove the tumor completely without damaging healthy brain structures and thereby impairing the patient's neurological functions. This requires careful planning and execution of the operation by experienced neurosurgeons using the latest intraoperative technologies to achieve safe and rapid tumor reduction without harming the patient. To achieve this goal, a standard ultrasonic aspirator designed for tissue removal is equipped with additional intraoperative tissue detection using machine learning methods.Since decision-making in a clinical context must be fast, online contact detection is critical. Data are generated on three types of artificial tissue models in a CNC machine-controlled environment with four different ultrasonic aspirator settings. Contact classification on artificial tissue models is evaluated on four classification algorithms: change point detection (CPD), random forest (RF), recurrent neural network (RNN) and temporal convolutional network (TCN). Data preprocessing steps are applied, and their impacts are investigated. All methods are evaluated on five-fold cross-validation and provide generally good results with a performance of up to 0.977±0.007 in mean F1-score. Preprocessing the data has a positive effect on the classification processes for all methods and consistently improves the metrics. Thus, this work indicates in a first step that contact classification is feasible in an online context for an ultrasonic aspirator. Further research is necessary on different tissue types, as well as hand-held use to more closely resemble the intraoperative clinical conditions.

Brain contrast-enhanced ultrasonography and elastography in infants

Advanced ultrasound techniques, including brain contrast-enhanced ultrasonography and elastography, are increasingly being explored to better understand infant brain health. While conventional brain ultrasonography provides a convenient, noninvasive means of assessing major intracranial pathologies, its value in revealing functional and physiologic insights into the brain lags behind advanced imaging techniques such as magnetic resonance imaging. In this regard, contrast-enhanced ultrasonography provides highly precise functional information on macrovascular and microvascular perfusion, while brain elastography offers information on brain stiffness that may be associated with relevant physiological factors of diagnostic, therapeutic, and/or prognostic utility. This review details the technical background, current understanding and utility, and future directions of these two emerging advanced ultrasound techniques for neonatal brain applications.

Toward intraoperative tissue classification: exploiting signal feedback from an ultrasonic aspirator for brain tissue differentiation

PURPOSE

During brain tumor surgery, care must be taken to accurately differentiate between tumorous and healthy tissue, as inadvertent resection of functional brain areas can cause severe consequences. Since visual assessment can be difficult during tissue resection, neurosurgeons have to rely on the mechanical perception of tissue, which in itself is inherently challenging. A commonly used instrument for tumor resection is the ultrasonic aspirator, whose system behavior is already dependent on tissue properties. Using data recorded during tissue fragmentation, machine learning-based tissue differentiation is investigated for the first time utilizing ultrasonic aspirators.

METHODS

Artificial tissue model with two different mechanical properties is synthesized to represent healthy and tumorous tissue. 40,000 temporal measurement points of electrical data are recorded in a laboratory environment using a CNC machine. Three different machine learning approaches are applied: a random forest (RF), a fully connected neural network (NN) and a 1D convolutional neural network (CNN). Additionally, different preprocessing steps are investigated.

RESULTS

Fivefold cross-validation is conducted over the data and evaluated with the metrics F1, accuracy, positive predictive value, true positive rate and area under the receiver operating characteristic. Results show a generally good performance with a mean F1 of up to 0.900 ± 0.096 using a NN approach. Temporal information indicates low impact on classification performance, while a low-pass filter preprocessing step leads to superior results.

CONCLUSION

This work demonstrates the first steps to successfully differentiate healthy brain and tumor tissue using an ultrasonic aspirator during tissue fragmentation. Evaluation shows that both neural network-based classifiers outperform the RF. In addition, the effects of temporal dependencies are found to be reduced when adequate data preprocessing is performed. To ensure subsequent implementation in the clinic, handheld ultrasonic aspirator use needs to be investigated in the future as well as the addition of data to reflect tissue diversity during neurosurgical operations.

Intraoperative tissue classification methods in orthopedic and neurological surgeries: A systematic review

Accurate tissue differentiation during orthopedic and neurological surgeries is critical, given that such surgeries involve operations on or in the vicinity of vital neurovascular structures and erroneous surgical maneuvers can lead to surgical complications. By now, the number of emerging technologies tackling the problem of intraoperative tissue classification methods is increasing. Therefore, this systematic review paper intends to give a general overview of existing technologies. The review was done based on the PRISMA principle and two databases: PubMed and IEEE Xplore. The screening process resulted in 60 full-text papers. The general characteristics of the methodology from extracted papers included data processing pipeline, machine learning methods if applicable, types of tissues that can be identified with them, phantom used to conduct the experiment, and evaluation results. This paper can be useful in identifying the problems in the current status of the state-of-the-art intraoperative tissue classification methods and designing new enhanced techniques.

Improved Delivery Performance of n-Butylidenephthalide-Polyethylene Glycol-Gold Nanoparticles Efficient for Enhanced Anti-Cancer Activity in Brain Tumor

n-butylidenephthalide (BP) has been verified as having the superior characteristic of cancer cell toxicity. Furthermore, gold (Au) nanoparticles are biocompatible materials, as well as effective carriers for delivering bio-active molecules for cancer therapeutics. In the present research, Au nanoparticles were first conjugated with polyethylene glycol (PEG), and then cross-linked with BP to obtain PEG-Au-BP nanodrugs. The physicochemical properties were characterized through ultraviolet-visible spectroscopy (UV-Vis), Fourier-transform infrared spectroscopy (FTIR), and dynamic light scattering (DLS) to confirm the combination of PEG, Au, and BP. In addition, both the size and structure of Au nanoparticles were observed through scanning electron microscopy (SEM) and transmission electron microscopy (TEM), where the size of Au corresponded to the results of DLS assay. Through in vitro assessments, non-transformed BAEC and DBTRG human glioma cells were treated with PEG-Au-BP drugs to investigate the tumor-cell selective cytotoxicity, cell uptake efficiency, and mechanism of endocytic routes. According to the results of MTT assay, PEG-Au-BP was able to significantly inhibit DBTRG brain cancer cell proliferation. Additionally, cell uptake efficiency and potential cellular transportation in both BAEC and DBTRG cell lines were observed to be significantly higher at 2 and 24 h. Moreover, the mechanisms of endocytosis, clathrin-mediated endocytosis, and cell autophagy were explored and determined to be favorable routes for BAEC and DBTRG cells to absorb PEG-Au-BP nanodrugs. Next, the cell progression and apoptosis of DBTRG cells after PEG-Au-BP treatment was investigated by flow cytometry. The results show that PEG-Au-BP could remarkably regulate the DBTRG cell cycle at the Sub-G1 phase, as well as induce more apoptotic cells. The expression of apoptotic-related proteins in DBTRG cells was determined through Western blotting assay. After treatment with PEG-Au-BP, the apoptotic cascade proteins p21, Bax, and Act-caspase-3 were all significantly expressed in DBTRG brain cancer cells. Through in vivo assessments, the tissue morphology and particle distribution in a mouse model were examined after a retro-orbital sinus injection containing PEG-Au-BP nanodrugs. The results demonstrate tissue integrity in the brain (forebrain, cerebellum, and midbrain), heart, liver, spleen, lung, and kidney, as they did not show significant destruction due to PEG-Au-BP treatment. Simultaneously, the extended retention period for PEG-Au-BP nanodrugs was discovered, particularly in brain tissue. The above findings identify PEG-Au-BP as a potential nanodrug for brain cancer therapies.

Colorectal cancer: Artificial intelligence and its role in surgical decision making

Despite several advances in the oncological management of colorectal cancer (CRC), there still remains a lacuna in the treatment strategy, which differs from center to center and on the philosophy of the treating clinician that is not without bias. Personalized treatment is essential for the treatment of CRC to achieve better long-term outcomes and to reduce morbidity. Surgery has an important role to play in the treatment. Surgical treatment of CRC is decided based on clinical parameters and investigations and hence likely to have judgmental errors. Artificial intelligence has been reported to be useful in the surveillance, diagnosis, treatment, and follow-up with accuracy in several malignancies. However, it is still evolving and yet to be established in surgical decision making in CRC. It is not only useful preoperatively but also intraoperatively. Artificial intelligence helps to rectify the human surgical decision when clinical data and radiological and laboratory parameters are fed into the computer and may guide correct surgical treatment.

Artificial intelligence-enhanced intraoperative neurosurgical workflow: state of the art and future perspectives

BACKGROUND

Artificial Intelligence (AI) and Machine Learning (ML) augment decision-making processes and productivity by supporting surgeons over a range of clinical activities: from diagnosis and preoperative planning to intraoperative surgical assistance. We reviewed the literature to identify current AI platforms applied to neurosurgical perioperative and intraoperative settings and describe their role in multiple subspecialties.

METHODS

A systematic review of the literature was conducted following the PRISMA guidelines. PubMed, EMBASE, and Scopus databases were searched from inception to December 31, 2020. Original articles were included if they: presented AI platforms implemented in perioperative, intraoperative settings and reported ML models' performance metrics. Due to the heterogeneity in neurosurgical applications, a qualitative synthesis was deemed appropriate. The risk of bias and applicability of predicted outcomes were assessed using the PROBAST tool.

RESULTS

41 articles were included. All studies evaluated a supervised learning algorithm. A total of 10 ML models were described; the most frequent were neural networks (n = 15) and tree-based models (n = 13). Overall, the risk of bias was medium-high, but applicability was considered positive for all studies. Articles were grouped into 4 categories according to the subspecialty of interest: neuro-oncology, spine, functional and other. For each category, different prediction tasks were identified.

CONCLUSIONS

In this review, we summarize the state-of-art applications of AI for the intraoperative augmentation of neurosurgical workflows across multiple subspecialties. ML models may boost surgical team performances by reducing human errors and providing patient-tailored surgical plans, but further and higher-quality studies need to be conducted.

Segmentation of brain tumour in 3D Intraoperative Ultrasound imaging

BACKGROUND

Intraoperative ultrasound (iUS), using a navigation system and preoperative magnetic resonance imaging (pMRI), supports the surgeon intraoperatively in identifying tumour margins. Therefore, visual tumour enhancement can be supported by efficient segmentation methods.

METHODS

A semi-automatic and two registration-based segmentation methods are evaluated to extract brain tumours from 3D-iUS data. The registration-based methods estimated the brain deformation after craniotomy based on pMRI and 3D-iUS data. Both approaches use the normalised gradient field and linear correlation of linear combinations metrics. Proposed methods were evaluated on 66 B-mode and contrast-mode 3D-iUS data with metastasis and glioblastoma.

RESULTS

The semi-automatic segmentation achieved superior results with dice similarity index (DSI) values between [85.34, 86.79]% and contour mean distance values between [1.05, 1.11] mm for both modalities and tumour classes.

CONCLUSIONS

Better segmentation results were obtained for metastasis detection than glioblastoma, preferring 3D-intraoperative B-mode over 3D-intraoperative contrast-mode.

Current applications of artificial intelligence for intraoperative decision support in surgery

Research into medical artificial intelligence (AI) has made significant advances in recent years, including surgical applications. This scoping review investigated AI-based decision support systems targeted at the intraoperative phase of surgery and found a wide range of technological approaches applied across several surgical specialties. Within the twenty-one (n = 21) included papers, three main categories of motivations were identified for developing such technologies: (1) augmenting the information available to surgeons, (2) accelerating intraoperative pathology, and (3) recommending surgical steps. While many of the proposals hold promise for improving patient outcomes, important methodological shortcomings were observed in most of the reviewed papers that made it difficult to assess the clinical significance of the reported performance statistics. Despite limitations, the current state of this field suggests that a number of opportunities exist for future researchers and clinicians to work on AI for surgical decision support with exciting implications for improving surgical care.

Automatic segmentation of brain tumor resections in intraoperative ultrasound images using U-Net

To compensate for the intraoperative brain tissue deformation, computer-assisted intervention methods have been used to register preoperative magnetic resonance images with intraoperative images. In order to model the deformation due to tissue resection, the resection cavity needs to be segmented in intraoperative images. We present an automatic method to segment the resection cavity in intraoperative ultrasound (iUS) images. We trained and evaluated two-dimensional (2-D) and three-dimensional (3-D) U-Net networks on two datasets of 37 and 13 cases that contain images acquired from different ultrasound systems. The best overall performing method was the 3-D network, which resulted in a 0.72 mean and 0.88 median Dice score over the whole dataset. The 2-D network also had good results with less computation time, with a median Dice score over 0.8. We also evaluated the sensitivity of network performance to training and testing with images from different ultrasound systems and image field of view. In this application, we found specialized networks to be more accurate for processing similar images than a general network trained with all the data. Overall, promising results were obtained for both datasets using specialized networks. This motivates further studies with additional clinical data, to enable training and validation of a clinically viable deep-learning model for automated delineation of the tumor resection cavity in iUS images.

Contrast-Enhanced Ultrasound Quantification: From Kinetic Modeling to Machine Learning

Ultrasound contrast agents (UCAs) have opened up immense diagnostic possibilities by combined use of indicator dilution principles and dynamic contrast-enhanced ultrasound (DCE-US) imaging. UCAs are microbubbles encapsulated in a biocompatible shell. With a rheology comparable to that of red blood cells, UCAs provide an intravascular indicator for functional imaging of the (micro)vasculature by quantitative DCE-US. Several models of the UCA intravascular kinetics have been proposed to provide functional quantitative maps, aiding diagnosis of different pathological conditions. This article is a comprehensive review of the available methods for quantitative DCE-US imaging based on temporal, spatial and spatiotemporal analysis of the UCA kinetics. The recent introduction of novel UCAs that are targeted to specific vascular receptors has advanced DCE-US to a molecular imaging modality. In parallel, new kinetic models of increased complexity have been developed. The extraction of multiple quantitative maps, reflecting complementary variables of the underlying physiological processes, requires an integrative approach to their interpretation. A probabilistic framework based on emerging machine-learning methods represents nowadays the ultimate approach, improving the diagnostic accuracy of DCE-US imaging by optimal combination of the extracted complementary information. The current value and future perspective of all these advances are critically discussed.

The role of contrast-enhanced ultrasound in neurosurgical disease

Contrast-enhanced ultrasound (CEUS) is a relatively new imaging modality in the realm of neurosurgical disease. CEUS permits the examination of blood flow through arteries, veins, and capillaries via intravascular contrast agents and allows vascular architectural mapping with extreme sensitivity and specificity. While it has established utility in other organ systems such as the liver and kidneys, CEUS has not been studied extensively in the brain. This report presents a review of the literature on the neurosurgical applications of CEUS and provides an outline of the imaging modality’s role in the diagnosis, evaluation, and treatment of neurosurgical disease.

Patient-specific model-based segmentation of brain tumors in 3D intraoperative ultrasound images

PURPOSE

Intraoperative ultrasound (iUS) imaging is commonly used to support brain tumor operation. The tumor segmentation in the iUS images is a difficult task and still under improvement because of the low signal-to-noise ratio. The success of automatic methods is also limited due to the high noise sensibility. Therefore, an alternative brain tumor segmentation method in 3D-iUS data using a tumor model obtained from magnetic resonance (MR) data for local MR-iUS registration is presented in this paper. The aim is to enhance the visualization of the brain tumor contours in iUS.

METHODS

A multistep approach is proposed. First, a region of interest (ROI) based on the specific patient tumor model is defined. Second, hyperechogenic structures, mainly tumor tissues, are extracted from the ROI of both modalities by using automatic thresholding techniques. Third, the registration is performed over the extracted binary sub-volumes using a similarity measure based on gradient values, and rigid and affine transformations. Finally, the tumor model is aligned with the 3D-iUS data, and its contours are represented.

RESULTS

Experiments were successfully conducted on a dataset of 33 patients. The method was evaluated by comparing the tumor segmentation with expert manual delineations using two binary metrics: contour mean distance and Dice index. The proposed segmentation method using local and binary registration was compared with two grayscale-based approaches. The outcomes showed that our approach reached better results in terms of computational time and accuracy than the comparative methods.

CONCLUSION

The proposed approach requires limited interaction and reduced computation time, making it relevant for intraoperative use. Experimental results and evaluations were performed offline. The developed tool could be useful for brain tumor resection supporting neurosurgeons to improve tumor border visualization in the iUS volumes.

Intraoperative Contrast Enhanced Ultrasound Evaluates the Grade of Glioma

Objective. The aim of our study was to investigate the value of intraoperative contrast enhanced ultrasound (CEUS) for evaluating the grade of glioma and the correlation between microvessel density (MVD) and vascular endothelial growth factor (VEGF). Methods. We performed intraoperative conventional ultrasound (CUS) and CEUS on 88 patients with gliomas. All of the patients have undergone surgery and obtained the results of pathology. All patients have undergone intraoperative CUS and CEUS to compare the characteristics of different grade gliomas and the results of CUS and CEUS were compared with pathological results. Results. The time to start (TTS) and time to peak (TTP) of low grade glioma (LGG) were similar to those of edema and normal brain surrounding glioma. The enhanced extent of LGG was higher than that of the normal brain and edema. The TTS and TTP of high grade glioma were earlier than those of the edema and normal brain surrounding glioma. The enhancement of HGG was higher than that of LGG. The absolute peak intensity (API) was correlated with MVD and VEGF. Conclusion. Intraoperative CEUS could help in determining boundary of peritumoral brain edema of glioma. Intraoperative CEUS parameters in cerebral gliomas could indirectly reflect the information of MVD and VEGF.

Pixel-based approach to assess contrast-enhanced ultrasound kinetics parameters for differential diagnosis of rheumatoid arthritis

Inflammatory rheumatic diseases are the leading causes of disability and constitute a frequent medical disorder, leading to inability to work, high comorbidity, and increased mortality. The standard for diagnosing and differentiating arthritis is based on clinical examination, laboratory exams, and imaging findings, such as synovitis, bone edema, or joint erosions. Contrast-enhanced ultrasound (CEUS) examination of the small joints is emerging as a sensitive tool for assessing vascularization and disease activity. Quantitative assessment is mostly performed at the region of interest level, where the mean intensity curve is fitted with an exponential function. We showed that using a more physiologically motivated perfusion curve, and by estimating the kinetic parameters separately pixel by pixel, the quantitative information gathered is able to more effectively characterize the different perfusion patterns. In particular, we demonstrated that a random forest classifier based on pixelwise quantification of the kinetic contrast agent perfusion features can discriminate rheumatoid arthritis from different arthritis forms (psoriatic arthritis, spondyloarthritis, and arthritis in connective tissue disease) with an average accuracy of 97%. On the contrary, clinical evaluation (DAS28), semiquantitative CEUS assessment, serological markers, or region-based parameters do not allow such a high diagnostic accuracy.

The Art of Intraoperative Glioma Identification

A major dilemma in brain-tumor surgery is the identification of tumor boundaries to maximize tumor excision and minimize postoperative neurological damage. Gliomas, especially low-grade tumors, and normal brain have a similar color and texture, which poses a challenge to the neurosurgeon. Advances in glioma resection techniques combine the experience of the neurosurgeon and various advanced technologies. Intraoperative methods to delineate gliomas from normal tissue consist of (1) image-based navigation, (2) intraoperative sampling, (3) electrophysiological monitoring, and (4) enhanced visual tumor demarcation. The advantages and disadvantages of each technique are discussed. A combination of these methods is becoming widely accepted in routine glioma surgery. Gross total resection in conjunction with radiation, chemotherapy, or immune/gene therapy may increase the rates of cure in this devastating disease.

Optimizing glioblastoma resection: intraoperative mapping and beyond

The management of glioblastomas starts with surgical resection if possible, along with subsequent chemotherapy and radiation therapy. Several retrospective studies have suggested that extent of resection plays a role in the prognosis of glioblastoma patients. The importance of extent of resection must be balanced with preserving patient's functional status for tumors in eloquent areas. Here we review the preoperative imaging modalities such as functional MRI and magnetoencephalography (MEG), and the intraoperative techniques such as motor and language mapping, intraoperative MRI, and intraoperative techniques such as 5-aminolevulinic acid administration, that allow maximal safe operative resection of glioblastomas.