Supervised CNN Strategies for Optical Image Segmentation and Classification in Interventional Medicine

Real-time vessel segmentation and reconstruction for virtual fixtures for an active handheld microneurosurgical instrument

PURPOSE

Complications related to vascular damage such as intra-operative bleeding may be avoided during neurosurgical procedures such as petroclival meningioma surgery. To address this and improve the patient's safety, we designed a real-time blood vessel avoidance strategy that enables operation on deformable tissue during petroclival meningioma surgery using Micron, a handheld surgical robotic tool.

METHODS

We integrated real-time intra-operative blood vessel segmentation of brain vasculature using deep learning, with a 3D reconstruction algorithm to obtain the vessel point cloud in real time. We then implemented a virtual-fixture-based strategy that prevented Micron's tooltip from entering a forbidden region around the vessel, thus avoiding damage to it.

RESULTS

We achieved a median Dice similarity coefficient of 0.97, 0.86, 0.87 and 0.77 on datasets of phantom blood vessels, petrosal vein, internal carotid artery and superficial vessels, respectively. We conducted trials with deformable clay vessel phantoms, keeping the forbidden region 400 [Formula: see text]m outside and 400 [Formula: see text]m inside the vessel. Micron's tip entered the forbidden region with a median penetration of just 8.84 [Formula: see text]m and 9.63 [Formula: see text]m, compared to 148.74 [Formula: see text]m and 117.17 [Formula: see text]m without our strategy, for the former and latter trials, respectively.

CONCLUSION

Real-time control of Micron was achieved at 33.3 fps. We achieved improvements in real-time segmentation of brain vasculature from intra-operative images and showed that our approach works even on non-stationary vessel phantoms. The results suggest that by enabling precise, real-time control, we are one step closer to using Micron in real neurosurgical procedures.

Anatomical classification of upper gastrointestinal organs under various image capture conditions using AlexNet

BACKGROUND

Machine learning has led to several endoscopic studies about the automated localization of digestive lesions and prediction of cancer invasion depth. Training and validation dataset collection are required for a disease in each digestive organ under a similar image capture condition; this is the first step in system development. This data cleansing task in data collection causes a great burden among experienced endoscopists. Thus, this study classified upper gastrointestinal (GI) organ images obtained via routine esophagogastroduodenoscopy (EGD) into precise anatomical categories using AlexNet.

METHOD

In total, 85,246 raw upper GI endoscopic images from 441 patients with gastric cancer were collected retrospectively. The images were manually classified into 14 categories: 0) white-light (WL) stomach with indigo carmine (IC); 1) WL esophagus with iodine; 2) narrow-band (NB) esophagus; 3) NB stomach with IC; 4) NB stomach; 5) WL duodenum; 6) WL esophagus; 7) WL stomach; 8) NB oral-pharynx-larynx; 9) WL oral-pharynx-larynx; 10) WL scaling paper; 11) specimens; 12) WL muscle fibers during endoscopic submucosal dissection (ESD); and 13) others. AlexNet is a deep learning framework and was trained using 49,174 datasets and validated using 36,072 independent datasets.

RESULTS

The accuracy rates of the training and validation dataset were 0.993 and 0.965, respectively.

CONCLUSIONS

A simple anatomical organ classifier using AlexNet was developed and found to be effective in data cleansing task for collection of EGD images. Moreover, it could be useful to both expert and non-expert endoscopists as well as engineers in retrospectively assessing upper GI images.