Cost-effective 3D scanning and printing technologies for outer ear reconstruction: current status

Current 3D scanning and printing technologies offer not only state-of-the-art developments in the field of medical imaging and bio-engineering, but also cost and time effective solutions for surgical reconstruction procedures. Besides tissue engineering, where living cells are used, bio-compatible polymers or synthetic resin can be applied. The combination of 3D handheld scanning devices or volumetric imaging, (open-source) image processing packages, and 3D printers form a complete workflow chain that is capable of effective rapid prototyping of outer ear replicas. This paper reviews current possibilities and latest use cases for 3D-scanning, data processing and printing of outer ear replicas with a focus on low-cost solutions for rehabilitation engineering.

YOLO networks for polyp detection: A human-in-the-loop training approach

Introduction: Early detection of adenomas and polyps is one central goal of colonoscopic screening programs. As the adenoma detection rate (ADR) depends on the experience of the endoscopist, AI-based polyp detection systems can be used for real-time assistance. Hence, to support the physicians such AI-based systems using deep-convolutional neural networks (DCNNS) have been introduced in the past years. One disadvantage of these techniques is the need of a huge amount of labeled training data. Method: We investigate a "human-in-the-loop approach" to minimize the required time to generate labeled training data. The approach is evaluated within the training a YOLOv4 neural network to detect polyps in colonoscopic image data. The performance metrics of the neural network are evaluated on three public datasets. Results: The performance of the YOLO network increased from a precision of 0.88, recall of 0.83, F1 score of 0.86, and a F2 score of 0.86 to a precision of 0.91, recall of 0.87, F1 = of 0.89, and F2 = of 0.88. The interactive labelling of 1,000 images only takes one hours. Conclusion: The proposed "human-in-theloop approach" is capable of generating labelled image data in a minimum of time while increasing the performance metrics as well. For higher performance increase more data can now be labeled within this new approach.

Automatic detection of foreign objects and contaminants in colonoscopic video data using deep learning

Introduction: In the field of colonoscopy results of automated or supporting procedures using medical image processing are often compromised by contaminants such as blood or stool, but also due to medical instruments. Such objects or contaminations often obscure relevant areas or cause disturbances in various algorithms and also prevent physicians from recognizing relevant information in the video-stream. Hence, automatic detection and segmentation of such contaminations is desirable, by obtaining proper segmentation masks to an algorithm yields the possibility to exclude such areas from analysis. Method: We created a training and validation dataset by collecting (and annotating, where necessary) endoscopic video sequences from various public available datasets and a privat collection. An encoder-decoder deep convolutional neural network was trained with this data to predict segmentation masks for blood, stool, medical instruments and other objects. Results: Depending on the class (blood, stool, ... ), we were able to obtain a mean Dice score in the range between 0.88 to 0.93 on the evaluation dataset. Conclusion: The proposed approach hence allows us providing segmentation masks that are able to reduce problems in the subsequent image analysis and also find regions that were not properly observed.

Panorama mapping of the colon – a phantom study with robotic endoscopy

When documenting colonoscopic examinations, the challenge arises, that the exact location of the detected lesions (such as polyps) can usually only be determined with low precision. In order to improve the image-based documentation of such lesions with respect to their location and context, a system is currently investigated and developed, allowing the creation of a panoramic map of the interior of the colon’s lumen during an endoscopic examination. To evaluate the proposed algorithms, a 3D-printed colon phantom with known geometry has been developed. Furthermore, to obtain image sequences with precise and known motion during a simulated examination for test purposes, a robot was used to perform the insertion and withdrawal of the endoscope with constant speed and a steady viewing angle. The thus obtained image data is used to generate an ideal panoramic map using our the proposed stitching algorithm. Hence, it is possible to obtain reference maps of the phantom, which can be used to evaluate the quality of the algorithm.

Vorstellung und Evaluation des haptisch-visuellen, ohrchirurgischen Trainingssystems „HaptiVisT“

Hintergrund Operationen am Felsenbein stellen eine besondere Herausforderung für HNO-Chirurgen dar. Ziel des BMBF-geförderten Projektes war die Entwicklung eines realitätsnahen Trainingssystems für Ohroperationen in Form eines „Serious Game“.

Methodik Der vorgestellte Prototyp des HaptiVisT-Systems fungiert als ohrchirurgisches Trainingssystem mit visueller Rückkopplung durch einen brillenlosen 3D-Monitor und haptischer Rückkopplung durch einen den Bohrer simulierenden Haptik-Arm. Eine Vielfalt von Trainingsmöglichkeiten wird durch 3 verfügbare chirurgische Prozeduren (Antrotromie, Mastoidektomie, posteriore Tympanotomie) gewährleistet. Ein gewichtetes Punktesystem ermöglicht die Messbarkeit des Trainingserfolgs. Im Zuge der technischen Entwicklung des Prototyps erfolgte eine prospektive Evaluation durch 8 HNO-Ärzte und 4 Studierende u.a. hinsichtlich der Lerninhalte und Benutzerfreundlichkeit. Zur Anwendung kam ein standardisierter Fragebogen (Ordinalskala: 1=sehr gut bis 5=sehr schlecht).

Ergebnisse Hinsichtlich der Lerninhalte ergaben die Aspekte „Festigung Anatomie (Mittelwert=1,58)“, „Training Hand-Augen-Koordination (1,67)“, „Übertragbarkeit in die Praxis (1,83)“ und „Nützlichkeit für die Praxis (1,33)“ gute bis sehr gute Werte. Die Benutzerfreundlichkeit zeigte für die Aspekte „Realitätsnähe (2,29)“, „Zusammenspiel Haptik und Optik (2,33)“ sowie „Immersion in das Trainingssystem (1,89)“ ebenfalls gute Werte. Der „Motivationsfaktor“ war bei allen Testpersonen sehr hoch (1,2).

Schlussfolgerung Das ohrchirurgische Trainingssystem HaptiVisT bietet die Möglichkeit des immersiven Trainings von Ohroperationen. Eine Integration in den klinischen Alltag und insbesondere in die ärztliche Weiterbildung zum HNO-Facharzt erscheint daher sinnvoll.

Partial 3D-reconstruction of the colon from monoscopic colonoscopy videos using shape-from-motion and deep learning

For the image-based documentation of a colonoscopy procedure, a 3D-reconstuction of the hollow colon structure from endoscopic video streams is desirable. To obtain this reconstruction, 3D information about the colon has to be extracted from monocular colonoscopy image sequences. This information can be provided by estimating depth through shape-from-motion approaches, using the image information from two successive image frames and the exact knowledge of their disparity. Nevertheless, during a standard colonoscopy the spatial offset between successive frames is continuously changing. Thus, in this work deep convolutional neural networks (DCNNs) are applied in order to obtain piecewise depth maps and point clouds of the colon. These pieces can then be fused for a partial 3D reconstruction.

Acquisition of Semantics for AI-based Applications in Medical Technologies

For the development, training, and validation of AIbased procedures, such as the analysis of clinical data, prediction of critical events, or planning of healthcare procedures, a lot of data is needed. In addition to this data of any origin (image data, bio-signals, health records, machine states, …) adequate supplementary information about the meaning encoded in the data is required. With this additional information - the semantic or knowledge - a tight relation between the raw data and the human-understandable concepts from the real world can be established. Nevertheless, as the amount of data needed to develop robust AI-based methods is strongly increasing, the assessment and acquisition of the related knowledge becomes more and more challenging. Within this work, an overview of currently available concepts of knowledge acquisition are described and evaluated. Four main groups of knowledge acquisition related to AI-based technologies have been identified. For image data mainly iconic annotation methods are used, where experienced users mark or draw depicted entities in the images and label them using predefined sets of classifications. Similarly, bio-signals are manually labelled, whereby important events along the timeline are marked. If no sufficient data is available, augmentation and simulation techniques are applied yielding data and semantics at the same time. In applications, where expensive sensors are replaced by low-cost devices, the high-grade data can be used as semantics. Finally, classic rule-based approaches are used, where human factual and procedural knowledge about the data and its context is translated into machine-understandable procedures. All these methods are depending on the involvement of human experts. To reduce this, more intelligent and hybrid approaches are needed, shifting the focus from the-human-in-the-loop to the-machine- in-the-loop.

Initial experiments of eye-tracking during AIassisted polyp-detection in colonoscopy

Currently, various AI-based systems for computerassisted adenoma- and polyp-detection during colonoscopy have been brought to the market and are under clinical investigation. With these systems available to be used during routine screening colonoscopy and first results published about experiments and findings, it has become of interest how and to which extend such systems are used during the examination. Specifically, similarly to automotive navigation, it is of interest of how much visual focus is put onto the augmented image of the above-mentioned devices, signalling possible hypothesis of adenomas or polyps, and how much time-of-attention remains on the original colonoscopic video data. Thus, within a study, N = 36 participants using a prototype of a polypdetection system have been observed with an eye-tracker-system, to capture and evaluate the relative time of attention with respect to the original and augmented video data and differentiate these values between various sub-groups based on experience, education and gender. T-tests were conducted to identify potential significant differences. Based on the obtained data, the augmented video data is used with a very high attention (up to 75%) depending on the regarded sub-group. Experienced as well as less-experienced users (with > 500 colonoscopies) both preferred looking at the original data. In contrast, gastroenterologists (in contrast to nurses, students, engineers) were more interested in the outcome of the novel AIsystem. The female group preferred looking at the unobstructed data, while the male group was highly interested in the AI-based data.

Interactive Image Segmentation for Cochlea Implant Planning based on DVT Data

Cochlea Implant (CI) planning is usually based on preoperative obtained CT or MRI data, visualising risk structures in the petrosal bone. In the past years, Digital Volume Tomography (DVT) with an increased spatial resolution and reduced radiation has become more important in the clinical routine for otology. In this work we propose an extension of our interactive “wizard”-guided approach for the interactive segmentation of the middle and inner ear structures for the use with DVT data. Different filter pipelines enable the user to interactive segment the acoustic canal, ossicles, tympanic cavity, facial nerve, chorda tympani, round window, cochlea and semicircular canals. The approach has been evaluated on six pre-operative acquired DVT datasets by an ENT expert. Results imply that the proposed method can handle DVT well and can potentially be used for interactive OR planning.