Comparing ensemble methods combined with different aggregating models using micrograph cell segmentation as an initial application example

Strategies such as ensemble learning and averaging techniques try to reduce the variance of single deep neural networks. The focus of this study is on ensemble averaging techniques, fusing the results of differently initialized and trained networks. Thereby, using micrograph cell segmentation as an application example, various ensembles have been initialized and formed during network training, whereby the following methods have been applied: (a) random seeds, (b) L ₁-norm pruning, (c) variable numbers of training examples, and (d) a combination of the latter 2 items. Furthermore, different averaging methods are in common use and were evaluated in this study. As averaging methods, the mean, the median, and the location parameter of an alpha-stable distribution, fit to the histograms of class membership probabilities (CMPs), as well as a majority vote of the members of an ensemble were considered. The performance of these methods is demonstrated and evaluated on a micrograph cell segmentation use case, employing a common state-of-the art deep convolutional neural network (DCNN) architecture exploiting the principle of the common VGG-architecture. The study demonstrates that for this data set, the choice of the ensemble averaging method only has a marginal influence on the evaluation metrics (accuracy and Dice coefficient) used to measure the segmentation performance. Nevertheless, for practical applications, a simple and fast estimate of the mean of the distribution is highly competitive with respect to the most sophisticated representation of the CMP distributions by an alpha-stable distribution, and hence seems the most proper ensemble averaging method to be used for this application.

Number of necessary training examples for Neural Networks with different number of trainable parameters

In this work, the network complexity should be reduced with a concomitant reduction in the number of necessary training examples. The focus thus was on the dependence of proper evaluation metrics on the number of adjustable parameters of the considered deep neural network. The used data set encompassed Hematoxylin and Eosin (H&E) colored cell images provided by various clinics. We used a deep convolutional neural network to get the relation between a model’s complexity, its concomitant set of parameters, and the size of the training sample necessary to achieve a certain classification accuracy. The complexity of the deep neural networks was reduced by pruning a certain amount of filters in the network. As expected, the unpruned neural network showed best performance. The network with the highest number of trainable parameter achieved, within the estimated standard error of the optimized cross-entropy loss, best results up to 30% pruning. Strongly pruned networks are highly viable and the classification accuracy declines quickly with decreasing number of training patterns. However, up to a pruning ratio of 40%, we found a comparable performance of pruned and unpruned deep convolutional neural networks (DCNN) and densely connected convolutional networks (DCCN).

Evaluation of an algorithm for optical pulse detection in children for application to the Pepper robot

To engage in socio-emotional interactions, children with autism spectrum conditions (ASC) need support to understand and convey emotions. In our approach, a humanoid robot (Pepper, Softbanks Robotics) acts as a tutor for the child within autism care. The robot, equipped with multimodal sensor technology to acquire the emotional feedback of the child, stimulates the child to perform tasks, adapted to its current arousal state. By in-, or decreasing the difficulties of implemented training modules, the child can be given the appropriate task according to its emotional state. The child’s arousal is measured with different techniques implemented in and on the robot: emotion detection based on audio recordings of the speech signal and camera detected facial expressions, or heart rate. To this end, the remote Photoplethysmography (rPPG) signal from camera recordings of the subjects’ face is acquired. While its unintrusive measurement is an advantage, a major drawback for rPPG is its proneness to motion and light artefacts requiring de-noising steps. A wavelet transform based on log-Gabor wavelets and a filter bank with 32 filters was implemented. The signal was filtered with a prior filter and afterwards with a Markov chain in order to extract the underlying pulse rate. Within an initial study, five children were observed watching videos with different co-notated emotions. As reference for the heart rate (HR), a wristband (empatica E4) was used. The captured emotions of all subjects were annotated to identify low and high arousal parts and positive and negative emotions. Extracted HR from rPPG-data indicated a correlation with the annotated emotions.

Rapid Progression After 177Lu-DOTATATE in Patients With Neuroendocrine Tumors

ABSTRACT

Peptide receptor radionuclide therapy (PRRT) is a treatment option for somatostatin receptor-positive, unresectable or metastatic neuroendocrine tumors (NETs). Despite high disease control rates seen with PRRT, a subset of the NET population seems to have a short progression-free interval. We hypothesize that patients with NETs with rapid progression post-PRRT may have mixed low- and high-grade cell populations, and PRRT treats the lower-grade component, allowing the more aggressive high-grade component to progress.We report 7 patients with biopsy-proven NET who received PRRT with 177Lu-DOTATATE at the Stanford Cancer Center who had evidence of progressive disease (PD) on or within 6 months of therapy.All patients had primary pancreatic, metastatic, well-differentiated NET on diagnosis and were heavily pretreated before receiving PRRT. Two patients had PD while on PRRT; 5 had PD within 6 months of completing PRRT. The median time from the last cycle to PD was 3.2 months (range, 1.1-4.6 months). The median progression-free survival was 7.7 months (95% confidence interval, 5.7-9.8 months). Three patients had a repeat biopsy post-PRRT, 2 of which demonstrated higher disease grade compared with their initial pathology. Further evaluation in larger patient cohorts is warranted to elucidate predictive factors of PRRT response/nonresponse to enable better patient selection.

First results of computer-enhanced optical diagnosis of bladder cancer

Bladder cancer is the sixth leading cancer cause worldwide. Non-muscle invasive tumors can be diagnosed and treated endoscopically. Based on biopsies alone, pathologists cannot determine the spatial organization of specimens, their relationship with each other, or their complete removal. To extend white light cystoscopy as the gold standard for bladder cancer detection, diagnosis and removal of small or flat lesions, new image-based technologies have been investigated. These include a stereo-cystoscope for improved orientation and navigation, computation of 2D and 3D panoramic images for extended visualization and documentation, as well as label-free fiber-based fluorescence-lifetime imaging (FLIM) and Raman-spectroscopy in combination with statistical data analysis. Combining all these technologies, cystoscopy can will be further enhanced to include new diagnostic possibilities.

Establishing an International Reference Image Database for Research and Development in Medical Image Processing

Introduction: The lack of comparability of evaluation results is one of the major obstacles of research and development in Medical Image Processing (MIP). The main reason for that is the usage of different image datasets with different quality, size and Gold standard.

Objectives: Therefore, one of the goals of the Working Group on Medical Image Processing of the European Federation for Medical Informatics (EFMI WG MIP) is to develop first parts of a Reference Image Database.

Methods: Kernel of the concept is to identify highly relevant medical problems with significant potential for improvement by MIP, and then to provide respective reference datasets. The EFMI WG MIP has primarily the role of a specifying group and an information broker, while the provider user relationships are defined by bilateral co-operation or license agreements.

Results: An explorative database prototype has been implemented using the MySQL database software on the Web. Templates for provider user agreements have been worked out and already applied for own ‘pre-RID-MIP’ co-operations of the authors.

Discussion and Conclusion: First steps towards a comprehensive reference image database have been done. Issues like funding, motivation, management, provision of Gold standards and evaluation guidelines are to be solved. Due to the interest from research groups and industry the efforts will be continued.

A Semantic Approach to Segmentation of Overlapping Objects

Objectives: This paper aims at introducing a novel approach for segmentation of overlapping objects and at demonstrating its applicability to medical images.

Methods: This work details a novel approach enhancing the known theory of full-segmentation of an image into regions by lifting it to a semantic segmentation into objects. Our theory allows the formal description of partitioning an image into regions on the first level and allowing the occurrence of overlaps and occlusions of objects on a second, semantic level. Possible applications for the use of this ‘semantical segmentation‘ are the analysis of radiographs and micrographs. We demonstrate our approach by the example of segmentation and separation of overlapping cervical cells and cell clusters on a set of 787 image pairs of registered PAP- and DAPI-stained micrographs. The semantical cell segmentation yielding areas of cell plasmas and nuclei are compared to a manual segmentation of the same images, where 2212 cells have been labeled. A direct comparison of over and under-segmentation between the two segmentation sets yields a mean difference value of 10.15% for the nuclei and 10.80% for the plasma.

Results: Using the proposed theory of semantical segmentation of images in combination with adequate models of the image contents, our approach allows identifying, separating and distinguishing several overlapping, occluding objects in medical images. Applying the proposed theory to the application of cervical cell segmentation from overlapping cell clusters and aggregates, it can be seen that it is possible to formally describe the complex image contents.

Conclusions: The proposed method of semantical segmentation is a mighty tool and under the assumption of the subtractive transparency model can be used in different medical image processing applications such as radiology and microscopy. By using alternative models to solve the ambiguities attached to overlaps and occlusions, further fields of application can be addressed.

Automatic texture-based analysis in ultrasound imaging of ovarian masses

PURPOSE

To assess the diagnostic accuracy of a new automatic texture-based algorithm (ATBA) in ultrasound imaging of ovarian masses and to compare its performance to subjective assessment by examiners with different levels of ultrasound experience.

MATERIALS AND METHODS

A total of 105 ultrasound images from three different groups of ovarian lesions (malignancies, functional cysts, and dermoid cysts) were evaluated using ATBA and by a total of 36 examiners with four different levels of experience (9 junior trainees, 8 senior trainees, 11 senior gynecologists, and 8 experts). Cohen's κ, Youden's indices, and the sensitivity and specificity of ATBA and of each observer were calculated for every subgroup of ovarian lesions.

RESULTS

ATBA classified 78 of the 105 masses correctly (κ = 0.62) - results that were significantly better than those of the junior and senior trainees (p = 0.02 and p < 0.01), while differences from the group of level II examiners did not reach statistical significance (p = 0.27). The best diagnostic performance (κ = 0.70) was obtained by the group of expert level III ultrasonographers. The best classification rates overall, including both ATBA and subjective assessments, were achieved in the detection of functional cysts (Youden's indices from 0.73 to 0.85), while the poorest diagnostic performance was obtained for the classification of dermoid cysts (Youden's indices from 0.28 to 0.55).

CONCLUSION

ATBA showed a significantly better diagnostic performance than observers with low or medium levels of experience, emphasizing its potential value for training purposes and in providing additional diagnostic assistance for inexperienced observers.

Computer-assisted diagnosis for precancerous lesions in the esophagus

OBJECTIVES

The interpretation of endoscopic findings by gastroenterologists is still a difficult and highly subjective task. Despite important developments such as chromo-endoscopy, pit pattern analysis, fluorescence imaging as well as narrow band imaging it still requires lots of experience and training with a certain tentativeness until the final biopsy. By the development of computer-assisted diagnosis (CAD) systems this process can be supported.

METHODS

This paper presents a new approach to CAD for precancerous lesions in the esophagus based on color-texture analysis in a content-based image retrieval (CBIR) framework. The novelty of our approach lies in the combination of newly developed color-texture features with the interactive feedback loop provided by a relevance feedback algorithm. This allows the expert to steer the query and is still robust against accidental false decisions.

RESULTS

We reached an inter-rater reliability of kappa = 0.71 on a database of 390 endoscopic images. The retrieval accuracy didn't change significantly until a wrong decision rate of 20%.

CONCLUSIONS

Thus, the system could be able to support practitioners with less experience or in private practice. In combination with a connected case database it can also support case-based reasoning for the diagnostic decision process.

The prediction of breast cancer biopsy outcomes using two CAD approaches that both emphasize an intelligible decision process

Mammography is the most effective method for breast cancer screening available today. However, the low positive predictive value of breast biopsy resulting from mammogram interpretation leads to approximately 70% unnecessary biopsies with benign outcomes. To reduce the high number of unnecessary breast biopsies, several computer-aided diagnosis (CAD) systems have been proposed in the last several years. These systems help physicians in their decision to perform a breast biopsy on a suspicious lesion seen in a mammogram or to perform a short term follow-up examination instead. We present two novel CAD approaches that both emphasize an intelligible decision process to predict breast biopsy outcomes from BI-RADS findings. An intelligible reasoning process is an important requirement for the acceptance of CAD systems by physicians. The first approach induces a global model based on decison-tree learning. The second approach is based on case-based reasoning and applies an entropic similarity measure. We have evaluated the performance of both CAD approaches on two large publicly available mammography reference databases using receiver operating characteristic (ROC) analysis, bootstrap sampling, and the ANOVA statistical significance test. Both approaches outperform the diagnosis decisions of the physicians. Hence, both systems have the potential to reduce the number of unnecessary breast biopsies in clinical practice. A comparison of the performance of the proposed decision tree and CBR approaches with a state of the art approach based on artificial neural networks (ANN) shows that the CBR approach performs slightly better than the ANN approach, which in turn results in slightly better performance than the decision-tree approach. The differences are statistically significant (p value < 0.001). On 2100 masses extracted from the DDSM database, the CRB approach for example resulted in an area under the ROC curve of A(z) = 0.89 +/- 0.01, the decision-tree approach in A(z) = 0.87 +/- 0.01, and the ANN approach in A(z) = 0.88 +/- 0.01.

[Experiences with 'optical' biopsies of leukoplakia of the vocal folds]

BACKGROUND

It is often difficult to distinguish leucoplakia from carcinoma or chronic laryngitis. In this study, we examined if color texture optical biopsies are able to detect leukoplakia reliably and to distinguish this from normal vocal fold tissue.

METHODS

25 images from patients with a normal larynx and 25 images from patients with leucoplakia were analysed retrospectively. The images were recorded with a rigid 90 degrees -laryngoscope (Wolf Typ 4450.571) during a clinical setting and were recorded by a S-VHS-videorecorder. With a software program (InSegT), regions with leucoplakia, normal tissue and suspicious tissue were manually marked. Within each marked region, the use of color texture analysis numeric features were calculated to characterize the surface in texture and color. Color histogramms (HST), Sum- and Difference histogramms (S/D), Statistical Geometric Features (SGF) and Grey-Value-Dependent-Matrix (GLDM) were used.

PATIENTS

29 women and 21 men (age 18 to 81 years, average of 53 years), who were examined in our clinic from 10/1999 - 8/2003, took part in this study.

RESULTS

The automatized classification for color texture analysis resulted in 71 % for leucoplakia and 97 % for normal tissue.

CONCLUSION

Optical biopsies can contribute to find the correct diagnosis. However, classification results must still get better when optical biopsies should be used in clinical practice.

A semantic approach to segmentation of overlapping objects

OBJECTIVES

This paper aims at introducing a novel approach for segmentation of overlapping objects and at demonstrating its applicability to medical images.

METHODS

This work details a novel approach enhancing the known theory of full-segmentation of an image into regions by lifting it to a semantic segmentation into objects. Our theory allows the formal description of partitioning an image into regions on the first level and allowing the occurrence of overlaps and occlusions of objects on a second, semantic level. Possible applications for the use of this 'semantical segmentation' are the analysis of radiographs and micrographs. We demonstrate our approach by the example of segmentation and separation of overlapping cervical cells and cell clusters on a set of 787 image pairs of registered PAP- and DAPI-stained micrographs. The semantical cell segmentation yielding areas of cell plasmas and nuclei are compared to a manual segmentation of the same images, where 2212 cells have been labeled. A direct comparison of over and under-segmentation between the two segmentation sets yields a mean difference value of 10.15% for the nuclei and 10.80% for the plasma.

RESULTS

Using the proposed theory of semantical segmentation of images in combination with adequate models of the image contents, our approach allows identifying, separating and distinguishing several overlapping, occluding objects in medical images. Applying the proposed theory to the application of cervical cell segmentation from overlapping cell clusters and aggregates, it can be seen that it is possible to formally describe the complex image contents.

CONCLUSIONS

The proposed method of semantical segmentation is a mighty tool and under the assumption of the subtractive transparency model can be used in different medical image processing applications such as radiology and microscopy. By using alternative models to solve the ambiguities attached to overlaps and occlusions, further fields of application can be addressed.

Establishing an international reference image database for research and development in medical image processing

INTRODUCTION

The lack of comparability of evaluation results is one of the major obstacles of research and development in Medical Image Processing (MIP). The main reason for that is the usage of different image datasets with different quality, size and Gold standard.

OBJECTIVES

Therefore, one of the goals of the Working Group on Medical Image Processing of the European Federation for Medical Informatics (EFMI WG MIP) is to develop first parts of a Reference Image Database.

METHODS

Kernel of the concept is to identify highly relevant medical problems with significant potential for improvement by MIP, and then to provide respective reference datasets. The EFMI WG MIP has primarily the role of a specifying group and an information broker, while the provider user relationships are defined by bilateral co-operation or license agreements.

RESULTS

An explorative database prototype has been implemented using the MySQL database software on the Web. Templates for provider user agreements have been worked out and already applied for own 'pre-RID-MIP' co-operations of the authors.

DISCUSSION AND CONCLUSION

First steps towards a comprehensive reference image database have been done. Issues like funding, motivation, management, provision of Gold standards and evaluation guidelines are to be solved. Due to the interest from research groups and industry the efforts will be continued.

Functional imaging of vocal fold vibration: digital multislice high-speed kymography

For the diagnosis of voice disorders, and especially for the classification of hoarseness, direct observation of vocal fold vibration is essential. Furthermore, a quantitative description of the movement of the vocal fold becomes increasingly necessary to document and compare findings as well as the progression of speech therapy. On the base of digital high-speed sequences of vocal fold vibration, multiple "functional images"-also called digital kymograms-are obtained using image- and signal-processing algorithms. Digital kymograms can serve as a powerful aid for visualization, description, and classification of vocal fold vibration and as an intermediate step for image interpretation by biomechanical modeling. This visualization technique will be discussed and compared to other techniques currently available: videokymography and videostroboscopy. The technique is applied to several clinical examples: aperiodic processes (phonation onset), irregular vocal fold vibration (paralysis of the recurrent nerve), particular vibration modes (anterior-posterior modes), and running speech.

Imaging of vocal fold vibration by digital multi-plane kymography

Digital multi-plane kymography is presented as a new method to demonstrate vocal fold vibration from digital high-speed recordings. Single lines from digital high-speed sequences of laryngoscopical examinations are concatenated to images, which are called kymograms. In order to reveal anterior-posterior (AP) modes of vibration several kymograms from different location of the glottis can be obtained from a single recording. Problems due to rotation of the endoscope or relative movements of patient or examiner can be solved by image processing algorithms specifically designed for this application. Different types of phonation onset and examples of voice disorders are given.

High-speed digital imaging of neoglottic vibration after total laryngectomy

OBJECTIVES

To establish the applicability of digital high-speed imaging in studying neoglottic mucosal vibration after total laryngectomy and to perform a structured evaluation of the recordings using a standardized assessment form to gain insight about the anatomical and morphologic characteristics of the neoglottis.

DESIGN

Evaluation of a new clinical tool and description of clinical disorders in a patient survey.

SETTING

The Netherlands Cancer Institute, Amsterdam.

PATIENTS

Forty-six patients who underwent laryngectomy, 36 who underwent standard total laryngectomy and 10 who underwent a partial or total pharynx reconstruction (ie, myocutaneous pectoralis major flap [n = 4], free radial forearm flap [n = 2], tubed gastric pull-up [n = 3], and full gastric pull-up [n = 1]).

INTERVENTION

Digital high-speed imaging, using a 90 degrees rigid laryngoscope, of the neoglottic vibration in prosthetic tracheoesophageal speakers after total laryngectomy.

MAIN OUTCOME MEASURES

Digital high-speed imaging might overcome some of the problems of stroboscopy in studying irregular voices and could, therefore, be expected to give more insight into the anatomical and morphologic characteristics of the neoglottis.

RESULTS

Digital high-speed recordings could be obtained in 44 of 46 patients. Using a structured evaluation form, a wide variability in anatomical and morphologic features could be established.

CONCLUSIONS

Digital high-speed imaging appeared to be a useful tool in studying the irregular vibrations of the neoglottis. Evaluation by the structured evaluation form gives a good idea about the wide variability in anatomical and morphologic features of the neoglottis.

Phonation onset: vocal fold modeling and high-speed glottography

Phonation onset is discussed in the framework of dynamical systems as a Hopf bifurcation, i.e., as a transition from damped to sustained vocal fold oscillations due to changes of parameters defining the underlying laryngeal configuration (e.g., adduction, subglottal pressure, muscular activity). An analytic envelope curve of the oscillation onset is deduced by analyzing the Hopf bifurcation in mathematical models of the vocal folds. It is governed by a single time constant which can be identified with the physiological parameter phonation onset time. This parameter reflects the laryngeal state prior to phonation and can be used as a quantitative classification criterion in order to assess the phonation onset in clinical diagnosis. The extraction of the phonation onset time from simulated time series using a simplified two-mass model and from digital high-speed videos is described in detail. It shows a good agreement between theory and measurement.

Direct evaluation of high-speed recordings of vocal fold vibrations

A digital high-speed camera system for the endoscopic examination of the larynx allows recording speeds of up to 5,600 frames/s. Recordings of up to 1 s duration can be stored and used for further evaluation. Combined with an image processing program the system is able to render x-t diagrams of vocal cord movement. Data acquired from different locations of each vocal cord can be plotted separately. All of the known objective parameters of the voice can be derived from highspeed glottograms.

Mutant of rhodopseudomonas spheroides unable to grow aerobically

We isolated a mutant of Rhodopseudomonas spheroides that grows normally under photosynthetic conditions but is unable to grow exponentially under aerobic conditions. Photosynthetically grown cultures of the mutant increase in mass and cell number for about 10 hr after transfer to aerobic conditions. During this time, no heme pigments are synthesized and the Q(O(2)) declines. In the mutant, synthesis of heme pigments is obligatorily coupled to synthesis of bacteriochlorophyll.