1
|
Deike K, Decker A, Scheyhing P, Harten J, Zimmermann N, Paech D, Peters O, Freiesleben SD, Schneider LS, Preis L, Priller J, Spruth E, Altenstein S, Lohse A, Fliessbach K, Kimmich O, Wiltfang J, Bartels C, Hansen N, Jessen F, Rostamzadeh A, Düzel E, Glanz W, Incesoy EI, Butryn M, Buerger K, Janowitz D, Ewers M, Perneczky R, Rauchmann BS, Teipel S, Kilimann I, Goerss D, Laske C, Munk MH, Spottke A, Roy N, Wagner M, Roeske S, Heneka MT, Brosseron F, Ramirez A, Dobisch L, Wolfsgruber S, Kleineidam L, Yakupov R, Stark M, Schmid MC, Berger M, Hetzer S, Dechent P, Scheffler K, Petzold GC, Schneider A, Effland A, Radbruch A. Machine Learning-Based Perivascular Space Volumetry in Alzheimer Disease. Invest Radiol 2024:00004424-990000000-00211. [PMID: 38652067 DOI: 10.1097/rli.0000000000001077] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/25/2024]
Abstract
OBJECTIVES Impaired perivascular clearance has been suggested as a contributing factor to the pathogenesis of Alzheimer disease (AD). However, it remains unresolved when the anatomy of the perivascular space (PVS) is altered during AD progression. Therefore, this study investigates the association between PVS volume and AD progression in cognitively unimpaired (CU) individuals, both with and without subjective cognitive decline (SCD), and in those clinically diagnosed with mild cognitive impairment (MCI) or mild AD. MATERIALS AND METHODS A convolutional neural network was trained using manually corrected, filter-based segmentations (n = 1000) to automatically segment the PVS in the centrum semiovale from interpolated, coronal T2-weighted magnetic resonance imaging scans (n = 894). These scans were sourced from the national German Center for Neurodegenerative Diseases Longitudinal Cognitive Impairment and Dementia Study. Convolutional neural network-based segmentations and those performed by a human rater were compared in terms of segmentation volume, identified PVS clusters, as well as Dice score. The comparison revealed good segmentation quality (Pearson correlation coefficient r = 0.70 with P < 0.0001 for PVS volume, detection rate in cluster analysis = 84.3%, and Dice score = 59.0%). Subsequent multivariate linear regression analysis, adjusted for participants' age, was performed to correlate PVS volume with clinical diagnoses, disease progression, cerebrospinal fluid biomarkers, lifestyle factors, and cognitive function. Cognitive function was assessed using the Mini-Mental State Examination, the Comprehensive Neuropsychological Test Battery, and the Cognitive Subscale of the 13-Item Alzheimer's Disease Assessment Scale. RESULTS Multivariate analysis, adjusted for age, revealed that participants with AD and MCI, but not those with SCD, had significantly higher PVS volumes compared with CU participants without SCD (P = 0.001 for each group). Furthermore, CU participants who developed incident MCI within 4.5 years after the baseline assessment showed significantly higher PVS volumes at baseline compared with those who did not progress to MCI (P = 0.03). Cognitive function was negatively correlated with PVS volume across all participant groups (P ≤ 0.005 for each). No significant correlation was found between PVS volume and any of the following parameters: cerebrospinal fluid biomarkers, sleep quality, body mass index, nicotine consumption, or alcohol abuse. CONCLUSIONS The very early changes of PVS volume may suggest that alterations in PVS function are involved in the pathophysiology of AD. Overall, the volumetric assessment of centrum semiovale PVS represents a very early imaging biomarker for AD.
Collapse
Affiliation(s)
- Katerina Deike
- From the German Center for Neurodegenerative Diseases (DZNE), Bonn, Germany (K.D., A.D., K.F., O.K., F.J., Annika Spottke, N.R., M.W., S.R., M.T.H., F.B., Alfredo Ramirez, S.W., L.K., M.S., M.C.S., G.C.P., Anja Schneider, Alexander Radbruch); Department of Neuroradiology, University Hospital, Bonn, Germany (K.D., P.S., D.P., Alexander Radbruch); Department of Neurodegenerative Disease and Geriatric Psychiatry/Psychiatry, University Hospital Bonn, Bonn, Germany (J.H., N.Z., K.F., M.W., Alfredo Ramirez, S.W., L.K., Anja Schneider); Department of Radiology, German Cancer Research Center (DKFZ), Heidelberg, Germany (D.P.); German Center for Neurodegenerative Diseases (DZNE), Berlin, Germany (O.P., S.D.F., J.P., E.S., S.A.); Institute of Psychiatry and Psychotherapy, Charité-Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany (O.P., S.D.F., L.-S.S., L.P.); Department of Psychiatry and Psychotherapy, Charité, Berlin, Germany (J.P., E.S., S.A., A.L.); Department of Psychiatry and Psychotherapy, School of Medicine, Munich, Germany (J.P.); University of Edinburgh and UK DRI, Edinburgh, United Kingdom (J.P.); German Center for Neurodegenerative Diseases (DZNE), Goettingen, Germany (J.W.); Department of Psychiatry and Psychotherapy, University Medical Center, Goettingen, Germany (J.W., C.B., N.H.); Neurosciences and Signaling Group, Institute of Biomedicine (iBiMED), University of Aveiro, Aveiro, Portugal (J.W.); Department of Psychiatry, University of Cologne, Cologne, Germany (F.J., Ayda Rostamzadeh); Excellence Cluster on Cellular Stress Responses in Aging-Associated Diseases, University of Cologne, Cologne, Germany (F.J., Alfredo Ramirez); German Center for Neurodegenerative Diseases (DZNE), Magdeburg, Germany (E.D., W.G., E.I.I., Michaela Butryn, L.D., R.Y.); Institute of Cognitive Neurology and Dementia Research, Otto-von-Guericke University, Magdeburg, Germany (E.D., W.G., E.I.I., Michaela Butryn); Department for Psychiatry and Psychotherapy, University Clinic Magdeburg, Magdeburg, Germany (E.I.I.); German Center for Neurodegenerative Diseases (DZNE), Munich, Germany (K.B., M.E., R.P.); Institute for Stroke and Dementia Research, LMU Munich, Germany (K.B., D.J., M.E.); Department of Psychiatry and Psychotherapy, LMU Munich, Germany (R.P., B.-S.R.); Munich Cluster for Systems Neurology (SyNergy), Munich, Germany (R.P.); Ageing Epidemiology Research Unit, School of Public Health, Imperial College London, London, United Kingdom (R.P.); Sheffield Institute for Translational Neuroscience, University of Sheffield, Sheffield, United Kingdom (R.P., B.-S.R.); Department of Neuroradiology, University Hospital Munich, Munich, Germany (B.-S.R.); German Center for Neurodegenerative Diseases (DZNE), Rostock, Germany (S.T., I.K., D.G.); Department of Psychosomatic Medicine, Rostock University Medical Center, Rostock, Germany (S.T., I.K., D.G.); German Center for Neurodegenerative Diseases (DZNE), Tübingen, Germany (C.L., M.H.M.); Section for Dementia Research, Hertie Institute for Clinical Brain Research and Department of Psychiatry and Psychotherapy, Tübingen, Germany (C.L.); Department of Psychiatry and Psychotherapy, University of Tübingen, Tübingen Germany (M.H.M.); Department of Neurology, University of Bonn, Bonn, Germany (Annika Spottke); Division of Neurogenetics and Molecular Psychiatry, Department of Psychiatry and Psychotherapy, Cologne, Germany (Alfredo Ramirez); Department of Psychiatry and Glenn Biggs Institute for Alzheimer's and Neurodegenerative Diseases, San Antonio, TX (Alfredo Ramirez); Institute for Medical Biometry, Informatics, and Epidemiology, University Hospital Bonn, Bonn, Germany (M.C.S., Moritz Berger); Berlin Center for Advanced Neuroimaging, Charité-Universitätsmedizin, Berlin, Germany (S.H.); MR-Research in Neurosciences, Department of Cognitive Neurology, Göttingen, Germany (P.D.); Department for Biomedical Magnetic Resonance, University of Tübingen, Tübingen, Germany (K.S.); Division of Vascular Neurology, Department of Neurology, University Hospital Bonn, Bonn, Germany (G.C.P.); and Institute for Applied Mathematics, University of Bonn, Bonn, Germany (A.E.)
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
Collapse
|
2
|
Grandits T, Verhulsdonk J, Haase G, Effland A, Pezzuto S. Digital Twinning of Cardiac Electrophysiology Models From the Surface ECG: A Geodesic Backpropagation Approach. IEEE Trans Biomed Eng 2024; 71:1281-1288. [PMID: 38048238 DOI: 10.1109/tbme.2023.3331876] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/06/2023]
Abstract
The eikonal equation has become an indispensable tool for modeling cardiac electrical activation accurately and efficiently. In principle, by matching clinically recorded and eikonal-based electrocardiograms (ECGs), it is possible to build patient-specific models of cardiac electrophysiology in a purely non-invasive manner. Nonetheless, the fitting procedure remains a challenging task. The present study introduces a novel method, Geodesic-BP, to solve the inverse eikonal problem. Geodesic-BP is well-suited for GPU-accelerated machine learning frameworks, allowing us to optimize the parameters of the eikonal equation to reproduce a given ECG. We show that Geodesic-BP can reconstruct a simulated cardiac activation with high accuracy in a synthetic test case, even in the presence of modeling inaccuracies. Furthermore, we apply our algorithm to a publicly available dataset of a biventricular rabbit model, with promising results. Given the future shift towards personalized medicine, Geodesic-BP has the potential to help in future functionalizations of cardiac models meeting clinical time constraints while maintaining the physiological accuracy of state-of-the-art cardiac models.
Collapse
|
3
|
Böhner AMC, Effland A, Jacob AM, Böhner KAM, Abdullah Z, Brähler S, Attenberger UI, Rumpf M, Kurts C. Determining individual glomerular proteinuria and periglomerular infiltration in a cleared murine kidney by a 3-dimensional fast marching algorithm. Kidney Int 2024:S0085-2538(24)00172-8. [PMID: 38458475 DOI: 10.1016/j.kint.2024.01.043] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2023] [Revised: 11/30/2023] [Accepted: 01/09/2024] [Indexed: 03/10/2024]
Abstract
Three-dimensional (3D) imaging has advanced basic research and clinical medicine. However, limited resolution and imperfections of real-world 3D image material often preclude algorithmic image analysis. Here, we present a methodologic framework for such imaging and analysis for functional and spatial relations in experimental nephritis. First, optical tissue-clearing protocols were optimized to preserve fluorescence signals for light sheet fluorescence microscopy and compensated attenuation effects using adjustable 3D correction fields. Next, we adapted the fast marching algorithm to conduct backtracking in 3D environments and developed a tool to determine local concentrations of extractable objects. As a proof-of-concept application, we used this framework to determine in a glomerulonephritis model the individual proteinuria and periglomerular immune cell infiltration for all glomeruli of half a mouse kidney. A correlation between these parameters surprisingly did not support the intuitional assumption that the most inflamed glomeruli are the most proteinuric. Instead, the spatial density of adjacent glomeruli positively correlated with the proteinuria of a given glomerulus. Because proteinuric glomeruli appear clustered, this suggests that the exact location of a kidney biopsy may affect the observed severity of glomerular damage. Thus, our algorithmic pipeline described here allows analysis of various parameters of various organs composed of functional subunits, such as the kidney, and can theoretically be adapted to processing other image modalities.
Collapse
Affiliation(s)
- Alexander M C Böhner
- Institute for Molecular Medicine and Experimental Immunology, University Hospital Bonn, Bonn, Germany; Department of Diagnostic and Interventional Radiology, University Hospital Bonn, Bonn, Germany
| | - Alexander Effland
- Institute for Applied Mathematics, University of Bonn, Bonn, Germany
| | - Alice M Jacob
- Institute for Molecular Medicine and Experimental Immunology, University Hospital Bonn, Bonn, Germany
| | - Karin A M Böhner
- Institute for Molecular Medicine and Experimental Immunology, University Hospital Bonn, Bonn, Germany
| | - Zeinab Abdullah
- Institute for Molecular Medicine and Experimental Immunology, University Hospital Bonn, Bonn, Germany
| | - Sebastian Brähler
- Department of Internal Medicine II, University Hospital Cologne, Cologne, Germany
| | - Ulrike I Attenberger
- Department of Diagnostic and Interventional Radiology, University Hospital Bonn, Bonn, Germany
| | - Martin Rumpf
- Institute for Numerical Simulation, University of Bonn, Bonn, Germany
| | - Christian Kurts
- Institute for Molecular Medicine and Experimental Immunology, University Hospital Bonn, Bonn, Germany; Department of Microbiology and Immunology, Doherty Institute for Infection and Immunity, University of Melbourne, Melbourne, Victoria, Australia.
| |
Collapse
|
4
|
Ortiz-Gonzalez A, Kobler E, Simon S, Bischoff L, Nowak S, Isaak A, Block W, Sprinkart AM, Attenberger U, Luetkens JA, Bayro-Corrochano E, Effland A. Optical Flow-Guided Cine MRI Segmentation With Learned Corrections. IEEE Trans Med Imaging 2024; 43:940-953. [PMID: 37856267 DOI: 10.1109/tmi.2023.3325766] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/21/2023]
Abstract
In cardiac cine magnetic resonance imaging (MRI), the heart is repeatedly imaged at numerous time points during the cardiac cycle. Frequently, the temporal evolution of a certain region of interest such as the ventricles or the atria is highly relevant for clinical diagnosis. In this paper, we devise a novel approach that allows for an automatized propagation of an arbitrary region of interest (ROI) along the cardiac cycle from respective annotated ROIs provided by medical experts at two different points in time, most frequently at the end-systolic (ES) and the end-diastolic (ED) cardiac phases. At its core, a 3D TV- L1 -based optical flow algorithm computes the apparent motion of consecutive MRI images in forward and backward directions. Subsequently, the given terminal annotated masks are propagated by this bidirectional optical flow in 3D, which results, however, in improper initial estimates of the segmentation masks due to numerical inaccuracies. These initially propagated segmentation masks are then refined by a 3D U-Net-based convolutional neural network (CNN), which was trained to enforce consistency with the forward and backward warped masks using a novel loss function. Moreover, a penalization term in the loss function controls large deviations from the initial segmentation masks. This method is benchmarked both on a new dataset with annotated single ventricles containing patients with severe heart diseases and on a publicly available dataset with different annotated ROIs. We emphasize that our novel loss function enables fine-tuning the CNN on a single patient, thereby yielding state-of-the-art results along the complete cardiac cycle.
Collapse
|
5
|
Haase R, Pinetz T, Kobler E, Paech D, Effland A, Radbruch A, Deike-Hofmann K. Artificial Contrast: Deep Learning for Reducing Gadolinium-Based Contrast Agents in Neuroradiology. Invest Radiol 2023; 58:539-547. [PMID: 36822654 DOI: 10.1097/rli.0000000000000963] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/25/2023]
Abstract
ABSTRACT Deep learning approaches are playing an ever-increasing role throughout diagnostic medicine, especially in neuroradiology, to solve a wide range of problems such as segmentation, synthesis of missing sequences, and image quality improvement. Of particular interest is their application in the reduction of gadolinium-based contrast agents, the administration of which has been under cautious reevaluation in recent years because of concerns about gadolinium deposition and its unclear long-term consequences. A growing number of studies are investigating the reduction (low-dose approach) or even complete substitution (zero-dose approach) of gadolinium-based contrast agents in diverse patient populations using a variety of deep learning methods. This work aims to highlight selected research and discusses the advantages and limitations of recent deep learning approaches, the challenges of assessing its output, and the progress toward clinical applicability distinguishing between the low-dose and zero-dose approach.
Collapse
Affiliation(s)
| | - Thomas Pinetz
- Institute of Applied Mathematics, Rheinische Friedrich-Wilhelms-Universität Bonn, Bonn, Germany
| | - Erich Kobler
- From the Department of Neuroradiology, University Medical Center Bonn, Rheinische Friedrich-Wilhelms-Universität Bonn
| | | | - Alexander Effland
- Institute of Applied Mathematics, Rheinische Friedrich-Wilhelms-Universität Bonn, Bonn, Germany
| | | | | |
Collapse
|
6
|
Haase R, Pinetz T, Bendella Z, Kobler E, Paech D, Block W, Effland A, Radbruch A, Deike-Hofmann K. Reduction of Gadolinium-Based Contrast Agents in MRI Using Convolutional Neural Networks and Different Input Protocols: Limited Interchangeability of Synthesized Sequences With Original Full-Dose Images Despite Excellent Quantitative Performance. Invest Radiol 2023; 58:420-430. [PMID: 36735399 DOI: 10.1097/rli.0000000000000955] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/04/2023]
Abstract
OBJECTIVES The purpose of this study was to implement a state-of-the-art convolutional neural network used to synthesize artificial T1-weighted (T1w) full-dose images from corresponding noncontrast and low-dose images (using various settings of input sequences) and test its performance on a patient population acquired prospectively. MATERIALS AND METHODS In this monocentric, institutional review board-approved study, a total of 138 participants were included who received an adapted imaging protocol with acquisition of a T1w low dose after administration of 10% of the standard dose and acquisition of a T1w full dose after administration of the remaining 90% of the standard dose of a gadolinium-containing contrast agent. A total of 83 participants formed the training sample (51.7 ± 16.5 years, 36 women), 25 the validation sample (55.3 ± 16.4 years, 11 women), and 30 the test sample (55.0 ± 15.0 years, 9 women). Four input settings were differentiated: only the T1w noncontrast and T1w low-dose images (standard setting), only the T1w noncontrast and T1w low-dose images with a prolonged postinjection time of 5 minutes (5-minute setting), multiple noncontrast sequences (T1w, T2w, diffusion) and the T1w low-dose images (extended setting), and only noncontrast sequences (T1w, T2w, diffusion) were used (zero-dose setting). For each setting, a deep neural network was trained to synthesize artificial T1w full-dose images, which were assessed on the test sample using an objective evaluation based on quantitative metrics and a subjective evaluation through a reader-based study. Three readers scored the overall image quality, the interchangeability in regard to the clinical conclusion compared with the true T1w full-dose sequence, the contrast enhancement of lesions, and their conformity to the respective references in the true T1w full dose. RESULTS Quantitative analysis of the artificial T1w full-dose images of the standard setting provided a peak signal-to-noise ratio of 33.39 ± 0.62 (corresponding to an average improvement of the low-dose sequences of 5.2 dB) and a structural similarity index measure of 0.938 ± 0.005. In the 4-fold cross-validation, the extended setting yielded similar performance to the standard setting in terms of peak signal-to-noise ratio ( P = 0.20), but a slight improvement in structural similarity index measure ( P < 0.0001). For all settings, the reader study found comparable overall image quality between the original and artificial T1w full-dose images. The proportion of scans scored as fully or mostly interchangeable was 55%, 58%, 43%, and 3% and the average counts of false positives per case were 0.42 ± 0.83, 0.34 ± 0.71, 0.82 ± 1.15, and 2.00 ± 1.07 for the standard, 5-minute, extended, and zero-dose setting, respectively. Using a 5-point Likert scale (0 to 4, 0 being the worst), all settings of synthesized full-dose images showed significantly poorer contrast enhancement of lesions compared with the original full-dose sequence (difference of average degree of contrast enhancement-standard: -0.97 ± 0.83, P = <0.001; 5-minute: -0.93 ± 0.91, P = <0.001; extended: -0.96 ± 0.97, P = <0.001; zero-dose: -2.39 ± 1.14, P = <0.001). The average scores of conformity of the lesions compared with the original full-dose sequence were 2.25 ± 1.21, 2.22 ± 1.27, 2.24 ± 1.25, and 0.73 ± 0.93 for the standard, 5-minute, extended, and zero-dose setting, respectively. CONCLUSIONS The tested deep learning algorithm for synthesis of artificial T1w full-dose sequences based on images after administration of only 10% of the standard dose of a gadolinium-based contrast agent showed very good quantitative performance. Despite good image quality in all settings, both false-negative and false-positive signals resulted in significantly limited interchangeability of the synthesized sequences with the original full-dose sequences.
Collapse
Affiliation(s)
| | - Thomas Pinetz
- Institute of Applied Mathematics, Rheinische Friedrich-Wilhelms-Universität Bonn, Bonn, Germany
| | - Zeynep Bendella
- From the Department of Neuroradiology, University Medical Center Bonn, Rheinische Friedrich-Wilhelms-Universität Bonn
| | - Erich Kobler
- From the Department of Neuroradiology, University Medical Center Bonn, Rheinische Friedrich-Wilhelms-Universität Bonn
| | | | - Wolfgang Block
- From the Department of Neuroradiology, University Medical Center Bonn, Rheinische Friedrich-Wilhelms-Universität Bonn
| | - Alexander Effland
- Institute of Applied Mathematics, Rheinische Friedrich-Wilhelms-Universität Bonn, Bonn, Germany
| | | | | |
Collapse
|
7
|
Gdoura A, Degünther M, Lorenz B, Effland A. Combining CNNs and Markov-like Models for Facial Landmark Detection with Spatial Consistency Estimates. J Imaging 2023; 9:jimaging9050104. [PMID: 37233323 DOI: 10.3390/jimaging9050104] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2023] [Revised: 05/09/2023] [Accepted: 05/11/2023] [Indexed: 05/27/2023] Open
Abstract
The accurate localization of facial landmarks is essential for several tasks, including face recognition, head pose estimation, facial region extraction, and emotion detection. Although the number of required landmarks is task-specific, models are typically trained on all available landmarks in the datasets, limiting efficiency. Furthermore, model performance is strongly influenced by scale-dependent local appearance information around landmarks and the global shape information generated by them. To account for this, we propose a lightweight hybrid model for facial landmark detection designed specifically for pupil region extraction. Our design combines a convolutional neural network (CNN) with a Markov random field (MRF)-like process trained on only 17 carefully selected landmarks. The advantage of our model is the ability to run different image scales on the same convolutional layers, resulting in a significant reduction in model size. In addition, we employ an approximation of the MRF that is run on a subset of landmarks to validate the spatial consistency of the generated shape. This validation process is performed against a learned conditional distribution, expressing the location of one landmark relative to its neighbor. Experimental results on popular facial landmark localization datasets such as 300 w, WFLW, and HELEN demonstrate the accuracy of our proposed model. Furthermore, our model achieves state-of-the-art performance on a well-defined robustness metric. In conclusion, the results demonstrate the ability of our lightweight model to filter out spatially inconsistent predictions, even with significantly fewer training landmarks.
Collapse
Affiliation(s)
- Ahmed Gdoura
- Department of Ophthalmology, Justus-Liebig-University Gießen, 35392 Gießen, Germany
- Department of Mathematics, Natural Sciences and Data Processing, Technische Hochschule Mittelhessen, 61169 Friedberg, Germany
| | - Markus Degünther
- Department of Mathematics, Natural Sciences and Data Processing, Technische Hochschule Mittelhessen, 61169 Friedberg, Germany
| | - Birgit Lorenz
- Department of Ophthalmology, Justus-Liebig-University Gießen, 35392 Gießen, Germany
- Department of Ophthalmology, University Hospital Bonn, 53127 Bonn, Germany
| | - Alexander Effland
- Institute of Applied Mathematics, University of Bonn, 53115 Bonn, Germany
| |
Collapse
|
8
|
Kobler E, Effland A, Kunisch K, Pock T. Total Deep Variation: A Stable Regularization Method for Inverse Problems. IEEE Trans Pattern Anal Mach Intell 2022; 44:9163-9180. [PMID: 34727026 DOI: 10.1109/tpami.2021.3124086] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
Various problems in computer vision and medical imaging can be cast as inverse problems. A frequent method for solving inverse problems is the variational approach, which amounts to minimizing an energy composed of a data fidelity term and a regularizer. Classically, handcrafted regularizers are used, which are commonly outperformed by state-of-the-art deep learning approaches. In this work, we combine the variational formulation of inverse problems with deep learning by introducing the data-driven general-purpose total deep variation regularizer. In its core, a convolutional neural network extracts local features on multiple scales and in successive blocks. This combination allows for a rigorous mathematical analysis including an optimal control formulation of the training problem in a mean-field setting and a stability analysis with respect to the initial values and the parameters of the regularizer. In addition, we experimentally verify the robustness against adversarial attacks and numerically derive upper bounds for the generalization error. Finally, we achieve state-of-the-art results for several imaging tasks.
Collapse
|
9
|
Narnhofer D, Effland A, Kobler E, Hammernik K, Knoll F, Pock T. Bayesian Uncertainty Estimation of Learned Variational MRI Reconstruction. IEEE Trans Med Imaging 2022; 41:279-291. [PMID: 34506279 PMCID: PMC8941176 DOI: 10.1109/tmi.2021.3112040] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
Recent deep learning approaches focus on improving quantitative scores of dedicated benchmarks, and therefore only reduce the observation-related (aleatoric) uncertainty. However, the model-immanent (epistemic) uncertainty is less frequently systematically analyzed. In this work, we introduce a Bayesian variational framework to quantify the epistemic uncertainty. To this end, we solve the linear inverse problem of undersampled MRI reconstruction in a variational setting. The associated energy functional is composed of a data fidelity term and the total deep variation (TDV) as a learned parametric regularizer. To estimate the epistemic uncertainty we draw the parameters of the TDV regularizer from a multivariate Gaussian distribution, whose mean and covariance matrix are learned in a stochastic optimal control problem. In several numerical experiments, we demonstrate that our approach yields competitive results for undersampled MRI reconstruction. Moreover, we can accurately quantify the pixelwise epistemic uncertainty, which can serve radiologists as an additional resource to visualize reconstruction reliability.
Collapse
|
10
|
Böhner AM, Jacob AM, Heuser C, Stumpf NE, Effland A, Abdullah Z, Meyer-Schwesiger C, von Vietinghoff S, Kurts C. Renal Denervation Exacerbates LPS- and Antibody-induced Acute Kidney Injury, but Protects from Pyelonephritis in Mice. J Am Soc Nephrol 2021; 32:2445-2453. [PMID: 34599036 PMCID: PMC8722799 DOI: 10.1681/asn.2021010110] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/27/2021] [Accepted: 06/01/2021] [Indexed: 02/04/2023] Open
Abstract
BACKGROUND Renal denervation (RDN) is an invasive intervention to treat drug-resistant arterial hypertension. Its therapeutic value is contentious. Here we examined the effects of RDN on inflammatory and infectious kidney disease models in mice. METHODS Mice were unilaterally or bilaterally denervated, or sham operated, then three disease models were induced: nephrotoxic nephritis (NTN, a model for crescentic GN), pyelonephritis, and acute endotoxemic kidney injury (as a model for septic kidney injury). Analytical methods included measurement of renal glomerular filtration, proteinuria, flow cytometry of renal immune cells, immunofluorescence microscopy, and three-dimensional imaging of optically cleared kidney tissue by light-sheet fluorescence microscopy followed by algorithmic analysis. RESULTS Unilateral RDN increased glomerular filtration in denervated kidneys, but decreased it in the contralateral kidneys. In the NTN model, more nephritogenic antibodies were deposited in glomeruli of denervated kidneys, resulting in stronger inflammation and injury in denervated compared with contralateral nondenervated kidneys. Also, intravenously injected LPS increased neutrophil influx and inflammation in the denervated kidneys, both after unilateral and bilateral RDN. When we induced pyelonephritis in bilaterally denervated mice, both kidneys contained less bacteria and neutrophils. In unilaterally denervated mice, pyelonephritis was attenuated and intrarenal neutrophil numbers were lower in the denervated kidneys. The nondenervated contralateral kidneys harbored more bacteria, even compared with sham-operated mice, and showed the strongest influx of neutrophils. CONCLUSIONS Our data suggest that the increased perfusion and filtration in denervated kidneys can profoundly influence concomitant inflammatory diseases. Renal deposition of circulating nephritic material is higher, and hence antibody- and endotoxin-induced kidney injury was aggravated in mice. Pyelonephritis was attenuated in denervated murine kidneys, because the higher glomerular filtration facilitated better flushing of bacteria with the urine, at the expense of contralateral, nondenervated kidneys after unilateral denervation.
Collapse
Affiliation(s)
- Alexander M.C. Böhner
- Institute for Molecular Medicine and Experimental Immunology, University Hospital of Bonn, Germany
- Department of Radiation Oncology, University Hospital of Bonn, Germany
| | - Alice M. Jacob
- Institute for Molecular Medicine and Experimental Immunology, University Hospital of Bonn, Germany
| | - Christoph Heuser
- Institute for Molecular Medicine and Experimental Immunology, University Hospital of Bonn, Germany
| | - Natascha E. Stumpf
- Institute for Molecular Medicine and Experimental Immunology, University Hospital of Bonn, Germany
| | | | - Zeinab Abdullah
- Institute for Molecular Medicine and Experimental Immunology, University Hospital of Bonn, Germany
| | | | | | - Christian Kurts
- Institute for Molecular Medicine and Experimental Immunology, University Hospital of Bonn, Germany
- Department of Microbiology and Immunology, Doherty Institute for Infection and Immunity, Victoria, Australia
| |
Collapse
|
11
|
Grandits T, Effland A, Pock T, Krause R, Plank G, Pezzuto S. GEASI: Geodesic-based earliest activation sites identification in cardiac models. Int J Numer Method Biomed Eng 2021; 37:e3505. [PMID: 34170082 PMCID: PMC8459297 DOI: 10.1002/cnm.3505] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/08/2021] [Revised: 06/19/2021] [Accepted: 06/22/2021] [Indexed: 05/18/2023]
Abstract
The identification of the initial ventricular activation sequence is a critical step for the correct personalization of patient-specific cardiac models. In healthy conditions, the Purkinje network is the main source of the electrical activation, but under pathological conditions the so-called earliest activation sites (EASs) are possibly sparser and more localized. Yet, their number, location and timing may not be easily inferred from remote recordings, such as the epicardial activation or the 12-lead electrocardiogram (ECG), due to the underlying complexity of the model. In this work, we introduce GEASI (Geodesic-based Earliest Activation Sites Identification) as a novel approach to simultaneously identify all EASs. To this end, we start from the anisotropic eikonal equation modeling cardiac electrical activation and exploit its Hamilton-Jacobi formulation to minimize a given objective function, for example, the quadratic mismatch to given activation measurements. This versatile approach can be extended to estimate the number of activation sites by means of the topological gradient, or fitting a given ECG. We conducted various experiments in 2D and 3D for in-silico models and an in-vivo intracardiac recording collected from a patient undergoing cardiac resynchronization therapy. The results demonstrate the clinical applicability of GEASI for potential future personalized models and clinical intervention.
Collapse
Affiliation(s)
- Thomas Grandits
- Institute of Computer Graphics and VisionTU GrazGrazAustria
- BioTechMed‐GrazGrazAustria
| | - Alexander Effland
- Institute of Computer Graphics and VisionTU GrazGrazAustria
- Silicon Austria Labs (TU Graz SAL DES Lab)GrazAustria
- Institute for Applied MathematicsUniversity of BonnBonnGermany
| | - Thomas Pock
- Institute of Computer Graphics and VisionTU GrazGrazAustria
- BioTechMed‐GrazGrazAustria
| | - Rolf Krause
- Center for Computational Medicine in Cardiology, Euler InstituteUniversità della Svizzera ItalianaLuganoSwitzerland
| | - Gernot Plank
- BioTechMed‐GrazGrazAustria
- Gottfried Schatz Research Center—Division of BiophysicsMedical University of GrazGrazAustria
| | - Simone Pezzuto
- Center for Computational Medicine in Cardiology, Euler InstituteUniversità della Svizzera ItalianaLuganoSwitzerland
| |
Collapse
|
12
|
Effland A, Kobler E, Pock T, Rajković M, Rumpf M. Image Morphing in Deep Feature Spaces: Theory and Applications. J Math Imaging Vis 2020; 63:309-327. [PMID: 33627956 PMCID: PMC7878289 DOI: 10.1007/s10851-020-00974-5] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/23/2019] [Accepted: 06/08/2020] [Indexed: 06/12/2023]
Abstract
This paper combines image metamorphosis with deep features. To this end, images are considered as maps into a high-dimensional feature space and a structure-sensitive, anisotropic flow regularization is incorporated in the metamorphosis model proposed by Miller and Younes (Int J Comput Vis 41(1):61-84, 2001) and Trouvé and Younes (Found Comput Math 5(2):173-198, 2005). For this model, a variational time discretization of the Riemannian path energy is presented and the existence of discrete geodesic paths minimizing this energy is demonstrated. Furthermore, convergence of discrete geodesic paths to geodesic paths in the time continuous model is investigated. The spatial discretization is based on a finite difference approximation in image space and a stable spline approximation in deformation space; the fully discrete model is optimized using the iPALM algorithm. Numerical experiments indicate that the incorporation of semantic deep features is superior to intensity-based approaches.
Collapse
Affiliation(s)
- Alexander Effland
- Institute of Computer Graphics and Vision, Graz University of Technology, Graz, Austria
| | - Erich Kobler
- Institute of Computer Graphics and Vision, Graz University of Technology, Graz, Austria
| | - Thomas Pock
- Institute of Computer Graphics and Vision, Graz University of Technology, Graz, Austria
| | - Marko Rajković
- Institute of Numerical Simulation, University of Bonn, Bonn, Germany
| | - Martin Rumpf
- Institute of Numerical Simulation, University of Bonn, Bonn, Germany
| |
Collapse
|
13
|
Effland A, Kobler E, Kunisch K, Pock T. Variational Networks: An Optimal Control Approach to Early Stopping Variational Methods for Image Restoration. J Math Imaging Vis 2020; 62:396-416. [PMID: 32300264 PMCID: PMC7138785 DOI: 10.1007/s10851-019-00926-8] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 07/17/2019] [Accepted: 10/29/2019] [Indexed: 06/11/2023]
Abstract
We investigate a well-known phenomenon of variational approaches in image processing, where typically the best image quality is achieved when the gradient flow process is stopped before converging to a stationary point. This paradox originates from a tradeoff between optimization and modeling errors of the underlying variational model and holds true even if deep learning methods are used to learn highly expressive regularizers from data. In this paper, we take advantage of this paradox and introduce an optimal stopping time into the gradient flow process, which in turn is learned from data by means of an optimal control approach. After a time discretization, we obtain variational networks, which can be interpreted as a particular type of recurrent neural networks. The learned variational networks achieve competitive results for image denoising and image deblurring on a standard benchmark data set. One of the key theoretical results is the development of first- and second-order conditions to verify optimal stopping time. A nonlinear spectral analysis of the gradient of the learned regularizer gives enlightening insights into the different regularization properties.
Collapse
Affiliation(s)
- Alexander Effland
- Institute of Computer Graphics and Vision, Graz University of Technology, Graz, Austria
| | - Erich Kobler
- Institute of Computer Graphics and Vision, Graz University of Technology, Graz, Austria
| | - Karl Kunisch
- Institute of Mathematics and Scientific Computing, University of Graz, Graz, Austria
| | - Thomas Pock
- Institute of Computer Graphics and Vision, Graz University of Technology, Graz, Austria
| |
Collapse
|
14
|
Effland A, Kobler E, Brandenburg A, Klatzer T, Neuhäuser L, Hölzel M, Landsberg J, Pock T, Rumpf M. Joint reconstruction and classification of tumor cells and cell interactions in melanoma tissue sections with synthesized training data. Int J Comput Assist Radiol Surg 2019; 14:587-599. [PMID: 30779021 PMCID: PMC6420907 DOI: 10.1007/s11548-019-01919-z] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2018] [Accepted: 01/21/2019] [Indexed: 01/02/2023]
Abstract
Purpose Cancers are almost always diagnosed by morphologic features in tissue sections. In this context, machine learning tools provide new opportunities to describe tumor immune cell interactions within the tumor microenvironment and thus provide phenotypic information that might be predictive for the response to immunotherapy. Methods We develop a machine learning approach using variational networks for joint image denoising and classification of tissue sections for melanoma, which is an established model tumor for immuno-oncology research. The manual annotation of real training data would require substantial user interaction of experienced pathologists for each single training image, and the training of larger networks would rely on a very large number of such data sets with ground truth annotation. To overcome this bottleneck, we synthesize training data together with a proper tissue structure classification. To this end, a stochastic data generation process is used to mimic cell morphology, cell distribution and tissue architecture in the tumor microenvironment. Particular components of this tool are random placement and rotation of a large number of patches for presegmented cell nuclei, a stochastic fast marching approach to mimic the geometry of cells and texture generation based on a color covariance analysis of real data. Here, the generated training data reflect a large range of interaction patterns. Results In several applications to histological tissue sections, we analyze the efficiency and accuracy of the proposed approach. As a result, depending on the scenario considered, almost all cells and nuclei which ought to be detected are actually marked as classified and hardly any misclassifications occur. Conclusions The proposed method allows for a computer-aided screening of histological tissue sections utilizing variational networks with a particular emphasis on tumor immune cell interactions and on the robust cell nuclei classification.
Collapse
Affiliation(s)
- Alexander Effland
- Institute for Numerical Simulation, University of Bonn, Bonn, Germany.
| | - Erich Kobler
- Institute of Computer Graphics and Vision, Graz University of Technology, Graz, Austria
| | - Anne Brandenburg
- Department of Dermatology and Allergy, University of Bonn, Bonn, Germany
| | - Teresa Klatzer
- Institute of Computer Graphics and Vision, Graz University of Technology, Graz, Austria
| | - Leonie Neuhäuser
- Institute for Numerical Simulation, University of Bonn, Bonn, Germany
| | - Michael Hölzel
- Institute of Clinical Chemistry and Clinical Pharmacology, University of Bonn, Bonn, Germany
| | - Jennifer Landsberg
- Department of Dermatology and Allergy, University of Bonn, Bonn, Germany
| | - Thomas Pock
- Institute of Computer Graphics and Vision, Graz University of Technology, Graz, Austria
| | - Martin Rumpf
- Institute for Numerical Simulation, University of Bonn, Bonn, Germany
| |
Collapse
|