1
|
Kofler A, Wald C, Kolbitsch C, V Tycowicz C, Ambellan F. Joint reconstruction and segmentation in undersampled 3D knee MRI combining shape knowledge and deep learning. Phys Med Biol 2024; 69:095022. [PMID: 38527376 DOI: 10.1088/1361-6560/ad3797] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/16/2023] [Accepted: 03/25/2024] [Indexed: 03/27/2024]
Abstract
Objective.Task-adapted image reconstruction methods using end-to-end trainable neural networks (NNs) have been proposed to optimize reconstruction for subsequent processing tasks, such as segmentation. However, their training typically requires considerable hardware resources and thus, only relatively simple building blocks, e.g. U-Nets, are typically used, which, albeit powerful, do not integrate model-specific knowledge.Approach.In this work, we extend an end-to-end trainable task-adapted image reconstruction method for a clinically realistic reconstruction and segmentation problem of bone and cartilage in 3D knee MRI by incorporating statistical shape models (SSMs). The SSMs model the prior information and help to regularize the segmentation maps as a final post-processing step. We compare the proposed method to a simultaneous multitask learning approach for image reconstruction and segmentation (MTL) and to a complex SSMs-informed segmentation pipeline (SIS).Main results.Our experiments show that the combination of joint end-to-end training and SSMs to further regularize the segmentation maps obtained by MTL highly improves the results, especially in terms of mean and maximal surface errors. In particular, we achieve the segmentation quality of SIS and, at the same time, a substantial model reduction that yields a five-fold decimation in model parameters and a computational speedup of an order of magnitude.Significance.Remarkably, even for undersampling factors of up toR= 8, the obtained segmentation maps are of comparable quality to those obtained by SIS from ground-truth images.
Collapse
Affiliation(s)
- A Kofler
- Physikalisch-Technische Bundesanstalt, Braunschweig and Berlin, Germany
| | - C Wald
- Department of Mathematics, Technical University of Berlin, Berlin, Germany
| | - C Kolbitsch
- Physikalisch-Technische Bundesanstalt, Braunschweig and Berlin, Germany
| | - C V Tycowicz
- Department of Visual and Data-Centric Computing, Zuse Institute Berlin, Berlin, Germany
| | - F Ambellan
- Department of Visual and Data-Centric Computing, Zuse Institute Berlin, Berlin, Germany
| |
Collapse
|
2
|
Glatzeder K, Komnik I, Ambellan F, Zachow S, Potthast W. Author Correction: Dynamic pressure analysis of novel interpositional knee spacer implants in 3D-printed human knee models. Sci Rep 2022; 12:21954. [PMID: 36536063 PMCID: PMC9763326 DOI: 10.1038/s41598-022-26540-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022] Open
Affiliation(s)
- Korbinian Glatzeder
- grid.27593.3a0000 0001 2244 5164Institute of Biomechanics and Orthopaedics, German Sport University Cologne, Am Sportpark Müngersdorf 6, 50933 Cologne, Germany
| | - Igor Komnik
- grid.27593.3a0000 0001 2244 5164Institute of Biomechanics and Orthopaedics, German Sport University Cologne, Am Sportpark Müngersdorf 6, 50933 Cologne, Germany
| | - Felix Ambellan
- grid.425649.80000 0001 1010 926XZuse Institute Berlin (ZIB), Takustraße 7, 14195 Berlin, Germany ,grid.14095.390000 0000 9116 4836Freie Universität Berlin, Kaiserswerther Str. 16‑18, Berlin, Germany
| | - Stefan Zachow
- grid.425649.80000 0001 1010 926XZuse Institute Berlin (ZIB), Takustraße 7, 14195 Berlin, Germany
| | - Wolfgang Potthast
- grid.27593.3a0000 0001 2244 5164Institute of Biomechanics and Orthopaedics, German Sport University Cologne, Am Sportpark Müngersdorf 6, 50933 Cologne, Germany
| |
Collapse
|
3
|
Glatzeder K, Igor K, Ambellan F, Zachow S, Potthast W. Dynamic pressure analysis of novel interpositional knee spacer implants in 3D-printed human knee models. Sci Rep 2022; 12:16853. [PMID: 36207344 PMCID: PMC9546830 DOI: 10.1038/s41598-022-20463-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2021] [Accepted: 09/13/2022] [Indexed: 11/09/2022] Open
Abstract
Alternative treatment methods for knee osteoarthritis (OA) are in demand, to delay the young (< 50 Years) patient's need for osteotomy or knee replacement. Novel interpositional knee spacers shape based on statistical shape model (SSM) approach and made of polyurethane (PU) were developed to present a minimally invasive method to treat medial OA in the knee. The implant should be supposed to reduce peak strains and pain, restore the stability of the knee, correct the malalignment of a varus knee and improve joint function and gait. Firstly, the spacers were tested in artificial knee models. It is assumed that by application of a spacer, a significant reduction in stress values and a significant increase in the contact area in the medial compartment of the knee will be registered. Biomechanical analysis of the effect of novel interpositional knee spacer implants on pressure distribution in 3D-printed knee model replicas: the primary purpose was the medial joint contact stress-related biomechanics. A secondary purpose was a better understanding of medial/lateral redistribution of joint loading. Six 3D printed knee models were reproduced from cadaveric leg computed tomography. Each of four spacer implants was tested in each knee geometry under realistic arthrokinematic dynamic loading conditions, to examine the pressure distribution in the knee joint. All spacers showed reduced mean stress values by 84-88% and peak stress values by 524-704% in the medial knee joint compartment compared to the non-spacer test condition. The contact area was enlarged by 462-627% as a result of the inserted spacers. Concerning the appreciable contact stress reduction and enlargement of the contact area in the medial knee joint compartment, the premises are in place for testing the implants directly on human knee cadavers to gain further insights into a possible tool for treating medial knee osteoarthritis.
Collapse
Affiliation(s)
- Korbinian Glatzeder
- Institute of Biomechanics and Orthopaedics, German Sport University Cologne, Am Sportpark Müngersdorf 6, 50933, Cologne, Germany.
| | - Komnik Igor
- Institute of Biomechanics and Orthopaedics, German Sport University Cologne, Am Sportpark Müngersdorf 6, 50933, Cologne, Germany
| | - Felix Ambellan
- Zuse Institute Berlin (ZIB), Takustraße 7, 14195, Berlin, Germany.,Freie Universität Berlin, Kaiserswerther Str. 16-18, Berlin, Germany
| | - Stefan Zachow
- Zuse Institute Berlin (ZIB), Takustraße 7, 14195, Berlin, Germany
| | - Wolfgang Potthast
- Institute of Biomechanics and Orthopaedics, German Sport University Cologne, Am Sportpark Müngersdorf 6, 50933, Cologne, Germany
| |
Collapse
|
4
|
Tack A, Ambellan F, Zachow S. Towards novel osteoarthritis biomarkers: Multi-criteria evaluation of 46,996 segmented knee MRI data from the Osteoarthritis Initiative. PLoS One 2021; 16:e0258855. [PMID: 34673842 PMCID: PMC8530341 DOI: 10.1371/journal.pone.0258855] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2021] [Accepted: 10/06/2021] [Indexed: 01/16/2023] Open
Abstract
Convolutional neural networks (CNNs) are the state-of-the-art for automated assessment of knee osteoarthritis (KOA) from medical image data. However, these methods lack interpretability, mainly focus on image texture, and cannot completely grasp the analyzed anatomies' shapes. In this study we assess the informative value of quantitative features derived from segmentations in order to assess their potential as an alternative or extension to CNN-based approaches regarding multiple aspects of KOA. Six anatomical structures around the knee (femoral and tibial bones, femoral and tibial cartilages, and both menisci) are segmented in 46,996 MRI scans. Based on these segmentations, quantitative features are computed, i.e., measurements such as cartilage volume, meniscal extrusion and tibial coverage, as well as geometric features based on a statistical shape encoding of the anatomies. The feature quality is assessed by investigating their association to the Kellgren-Lawrence grade (KLG), joint space narrowing (JSN), incident KOA, and total knee replacement (TKR). Using gold standard labels from the Osteoarthritis Initiative database the balanced accuracy (BA), the area under the Receiver Operating Characteristic curve (AUC), and weighted kappa statistics are evaluated. Features based on shape encodings of femur, tibia, and menisci plus the performed measurements showed most potential as KOA biomarkers. Differentiation between non-arthritic and severely arthritic knees yielded BAs of up to 99%, 84% were achieved for diagnosis of early KOA. Weighted kappa values of 0.73, 0.72, and 0.78 were achieved for classification of the grade of medial JSN, lateral JSN, and KLG, respectively. The AUC was 0.61 and 0.76 for prediction of incident KOA and TKR within one year, respectively. Quantitative features from automated segmentations provide novel biomarkers for KLG and JSN classification and show potential for incident KOA and TKR prediction. The validity of these features should be further evaluated, especially as extensions of CNN-based approaches. To foster such developments we make all segmentations publicly available together with this publication.
Collapse
Affiliation(s)
| | | | - Stefan Zachow
- Zuse Institute Berlin, Berlin, Germany
- Charité – Universitätsmedizin Berlin, Berlin, Germany
| |
Collapse
|
5
|
Sekuboyina A, Husseini ME, Bayat A, Löffler M, Liebl H, Li H, Tetteh G, Kukačka J, Payer C, Štern D, Urschler M, Chen M, Cheng D, Lessmann N, Hu Y, Wang T, Yang D, Xu D, Ambellan F, Amiranashvili T, Ehlke M, Lamecker H, Lehnert S, Lirio M, Olaguer NPD, Ramm H, Sahu M, Tack A, Zachow S, Jiang T, Ma X, Angerman C, Wang X, Brown K, Kirszenberg A, Puybareau É, Chen D, Bai Y, Rapazzo BH, Yeah T, Zhang A, Xu S, Hou F, He Z, Zeng C, Xiangshang Z, Liming X, Netherton TJ, Mumme RP, Court LE, Huang Z, He C, Wang LW, Ling SH, Huỳnh LD, Boutry N, Jakubicek R, Chmelik J, Mulay S, Sivaprakasam M, Paetzold JC, Shit S, Ezhov I, Wiestler B, Glocker B, Valentinitsch A, Rempfler M, Menze BH, Kirschke JS. VerSe: A Vertebrae labelling and segmentation benchmark for multi-detector CT images. Med Image Anal 2021; 73:102166. [PMID: 34340104 DOI: 10.1016/j.media.2021.102166] [Citation(s) in RCA: 64] [Impact Index Per Article: 21.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2020] [Revised: 06/25/2021] [Accepted: 07/06/2021] [Indexed: 11/25/2022]
Abstract
Vertebral labelling and segmentation are two fundamental tasks in an automated spine processing pipeline. Reliable and accurate processing of spine images is expected to benefit clinical decision support systems for diagnosis, surgery planning, and population-based analysis of spine and bone health. However, designing automated algorithms for spine processing is challenging predominantly due to considerable variations in anatomy and acquisition protocols and due to a severe shortage of publicly available data. Addressing these limitations, the Large Scale Vertebrae Segmentation Challenge (VerSe) was organised in conjunction with the International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) in 2019 and 2020, with a call for algorithms tackling the labelling and segmentation of vertebrae. Two datasets containing a total of 374 multi-detector CT scans from 355 patients were prepared and 4505 vertebrae have individually been annotated at voxel level by a human-machine hybrid algorithm (https://osf.io/nqjyw/, https://osf.io/t98fz/). A total of 25 algorithms were benchmarked on these datasets. In this work, we present the results of this evaluation and further investigate the performance variation at the vertebra level, scan level, and different fields of view. We also evaluate the generalisability of the approaches to an implicit domain shift in data by evaluating the top-performing algorithms of one challenge iteration on data from the other iteration. The principal takeaway from VerSe: the performance of an algorithm in labelling and segmenting a spine scan hinges on its ability to correctly identify vertebrae in cases of rare anatomical variations. The VerSe content and code can be accessed at: https://github.com/anjany/verse.
Collapse
Affiliation(s)
- Anjany Sekuboyina
- Department of Informatics, Technical University of Munich, Germany; Munich School of BioEngineering, Technical University of Munich, Germany; Department of Neuroradiology, Klinikum Rechts der Isar, Germany.
| | - Malek E Husseini
- Department of Informatics, Technical University of Munich, Germany; Department of Neuroradiology, Klinikum Rechts der Isar, Germany
| | - Amirhossein Bayat
- Department of Informatics, Technical University of Munich, Germany; Department of Neuroradiology, Klinikum Rechts der Isar, Germany
| | | | - Hans Liebl
- Department of Neuroradiology, Klinikum Rechts der Isar, Germany
| | - Hongwei Li
- Department of Informatics, Technical University of Munich, Germany
| | - Giles Tetteh
- Department of Informatics, Technical University of Munich, Germany
| | - Jan Kukačka
- Institute of Biological and Medical Imaging, Helmholtz Zentrum München, Germany
| | - Christian Payer
- Institute of Computer Graphics and Vision, Graz University of Technology, Austria
| | - Darko Štern
- Gottfried Schatz Research Center: Biophysics, Medical University of Graz, Austria
| | - Martin Urschler
- School of Computer Science, The University of Auckland, New Zealand
| | - Maodong Chen
- Computer Vision Group, iFLYTEK Research South China, China
| | - Dalong Cheng
- Computer Vision Group, iFLYTEK Research South China, China
| | - Nikolas Lessmann
- Department of Radiology and Nuclear Medicine, Radboud University Medical Center Nijmegen, The Netherlands
| | - Yujin Hu
- Shenzhen Research Institute of Big Data, China
| | - Tianfu Wang
- School of Biomedical Engineering, Health Science Center, Shenzhen University, China
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Xin Wang
- Department of Electronic Engineering, Fudan University, China; Department of Radiology, University of North Carolina at Chapel Hill, USA
| | | | | | | | | | | | | | | | | | | | - Feng Hou
- Institute of Computing Technology, Chinese Academy of Sciences, China
| | | | | | - Zheng Xiangshang
- College of Computer Science and Technology, Zhejiang University, China; Real Doctor AI Research Centre, Zhejiang University, China
| | - Xu Liming
- College of Computer Science and Technology, Zhejiang University, China
| | | | | | | | - Zixun Huang
- Department of Electronic and Information Engineering, The Hong Kong Polytechnic University, China
| | - Chenhang He
- Department of Computing, The Hong Kong Polytechnic University, China
| | - Li-Wen Wang
- Department of Electronic and Information Engineering, The Hong Kong Polytechnic University, China
| | - Sai Ho Ling
- The School of Biomedical Engineering, University of Technology Sydney, Australia
| | - Lê Duy Huỳnh
- EPITA Research and Development Laboratory (LRDE), France
| | - Nicolas Boutry
- EPITA Research and Development Laboratory (LRDE), France
| | - Roman Jakubicek
- Department of Biomedical Engineering, Brno University of Technology, Czech Republic
| | - Jiri Chmelik
- Department of Biomedical Engineering, Brno University of Technology, Czech Republic
| | - Supriti Mulay
- Indian Institute of Technology Madras, India; Healthcare Technology Innovation Centre, India
| | | | | | - Suprosanna Shit
- Department of Informatics, Technical University of Munich, Germany
| | - Ivan Ezhov
- Department of Informatics, Technical University of Munich, Germany
| | | | - Ben Glocker
- Department of Computing, Imperial College London, UK
| | | | - Markus Rempfler
- Friedrich Miescher Institute for Biomedical Engineering, Switzerland
| | - Björn H Menze
- Department of Informatics, Technical University of Munich, Germany; Department for Quantitative Biomedicine, University of Zurich, Switzerland
| | - Jan S Kirschke
- Department of Neuroradiology, Klinikum Rechts der Isar, Germany
| |
Collapse
|
6
|
Abstract
In our chapter we are describing how to reconstruct three-dimensional anatomy from medical image data and how to build Statistical 3D Shape Models out of many such reconstructions yielding a new kind of anatomy that not only allows quantitative analysis of anatomical variation but also a visual exploration and educational visualization. Future digital anatomy atlases will not only show a static (average) anatomy but also its normal or pathological variation in three or even four dimensions, hence, illustrating growth and/or disease progression.Statistical Shape Models (SSMs) are geometric models that describe a collection of semantically similar objects in a very compact way. SSMs represent an average shape of many three-dimensional objects as well as their variation in shape. The creation of SSMs requires a correspondence mapping, which can be achieved e.g. by parameterization with a respective sampling. If a corresponding parameterization over all shapes can be established, variation between individual shape characteristics can be mathematically investigated.We will explain what Statistical Shape Models are and how they are constructed. Extensions of Statistical Shape Models will be motivated for articulated coupled structures. In addition to shape also the appearance of objects will be integrated into the concept. Appearance is a visual feature independent of shape that depends on observers or imaging techniques. Typical appearances are for instance the color and intensity of a visual surface of an object under particular lighting conditions, or measurements of material properties with computed tomography (CT) or magnetic resonance imaging (MRI). A combination of (articulated) Statistical Shape Models with statistical models of appearance lead to articulated Statistical Shape and Appearance Models (a-SSAMs).After giving various examples of SSMs for human organs, skeletal structures, faces, and bodies, we will shortly describe clinical applications where such models have been successfully employed. Statistical Shape Models are the foundation for the analysis of anatomical cohort data, where characteristic shapes are correlated to demographic or epidemiologic data. SSMs consisting of several thousands of objects offer, in combination with statistical methods or machine learning techniques, the possibility to identify characteristic clusters, thus being the foundation for advanced diagnostic disease scoring.
Collapse
Affiliation(s)
| | - Hans Lamecker
- Zuse Institute Berlin, Berlin, Germany.,1000 Shapes GmbH, Berlin, Germany
| | | | - Stefan Zachow
- Zuse Institute Berlin, Berlin, Germany. .,1000 Shapes GmbH, Berlin, Germany.
| |
Collapse
|