1
|
Kebaili A, Lapuyade-Lahorgue J, Ruan S. Deep Learning Approaches for Data Augmentation in Medical Imaging: A Review. J Imaging 2023; 9:81. [PMID: 37103232 PMCID: PMC10144738 DOI: 10.3390/jimaging9040081] [Citation(s) in RCA: 14] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2023] [Revised: 03/31/2023] [Accepted: 04/07/2023] [Indexed: 04/28/2023] Open
Abstract
Deep learning has become a popular tool for medical image analysis, but the limited availability of training data remains a major challenge, particularly in the medical field where data acquisition can be costly and subject to privacy regulations. Data augmentation techniques offer a solution by artificially increasing the number of training samples, but these techniques often produce limited and unconvincing results. To address this issue, a growing number of studies have proposed the use of deep generative models to generate more realistic and diverse data that conform to the true distribution of the data. In this review, we focus on three types of deep generative models for medical image augmentation: variational autoencoders, generative adversarial networks, and diffusion models. We provide an overview of the current state of the art in each of these models and discuss their potential for use in different downstream tasks in medical imaging, including classification, segmentation, and cross-modal translation. We also evaluate the strengths and limitations of each model and suggest directions for future research in this field. Our goal is to provide a comprehensive review about the use of deep generative models for medical image augmentation and to highlight the potential of these models for improving the performance of deep learning algorithms in medical image analysis.
Collapse
Affiliation(s)
| | | | - Su Ruan
- Université Rouen Normandie, INSA Rouen Normandie, Université Le Havre Normandie, Normandie Univ, LITIS UR 4108, F-76000 Rouen, France
| |
Collapse
|
2
|
Moya‐Sáez E, Navarro‐González R, Cepeda S, Pérez‐Núñez Á, de Luis‐García R, Aja‐Fernández S, Alberola‐López C. Synthetic MRI improves radiomics-based glioblastoma survival prediction. NMR IN BIOMEDICINE 2022; 35:e4754. [PMID: 35485596 PMCID: PMC9542221 DOI: 10.1002/nbm.4754] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/24/2022] [Revised: 04/26/2022] [Accepted: 04/26/2022] [Indexed: 06/14/2023]
Abstract
Glioblastoma is an aggressive and fast-growing brain tumor with poor prognosis. Predicting the expected survival of patients with glioblastoma is a key task for efficient treatment and surgery planning. Survival predictions could be enhanced by means of a radiomic system. However, these systems demand high numbers of multicontrast images, the acquisitions of which are time consuming, giving rise to patient discomfort and low healthcare system efficiency. Synthetic MRI could favor deployment of radiomic systems in the clinic by allowing practitioners not only to reduce acquisition time, but also to retrospectively complete databases or to replace artifacted images. In this work we analyze the replacement of an actually acquired MR weighted image by a synthesized version to predict survival of glioblastoma patients with a radiomic system. Each synthesized version was realistically generated from two acquired images with a deep learning synthetic MRI approach based on a convolutional neural network. Specifically, two weighted images were considered for the replacement one at a time, a T2w and a FLAIR, which were synthesized from the pairs T1w and FLAIR, and T1w and T2w, respectively. Furthermore, a radiomic system for survival prediction, which can classify patients into two groups (survival >480 days and ≤ 480 days), was built. Results show that the radiomic system fed with the synthesized image achieves similar performance compared with using the acquired one, and better performance than a model that does not include this image. Hence, our results confirm that synthetic MRI does add to glioblastoma survival prediction within a radiomics-based approach.
Collapse
Affiliation(s)
- Elisa Moya‐Sáez
- Laboratorio de Procesado de ImagenUniversidad de ValladolidValladolid
| | | | - Santiago Cepeda
- Departamento de NeurocirugíaHospital Universitario Río HortegaValladolidSpain
| | - Ángel Pérez‐Núñez
- Departamento de NeurocirugíaHospital Universitario 12 de OctubreMadridSpain
| | | | | | | |
Collapse
|
3
|
Jeong JJ, Tariq A, Adejumo T, Trivedi H, Gichoya JW, Banerjee I. Systematic Review of Generative Adversarial Networks (GANs) for Medical Image Classification and Segmentation. J Digit Imaging 2022; 35:137-152. [PMID: 35022924 PMCID: PMC8921387 DOI: 10.1007/s10278-021-00556-w] [Citation(s) in RCA: 31] [Impact Index Per Article: 15.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/05/2021] [Revised: 11/23/2021] [Accepted: 11/26/2021] [Indexed: 11/28/2022] Open
Abstract
In recent years, generative adversarial networks (GANs) have gained tremendous popularity for various imaging related tasks such as artificial image generation to support AI training. GANs are especially useful for medical imaging-related tasks where training datasets are usually limited in size and heavily imbalanced against the diseased class. We present a systematic review, following the PRISMA guidelines, of recent GAN architectures used for medical image analysis to help the readers in making an informed decision before employing GANs in developing medical image classification and segmentation models. We have extracted 54 papers that highlight the capabilities and application of GANs in medical imaging from January 2015 to August 2020 and inclusion criteria for meta-analysis. Our results show four main architectures of GAN that are used for segmentation or classification in medical imaging. We provide a comprehensive overview of recent trends in the application of GANs in clinical diagnosis through medical image segmentation and classification and ultimately share experiences for task-based GAN implementations.
Collapse
Affiliation(s)
- Jiwoong J Jeong
- Department of Biomedical Informatics, Emory School of Medicine, Atlanta, USA.
| | - Amara Tariq
- Department of Biomedical Informatics, Emory School of Medicine, Atlanta, USA
| | | | - Hari Trivedi
- Department of Radiology, Emory School of Medicine, Atlanta, USA
| | - Judy W Gichoya
- Department of Radiology, Emory School of Medicine, Atlanta, USA
| | - Imon Banerjee
- Department of Biomedical Informatics, Emory School of Medicine, Atlanta, USA.,Department of Radiology, Emory School of Medicine, Atlanta, USA
| |
Collapse
|
4
|
Chlap P, Min H, Vandenberg N, Dowling J, Holloway L, Haworth A. A review of medical image data augmentation techniques for deep learning applications. J Med Imaging Radiat Oncol 2021; 65:545-563. [PMID: 34145766 DOI: 10.1111/1754-9485.13261] [Citation(s) in RCA: 155] [Impact Index Per Article: 51.7] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2021] [Accepted: 05/23/2021] [Indexed: 12/21/2022]
Abstract
Research in artificial intelligence for radiology and radiotherapy has recently become increasingly reliant on the use of deep learning-based algorithms. While the performance of the models which these algorithms produce can significantly outperform more traditional machine learning methods, they do rely on larger datasets being available for training. To address this issue, data augmentation has become a popular method for increasing the size of a training dataset, particularly in fields where large datasets aren't typically available, which is often the case when working with medical images. Data augmentation aims to generate additional data which is used to train the model and has been shown to improve performance when validated on a separate unseen dataset. This approach has become commonplace so to help understand the types of data augmentation techniques used in state-of-the-art deep learning models, we conducted a systematic review of the literature where data augmentation was utilised on medical images (limited to CT and MRI) to train a deep learning model. Articles were categorised into basic, deformable, deep learning or other data augmentation techniques. As artificial intelligence models trained using augmented data make their way into the clinic, this review aims to give an insight to these techniques and confidence in the validity of the models produced.
Collapse
Affiliation(s)
- Phillip Chlap
- South Western Sydney Clinical School, University of New South Wales, Sydney, New South Wales, Australia.,Ingham Institute for Applied Medical Research, Sydney, New South Wales, Australia.,Liverpool and Macarthur Cancer Therapy Centre, Liverpool Hospital, Sydney, New South Wales, Australia
| | - Hang Min
- South Western Sydney Clinical School, University of New South Wales, Sydney, New South Wales, Australia.,Ingham Institute for Applied Medical Research, Sydney, New South Wales, Australia.,The Australian e-Health and Research Centre, CSIRO Health and Biosecurity, Brisbane, Queensland, Australia
| | - Nym Vandenberg
- Institute of Medical Physics, University of Sydney, Sydney, New South Wales, Australia
| | - Jason Dowling
- South Western Sydney Clinical School, University of New South Wales, Sydney, New South Wales, Australia.,The Australian e-Health and Research Centre, CSIRO Health and Biosecurity, Brisbane, Queensland, Australia
| | - Lois Holloway
- South Western Sydney Clinical School, University of New South Wales, Sydney, New South Wales, Australia.,Ingham Institute for Applied Medical Research, Sydney, New South Wales, Australia.,Liverpool and Macarthur Cancer Therapy Centre, Liverpool Hospital, Sydney, New South Wales, Australia.,Institute of Medical Physics, University of Sydney, Sydney, New South Wales, Australia.,Centre for Medical Radiation Physics, University of Wollongong, Wollongong, New South Wales, Australia
| | - Annette Haworth
- Institute of Medical Physics, University of Sydney, Sydney, New South Wales, Australia
| |
Collapse
|