1
|
Borowa A, Rymarczyk D, Żyła M, Kańdula M, Sánchez-Fernández A, Rataj K, Struski Ł, Tabor J, Zieliński B. Decoding phenotypic screening: A comparative analysis of image representations. Comput Struct Biotechnol J 2024; 23:1181-1188. [PMID: 38510976 PMCID: PMC10951426 DOI: 10.1016/j.csbj.2024.02.022] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2023] [Revised: 02/26/2024] [Accepted: 02/26/2024] [Indexed: 03/22/2024] Open
Abstract
Biomedical imaging techniques such as high content screening (HCS) are valuable for drug discovery, but high costs limit their use to pharmaceutical companies. To address this issue, The JUMP-CP consortium released a massive open image dataset of chemical and genetic perturbations, providing a valuable resource for deep learning research. In this work, we aim to utilize the JUMP-CP dataset to develop a universal representation model for HCS data, mainly data generated using U2OS cells and CellPainting protocol, using supervised and self-supervised learning approaches. We propose an evaluation protocol that assesses their performance on mode of action and property prediction tasks using a popular phenotypic screening dataset. Results show that the self-supervised approach that uses data from multiple consortium partners provides representation that is more robust to batch effects whilst simultaneously achieving performance on par with standard approaches. Together with other conclusions, it provides recommendations on the training strategy of a representation model for HCS images.
Collapse
Affiliation(s)
- Adriana Borowa
- Jagiellonian University, Faculty of Mathematics and Computer Science, Kraków, Poland
- Jagiellonian University, Doctoral School of Exact and Natural Sciences, Kraków, Poland
- Ardigen SA, Kraków, Poland
| | - Dawid Rymarczyk
- Jagiellonian University, Faculty of Mathematics and Computer Science, Kraków, Poland
- Ardigen SA, Kraków, Poland
| | | | | | | | | | - Łukasz Struski
- Jagiellonian University, Faculty of Mathematics and Computer Science, Kraków, Poland
| | - Jacek Tabor
- Jagiellonian University, Faculty of Mathematics and Computer Science, Kraków, Poland
| | - Bartosz Zieliński
- Jagiellonian University, Faculty of Mathematics and Computer Science, Kraków, Poland
- Ardigen SA, Kraków, Poland
| |
Collapse
|
2
|
Seal S, Trapotsi MA, Spjuth O, Singh S, Carreras-Puigvert J, Greene N, Bender A, Carpenter AE. A Decade in a Systematic Review: The Evolution and Impact of Cell Painting. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.05.04.592531. [PMID: 38766203 PMCID: PMC11100607 DOI: 10.1101/2024.05.04.592531] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/22/2024]
Abstract
High-content image-based assays have fueled significant discoveries in the life sciences in the past decade (2013-2023), including novel insights into disease etiology, mechanism of action, new therapeutics, and toxicology predictions. Here, we systematically review the substantial methodological advancements and applications of Cell Painting. Advancements include improvements in the Cell Painting protocol, assay adaptations for different types of perturbations and applications, and improved methodologies for feature extraction, quality control, and batch effect correction. Moreover, machine learning methods recently surpassed classical approaches in their ability to extract biologically useful information from Cell Painting images. Cell Painting data have been used alone or in combination with other - omics data to decipher the mechanism of action of a compound, its toxicity profile, and many other biological effects. Overall, key methodological advances have expanded Cell Painting's ability to capture cellular responses to various perturbations. Future advances will likely lie in advancing computational and experimental techniques, developing new publicly available datasets, and integrating them with other high-content data types.
Collapse
Affiliation(s)
- Srijit Seal
- Broad Institute of MIT and Harvard, Cambridge, Massachusetts, United States
- Yusuf Hamied Department of Chemistry, University of Cambridge, Lensfield Road, CB2 1EW, Cambridge, United Kingdom
| | - Maria-Anna Trapotsi
- Imaging and Data Analytics, Clinical Pharmacology & Safety Sciences, R&D, AstraZeneca, 1 Francis Crick Avenue, Cambridge, CB2 0AA, United Kingdom
| | - Ola Spjuth
- Department of Pharmaceutical Biosciences and Science for Life Laboratory, Uppsala University, Box 591, SE-75124, Uppsala, Sweden
| | - Shantanu Singh
- Imaging and Data Analytics, Clinical Pharmacology & Safety Sciences, R&D, AstraZeneca, 1 Francis Crick Avenue, Cambridge, CB2 0AA, United Kingdom
| | - Jordi Carreras-Puigvert
- Department of Pharmaceutical Biosciences and Science for Life Laboratory, Uppsala University, Box 591, SE-75124, Uppsala, Sweden
| | - Nigel Greene
- Imaging and Data Analytics, Clinical Pharmacology & Safety Sciences, R&D, AstraZeneca, 35 Gatehouse Drive, Waltham, MA 02451, USA
| | - Andreas Bender
- Yusuf Hamied Department of Chemistry, University of Cambridge, Lensfield Road, CB2 1EW, Cambridge, United Kingdom
| | - Anne E. Carpenter
- Broad Institute of MIT and Harvard, Cambridge, Massachusetts, United States
| |
Collapse
|
3
|
Ostovich E, Klaper R. Using a Novel Multiplexed Algal Cytological Imaging (MACI) Assay and Machine Learning as a Way to Characterize Complex Phenotypes in Plant-Type Organisms. ENVIRONMENTAL SCIENCE & TECHNOLOGY 2024; 58:4894-4903. [PMID: 38446593 DOI: 10.1021/acs.est.3c07733] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/08/2024]
Abstract
High-throughput phenotypic profiling assays, popular for their ability to characterize alternations in single-cell morphological feature data, have been useful in recent years for predicting cellular targets and mechanisms of action (MoAs) for different chemicals and novel drugs. However, this approach has not been extensively used in environmental toxicology due to the lack of studies and established methods for performing this kind of assay in environmentally relevant species. Here, we developed a multiplexed algal cytological imaging (MACI) assay, based on the subcellular structures of the unicellular microalgae, Raphidocelis subcapitata, a toxicology and ecological model species. Several different herbicides and antibiotics with unique MoAs were exposed to R. subcapitata cells, and MACI was used to characterize cellular impacts by measuring subtle changes in their morphological features, including metrics of area, shape, quantity, fluorescence intensity, and granularity of individual subcellular components. This study demonstrates that MACI offers a quick and effective framework for characterizing complex phenotypic responses to environmental chemicals that can be used for determining their MoAs and identifying their cellular targets in plant-type organisms.
Collapse
Affiliation(s)
- Eric Ostovich
- School of Freshwater Sciences, University of Wisconsin-Milwaukee, Milwaukee, Wisconsin 53204, United States
| | - Rebecca Klaper
- School of Freshwater Sciences, University of Wisconsin-Milwaukee, Milwaukee, Wisconsin 53204, United States
| |
Collapse
|
4
|
Lau TA, Mair E, Rabbitts BM, Lohith A, Lokey RS. High-Content Image-Based Screening and Deep Learning for the Detection of Anti-Inflammatory Drug Leads. Chembiochem 2024; 25:e202300136. [PMID: 37815526 PMCID: PMC11126213 DOI: 10.1002/cbic.202300136] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2023] [Revised: 10/02/2023] [Accepted: 10/10/2023] [Indexed: 10/11/2023]
Abstract
We developed a high-content image-based screen that utilizes the pro-inflammatory stimulus lipopolysaccharide (LPS) and murine macrophages (RAW264.7) with the goal of enabling the identification of novel anti-inflammatory lead compounds. We screened 2,259 bioactive compounds with annotated mechanisms of action (MOA) to identify compounds that block the LPS-induced phenotype in macrophages. We utilized a set of seven fluorescence microscopy probes to generate images that were used to train and optimize a deep neural network classifier to distinguish between unstimulated and LPS-stimulated macrophages. The top hits from the deep learning classifier were validated using a linear classifier trained on individual cells and subsequently investigated in a multiplexed cytokine secretion assay. All 12 hits significantly modulated the expression of at least one cytokine upon LPS stimulation. Seven of these were allosteric inhibitors of the mitogen-activated protein kinase kinase (MEK1/2) and showed similar effects on cytokine expression. This deep learning morphological assay identified compounds that modulate the innate immune response to LPS and may aid in identifying new anti-inflammatory drug leads.
Collapse
Affiliation(s)
- Tannia A Lau
- Department of Chemistry and Biochemistry, University of California Santa Cruz, Santa Cruz, CA 95064, USA
| | - Elmar Mair
- No affiliation, Santa Cruz, CA 95060, USA
| | - Beverley M Rabbitts
- Department of Chemistry and Biochemistry, University of California Santa Cruz, Santa Cruz, CA 95064, USA
| | - Akshar Lohith
- Department of Chemistry and Biochemistry, University of California Santa Cruz, Santa Cruz, CA 95064, USA
| | - R Scott Lokey
- Department of Chemistry and Biochemistry, University of California Santa Cruz, Santa Cruz, CA 95064, USA
| |
Collapse
|
5
|
Kumar N, Marée R, Geurts P, Muller M. Recent Advances in Bioimage Analysis Methods for Detecting Skeletal Deformities in Biomedical and Aquaculture Fish Species. Biomolecules 2023; 13:1797. [PMID: 38136667 PMCID: PMC10742266 DOI: 10.3390/biom13121797] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2023] [Revised: 12/05/2023] [Accepted: 12/09/2023] [Indexed: 12/24/2023] Open
Abstract
Detecting skeletal or bone-related deformities in model and aquaculture fish is vital for numerous biomedical studies. In biomedical research, model fish with bone-related disorders are potential indicators of various chemically induced toxins in their environment or poor dietary conditions. In aquaculture, skeletal deformities are affecting fish health, and economic losses are incurred by fish farmers. This survey paper focuses on showcasing the cutting-edge image analysis tools and techniques based on artificial intelligence that are currently applied in the analysis of bone-related deformities in aquaculture and model fish. These methods and tools play a significant role in improving research by automating various aspects of the analysis. This paper also sheds light on some of the hurdles faced when dealing with high-content bioimages and explores potential solutions to overcome these challenges.
Collapse
Affiliation(s)
- Navdeep Kumar
- Department of Computer Science and Electrical Engineering, Montefiore Institute, University of Liège, 4000 Liège, Belgium; (R.M.); (P.G.)
| | - Raphaël Marée
- Department of Computer Science and Electrical Engineering, Montefiore Institute, University of Liège, 4000 Liège, Belgium; (R.M.); (P.G.)
| | - Pierre Geurts
- Department of Computer Science and Electrical Engineering, Montefiore Institute, University of Liège, 4000 Liège, Belgium; (R.M.); (P.G.)
| | - Marc Muller
- Laboratory for Organogenesis and Regeneration (LOR), GIGA Institute, University of Liège, 4000 Liège, Belgium;
| |
Collapse
|
6
|
Way GP, Sailem H, Shave S, Kasprowicz R, Carragher NO. Evolution and impact of high content imaging. SLAS DISCOVERY : ADVANCING LIFE SCIENCES R & D 2023; 28:292-305. [PMID: 37666456 DOI: 10.1016/j.slasd.2023.08.009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/03/2023] [Revised: 08/09/2023] [Accepted: 08/29/2023] [Indexed: 09/06/2023]
Abstract
The field of high content imaging has steadily evolved and expanded substantially across many industry and academic research institutions since it was first described in the early 1990's. High content imaging refers to the automated acquisition and analysis of microscopic images from a variety of biological sample types. Integration of high content imaging microscopes with multiwell plate handling robotics enables high content imaging to be performed at scale and support medium- to high-throughput screening of pharmacological, genetic and diverse environmental perturbations upon complex biological systems ranging from 2D cell cultures to 3D tissue organoids to small model organisms. In this perspective article the authors provide a collective view on the following key discussion points relevant to the evolution of high content imaging: • Evolution and impact of high content imaging: An academic perspective • Evolution and impact of high content imaging: An industry perspective • Evolution of high content image analysis • Evolution of high content data analysis pipelines towards multiparametric and phenotypic profiling applications • The role of data integration and multiomics • The role and evolution of image data repositories and sharing standards • Future perspective of high content imaging hardware and software.
Collapse
Affiliation(s)
- Gregory P Way
- Department of Biomedical Informatics, University of Colorado Anschutz Medical Campus, Aurora, CO, USA
| | - Heba Sailem
- School of Cancer and Pharmaceutical Sciences, King's College London, UK
| | - Steven Shave
- GlaxoSmithKline Medicines Research Centre, Gunnels Wood Rd, Stevenage SG1 2NY, UK; Edinburgh Cancer Research, Cancer Research UK Scotland Centre, Institute of Genetics and Cancer, University of Edinburgh, UK
| | - Richard Kasprowicz
- GlaxoSmithKline Medicines Research Centre, Gunnels Wood Rd, Stevenage SG1 2NY, UK
| | - Neil O Carragher
- Edinburgh Cancer Research, Cancer Research UK Scotland Centre, Institute of Genetics and Cancer, University of Edinburgh, UK.
| |
Collapse
|
7
|
Depto DS, Rizvee MM, Rahman A, Zunair H, Rahman MS, Mahdy MRC. Quantifying imbalanced classification methods for leukemia detection. Comput Biol Med 2023; 152:106372. [PMID: 36516574 DOI: 10.1016/j.compbiomed.2022.106372] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2022] [Revised: 11/01/2022] [Accepted: 11/27/2022] [Indexed: 12/03/2022]
Abstract
Uncontrolled proliferation of B-lymphoblast cells is a common characterization of Acute Lymphoblastic Leukemia (ALL). B-lymphoblasts are found in large numbers in peripheral blood in malignant cases. Early detection of the cell in bone marrow is essential as the disease progresses rapidly if left untreated. However, automated classification of the cell is challenging, owing to its fine-grained variability with B-lymphoid precursor cells and imbalanced data points. Deep learning algorithms demonstrate potential for such fine-grained classification as well as suffer from the imbalanced class problem. In this paper, we explore different deep learning-based State-Of-The-Art (SOTA) approaches to tackle imbalanced classification problems. Our experiment includes input, GAN (Generative Adversarial Networks), and loss-based methods to mitigate the issue of imbalanced class on the challenging C-NMC and ALLIDB-2 dataset for leukemia detection. We have shown empirical evidence that loss-based methods outperform GAN-based and input-based methods in imbalanced classification scenarios.
Collapse
Affiliation(s)
- Deponker Sarker Depto
- Department of Electrical and Computer Engineering, North South University, Bashundhara, Dhaka, 1229, Bangladesh.
| | - Md Mashfiq Rizvee
- Department of Electrical and Computer Engineering, North South University, Bashundhara, Dhaka, 1229, Bangladesh; Texas Tech University, Lubbock, TX, United States of America.
| | - Aimon Rahman
- Department of Electrical and Computer Engineering, North South University, Bashundhara, Dhaka, 1229, Bangladesh.
| | | | - M Sohel Rahman
- Department of Computer Science and Engineering, Bangladesh University of Engineering and Technology, ECE Building, West Palasi, Dhaka 1205, Bangladesh.
| | - M R C Mahdy
- Department of Electrical and Computer Engineering, North South University, Bashundhara, Dhaka, 1229, Bangladesh.
| |
Collapse
|
8
|
Baručić D, Kaushik S, Kybic J, Stanková J, Džubák P, Hajdúch M. Characterization of drug effects on cell cultures from phase-contrast microscopy images. Comput Biol Med 2022; 151:106171. [PMID: 36306582 DOI: 10.1016/j.compbiomed.2022.106171] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2022] [Revised: 08/30/2022] [Accepted: 10/01/2022] [Indexed: 12/27/2022]
Abstract
In this work, we classify chemotherapeutic agents (topoisomerase inhibitors) based on their effect on U-2 OS cells. We use phase-contrast microscopy images, which are faster and easier to obtain than fluorescence images and support live cell imaging. We use a convolutional neural network (CNN) trained end-to-end directly on the input images without requiring for manual segmentations or any other auxiliary data. Our method can distinguish between tested cytotoxic drugs with an accuracy of 98%, provided that their mechanism of action differs, outperforming previous work. The results are even better when substance-specific concentrations are used. We show the benefit of sharing the extracted features over all classes (drugs). Finally, a 2D visualization of these features reveals clusters, which correspond well to known class labels, suggesting the possible use of our methodology for drug discovery application in analyzing new, unseen drugs.
Collapse
Affiliation(s)
- Denis Baručić
- Faculty of Electrical Engineering, Czech Technical University in Prague, Prague, Czech Republic.
| | - Sumit Kaushik
- Faculty of Electrical Engineering, Czech Technical University in Prague, Prague, Czech Republic.
| | - Jan Kybic
- Faculty of Electrical Engineering, Czech Technical University in Prague, Prague, Czech Republic.
| | - Jarmila Stanková
- Institute of Molecular and Translational Medicine, Faculty of Medicine and Dentistry, Palacky University, Olomouc, Czech Republic
| | - Petr Džubák
- Institute of Molecular and Translational Medicine, Faculty of Medicine and Dentistry, Palacky University, Olomouc, Czech Republic
| | - Marián Hajdúch
- Institute of Molecular and Translational Medicine, Faculty of Medicine and Dentistry, Palacky University, Olomouc, Czech Republic
| |
Collapse
|
9
|
Severin Y, Hale BD, Mena J, Goslings D, Frey BM, Snijder B. Multiplexed high-throughput immune cell imaging reveals molecular health-associated phenotypes. SCIENCE ADVANCES 2022; 8:eabn5631. [PMID: 36322666 PMCID: PMC9629716 DOI: 10.1126/sciadv.abn5631] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/05/2023]
Abstract
Phenotypic plasticity is essential to the immune system, yet the factors that shape it are not fully understood. Here, we comprehensively analyze immune cell phenotypes including morphology across human cohorts by single-round multiplexed immunofluorescence, automated microscopy, and deep learning. Using the uncertainty of convolutional neural networks to cluster the phenotypes of eight distinct immune cell subsets, we find that the resulting maps are influenced by donor age, gender, and blood pressure, revealing distinct polarization and activation-associated phenotypes across immune cell classes. We further associate T cell morphology to transcriptional state based on their joint donor variability and validate an inflammation-associated polarized T cell morphology and an age-associated loss of mitochondria in CD4+ T cells. Together, we show that immune cell phenotypes reflect both molecular and personal health information, opening new perspectives into the deep immune phenotyping of individual people in health and disease.
Collapse
Affiliation(s)
- Yannik Severin
- Department of Biology, Institute of Molecular Systems Biology, ETH Zürich, 8049 Zürich, Switzerland
| | - Benjamin D. Hale
- Department of Biology, Institute of Molecular Systems Biology, ETH Zürich, 8049 Zürich, Switzerland
| | - Julien Mena
- Department of Biology, Institute of Molecular Systems Biology, ETH Zürich, 8049 Zürich, Switzerland
| | - David Goslings
- Blood Transfusion Service Zürich, SRC, 8952 Schlieren, Switzerland
| | - Beat M. Frey
- Blood Transfusion Service Zürich, SRC, 8952 Schlieren, Switzerland
| | - Berend Snijder
- Department of Biology, Institute of Molecular Systems Biology, ETH Zürich, 8049 Zürich, Switzerland
- Corresponding author.
| |
Collapse
|
10
|
Machine learning phenomics (MLP) combining deep learning with time-lapse-microscopy for monitoring colorectal adenocarcinoma cells gene expression and drug-response. Sci Rep 2022; 12:8545. [PMID: 35595808 PMCID: PMC9123013 DOI: 10.1038/s41598-022-12364-5] [Citation(s) in RCA: 14] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/18/2021] [Accepted: 01/31/2022] [Indexed: 11/25/2022] Open
Abstract
High-throughput phenotyping is becoming increasingly available thanks to analytical and bioinformatics approaches that enable the use of very high-dimensional data and to the availability of dynamic models that link phenomena across levels: from genes to cells, from cells to organs, and through the whole organism. The combination of phenomics, deep learning, and machine learning represents a strong potential for the phenotypical investigation, leading the way to a more embracing approach, called machine learning phenomics (MLP). In particular, in this work we present a novel MLP platform for phenomics investigation of cancer-cells response to therapy, exploiting and combining the potential of time-lapse microscopy for cell behavior data acquisition and robust deep learning software architectures for the latent phenotypes extraction. A two-step proof of concepts is designed. First, we demonstrate a strict correlation among gene expression and cell phenotype with the aim to identify new biomarkers and targets for tailored therapy in human colorectal cancer onset and progression. Experiments were conducted on human colorectal adenocarcinoma cells (DLD-1) and their profile was compared with an isogenic line in which the expression of LOX-1 transcript was knocked down. In addition, we also evaluate the phenotypic impact of the administration of different doses of an antineoplastic drug over DLD-1 cells. Under the omics paradigm, proteomics results are used to confirm the findings of the experiments.
Collapse
|
11
|
Xu C, Zhang Y, Fan X, Lan X, Ye X, Wu T. An efficient fluorescence in situ hybridization (FISH)-based circulating genetically abnormal cells (CACs) identification method based on Multi-scale MobileNet-YOLO-V4. Quant Imaging Med Surg 2022; 12:2961-2976. [PMID: 35502367 PMCID: PMC9014158 DOI: 10.21037/qims-21-909] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2021] [Accepted: 02/11/2022] [Indexed: 11/06/2023]
Abstract
BACKGROUND Circulating tumor cells (CTCs) acting as "liquid biopsy" of cancer are cells that have been shed from the primary tumor, which cause the development of a secondary tumor in a distant organ site, leading to cancer metastasis. Recent research suggests that CTCs with abnormalities in gene copy numbers in mononuclear cell-enriched peripheral blood samples, namely circulating genetically abnormal cells (CACs), could be used as a non-invasive decision tool to detect patients with benign pulmonary nodules. Such cells are identified by counting the fluorescence signals of fluorescence in situ hybridization (FISH). However, owing to the rarity of CACs in the blood, identification of CACs using this technique is time-consuming and is a drawback of this method. METHODS This study has proposed an efficient and automatic FISH-based CACs identification approach which is based on a combination of the high accuracy of You Only Look Once (YOLO)-V4 and the lightweight and rapidness of MobileNet-V3. The backbone of YOLO-V4 was replaced with MobileNet-V3 to improve the detection efficiency and prevent overfitting, and the architecture of YOLO-V4 was optimized by utilizing a new feature map with a larger scale to enable the enhanced detection ability for small targets. RESULTS We trained and tested the proposed model using a dataset containing more than 7,000 cells based on five-fold cross-validation. All the images in the dataset were 2,448×2,048 (pixels) in size. The number of cells in each image was >70. The accuracy of four-color fluorescence signals detection for our proposed model were all approximately 98%, and the mean average precision (mAP) were close to 100%. The final outcome of the developed method was the type of cells, i.e., normal cells, CACs, gaining cells or deletion cells. The method had a CACs identification accuracy of 93.86% (similar to an expert pathologist), and a detection speed that was about 500 times greater than that of a pathologist. CONCLUSIONS The developed method could greatly increase the review accuracy, enhance the efficiency of reviewers, and reduce the review turnaround time during CACs identification.
Collapse
Affiliation(s)
- Chao Xu
- China Telecommunication Technology Labs, China Academy of Information and Communications Technology, Beijing, China
| | - Yi Zhang
- China Telecommunication Technology Labs, China Academy of Information and Communications Technology, Beijing, China
| | - Xianjun Fan
- Department of Product Development, Zhuhai Sanmed Biotech Ltd., Zhuhai, China
| | - Xingjie Lan
- Department of Data Operation, Zhuhai Sanmed Biotech Ltd., Zhuhai, China
| | - Xin Ye
- Department of Product Development, Zhuhai Sanmed Biotech Ltd., Zhuhai, China
| | - Tongning Wu
- China Telecommunication Technology Labs, China Academy of Information and Communications Technology, Beijing, China
| |
Collapse
|
12
|
Siegismund D, Fassler M, Heyse S, Steigele S. Benchmarking feature selection methods for compressing image information in high-content screening. SLAS Technol 2022; 27:85-93. [DOI: 10.1016/j.slast.2021.10.015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
|
13
|
Deep localization of subcellular protein structures from fluorescence microscopy images. Neural Comput Appl 2022. [DOI: 10.1007/s00521-021-06715-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
|
14
|
From imaging a single cell to implementing precision medicine: an exciting new era. Emerg Top Life Sci 2021; 5:837-847. [PMID: 34889448 PMCID: PMC8786301 DOI: 10.1042/etls20210219] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2021] [Revised: 11/24/2021] [Accepted: 11/24/2021] [Indexed: 11/17/2022]
Abstract
In the age of high-throughput, single-cell biology, single-cell imaging has evolved not only in terms of technological advancements but also in its translational applications. The synchronous advancements of imaging and computational biology have produced opportunities of merging the two, providing the scientific community with tools towards observing, understanding, and predicting cellular and tissue phenotypes and behaviors. Furthermore, multiplexed single-cell imaging and machine learning algorithms now enable patient stratification and predictive diagnostics of clinical specimens. Here, we provide an overall summary of the advances in single-cell imaging, with a focus on high-throughput microscopy phenomics and multiplexed proteomic spatial imaging platforms. We also review various computational tools that have been developed in recent years for image processing and downstream applications used in biomedical sciences. Finally, we discuss how harnessing systems biology approaches and data integration across disciplines can further strengthen the exciting applications and future implementation of single-cell imaging on precision medicine.
Collapse
|
15
|
Dursun G, Tandale SB, Gulakala R, Eschweiler J, Tohidnezhad M, Markert B, Stoffel M. Development of convolutional neural networks for recognition of tenogenic differentiation based on cellular morphology. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2021; 208:106279. [PMID: 34343743 DOI: 10.1016/j.cmpb.2021.106279] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/06/2020] [Accepted: 07/06/2021] [Indexed: 06/13/2023]
Abstract
BACKGROUND AND OBJECTIVE The use of automated systems for image recognition is highly preferred for regenerative medicine applications to evaluate stem cell differentiation early in the culturing state with non-invasive methodologies instead of invasive counterparts. Bone marrow-derived mesenchymal stem cells (BMSCs) are able to differentiate into desired cell phenotypes, and thereby promise a proper cell source for tendon regeneration. The therapeutic success of stem cell therapy requires cellular characterization prior to the implantation of cells. The foremost problem is that traditional characterization techniques require cellular material which would be more useful for cell therapy, complex laboratory procedures, and human expertise. Convolutional neural networks (CNNs), a class of deep neural networks, have recently made great improvements in image-based classifications, recognition, and detection tasks. We, therefore, aim to develop a potential CNN model in order to recognize differentiated stem cells by learning features directly from image data of unlabelled cells. METHODS The differentiation of bone marrow mesenchymal stem cells (BMSCs) into tenocytes was induced with the treatment of bone morphogenetic protein-12 (BMP-12). Following the treatment and incubation step, the phase-contrast images of cells were obtained and immunofluorescence staining has been applied to characterize the differentiated state of BMSCs. CNN models were developed and trained with the phase-contrast cell images. The comparison of CNN models was performed with respect to prediction performance and training time. Moreover, we have evaluated the effect of image enhancement method, data augmentation, and fine-tuning training strategy to increase classification accuracy of CNN models. The best model was integrated into a mobile application. RESULTS All the CNN models can fit the biological data extracted from immunofluorescence characterization. CNN models enable the cell classification with satisfactory accuracies. The best result in terms of accuracy and training time is achieved by the model proposed based on Inception-ResNet V2 trained from scratch using image enhancement and data augmentation strategies (96.80%, 434.55 sec). CONCLUSION Our study reveals that the CNN models show good performance by identifying stem cell differentiation. Importantly this technique provides a faster and real-time tool in comparison to traditional methods enabling the adjustment of culture conditions during cultivation to improve the yield of therapeutic stem cells.
Collapse
Affiliation(s)
- Gözde Dursun
- Institute of General Mechanics, RWTH Aachen University, Aachen, Germany
| | | | - Rutwik Gulakala
- Institute of General Mechanics, RWTH Aachen University, Aachen, Germany
| | - Jörg Eschweiler
- Department of Orthopaedic Surgery, RWTH Aachen University, Aachen, Germany
| | | | - Bernd Markert
- Institute of General Mechanics, RWTH Aachen University, Aachen, Germany
| | - Marcus Stoffel
- Institute of General Mechanics, RWTH Aachen University, Aachen, Germany.
| |
Collapse
|
16
|
Lee SMW, Shaw A, Simpson JL, Uminsky D, Garratt LW. Differential cell counts using center-point networks achieves human-level accuracy and efficiency over segmentation. Sci Rep 2021; 11:16917. [PMID: 34413367 PMCID: PMC8377024 DOI: 10.1038/s41598-021-96067-3] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2021] [Accepted: 08/03/2021] [Indexed: 11/08/2022] Open
Abstract
Differential cell counts is a challenging task when applying computer vision algorithms to pathology. Existing approaches to train cell recognition require high availability of multi-class segmentation and/or bounding box annotations and suffer in performance when objects are tightly clustered. We present differential count network ("DCNet"), an annotation efficient modality that utilises keypoint detection to locate in brightfield images the centre points of cells (not nuclei) and their cell class. The single centre point annotation for DCNet lowered burden for experts to generate ground truth data by 77.1% compared to bounding box labeling. Yet centre point annotation still enabled high accuracy when training DCNet on a multi-class algorithm on whole cell features, matching human experts in all 5 object classes in average precision and outperforming humans in consistency. The efficacy and efficiency of the DCNet end-to-end system represents a significant progress toward an open source, fully computationally approach to differential cell count based diagnosis that can be adapted to any pathology need.
Collapse
Affiliation(s)
- Sarada M W Lee
- Perth Machine Learning Group, Perth, WA, 6000, Australia
- School of Medicine and Public Health, University of Newcastle, Callaghan, NSW, 2308, Australia
| | - Andrew Shaw
- Data Institute, University of San Francisco, San Francisco, CA, 94117, USA
| | - Jodie L Simpson
- School of Medicine and Public Health, University of Newcastle, Callaghan, NSW, 2308, Australia
- Priority Research Centre for Healthy Lungs, University of Newcastle, Callaghan, NSW, 2308, Australia
| | - David Uminsky
- Department of Computer Science, University of Chicago, Chicago, IL, 60637, USA
| | - Luke W Garratt
- Wal-yan Respiratory Research Centre, Telethon Kids Institute, University of Western Australia, Nedlands, WA, 6009, Australia.
| |
Collapse
|
17
|
Leyh J, Paeschke S, Mages B, Michalski D, Nowicki M, Bechmann I, Winter K. Classification of Microglial Morphological Phenotypes Using Machine Learning. Front Cell Neurosci 2021; 15:701673. [PMID: 34267628 PMCID: PMC8276040 DOI: 10.3389/fncel.2021.701673] [Citation(s) in RCA: 75] [Impact Index Per Article: 25.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/28/2021] [Accepted: 06/07/2021] [Indexed: 12/17/2022] Open
Abstract
Microglia are the brain's immunocompetent macrophages with a unique feature that allows surveillance of the surrounding microenvironment and subsequent reactions to tissue damage, infection, or homeostatic perturbations. Thereby, microglia's striking morphological plasticity is one of their prominent characteristics and the categorization of microglial cell function based on morphology is well established. Frequently, automated classification of microglial morphological phenotypes is performed by using quantitative parameters. As this process is typically limited to a few and especially manually chosen criteria, a relevant selection bias may compromise the resulting classifications. In our study, we describe a novel microglial classification method by morphological evaluation using a convolutional neuronal network on the basis of manually selected cells in addition to classical morphological parameters. We focused on four microglial morphologies, ramified, rod-like, activated and amoeboid microglia within the murine hippocampus and cortex. The developed method for the classification was confirmed in a mouse model of ischemic stroke which is already known to result in microglial activation within affected brain regions. In conclusion, our classification of microglial morphological phenotypes using machine learning can serve as a time-saving and objective method for post-mortem characterization of microglial changes in healthy and disease mouse models, and might also represent a useful tool for human brain autopsy samples.
Collapse
Affiliation(s)
- Judith Leyh
- Institute of Anatomy, University of Leipzig, Leipzig, Germany
| | - Sabine Paeschke
- Institute of Anatomy, University of Leipzig, Leipzig, Germany
| | - Bianca Mages
- Institute of Anatomy, University of Leipzig, Leipzig, Germany
| | | | - Marcin Nowicki
- Institute of Anatomy, University of Leipzig, Leipzig, Germany
| | - Ingo Bechmann
- Institute of Anatomy, University of Leipzig, Leipzig, Germany
| | - Karsten Winter
- Institute of Anatomy, University of Leipzig, Leipzig, Germany
| |
Collapse
|
18
|
Mattiazzi Usaj M, Yeung CHL, Friesen H, Boone C, Andrews BJ. Single-cell image analysis to explore cell-to-cell heterogeneity in isogenic populations. Cell Syst 2021; 12:608-621. [PMID: 34139168 PMCID: PMC9112900 DOI: 10.1016/j.cels.2021.05.010] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2020] [Revised: 04/26/2021] [Accepted: 05/12/2021] [Indexed: 12/26/2022]
Abstract
Single-cell image analysis provides a powerful approach for studying cell-to-cell heterogeneity, which is an important attribute of isogenic cell populations, from microbial cultures to individual cells in multicellular organisms. This phenotypic variability must be explained at a mechanistic level if biologists are to fully understand cellular function and address the genotype-to-phenotype relationship. Variability in single-cell phenotypes is obscured by bulk readouts or averaging of phenotypes from individual cells in a sample; thus, single-cell image analysis enables a higher resolution view of cellular function. Here, we consider examples of both small- and large-scale studies carried out with isogenic cell populations assessed by fluorescence microscopy, and we illustrate the advantages, challenges, and the promise of quantitative single-cell image analysis.
Collapse
Affiliation(s)
- Mojca Mattiazzi Usaj
- Department of Chemistry and Biology, Ryerson University, Toronto, ON M5B 2K3, Canada
| | - Clarence Hue Lok Yeung
- The Donnelly Centre, University of Toronto, Toronto, ON M5S 3E1, Canada; Department of Molecular Genetics, University of Toronto, Toronto, ON M5S 3E1, Canada
| | - Helena Friesen
- The Donnelly Centre, University of Toronto, Toronto, ON M5S 3E1, Canada
| | - Charles Boone
- The Donnelly Centre, University of Toronto, Toronto, ON M5S 3E1, Canada; Department of Molecular Genetics, University of Toronto, Toronto, ON M5S 3E1, Canada; RIKEN Centre for Sustainable Resource Science, Wako, Saitama 351-0198, Japan
| | - Brenda J Andrews
- The Donnelly Centre, University of Toronto, Toronto, ON M5S 3E1, Canada; Department of Molecular Genetics, University of Toronto, Toronto, ON M5S 3E1, Canada.
| |
Collapse
|
19
|
Abstract
The cell cycle is an important process in cellular life. In recent years, some image processing methods have been developed to determine the cell cycle stages of individual cells. However, in most of these methods, cells have to be segmented, and their features need to be extracted. During feature extraction, some important information may be lost, resulting in lower classification accuracy. Thus, we used a deep learning method to retain all cell features. In order to solve the problems surrounding insufficient numbers of original images and the imbalanced distribution of original images, we used the Wasserstein generative adversarial network-gradient penalty (WGAN-GP) for data augmentation. At the same time, a residual network (ResNet) was used for image classification. ResNet is one of the most used deep learning classification networks. The classification accuracy of cell cycle images was achieved more effectively with our method, reaching 83.88%. Compared with an accuracy of 79.40% in previous experiments, our accuracy increased by 4.48%. Another dataset was used to verify the effect of our model and, compared with the accuracy from previous results, our accuracy increased by 12.52%. The results showed that our new cell cycle image classification system based on WGAN-GP and ResNet is useful for the classification of imbalanced images. Moreover, our method could potentially solve the low classification accuracy in biomedical images caused by insufficient numbers of original images and the imbalanced distribution of original images.
Collapse
|
20
|
Zhang E, Zhang B, Hu S, Zhang F, Liu Z, Wan X. Multi-labelled proteins recognition for high-throughput microscopy images using deep convolutional neural networks. BMC Bioinformatics 2021; 22:327. [PMID: 34130623 PMCID: PMC8207617 DOI: 10.1186/s12859-021-04196-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/26/2021] [Accepted: 05/13/2021] [Indexed: 11/24/2022] Open
Abstract
BACKGROUND Proteins are of extremely vital importance in the human body, and no movement or activity can be performed without proteins. Currently, microscopy imaging technologies developed rapidly are employed to observe proteins in various cells and tissues. In addition, due to the complex and crowded cellular environments as well as various types and sizes of proteins, a considerable number of protein images are generated every day and cannot be classified manually. Therefore, an automatic and accurate method should be designed to properly solve and analyse protein images with mixed patterns. RESULTS In this paper, we first propose a novel customized architecture with adaptive concatenate pooling and "buffering" layers in the classifier part, which could make the networks more adaptive to training and testing datasets, and develop a novel hard sampler at the end of our network to effectively mine the samples from small classes. Furthermore, a new loss is presented to handle the label imbalance based on the effectiveness of samples. In addition, in our method, several novel and effective optimization strategies are adopted to solve the difficult training-time optimization problem and further increase the accuracy by post-processing. CONCLUSION Our methods outperformed the SOTA method of multi-labelled protein classification on the HPA dataset, GapNet-PL, by above 2% in the F1 score. Therefore, experimental results based on the test set split from the Human Protein Atlas dataset show that our methods have good performance in automatically classifying multi-class and multi-labelled high-throughput microscopy protein images.
Collapse
Affiliation(s)
- Enze Zhang
- High Performance Computer Research Center, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China
- University of Chinese Academy of Sciences, Beijing, China
| | - Boheng Zhang
- Department of Automation, Tsinghua University, Beijing, China
| | - Shaohan Hu
- School of Software, Tsinghua University, Beijing, China
| | - Fa Zhang
- High Performance Computer Research Center, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China
- University of Chinese Academy of Sciences, Beijing, China
| | - Zhiyong Liu
- High Performance Computer Research Center, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China
- University of Chinese Academy of Sciences, Beijing, China
| | - Xiaohua Wan
- High Performance Computer Research Center, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China.
- University of Chinese Academy of Sciences, Beijing, China.
| |
Collapse
|
21
|
Chao JT, Roskelley CD, Loewen CJR. MAPS: machine-assisted phenotype scoring enables rapid functional assessment of genetic variants by high-content microscopy. BMC Bioinformatics 2021; 22:202. [PMID: 33879063 PMCID: PMC8056608 DOI: 10.1186/s12859-021-04117-4] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/12/2020] [Accepted: 04/02/2021] [Indexed: 12/20/2022] Open
Abstract
BACKGROUND Genetic testing is widely used in evaluating a patient's predisposition to hereditary diseases. In the case of cancer, when a functionally impactful mutation (i.e. genetic variant) is identified in a disease-relevant gene, the patient is at elevated risk of developing a lesion in their lifetime. Unfortunately, as the rate and coverage of genetic testing has accelerated, our ability to assess the functional status of new variants has fallen behind. Therefore, there is an urgent need for more practical, streamlined and cost-effective methods for classifying variants. RESULTS To directly address this issue, we designed a new approach that uses alterations in protein subcellular localization as a key indicator of loss of function. Thus, new variants can be rapidly functionalized using high-content microscopy (HCM). To facilitate the analysis of the large amounts of imaging data, we developed a new software toolkit, named MAPS for machine-assisted phenotype scoring, that utilizes deep learning to extract and classify cell-level features. MAPS helps users leverage cloud-based deep learning services that are easy to train and deploy to fit their specific experimental conditions. Model training is code-free and can be done with limited training images. Thus, MAPS allows cell biologists to easily incorporate deep learning into their image analysis pipeline. We demonstrated an effective variant functionalization workflow that integrates HCM and MAPS to assess missense variants of PTEN, a tumor suppressor that is frequently mutated in hereditary and somatic cancers. CONCLUSIONS This paper presents a new way to rapidly assess variant function using cloud deep learning. Since most tumor suppressors have well-defined subcellular localizations, our approach could be widely applied to functionalize variants of uncertain significance and help improve the utility of genetic testing.
Collapse
Affiliation(s)
- Jesse T Chao
- Department of Cellular and Physiological Sciences, Life Sciences Institute, University of British Columbia, Vancouver, V6T1Z3, Canada.
| | - Calvin D Roskelley
- Department of Cellular and Physiological Sciences, Life Sciences Institute, University of British Columbia, Vancouver, V6T1Z3, Canada
| | - Christopher J R Loewen
- Department of Cellular and Physiological Sciences, Life Sciences Institute, University of British Columbia, Vancouver, V6T1Z3, Canada
| |
Collapse
|
22
|
Talikka M, Belcastro V, Boué S, Marescotti D, Hoeng J, Peitsch MC. Applying Systems Toxicology Methods to Drug Safety. SYSTEMS MEDICINE 2021. [DOI: 10.1016/b978-0-12-801238-3.11522-3] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/26/2022] Open
|
23
|
Berryman S, Matthews K, Lee JH, Duffy SP, Ma H. Image-based phenotyping of disaggregated cells using deep learning. Commun Biol 2020; 3:674. [PMID: 33188302 PMCID: PMC7666170 DOI: 10.1038/s42003-020-01399-x] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/08/2019] [Accepted: 10/20/2020] [Indexed: 12/14/2022] Open
Abstract
The ability to phenotype cells is fundamentally important in biological research and medicine. Current methods rely primarily on fluorescence labeling of specific markers. However, there are many situations where this approach is unavailable or undesirable. Machine learning has been used for image cytometry but has been limited by cell agglomeration and it is currently unclear if this approach can reliably phenotype cells that are difficult to distinguish by the human eye. Here, we show disaggregated single cells can be phenotyped with a high degree of accuracy using low-resolution bright-field and non-specific fluorescence images of the nucleus, cytoplasm, and cytoskeleton. Specifically, we trained a convolutional neural network using automatically segmented images of cells from eight standard cancer cell-lines. These cells could be identified with an average F1-score of 95.3%, tested using separately acquired images. Our results demonstrate the potential to develop an "electronic eye" to phenotype cells directly from microscopy images.
Collapse
Affiliation(s)
- Samuel Berryman
- Department of Mechanical Engineering, University of British Columbia, Vancouver, BC, Canada
- Centre for Blood Research, University of British Columbia, Vancouver, BC, Canada
| | - Kerryn Matthews
- Department of Mechanical Engineering, University of British Columbia, Vancouver, BC, Canada
- Centre for Blood Research, University of British Columbia, Vancouver, BC, Canada
| | - Jeong Hyun Lee
- Department of Mechanical Engineering, University of British Columbia, Vancouver, BC, Canada
- Centre for Blood Research, University of British Columbia, Vancouver, BC, Canada
| | - Simon P Duffy
- Department of Mechanical Engineering, University of British Columbia, Vancouver, BC, Canada
- Centre for Blood Research, University of British Columbia, Vancouver, BC, Canada
- British Columbia Institute of Technology, Burnaby, BC, Canada
| | - Hongshen Ma
- Department of Mechanical Engineering, University of British Columbia, Vancouver, BC, Canada.
- Centre for Blood Research, University of British Columbia, Vancouver, BC, Canada.
- School of Biomedical Engineering, University of British Columbia, Vancouver, BC, Canada.
- Vancouver Prostate Centre, Vancouver General Hospital, Vancouver, BC, Canada.
| |
Collapse
|
24
|
Steigele S, Siegismund D, Fassler M, Kustec M, Kappler B, Hasaka T, Yee A, Brodte A, Heyse S. Deep Learning-Based HCS Image Analysis for the Enterprise. SLAS DISCOVERY 2020; 25:812-821. [PMID: 32432952 PMCID: PMC7372584 DOI: 10.1177/2472555220918837] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Drug discovery programs are moving increasingly toward phenotypic imaging assays to model disease-relevant pathways and phenotypes in vitro. These assays offer richer information than target-optimized assays by investigating multiple cellular pathways simultaneously and producing multiplexed readouts. However, extracting the desired information from complex image data poses significant challenges, preventing broad adoption of more sophisticated phenotypic assays. Deep learning-based image analysis can address these challenges by reducing the effort required to analyze large volumes of complex image data at a quality and speed adequate for routine phenotypic screening in pharmaceutical research. However, while general purpose deep learning frameworks are readily available, they are not readily applicable to images from automated microscopy. During the past 3 years, we have optimized deep learning networks for this type of data and validated the approach across diverse assays with several industry partners. From this work, we have extracted five essential design principles that we believe should guide deep learning-based analysis of high-content images and multiparameter data: (1) insightful data representation, (2) automation of training, (3) multilevel quality control, (4) knowledge embedding and transfer to new assays, and (5) enterprise integration. We report a new deep learning-based software that embodies these principles, Genedata Imagence, which allows screening scientists to reliably detect stable endpoints for primary drug response, assess toxicity and safety-relevant effects, and discover new phenotypes and compound classes. Furthermore, we show how the software retains expert knowledge from its training on a particular assay and successfully reapplies it to different, novel assays in an automated fashion.
Collapse
Affiliation(s)
| | | | | | | | | | | | - Ada Yee
- Genedata AG, Basel, Switzerland
| | | | | |
Collapse
|
25
|
Nagao Y, Sakamoto M, Chinen T, Okada Y, Takao D. Robust classification of cell cycle phase and biological feature extraction by image-based deep learning. Mol Biol Cell 2020; 31:1346-1354. [PMID: 32320349 PMCID: PMC7353138 DOI: 10.1091/mbc.e20-03-0187] [Citation(s) in RCA: 17] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022] Open
Abstract
Across the cell cycle, the subcellular organization undergoes major spatiotemporal changes that could in principle contain biological features that could potentially represent cell cycle phase. We applied convolutional neural network-based classifiers to extract such putative features from the fluorescence microscope images of cells stained for the nucleus, the Golgi apparatus, and the microtubule cytoskeleton. We demonstrate that cell images can be robustly classified according to G1/S and G2 cell cycle phases without the need for specific cell cycle markers. Grad-CAM analysis of the classification models enabled us to extract several pairs of quantitative parameters of specific subcellular features as good classifiers for the cell cycle phase. These results collectively demonstrate that machine learning-based image processing is useful to extract biological features underlying cellular phenomena of interest in an unbiased and data-driven manner.
Collapse
Affiliation(s)
- Yukiko Nagao
- Faculty of Pharmaceutical Sciences, The University of Tokyo, Tokyo 113-0033, Japan
| | - Mika Sakamoto
- Genome Informatics Laboratory, National Institute of Genetics, Mishima 411-8540, Japan
| | - Takumi Chinen
- Faculty of Pharmaceutical Sciences, The University of Tokyo, Tokyo 113-0033, Japan
| | - Yasushi Okada
- Department of Cell Biology and Anatomy and International Research Center for Neurointelligence (WPI-IRCN), Graduate School of Medicine, The University of Tokyo, Tokyo 113-0033, Japan.,Department of Physics and Universal Biology Institute (UBI), Graduate School of Science, The University of Tokyo, Tokyo 113-0033, Japan.,Laboratory for Cell Polarity Regulation, Center for Biosystems Dynamics Research (BDR), RIKEN, Osaka 565-0874, Japan
| | - Daisuke Takao
- Department of Cell Biology and Anatomy and International Research Center for Neurointelligence (WPI-IRCN), Graduate School of Medicine, The University of Tokyo, Tokyo 113-0033, Japan
| |
Collapse
|
26
|
Wang ZJ, Walsh AJ, Skala MC, Gitter A. Classifying T cell activity in autofluorescence intensity images with convolutional neural networks. JOURNAL OF BIOPHOTONICS 2020; 13:e201960050. [PMID: 31661592 PMCID: PMC7065628 DOI: 10.1002/jbio.201960050] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/19/2019] [Revised: 09/28/2019] [Accepted: 10/16/2019] [Indexed: 05/13/2023]
Abstract
The importance of T cells in immunotherapy has motivated developing technologies to improve therapeutic efficacy. One objective is assessing antigen-induced T cell activation because only functionally active T cells are capable of killing the desired targets. Autofluorescence imaging can distinguish T cell activity states in a non-destructive manner by detecting endogenous changes in metabolic co-enzymes such as NAD(P)H. However, recognizing robust activity patterns is computationally challenging in the absence of exogenous labels. We demonstrate machine learning methods that can accurately classify T cell activity across human donors from NAD(P)H intensity images. Using 8260 cropped single-cell images from six donors, we evaluate classifiers ranging from traditional models that use previously-extracted image features to convolutional neural networks (CNNs) pre-trained on general non-biological images. Adapting pre-trained CNNs for the T cell activity classification task provides substantially better performance than traditional models or a simple CNN trained with the autofluorescence images alone. Visualizing the images with dimension reduction provides intuition into why the CNNs achieve higher accuracy than other approaches. Our image processing and classifier training software is available at https://github.com/gitter-lab/t-cell-classification.
Collapse
Affiliation(s)
- Zijie J. Wang
- Department of Computer SciencesUniversity of Wisconsin‐MadisonMadisonWisconsin
- Morgridge Institute for ResearchMadisonWisconsin
| | | | - Melissa C. Skala
- Morgridge Institute for ResearchMadisonWisconsin
- Department of Biomedical EngineeringUniversity of Wisconsin‐MadisonMadisonWisconsin
| | - Anthony Gitter
- Department of Computer SciencesUniversity of Wisconsin‐MadisonMadisonWisconsin
- Morgridge Institute for ResearchMadisonWisconsin
- Department of Biostatistics and Medical InformaticsUniversity of Wisconsin‐MadisonMadisonWisconsin
| |
Collapse
|
27
|
Huss R, Coupland SE. Software‐assisted decision support in digital histopathology. J Pathol 2020; 250:685-692. [DOI: 10.1002/path.5388] [Citation(s) in RCA: 26] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/02/2019] [Revised: 01/09/2020] [Accepted: 01/17/2020] [Indexed: 12/16/2022]
Affiliation(s)
- Ralf Huss
- Institute of Pathology and Molecular Diagnostics University Hospital Augsburg Augsburg Germany
| | - Sarah E Coupland
- Department of Cellular and Molecular Pathology University of Liverpool Liverpool UK
| |
Collapse
|
28
|
D’Acunto M, Martinelli M, Moroni D. From human mesenchymal stromal cells to osteosarcoma cells classification by deep learning. JOURNAL OF INTELLIGENT & FUZZY SYSTEMS 2019. [DOI: 10.3233/jifs-179332] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/08/2023]
Affiliation(s)
- Mario D’Acunto
- Institute of Biophysics, National Research Council of Italy, Via Moruzzi, 1 – 56124-Pisa (IT)
| | - Massimo Martinelli
- Institute of Information Science and Technologies, National Research Council of Italy, Via Moruzzi, 1 – 56124-Pisa (IT)
| | - Davide Moroni
- Institute of Information Science and Technologies, National Research Council of Italy, Via Moruzzi, 1 – 56124-Pisa (IT)
| |
Collapse
|
29
|
Deep Learning Model Comparison for Vision-Based Classification of Full/Empty-Load Trucks in Earthmoving Operations. APPLIED SCIENCES-BASEL 2019. [DOI: 10.3390/app9224871] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Earthmoving is an integral civil engineering operation of significance, and tracking its productivity requires the statistics of loads moved by dump trucks. Since current truck loads’ statistics methods are laborious, costly, and limited in application, this paper presents the framework of a novel, automated, non-contact field earthmoving quantity statistics (FEQS) for projects with large earthmoving demands that use uniform and uncovered trucks. The proposed FEQS framework utilizes field surveillance systems and adopts vision-based deep learning for full/empty-load truck classification as the core work. Since convolutional neural network (CNN) and its transfer learning (TL) forms are popular vision-based deep learning models and numerous in type, a comparison study is conducted to test the framework’s core work feasibility and evaluate the performance of different deep learning models in implementation. The comparison study involved 12 CNN or CNN-TL models in full/empty-load truck classification, and the results revealed that while several provided satisfactory performance, the VGG16-FineTune provided the optimal performance. This proved the core work feasibility of the proposed FEQS framework. Further discussion provides model choice suggestions that CNN-TL models are more feasible than CNN prototypes, and models that adopt different TL methods have advantages in either working accuracy or speed for different tasks.
Collapse
|
30
|
David L, Arús-Pous J, Karlsson J, Engkvist O, Bjerrum EJ, Kogej T, Kriegl JM, Beck B, Chen H. Applications of Deep-Learning in Exploiting Large-Scale and Heterogeneous Compound Data in Industrial Pharmaceutical Research. Front Pharmacol 2019; 10:1303. [PMID: 31749705 PMCID: PMC6848277 DOI: 10.3389/fphar.2019.01303] [Citation(s) in RCA: 26] [Impact Index Per Article: 5.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2019] [Accepted: 10/14/2019] [Indexed: 12/21/2022] Open
Abstract
In recent years, the development of high-throughput screening (HTS) technologies and their establishment in an industrialized environment have given scientists the possibility to test millions of molecules and profile them against a multitude of biological targets in a short period of time, generating data in a much faster pace and with a higher quality than before. Besides the structure activity data from traditional bioassays, more complex assays such as transcriptomics profiling or imaging have also been established as routine profiling experiments thanks to the advancement of Next Generation Sequencing or automated microscopy technologies. In industrial pharmaceutical research, these technologies are typically established in conjunction with automated platforms in order to enable efficient handling of screening collections of thousands to millions of compounds. To exploit the ever-growing amount of data that are generated by these approaches, computational techniques are constantly evolving. In this regard, artificial intelligence technologies such as deep learning and machine learning methods play a key role in cheminformatics and bio-image analytics fields to address activity prediction, scaffold hopping, de novo molecule design, reaction/retrosynthesis predictions, or high content screening analysis. Herein we summarize the current state of analyzing large-scale compound data in industrial pharmaceutical research and describe the impact it has had on the drug discovery process over the last two decades, with a specific focus on deep-learning technologies.
Collapse
Affiliation(s)
- Laurianne David
- Hit Discovery, Discovery Sciences, Biopharmaceutical R&D, AstraZeneca, Gothenburg, Sweden
- Department of Life Science Informatics, B-IT, Rheinische Friedrich-Wilhelms-Universität Bonn, Bonn, Germany
| | - Josep Arús-Pous
- Hit Discovery, Discovery Sciences, Biopharmaceutical R&D, AstraZeneca, Gothenburg, Sweden
- Department of Chemistry and Biochemistry, University of Bern, Bern, Switzerland
| | - Johan Karlsson
- Quantitative Biology, Discovery Sciences, Biopharmaceutical R&D, AstraZeneca, Gothenburg, Sweden
| | - Ola Engkvist
- Hit Discovery, Discovery Sciences, Biopharmaceutical R&D, AstraZeneca, Gothenburg, Sweden
| | - Esben Jannik Bjerrum
- Hit Discovery, Discovery Sciences, Biopharmaceutical R&D, AstraZeneca, Gothenburg, Sweden
| | - Thierry Kogej
- Hit Discovery, Discovery Sciences, Biopharmaceutical R&D, AstraZeneca, Gothenburg, Sweden
| | - Jan M. Kriegl
- Department of Medicinal Chemistry, Boehringer Ingelheim Pharma GmbH & Co. KG, Biberach an der Riss, Germany
| | - Bernd Beck
- Department of Medicinal Chemistry, Boehringer Ingelheim Pharma GmbH & Co. KG, Biberach an der Riss, Germany
| | - Hongming Chen
- Hit Discovery, Discovery Sciences, Biopharmaceutical R&D, AstraZeneca, Gothenburg, Sweden
- Chemistry and Chemical Biology Centre, Guangzhou Regenerative Medicine and Health – Guangdong Laboratory, Guangzhou, China
| |
Collapse
|
31
|
Lao Q, Fevens T. Cell Phenotype Classification Using Deep Residual Network and Its Variants. INT J PATTERN RECOGN 2019. [DOI: 10.1142/s0218001419400172] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
Deep residual network (ResNet) is currently the basis of many popular state-of-the-art convolutional neural network models for image recognition, and its recent variants include wide residual network (WRN), aggregated deep residual network (ResNeXt) and deep pyramidal residual network (PyramidNet). Here, we demonstrate the potential application of deep residual network and its variants in high-content screening (i.e. cell phenotype classification) that can overcome issues associated with analyzing high-content screening data, such as exhaustive preprocessing and inefficient learning. Cell phenotype classification is an image-based method that can be used for drug high-content screening, in which complex cell states associated with chemical compound treatment can be characterized. Previous work on cell phenotype classification typically requires a routine yet cumbersome step of single cell segmentation before the classification task. In this paper, we present a segmentation-free method for image-based cell phenotype classification using deep ResNet and its variants. The cell images are samples treated with annotated compounds that can be mainly grouped into three clusters, giving three classes to be classified. Instead of single-cell phenotype classification, we use the raw images without segmentation for our training and evaluation directly. Compared to previous reference work, we significantly simplify the data preprocessing step and accelerate the training while still achieving high accuracy. Our trained models achieve a 98.2% accuracy rate on the three classes classification problem (three compound clusters only), and a 93.8% accuracy rate on the four classes classification problem (three compound clusters plus the mock class) based on five-fold cross-validation.
Collapse
Affiliation(s)
- Qicheng Lao
- Department of Computer Science and Software Engineering, Concordia University, Montréal, Québec, Canada
| | - Thomas Fevens
- Department of Computer Science and Software Engineering, Concordia University, Montréal, Québec, Canada
| |
Collapse
|
32
|
Deep learning in drug discovery: opportunities, challenges and future prospects. Drug Discov Today 2019; 24:2017-2032. [DOI: 10.1016/j.drudis.2019.07.006] [Citation(s) in RCA: 104] [Impact Index Per Article: 20.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2019] [Revised: 06/11/2019] [Accepted: 07/18/2019] [Indexed: 12/27/2022]
|
33
|
Smith K, Piccinini F, Balassa T, Koos K, Danka T, Azizpour H, Horvath P. Phenotypic Image Analysis Software Tools for Exploring and Understanding Big Image Data from Cell-Based Assays. Cell Syst 2019; 6:636-653. [PMID: 29953863 DOI: 10.1016/j.cels.2018.06.001] [Citation(s) in RCA: 54] [Impact Index Per Article: 10.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/13/2017] [Revised: 03/07/2018] [Accepted: 06/01/2018] [Indexed: 01/01/2023]
Abstract
Phenotypic image analysis is the task of recognizing variations in cell properties using microscopic image data. These variations, produced through a complex web of interactions between genes and the environment, may hold the key to uncover important biological phenomena or to understand the response to a drug candidate. Today, phenotypic analysis is rarely performed completely by hand. The abundance of high-dimensional image data produced by modern high-throughput microscopes necessitates computational solutions. Over the past decade, a number of software tools have been developed to address this need. They use statistical learning methods to infer relationships between a cell's phenotype and data from the image. In this review, we examine the strengths and weaknesses of non-commercial phenotypic image analysis software, cover recent developments in the field, identify challenges, and give a perspective on future possibilities.
Collapse
Affiliation(s)
- Kevin Smith
- KTH Royal Institute of Technology, School of Electrical Engineering and Computer Science, Lindstedtsvägen 3, 10044 Stockholm, Sweden; Science for Life Laboratory, Tomtebodavägen 23A, 17165 Solna, Sweden
| | - Filippo Piccinini
- Istituto Scientifico Romagnolo per lo Studio e la Cura dei Tumori (IRST) IRCCS, Via P. Maroncelli 40, Meldola, FC 47014, Italy
| | - Tamas Balassa
- Synthetic and Systems Biology Unit, Hungarian Academy of Sciences, Biological Research Center (BRC), Temesvári krt. 62, 6726 Szeged, Hungary
| | - Krisztian Koos
- Synthetic and Systems Biology Unit, Hungarian Academy of Sciences, Biological Research Center (BRC), Temesvári krt. 62, 6726 Szeged, Hungary
| | - Tivadar Danka
- Synthetic and Systems Biology Unit, Hungarian Academy of Sciences, Biological Research Center (BRC), Temesvári krt. 62, 6726 Szeged, Hungary
| | - Hossein Azizpour
- KTH Royal Institute of Technology, School of Electrical Engineering and Computer Science, Lindstedtsvägen 3, 10044 Stockholm, Sweden; Science for Life Laboratory, Tomtebodavägen 23A, 17165 Solna, Sweden
| | - Peter Horvath
- Synthetic and Systems Biology Unit, Hungarian Academy of Sciences, Biological Research Center (BRC), Temesvári krt. 62, 6726 Szeged, Hungary; Institute for Molecular Medicine Finland, University of Helsinki, Tukholmankatu 8, 00014 Helsinki, Finland.
| |
Collapse
|
34
|
Nishimoto S, Tokuoka Y, Yamada TG, Hiroi NF, Funahashi A. Predicting the future direction of cell movement with convolutional neural networks. PLoS One 2019; 14:e0221245. [PMID: 31483827 PMCID: PMC6726366 DOI: 10.1371/journal.pone.0221245] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/22/2018] [Accepted: 08/02/2019] [Indexed: 12/16/2022] Open
Abstract
Image-based deep learning systems, such as convolutional neural networks (CNNs), have recently been applied to cell classification, producing impressive results; however, application of CNNs has been confined to classification of the current cell state from the image. Here, we focused on cell movement where current and/or past cell shape can influence the future cell movement. We demonstrate that CNNs prospectively predicted the future direction of cell movement with high accuracy from a single image patch of a cell at a certain time. Furthermore, by visualizing the image features that were learned by the CNNs, we could identify morphological features, e.g., the protrusions and trailing edge that have been experimentally reported to determine the direction of cell movement. Our results indicate that CNNs have the potential to predict the future direction of cell movement from current cell shape, and can be used to automatically identify those morphological features that influence future cell movement.
Collapse
Affiliation(s)
- Shori Nishimoto
- Department of Biosciences and Informatics, Keio University, Yokohama-shi, Kanagawa, Japan
| | - Yuta Tokuoka
- Department of Biosciences and Informatics, Keio University, Yokohama-shi, Kanagawa, Japan
| | - Takahiro G. Yamada
- Department of Biosciences and Informatics, Keio University, Yokohama-shi, Kanagawa, Japan
| | - Noriko F. Hiroi
- Department of Biosciences and Informatics, Keio University, Yokohama-shi, Kanagawa, Japan
- Faculty of Pharmacy, Sanyo-Onoda City University, Sanyo-Onoda, Yamaguchi, Japan
| | - Akira Funahashi
- Department of Biosciences and Informatics, Keio University, Yokohama-shi, Kanagawa, Japan
- * E-mail:
| |
Collapse
|
35
|
Lu AX, Kraus OZ, Cooper S, Moses AM. Learning unsupervised feature representations for single cell microscopy images with paired cell inpainting. PLoS Comput Biol 2019; 15:e1007348. [PMID: 31479439 PMCID: PMC6743779 DOI: 10.1371/journal.pcbi.1007348] [Citation(s) in RCA: 45] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/15/2019] [Revised: 09/13/2019] [Accepted: 08/20/2019] [Indexed: 12/03/2022] Open
Abstract
Cellular microscopy images contain rich insights about biology. To extract this information, researchers use features, or measurements of the patterns of interest in the images. Here, we introduce a convolutional neural network (CNN) to automatically design features for fluorescence microscopy. We use a self-supervised method to learn feature representations of single cells in microscopy images without labelled training data. We train CNNs on a simple task that leverages the inherent structure of microscopy images and controls for variation in cell morphology and imaging: given one cell from an image, the CNN is asked to predict the fluorescence pattern in a second different cell from the same image. We show that our method learns high-quality features that describe protein expression patterns in single cells both yeast and human microscopy datasets. Moreover, we demonstrate that our features are useful for exploratory biological analysis, by capturing high-resolution cellular components in a proteome-wide cluster analysis of human proteins, and by quantifying multi-localized proteins and single-cell variability. We believe paired cell inpainting is a generalizable method to obtain feature representations of single cells in multichannel microscopy images. To understand the cell biology captured by microscopy images, researchers use features, or measurements of relevant properties of cells, such as the shape or size of cells, or the intensity of fluorescent markers. Features are the starting point of most image analysis pipelines, so their quality in representing cells is fundamental to the success of an analysis. Classically, researchers have relied on features manually defined by imaging experts. In contrast, deep learning techniques based on convolutional neural networks (CNNs) automatically learn features, which can outperform manually-defined features at image analysis tasks. However, most CNN methods require large manually-annotated training datasets to learn useful features, limiting their practical application. Here, we developed a new CNN method that learns high-quality features for single cells in microscopy images, without the need for any labeled training data. We show that our features surpass other comparable features in identifying protein localization from images, and that our method can generalize to diverse datasets. By exploiting our method, researchers will be able to automatically obtain high-quality features customized to their own image datasets, facilitating many downstream analyses, as we highlight by demonstrating many possible use cases of our features in this study.
Collapse
Affiliation(s)
- Alex X. Lu
- Department of Computer Science, University of Toronto, Toronto, Canada
| | | | | | - Alan M. Moses
- Department of Computer Science, University of Toronto, Toronto, Canada
- Department of Cell and Systems Biology, University of Toronto, Toronto, Canada
- Center for Analysis of Genome Evolution and Function, University of Toronto, Toronto, Canada
- * E-mail:
| |
Collapse
|
36
|
Artificial intelligence for microscopy: what you should know. Biochem Soc Trans 2019; 47:1029-1040. [PMID: 31366471 DOI: 10.1042/bst20180391] [Citation(s) in RCA: 51] [Impact Index Per Article: 10.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/19/2019] [Revised: 07/09/2019] [Accepted: 07/11/2019] [Indexed: 02/06/2023]
Abstract
Artificial Intelligence based on Deep Learning (DL) is opening new horizons in biomedical research and promises to revolutionize the microscopy field. It is now transitioning from the hands of experts in computer sciences to biomedical researchers. Here, we introduce recent developments in DL applied to microscopy, in a manner accessible to non-experts. We give an overview of its concepts, capabilities and limitations, presenting applications in image segmentation, classification and restoration. We discuss how DL shows an outstanding potential to push the limits of microscopy, enhancing resolution, signal and information content in acquired data. Its pitfalls are discussed, along with the future directions expected in this field.
Collapse
|
37
|
Chessel A, Carazo Salas RE. From observing to predicting single-cell structure and function with high-throughput/high-content microscopy. Essays Biochem 2019; 63:197-208. [PMID: 31243141 PMCID: PMC6610450 DOI: 10.1042/ebc20180044] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/11/2019] [Revised: 05/24/2019] [Accepted: 05/24/2019] [Indexed: 02/08/2023]
Abstract
In the past 15 years, cell-based microscopy has evolved its focus from observing cell function to aiming to predict it. In particular-powered by breakthroughs in computer vision, large-scale image analysis and machine learning-high-throughput and high-content microscopy imaging have enabled to uniquely harness single-cell information to systematically discover and annotate genes and regulatory pathways, uncover systems-level interactions and causal links between cellular processes, and begin to clarify and predict causal cellular behaviour and decision making. Here we review these developments, discuss emerging trends in the field, and describe how single-cell 'omics and single-cell microscopy are imminently in an intersecting trajectory. The marriage of these two fields will make possible an unprecedented understanding of cell and tissue behaviour and function.
Collapse
Affiliation(s)
- Anatole Chessel
- École polytechnique, Université Paris-Saclay, 91128 Palaiseau Cedex, France
| | | |
Collapse
|
38
|
Gupta A, Harrison PJ, Wieslander H, Pielawski N, Kartasalo K, Partel G, Solorzano L, Suveer A, Klemm AH, Spjuth O, Sintorn I, Wählby C. Deep Learning in Image Cytometry: A Review. Cytometry A 2019; 95:366-380. [PMID: 30565841 PMCID: PMC6590257 DOI: 10.1002/cyto.a.23701] [Citation(s) in RCA: 86] [Impact Index Per Article: 17.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2018] [Revised: 11/07/2018] [Accepted: 11/29/2018] [Indexed: 12/18/2022]
Abstract
Artificial intelligence, deep convolutional neural networks, and deep learning are all niche terms that are increasingly appearing in scientific presentations as well as in the general media. In this review, we focus on deep learning and how it is applied to microscopy image data of cells and tissue samples. Starting with an analogy to neuroscience, we aim to give the reader an overview of the key concepts of neural networks, and an understanding of how deep learning differs from more classical approaches for extracting information from image data. We aim to increase the understanding of these methods, while highlighting considerations regarding input data requirements, computational resources, challenges, and limitations. We do not provide a full manual for applying these methods to your own data, but rather review previously published articles on deep learning in image cytometry, and guide the readers toward further reading on specific networks and methods, including new methods not yet applied to cytometry data. © 2018 The Authors. Cytometry Part A published by Wiley Periodicals, Inc. on behalf of International Society for Advancement of Cytometry.
Collapse
Affiliation(s)
- Anindya Gupta
- Centre for Image AnalysisUppsala UniversityUppsala75124Sweden
| | - Philip J. Harrison
- Department of Pharmaceutical BiosciencesUppsala UniversityUppsala75124Sweden
| | | | | | - Kimmo Kartasalo
- Faculty of Medicine and Life SciencesUniversity of TampereTampere33014Finland
- Faculty of Biomedical Sciences and EngineeringTampere University of TechnologyTampere33720Finland
| | - Gabriele Partel
- Centre for Image AnalysisUppsala UniversityUppsala75124Sweden
| | | | - Amit Suveer
- Centre for Image AnalysisUppsala UniversityUppsala75124Sweden
| | - Anna H. Klemm
- Centre for Image AnalysisUppsala UniversityUppsala75124Sweden
- BioImage Informatics Facility of SciLifeLabUppsala75124Sweden
| | - Ola Spjuth
- Department of Pharmaceutical BiosciencesUppsala UniversityUppsala75124Sweden
| | | | - Carolina Wählby
- Centre for Image AnalysisUppsala UniversityUppsala75124Sweden
- BioImage Informatics Facility of SciLifeLabUppsala75124Sweden
| |
Collapse
|
39
|
Matuszewski DJ, Wählby C, Krona C, Nelander S, Sintorn IM. Image-Based Detection of Patient-Specific Drug-Induced Cell-Cycle Effects in Glioblastoma. SLAS DISCOVERY 2018; 23:1030-1039. [PMID: 30074852 DOI: 10.1177/2472555218791414] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Image-based analysis is an increasingly important tool to characterize the effect of drugs in large-scale chemical screens. Herein, we present image and data analysis methods to investigate population cell-cycle dynamics in patient-derived brain tumor cells. Images of glioblastoma cells grown in multiwell plates were used to extract per-cell descriptors, including nuclear DNA content. We reduced the DNA content data from per-cell descriptors to per-well frequency distributions, which were used to identify compounds affecting cell-cycle phase distribution. We analyzed cells from 15 patient cases representing multiple subtypes of glioblastoma and searched for clusters of cell-cycle phase distributions characterizing similarities in response to 249 compounds at 11 doses. We show that this approach applied in a blind analysis with unlabeled substances identified drugs that are commonly used for treating solid tumors as well as other compounds that are well known for inducing cell-cycle arrest. Redistribution of nuclear DNA content signals is thus a robust metric of cell-cycle arrest in patient-derived glioblastoma cells.
Collapse
Affiliation(s)
- Damian J Matuszewski
- 1 Science for Life Laboratory, Uppsala University, Uppsala, Sweden.,2 Centre for Image Analysis, Uppsala University, Uppsala, Sweden
| | - Carolina Wählby
- 1 Science for Life Laboratory, Uppsala University, Uppsala, Sweden.,2 Centre for Image Analysis, Uppsala University, Uppsala, Sweden
| | - Cecilia Krona
- 3 Department of Immunology, Genetics and Pathology, Uppsala University, Sweden
| | - Sven Nelander
- 1 Science for Life Laboratory, Uppsala University, Uppsala, Sweden.,3 Department of Immunology, Genetics and Pathology, Uppsala University, Sweden
| | | |
Collapse
|
40
|
Dürr O, Murina E, Siegismund D, Tolkachev V, Steigele S, Sick B. Know When You Don't Know: A Robust Deep Learning Approach in the Presence of Unknown Phenotypes. Assay Drug Dev Technol 2018; 16:343-349. [DOI: 10.1089/adt.2018.859] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/20/2022] Open
Affiliation(s)
- Oliver Dürr
- Institute of Data Analysis and Process Design, ZHAW Winterthur, Winterthur, Switzerland
- Institute for Optical Systems, HTWG Konstanz, Konstanz, Germany
| | - Elvis Murina
- Institute of Data Analysis and Process Design, ZHAW Winterthur, Winterthur, Switzerland
| | | | - Vasily Tolkachev
- Institute of Data Analysis and Process Design, ZHAW Winterthur, Winterthur, Switzerland
| | | | - Beate Sick
- Institute of Data Analysis and Process Design, ZHAW Winterthur, Winterthur, Switzerland
- EBPI, University of Zurich, Zurich, Switzerland
| |
Collapse
|
41
|
Scheeder C, Heigwer F, Boutros M. Machine learning and image-based profiling in drug discovery. CURRENT OPINION IN SYSTEMS BIOLOGY 2018; 10:43-52. [PMID: 30159406 PMCID: PMC6109111 DOI: 10.1016/j.coisb.2018.05.004] [Citation(s) in RCA: 67] [Impact Index Per Article: 11.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Indexed: 12/21/2022]
Abstract
The increase in imaging throughput, new analytical frameworks and high-performance computational resources open new avenues for data-rich phenotypic profiling of small molecules in drug discovery. Image-based profiling assays assessing single-cell phenotypes have been used to explore mechanisms of action, target efficacy and toxicity of small molecules. Technological advances to generate large data sets together with new machine learning approaches for the analysis of high-dimensional profiling data create opportunities to improve many steps in drug discovery. In this review, we will discuss how recent studies applied machine learning approaches in functional profiling workflows with a focus on chemical genetics. While their utility in image-based screening and profiling is predictably evident, examples of novel insights beyond the status quo based on the applications of machine learning approaches are just beginning to emerge. To enable discoveries, future studies also need to develop methodologies that lower the entry barriers to high-throughput profiling experiments by streamlining image-based profiling assays and providing applications for advanced learning technologies such as easy to deploy deep neural networks.
Collapse
Affiliation(s)
| | | | - Michael Boutros
- German Cancer Research Center (DKFZ), Division Signaling and Functional Genomics and Heidelberg University, Department of Cell and Molecular Biology, Medical Faculty Mannheim, D-69120 Heidelberg, Germany
| |
Collapse
|
42
|
Construction of a system using a deep learning algorithm to count cell numbers in nanoliter wells for viable single-cell experiments. Sci Rep 2017; 7:16831. [PMID: 29203784 PMCID: PMC5715092 DOI: 10.1038/s41598-017-17012-x] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/04/2017] [Accepted: 11/20/2017] [Indexed: 12/22/2022] Open
Abstract
For single-cell experiments, it is important to accurately count the number of viable cells in a nanoliter well. We used a deep learning-based convolutional neural network (CNN) on a large amount of digital data obtained as microscopic images. The training set consisted of 103 019 samples, each representing a microscopic grayscale image. After extensive training, the CNN was able to classify the samples into four categories, i.e., 0, 1, 2, and more than 2 cells per well, with an accuracy of 98.3% when compared to determination by two trained technicians. By analyzing the samples for which judgments were discordant, we found that the judgment by technicians was relatively correct although cell counting was often difficult by the images of discordant samples. Based on the results, the system was further enhanced by introducing a new algorithm in which the highest outputs from CNN were used, increasing the accuracy to higher than 99%. Our system was able to classify the data even from wells with a different shape. No other tested machine learning algorithm showed a performance higher than that of our system. The presented CNN system is expected to be useful for various single-cell experiments, and for high-throughput and high-content screening.
Collapse
|
43
|
Sommer C, Hoefler R, Samwer M, Gerlich DW. A deep learning and novelty detection framework for rapid phenotyping in high-content screening. Mol Biol Cell 2017; 28:3428-3436. [PMID: 28954863 PMCID: PMC5687041 DOI: 10.1091/mbc.e17-05-0333] [Citation(s) in RCA: 61] [Impact Index Per Article: 8.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2017] [Revised: 08/31/2017] [Accepted: 09/18/2017] [Indexed: 11/16/2022] Open
Abstract
Supervised machine learning is a powerful and widely used method for analyzing high-content screening data. Despite its accuracy, efficiency, and versatility, supervised machine learning has drawbacks, most notably its dependence on a priori knowledge of expected phenotypes and time-consuming classifier training. We provide a solution to these limitations with CellCognition Explorer, a generic novelty detection and deep learning framework. Application to several large-scale screening data sets on nuclear and mitotic cell morphologies demonstrates that CellCognition Explorer enables discovery of rare phenotypes without user training, which has broad implications for improved assay development in high-content screening.
Collapse
Affiliation(s)
- Christoph Sommer
- Institute of Molecular Biotechnology of the Austrian Academy of Sciences (IMBA), Vienna Biocenter (VBC), 1030 Vienna, Austria
| | - Rudolf Hoefler
- Institute of Molecular Biotechnology of the Austrian Academy of Sciences (IMBA), Vienna Biocenter (VBC), 1030 Vienna, Austria
| | - Matthias Samwer
- Institute of Molecular Biotechnology of the Austrian Academy of Sciences (IMBA), Vienna Biocenter (VBC), 1030 Vienna, Austria
| | - Daniel W Gerlich
- Institute of Molecular Biotechnology of the Austrian Academy of Sciences (IMBA), Vienna Biocenter (VBC), 1030 Vienna, Austria
| |
Collapse
|
44
|
Reconstructing cell cycle and disease progression using deep learning. Nat Commun 2017; 8:463. [PMID: 28878212 PMCID: PMC5587733 DOI: 10.1038/s41467-017-00623-3] [Citation(s) in RCA: 142] [Impact Index Per Article: 20.3] [Reference Citation Analysis] [Abstract] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2016] [Accepted: 07/14/2017] [Indexed: 12/21/2022] Open
Abstract
We show that deep convolutional neural networks combined with nonlinear dimension reduction enable reconstructing biological processes based on raw image data. We demonstrate this by reconstructing the cell cycle of Jurkat cells and disease progression in diabetic retinopathy. In further analysis of Jurkat cells, we detect and separate a subpopulation of dead cells in an unsupervised manner and, in classifying discrete cell cycle stages, we reach a sixfold reduction in error rate compared to a recent approach based on boosting on image features. In contrast to previous methods, deep learning based predictions are fast enough for on-the-fly analysis in an imaging flow cytometer. The interpretation of information-rich, high-throughput single-cell data is a challenge requiring sophisticated computational tools. Here the authors demonstrate a deep convolutional neural network that can classify cell cycle status on-the-fly.
Collapse
|
45
|
Kraus OZ, Grys BT, Ba J, Chong Y, Frey BJ, Boone C, Andrews BJ. Automated analysis of high-content microscopy data with deep learning. Mol Syst Biol 2017; 13:924. [PMID: 28420678 PMCID: PMC5408780 DOI: 10.15252/msb.20177551] [Citation(s) in RCA: 142] [Impact Index Per Article: 20.3] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/31/2023] Open
Abstract
Existing computational pipelines for quantitative analysis of high‐content microscopy data rely on traditional machine learning approaches that fail to accurately classify more than a single dataset without substantial tuning and training, requiring extensive analysis. Here, we demonstrate that the application of deep learning to biological image data can overcome the pitfalls associated with conventional machine learning classifiers. Using a deep convolutional neural network (DeepLoc) to analyze yeast cell images, we show improved performance over traditional approaches in the automated classification of protein subcellular localization. We also demonstrate the ability of DeepLoc to classify highly divergent image sets, including images of pheromone‐arrested cells with abnormal cellular morphology, as well as images generated in different genetic backgrounds and in different laboratories. We offer an open‐source implementation that enables updating DeepLoc on new microscopy datasets. This study highlights deep learning as an important tool for the expedited analysis of high‐content microscopy data.
Collapse
Affiliation(s)
- Oren Z Kraus
- Department of Electrical and Computer Engineering, University of Toronto, Toronto, ON, Canada.,Donnelly Centre for Cellular and Biomolecular Research, University of Toronto, Toronto, ON, Canada
| | - Ben T Grys
- Donnelly Centre for Cellular and Biomolecular Research, University of Toronto, Toronto, ON, Canada.,Department of Molecular Genetics, University of Toronto, Toronto, ON, Canada
| | - Jimmy Ba
- Department of Electrical and Computer Engineering, University of Toronto, Toronto, ON, Canada
| | - Yolanda Chong
- Cellular Pharmacology, Discovery Sciences, Janssen Pharmaceutical Companies, Johnson & Johnson, Beerse, Belgium
| | - Brendan J Frey
- Department of Electrical and Computer Engineering, University of Toronto, Toronto, ON, Canada.,Donnelly Centre for Cellular and Biomolecular Research, University of Toronto, Toronto, ON, Canada.,Canadian Institute for Advanced Research, Program on Genetic Networks, Toronto, ON, Canada.,Canadian Institute for Advanced Research, Program on Learning in Machines & Brains, Toronto, ON, Canada
| | - Charles Boone
- Donnelly Centre for Cellular and Biomolecular Research, University of Toronto, Toronto, ON, Canada .,Department of Molecular Genetics, University of Toronto, Toronto, ON, Canada.,Canadian Institute for Advanced Research, Program on Genetic Networks, Toronto, ON, Canada
| | - Brenda J Andrews
- Donnelly Centre for Cellular and Biomolecular Research, University of Toronto, Toronto, ON, Canada .,Department of Molecular Genetics, University of Toronto, Toronto, ON, Canada.,Canadian Institute for Advanced Research, Program on Genetic Networks, Toronto, ON, Canada
| |
Collapse
|
46
|
An Overview of data science uses in bioimage informatics. Methods 2017; 115:110-118. [DOI: 10.1016/j.ymeth.2016.12.014] [Citation(s) in RCA: 22] [Impact Index Per Article: 3.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2016] [Revised: 12/09/2016] [Accepted: 12/30/2016] [Indexed: 01/17/2023] Open
|