1
|
Warchol S, Troidl J, Muhlich JL, Krueger R, Hoffer J, Lin T, Beyer J, Glassman E, Sorger PK, Pfister H. psudo: Exploring Multi-Channel Biomedical Image Data with Spatially and Perceptually Optimized Pseudocoloring. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.04.11.589087. [PMID: 38659870 PMCID: PMC11042212 DOI: 10.1101/2024.04.11.589087] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/26/2024]
Abstract
Over the past century, multichannel fluorescence imaging has been pivotal in myriad scientific breakthroughs by enabling the spatial visualization of proteins within a biological sample. With the shift to digital methods and visualization software, experts can now flexibly pseudocolor and combine image channels, each corresponding to a different protein, to explore their spatial relationships. We thus propose psudo, an interactive system that allows users to create optimal color palettes for multichannel spatial data. In psudo, a novel optimization method generates palettes that maximize the perceptual differences between channels while mitigating confusing color blending in overlapping channels. We integrate this method into a system that allows users to explore multi-channel image data and compare and evaluate color palettes for their data. An interactive lensing approach provides on-demand feedback on channel overlap and a color confusion metric while giving context to the underlying channel values. Color palettes can be applied globally or, using the lens, to local regions of interest. We evaluate our palette optimization approach using three graphical perception tasks in a crowdsourced user study with 150 participants, showing that users are more accurate at discerning and comparing the underlying data using our approach. Additionally, we showcase psudo in a case study exploring the complex immune responses in cancer tissue data with a biologist.
Collapse
|
2
|
Nambiar N, Rajesh V, Nair A, Nambiar S, Nair R, Uthamanthil R, Lotodo T, Mittal S, Kussick S. An AI based, open access screening tool for early diagnosis of Burkitt lymphoma. Front Med (Lausanne) 2024; 11:1345611. [PMID: 38903819 PMCID: PMC11187324 DOI: 10.3389/fmed.2024.1345611] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2023] [Accepted: 04/08/2024] [Indexed: 06/22/2024] Open
Abstract
Burkitt Lymphoma (BL) is a highly treatable cancer. However, delayed diagnosis of BL contributes to high mortality in BL endemic regions of Africa. Lack of enough pathologists in the region is a major reason for delayed diagnosis. The work described in this paper is a proof-of-concept study to develop a targeted, open access AI tool for screening of histopathology slides in suspected BL cases. Slides were obtained from a total of 90 BL patients. 70 Tonsillectomy samples were used as controls. We fine-tuned 6 pre-trained models and evaluated the performance of all 6 models across different configurations. An ensemble-based consensus approach ensured a balanced and robust classification. The tool applies novel features to BL diagnosis including use of multiple image magnifications, thus enabling use of different magnifications of images based on the microscope/scanner available in remote clinics, composite scoring of multiple models and utilizing MIL with weak labeling and image augmentation, enabling use of relatively low sample size to achieve good performance on the inference set. The open access model allows free access to the AI tool from anywhere with an internet connection. The ultimate aim of this work is making pathology services accessible, efficient and timely in remote clinics in regions where BL is endemic. New generation of low-cost slide scanners/microscopes is expected to make slide images available immediately for the AI tool for screening and thus accelerate diagnosis by pathologists available locally or online.
Collapse
Affiliation(s)
- Nikil Nambiar
- Burkitt’s Lymphoma Fund for Africa, Seattle, WA, United States
| | - Vineeth Rajesh
- Burkitt’s Lymphoma Fund for Africa, Seattle, WA, United States
| | - Akshay Nair
- Burkitt’s Lymphoma Fund for Africa, Seattle, WA, United States
| | - Sunil Nambiar
- Burkitt’s Lymphoma Fund for Africa, Seattle, WA, United States
| | - Renjini Nair
- Burkitt’s Lymphoma Fund for Africa, Seattle, WA, United States
| | - Rajesh Uthamanthil
- Burkitt’s Lymphoma Fund for Africa, Seattle, WA, United States
- Department of Comparative Medicine, University of Washington, Burkitt’s Lymphoma Fund for Africa, Seattle, WA, United States
| | - Teresa Lotodo
- Department Of Hematology and Blood Transfusion, Moi University/MTRH/AMPATH, Eldoret, Kenya
| | - Shachi Mittal
- Department of Chemical Engineering, University of Washington, Seattle, WA, United States
- Department of Laboratory Medicine and Pathology, University of Washington, Seattle, WA, United States
| | - Steven Kussick
- Burkitt’s Lymphoma Fund for Africa, Seattle, WA, United States
- Department of Laboratory Medicine and Pathology, University of Washington, Burkitt’s Lymphoma Fund for Africa, Seattle, WA, United States
| |
Collapse
|
3
|
Jiang W, Wang H, Dong X, Yu X, Zhao Y, Chen D, Yan B, Cheng J, Zhuo S, Wang H, Yan J. Pathomics Signature for Prognosis and Chemotherapy Benefits in Stage III Colon Cancer. JAMA Surg 2024; 159:519-528. [PMID: 38416471 PMCID: PMC10902777 DOI: 10.1001/jamasurg.2023.8015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/18/2023] [Accepted: 11/12/2023] [Indexed: 02/29/2024]
Abstract
Importance The current TNM staging system may not provide adequate information for prognostic purposes and to assess the potential benefits of chemotherapy for patients with stage III colon cancer. Objective To develop and validate a pathomics signature to estimate prognosis and benefit from chemotherapy using hematoxylin-eosin (H-E)-stained slides. Design, Setting, and Participants This retrospective prognostic study used data from consecutive patients with histologically confirmed stage III colon cancer at 2 medical centers between January 2012 and December 2015. A total of 114 pathomics features were extracted from digital H-E-stained images from Nanfang Hospital of Southern Medical University, Guangzhou, China, and a pathomics signature was constructed using a least absolute shrinkage and selection operator Cox regression model in the training cohort. The associations of the pathomics signature with disease-free survival (DFS) and overall survival (OS) were evaluated. Patients at the Sixth Affiliated Hospital, Sun Yat-sen University, Guangzhou, China, formed the validation cohort. Data analysis was conducted from September 2022 to March 2023. Main Outcomes and Measures The prognostic accuracy of the pathomics signature as well as its association with chemotherapy response were evaluated. Results This study included 785 patients (mean [SD] age, 62.7 [11.1] years; 437 [55.7%] male). A pathomics signature was constructed based on 4 features. Multivariable analysis revealed that the pathomics signature was an independent factor associated with DFS (hazard ratio [HR], 2.46 [95% CI, 2.89-4.13]; P < .001) and OS (HR, 2.78 [95% CI, 2.34-3.31]; P < .001) in the training cohort. Incorporating the pathomics signature into pathomics nomograms resulted in better performance for the estimation of prognosis than the traditional model in a concordance index comparison in the training cohort (DFS: HR, 0.88 [95% CI, 0.86-0.89] vs HR, 0.73 [95% CI, 0.71-0.75]; P < .001; OS: HR, 0.85 [95% CI, 0.84-0.86] vs HR, 0.74 [95% CI, 0.72-0.76]; P < .001) and validation cohort (DFS: HR, 0.83 [95% CI, 0.82-0.85] vs HR, 0.70 [95% CI, 0.67-0.72]; P < .001; OS: HR, 0.80 [95% CI, 0.78-0.82] vs HR, 0.69 [0.67-0.72]; P < .001). Further analysis revealed that patients with a low pathomics signature were more likely to benefit from chemotherapy (eg, combined cohort: DFS: HR, 0.44 [95% CI, 0.28-0.69]; P = .001; OS: HR, 0.43 [95% CI, 0.29-0.64]; P < .001). Conclusions and Relevance These findings suggest that a pathomics signature could help identify patients most likely to benefit from chemotherapy in stage III colon cancer.
Collapse
Affiliation(s)
- Wei Jiang
- Guangdong Provincial Key Laboratory of Precision Medicine for Gastrointestinal Tumor, Department of General Surgery, Nanfang Hospital, The First School of Clinical Medicine, Southern Medical University, Guangzhou, China
- School of Science, Jimei University, Xiamen, China
| | - Huaiming Wang
- Guangdong Provincial Key Laboratory of Colorectal and Pelvic Floor Diseases, Department of Colorectal Surgery & Guangdong Institute of Gastroenterology, the Sixth Affiliated Hospital, Supported by National Key Clinical Discipline, Sun Yat-sen University, Guangzhou, China
| | - Xiaoyu Dong
- Guangdong Provincial Key Laboratory of Precision Medicine for Gastrointestinal Tumor, Department of General Surgery, Nanfang Hospital, The First School of Clinical Medicine, Southern Medical University, Guangzhou, China
| | - Xian Yu
- Guangdong Provincial Key Laboratory of Precision Medicine for Gastrointestinal Tumor, Department of General Surgery, Nanfang Hospital, The First School of Clinical Medicine, Southern Medical University, Guangzhou, China
- Key Laboratory for Biorheological Science and Technology of Ministry of Education, Chongqing University Cancer Hospital & Chongqing Cancer Institute & Chongqing Cancer Hospital, Chongqing, China
| | - Yandong Zhao
- Department of Pathology, the Sixth Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Dexin Chen
- Guangdong Provincial Key Laboratory of Precision Medicine for Gastrointestinal Tumor, Department of General Surgery, Nanfang Hospital, The First School of Clinical Medicine, Southern Medical University, Guangzhou, China
| | - Botao Yan
- Guangdong Provincial Key Laboratory of Precision Medicine for Gastrointestinal Tumor, Department of General Surgery, Nanfang Hospital, The First School of Clinical Medicine, Southern Medical University, Guangzhou, China
| | - Jiaxin Cheng
- Guangdong Provincial Key Laboratory of Precision Medicine for Gastrointestinal Tumor, Department of General Surgery, Nanfang Hospital, The First School of Clinical Medicine, Southern Medical University, Guangzhou, China
| | | | - Hui Wang
- Guangdong Provincial Key Laboratory of Colorectal and Pelvic Floor Diseases, Department of Colorectal Surgery & Guangdong Institute of Gastroenterology, the Sixth Affiliated Hospital, Supported by National Key Clinical Discipline, Sun Yat-sen University, Guangzhou, China
| | - Jun Yan
- Guangdong Provincial Key Laboratory of Precision Medicine for Gastrointestinal Tumor, Department of General Surgery, Nanfang Hospital, The First School of Clinical Medicine, Southern Medical University, Guangzhou, China
- Department of Gastrointestinal Surgery, Shenzhen People’s Hospital, Second Clinical Medical College of Jinan University, First Affiliated Hospital of Southern University of Science and Technology, Shenzhen, China
| |
Collapse
|
4
|
Herzberger L, Hadwiger M, Kruger R, Sorger P, Pfister H, Groller E, Beyer J. Residency Octree: A Hybrid Approach for Scalable Web-Based Multi-Volume Rendering. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 2024; 30:1380-1390. [PMID: 37889813 PMCID: PMC10840607 DOI: 10.1109/tvcg.2023.3327193] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/29/2023]
Abstract
We present a hybrid multi-volume rendering approach based on a novel Residency Octree that combines the advantages of out-of-core volume rendering using page tables with those of standard octrees. Octree approaches work by performing hierarchical tree traversal. However, in octree volume rendering, tree traversal and the selection of data resolution are intrinsically coupled. This makes fine-grained empty-space skipping costly. Page tables, on the other hand, allow access to any cached brick from any resolution. However, they do not offer a clear and efficient strategy for substituting missing high-resolution data with lower-resolution data. We enable flexible mixed-resolution out-of-core multi-volume rendering by decoupling the cache residency of multi-resolution data from a resolution-independent spatial subdivision determined by the tree. Instead of one-to-one node-to-brick correspondences, each residency octree node is mapped to a set of bricks from different resolution levels. This makes it possible to efficiently and adaptively choose and mix resolutions, adapt sampling rates, and compensate for cache misses. At the same time, residency octrees support fine-grained empty-space skipping, independent of the data subdivision used for caching. Finally, to facilitate collaboration and outreach, and to eliminate local data storage, our implementation is a web-based, pure client-side renderer using WebGPU and WebAssembly. Our method is faster than prior approaches and efficient for many data channels with a flexible and adaptive choice of data resolution.
Collapse
|
5
|
Gaglia G, Burger ML, Ritch CC, Rammos D, Dai Y, Crossland GE, Tavana SZ, Warchol S, Jaeger AM, Naranjo S, Coy S, Nirmal AJ, Krueger R, Lin JR, Pfister H, Sorger PK, Jacks T, Santagata S. Lymphocyte networks are dynamic cellular communities in the immunoregulatory landscape of lung adenocarcinoma. Cancer Cell 2023; 41:871-886.e10. [PMID: 37059105 PMCID: PMC10193529 DOI: 10.1016/j.ccell.2023.03.015] [Citation(s) in RCA: 26] [Impact Index Per Article: 26.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/31/2022] [Revised: 01/31/2023] [Accepted: 03/22/2023] [Indexed: 04/16/2023]
Abstract
Lymphocytes are key for immune surveillance of tumors, but our understanding of the spatial organization and physical interactions that facilitate lymphocyte anti-cancer functions is limited. We used multiplexed imaging, quantitative spatial analysis, and machine learning to create high-definition maps of lung tumors from a Kras/Trp53-mutant mouse model and human resections. Networks of interacting lymphocytes ("lymphonets") emerged as a distinctive feature of the anti-cancer immune response. Lymphonets nucleated from small T cell clusters and incorporated B cells with increasing size. CXCR3-mediated trafficking modulated lymphonet size and number, but T cell antigen expression directed intratumoral localization. Lymphonets preferentially harbored TCF1+ PD-1+ progenitor CD8+ T cells involved in responses to immune checkpoint blockade (ICB) therapy. Upon treatment of mice with ICB or an antigen-targeted vaccine, lymphonets retained progenitor and gained cytotoxic CD8+ T cell populations, likely via progenitor differentiation. These data show that lymphonets create a spatial environment supportive of CD8+ T cell anti-tumor responses.
Collapse
Affiliation(s)
- Giorgio Gaglia
- Laboratory of Systems Pharmacology, Department of Systems Biology, Harvard Medical School, Boston, MA 02115, USA; Ludwig Center at Harvard, Harvard Medical School, Boston, MA 02115, USA; Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA 02115, USA
| | - Megan L Burger
- David H. Koch Institute for Integrative Cancer Research, Massachusetts Institute of Technology, Cambridge, MA 02139, USA; Department of Cell, Developmental and Cancer Biology, Oregon Health & Science University, Portland, OR 97212, USA; School of Medicine, Division of Hematology and Oncology, Oregon Health & Science University, Portland, OR 97212, USA
| | - Cecily C Ritch
- Laboratory of Systems Pharmacology, Department of Systems Biology, Harvard Medical School, Boston, MA 02115, USA; Ludwig Center at Harvard, Harvard Medical School, Boston, MA 02115, USA; Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA 02115, USA
| | - Danae Rammos
- Laboratory of Systems Pharmacology, Department of Systems Biology, Harvard Medical School, Boston, MA 02115, USA; Ludwig Center at Harvard, Harvard Medical School, Boston, MA 02115, USA; Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA 02115, USA
| | - Yang Dai
- Laboratory of Systems Pharmacology, Department of Systems Biology, Harvard Medical School, Boston, MA 02115, USA; Ludwig Center at Harvard, Harvard Medical School, Boston, MA 02115, USA; Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA 02115, USA
| | - Grace E Crossland
- David H. Koch Institute for Integrative Cancer Research, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Sara Z Tavana
- David H. Koch Institute for Integrative Cancer Research, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Simon Warchol
- Laboratory of Systems Pharmacology, Department of Systems Biology, Harvard Medical School, Boston, MA 02115, USA; School of Engineering and Applied Sciences, Harvard University, Boston, MA 02134, USA
| | - Alex M Jaeger
- David H. Koch Institute for Integrative Cancer Research, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Santiago Naranjo
- David H. Koch Institute for Integrative Cancer Research, Massachusetts Institute of Technology, Cambridge, MA 02139, USA; Department of Biology, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Shannon Coy
- Laboratory of Systems Pharmacology, Department of Systems Biology, Harvard Medical School, Boston, MA 02115, USA; Ludwig Center at Harvard, Harvard Medical School, Boston, MA 02115, USA; Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA 02115, USA
| | - Ajit J Nirmal
- Laboratory of Systems Pharmacology, Department of Systems Biology, Harvard Medical School, Boston, MA 02115, USA
| | - Robert Krueger
- Laboratory of Systems Pharmacology, Department of Systems Biology, Harvard Medical School, Boston, MA 02115, USA; School of Engineering and Applied Sciences, Harvard University, Boston, MA 02134, USA
| | - Jia-Ren Lin
- Laboratory of Systems Pharmacology, Department of Systems Biology, Harvard Medical School, Boston, MA 02115, USA; Ludwig Center at Harvard, Harvard Medical School, Boston, MA 02115, USA
| | - Hanspeter Pfister
- School of Engineering and Applied Sciences, Harvard University, Boston, MA 02134, USA
| | - Peter K Sorger
- Laboratory of Systems Pharmacology, Department of Systems Biology, Harvard Medical School, Boston, MA 02115, USA; Ludwig Center at Harvard, Harvard Medical School, Boston, MA 02115, USA
| | - Tyler Jacks
- David H. Koch Institute for Integrative Cancer Research, Massachusetts Institute of Technology, Cambridge, MA 02139, USA; Department of Biology, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Sandro Santagata
- Laboratory of Systems Pharmacology, Department of Systems Biology, Harvard Medical School, Boston, MA 02115, USA; Ludwig Center at Harvard, Harvard Medical School, Boston, MA 02115, USA; Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA 02115, USA.
| |
Collapse
|
6
|
Lin JR, Wang S, Coy S, Chen YA, Yapp C, Tyler M, Nariya MK, Heiser CN, Lau KS, Santagata S, Sorger PK. Multiplexed 3D atlas of state transitions and immune interaction in colorectal cancer. Cell 2023; 186:363-381.e19. [PMID: 36669472 PMCID: PMC10019067 DOI: 10.1016/j.cell.2022.12.028] [Citation(s) in RCA: 71] [Impact Index Per Article: 71.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2021] [Revised: 09/26/2022] [Accepted: 12/16/2022] [Indexed: 01/20/2023]
Abstract
Advanced solid cancers are complex assemblies of tumor, immune, and stromal cells characterized by high intratumoral variation. We use highly multiplexed tissue imaging, 3D reconstruction, spatial statistics, and machine learning to identify cell types and states underlying morphological features of known diagnostic and prognostic significance in colorectal cancer. Quantitation of these features in high-plex marker space reveals recurrent transitions from one tumor morphology to the next, some of which are coincident with long-range gradients in the expression of oncogenes and epigenetic regulators. At the tumor invasive margin, where tumor, normal, and immune cells compete, T cell suppression involves multiple cell types and 3D imaging shows that seemingly localized 2D features such as tertiary lymphoid structures are commonly interconnected and have graded molecular properties. Thus, while cancer genetics emphasizes the importance of discrete changes in tumor state, whole-specimen imaging reveals large-scale morphological and molecular gradients analogous to those in developing tissues.
Collapse
Affiliation(s)
- Jia-Ren Lin
- Ludwig Center at Harvard and Laboratory of Systems Pharmacology, Harvard Medical School, Boston, MA, USA
| | - Shu Wang
- Ludwig Center at Harvard and Laboratory of Systems Pharmacology, Harvard Medical School, Boston, MA, USA; Harvard Graduate Program in Biophysics, Harvard University, Cambridge, MA, USA
| | - Shannon Coy
- Ludwig Center at Harvard and Laboratory of Systems Pharmacology, Harvard Medical School, Boston, MA, USA; Department of Pathology, Brigham and Women's Hospital, Boston, MA, USA
| | - Yu-An Chen
- Ludwig Center at Harvard and Laboratory of Systems Pharmacology, Harvard Medical School, Boston, MA, USA
| | - Clarence Yapp
- Ludwig Center at Harvard and Laboratory of Systems Pharmacology, Harvard Medical School, Boston, MA, USA
| | - Madison Tyler
- Ludwig Center at Harvard and Laboratory of Systems Pharmacology, Harvard Medical School, Boston, MA, USA
| | - Maulik K Nariya
- Ludwig Center at Harvard and Laboratory of Systems Pharmacology, Harvard Medical School, Boston, MA, USA; Department of Systems Biology, Harvard Medical School, Boston, MA 02115, USA
| | - Cody N Heiser
- Program in Chemical & Physical Biology, Vanderbilt University School of Medicine, Nashville, TN, USA
| | - Ken S Lau
- Epithelial Biology Center and Department of Cell and Developmental Biology, Vanderbilt University School of Medicine, Nashville, TN, USA
| | - Sandro Santagata
- Ludwig Center at Harvard and Laboratory of Systems Pharmacology, Harvard Medical School, Boston, MA, USA; Department of Pathology, Brigham and Women's Hospital, Boston, MA, USA; Department of Systems Biology, Harvard Medical School, Boston, MA 02115, USA
| | - Peter K Sorger
- Ludwig Center at Harvard and Laboratory of Systems Pharmacology, Harvard Medical School, Boston, MA, USA; Department of Systems Biology, Harvard Medical School, Boston, MA 02115, USA.
| |
Collapse
|
7
|
Park J, Kim J, Lewy T, Rice CM, Elemento O, Rendeiro AF, Mason CE. Spatial omics technologies at multimodal and single cell/subcellular level. Genome Biol 2022; 23:256. [PMID: 36514162 PMCID: PMC9746133 DOI: 10.1186/s13059-022-02824-6] [Citation(s) in RCA: 28] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/06/2022] [Accepted: 11/29/2022] [Indexed: 12/15/2022] Open
Abstract
Spatial omics technologies enable a deeper understanding of cellular organizations and interactions within a tissue of interest. These assays can identify specific compartments or regions in a tissue with differential transcript or protein abundance, delineate their interactions, and complement other methods in defining cellular phenotypes. A variety of spatial methodologies are being developed and commercialized; however, these techniques differ in spatial resolution, multiplexing capability, scale/throughput, and coverage. Here, we review the current and prospective landscape of single cell to subcellular resolution spatial omics technologies and analysis tools to provide a comprehensive picture for both research and clinical applications.
Collapse
Affiliation(s)
- Jiwoon Park
- Department of Physiology, Biophysics and Systems Biology, Weill Cornell Medicine, New York, NY, USA
- Laboratory of Virology and Infectious Disease, The Rockefeller University, New York, NY, 10065, USA
| | - Junbum Kim
- Department of Physiology, Biophysics and Systems Biology, Weill Cornell Medicine, New York, NY, USA
| | - Tyler Lewy
- Laboratory of Virology and Infectious Disease, The Rockefeller University, New York, NY, 10065, USA
| | - Charles M Rice
- Laboratory of Virology and Infectious Disease, The Rockefeller University, New York, NY, 10065, USA
| | - Olivier Elemento
- Department of Physiology, Biophysics and Systems Biology, Weill Cornell Medicine, New York, NY, USA
- The HRH Prince Alwaleed Bin Talal Bin Abdulaziz Alsaud Institute for Computational Biomedicine, Weill Cornell Medicine, New York, NY, USA
- Caryl and Israel Englander Institute for Precision Medicine, Weill Cornell Medicine, New York, NY, USA
| | - André F Rendeiro
- The HRH Prince Alwaleed Bin Talal Bin Abdulaziz Alsaud Institute for Computational Biomedicine, Weill Cornell Medicine, New York, NY, USA
- Caryl and Israel Englander Institute for Precision Medicine, Weill Cornell Medicine, New York, NY, USA
- CeMM Research Center for Molecular Medicine of the Austrian Academy of Sciences, Vienna, Austria
| | - Christopher E Mason
- Department of Physiology, Biophysics and Systems Biology, Weill Cornell Medicine, New York, NY, USA.
- The HRH Prince Alwaleed Bin Talal Bin Abdulaziz Alsaud Institute for Computational Biomedicine, Weill Cornell Medicine, New York, NY, USA.
- The Feil Family Brain and Mind Research Institute, Weill Cornell Medicine, New York, NY, USA.
- The WorldQuant Initiative for Quantitative Prediction, Weill Cornell Medicine, New York, NY, USA.
| |
Collapse
|
8
|
Huang K, Lin B, Liu J, Liu Y, Li J, Tian G, Yang J. Predicting colorectal cancer tumor mutational burden from histopathological images and clinical information using multi-modal deep learning. Bioinformatics 2022; 38:5108-5115. [PMID: 36130268 DOI: 10.1093/bioinformatics/btac641] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2022] [Revised: 08/31/2022] [Accepted: 09/20/2022] [Indexed: 12/24/2022] Open
Abstract
MOTIVATION Tumor mutational burden (TMB) is an indicator of the efficacy and prognosis of immune checkpoint therapy in colorectal cancer (CRC). In general, patients with higher TMB values are more likely to benefit from immunotherapy. Though whole-exome sequencing is considered the gold standard for determining TMB, it is difficult to be applied in clinical practice due to its high cost. There are also a few DNA panel-based methods to estimate TMB; however, their detection cost is also high, and the associated wet-lab experiments usually take days, which emphasize the need for faster and cheaper alternatives. RESULTS In this study, we propose a multi-modal deep learning model based on a residual network (ResNet) and multi-modal compact bilinear pooling to predict TMB status (i.e. TMB high (TMB_H) or TMB low(TMB_L)) directly from histopathological images and clinical data. We applied the model to CRC data from The Cancer Genome Atlas and compared it with four other popular methods, namely, ResNet18, ResNet50, VGG19 and AlexNet. We tested different TMB thresholds, namely, percentiles of 10%, 14.3%, 15%, 16.3%, 20%, 30% and 50%, to differentiate TMB_H and TMB_L.For the percentile of 14.3% (i.e. TMB value 20) and ResNet18, our model achieved an area under the receiver operating characteristic curve of 0.817 after 5-fold cross-validation, which was better than that of other compared models. In addition, we also found that TMB values were significantly associated with the tumor stage and N and M stages. Our study shows that deep learning models can predict TMB status from histopathological images and clinical information only, which is worth clinical application.
Collapse
Affiliation(s)
- Kaimei Huang
- Department of Mathematics, Zhejiang Normal University, Jinghua 321004, China.,Department of Sciences, Geneis (Beijing) Co., Ltd, Beijing 100102, China.,Department of Sciences, Qingdao Geneis Institute of Big Data Mining and Precision Medicine, Qingdao 266000, China
| | - Binghu Lin
- Department of General Surgery of Third Ward, Xiangyang No.1 People's Hospital, Hubei University of Medicine, Xiangyang 441000, China
| | - Jinyang Liu
- Department of Sciences, Geneis (Beijing) Co., Ltd, Beijing 100102, China.,Department of Sciences, Qingdao Geneis Institute of Big Data Mining and Precision Medicine, Qingdao 266000, China
| | - Yankun Liu
- Cancer Institute, Tangshan People's Hospital, Tangshan 063001, China
| | - Jingwu Li
- Cancer Institute, Tangshan People's Hospital, Tangshan 063001, China
| | - Geng Tian
- Department of Sciences, Geneis (Beijing) Co., Ltd, Beijing 100102, China.,Department of Sciences, Qingdao Geneis Institute of Big Data Mining and Precision Medicine, Qingdao 266000, China
| | - Jialiang Yang
- Department of Sciences, Geneis (Beijing) Co., Ltd, Beijing 100102, China.,Department of Sciences, Qingdao Geneis Institute of Big Data Mining and Precision Medicine, Qingdao 266000, China
| |
Collapse
|
9
|
Peterson HM, Chin LK, Iwamoto Y, Oh J, Carlson JCT, Lee H, Im H, Weissleder R. Integrated Analytical System for Clinical Single-Cell Analysis. ADVANCED SCIENCE (WEINHEIM, BADEN-WURTTEMBERG, GERMANY) 2022; 9:e2200415. [PMID: 35508767 PMCID: PMC9284190 DOI: 10.1002/advs.202200415] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/21/2022] [Revised: 03/31/2022] [Indexed: 05/23/2023]
Abstract
High-dimensional analyses of cancers can potentially be used to better define cancer subtypes, analyze the complex tumor microenvironment, and perform cancer cell pathway analyses for drug trials. Unfortunately, integrated systems that allow such analyses in serial fine needle aspirates within a day or at point-of-care currently do not exist. To achieve this, an integrated immunofluorescence single-cell analyzer (i2SCAN) for deep profiling of directly harvested cells is developed. By combining a novel cellular imaging system, highly cyclable bioorthogonal FAST antibody panels, and integrated computational analysis, it is shown that same-day analysis is possible in thousands of harvested cells. It is demonstrated that the i2SCAN approach allows comprehensive analysis of breast cancer samples obtained by fine needle aspiration or core tissues. The method is a rapid, robust, and low-cost solution to high-dimensional analysis of scant clinical specimens.
Collapse
Affiliation(s)
- Hannah M. Peterson
- Center for Systems BiologyMassachusetts General HospitalBostonMA02114USA
| | - Lip Ket Chin
- Center for Systems BiologyMassachusetts General HospitalBostonMA02114USA
| | - Yoshi Iwamoto
- Center for Systems BiologyMassachusetts General HospitalBostonMA02114USA
| | - Juhyun Oh
- Center for Systems BiologyMassachusetts General HospitalBostonMA02114USA
| | - Jonathan C. T. Carlson
- Center for Systems BiologyMassachusetts General HospitalBostonMA02114USA
- Cancer CenterMassachusetts General HospitalBostonMA02114USA
| | - Hakho Lee
- Center for Systems BiologyMassachusetts General HospitalBostonMA02114USA
- Department of RadiologyMassachusetts General HospitalBostonMA02114USA
| | - Hyungsoon Im
- Center for Systems BiologyMassachusetts General HospitalBostonMA02114USA
- Department of RadiologyMassachusetts General HospitalBostonMA02114USA
| | - Ralph Weissleder
- Center for Systems BiologyMassachusetts General HospitalBostonMA02114USA
- Cancer CenterMassachusetts General HospitalBostonMA02114USA
- Department of RadiologyMassachusetts General HospitalBostonMA02114USA
- Department of Systems BiologyHarvard Medical SchoolBostonMA02115USA
| |
Collapse
|
10
|
Wang X, Yang S, Zhang J, Wang M, Zhang J, Yang W, Huang J, Han X. Transformer-based unsupervised contrastive learning for histopathological image classification. Med Image Anal 2022; 81:102559. [DOI: 10.1016/j.media.2022.102559] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/21/2022] [Revised: 06/24/2022] [Accepted: 07/25/2022] [Indexed: 10/16/2022]
|
11
|
Gaglia G, Kabraji S, Rammos D, Dai Y, Verma A, Wang S, Mills CE, Chung M, Bergholz JS, Coy S, Lin JR, Jeselsohn R, Metzger O, Winer EP, Dillon DA, Zhao JJ, Sorger PK, Santagata S. Temporal and spatial topography of cell proliferation in cancer. Nat Cell Biol 2022; 24:316-326. [PMID: 35292783 PMCID: PMC8959396 DOI: 10.1038/s41556-022-00860-9] [Citation(s) in RCA: 25] [Impact Index Per Article: 12.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2021] [Accepted: 01/31/2022] [Indexed: 02/06/2023]
Abstract
Proliferation is a fundamental trait of cancer cells, but its properties and spatial organization in tumours are poorly characterized. Here we use highly multiplexed tissue imaging to perform single-cell quantification of cell cycle regulators and then develop robust, multivariate, proliferation metrics. Across diverse cancers, proliferative architecture is organized at two spatial scales: large domains, and smaller niches enriched for specific immune lineages. Some tumour cells express cell cycle regulators in the (canonical) patterns expected of freely growing cells, a phenomenon we refer to as 'cell cycle coherence'. By contrast, the cell cycles of other tumour cell populations are skewed towards specific phases or exhibit non-canonical (incoherent) marker combinations. Coherence varies across space, with changes in oncogene activity and therapeutic intervention, and is associated with aggressive tumour behaviour. Thus, multivariate measures from high-plex tissue images capture clinically significant features of cancer proliferation, a fundamental step in enabling more precise use of anti-cancer therapies.
Collapse
Affiliation(s)
- Giorgio Gaglia
- Laboratory of Systems Pharmacology, Department of Systems Biology, Harvard Medical School, Boston, MA, USA
- Ludwig Center at Harvard, Harvard Medical School, Boston, MA, USA
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
| | - Sheheryar Kabraji
- Laboratory of Systems Pharmacology, Department of Systems Biology, Harvard Medical School, Boston, MA, USA.
- Ludwig Center at Harvard, Harvard Medical School, Boston, MA, USA.
- Department of Medical Oncology, Dana Farber Cancer Institute, Boston, MA, USA.
- Department of Cancer Biology, Dana Farber Cancer Institute, Boston, MA, USA.
- Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.
| | - Danae Rammos
- Laboratory of Systems Pharmacology, Department of Systems Biology, Harvard Medical School, Boston, MA, USA
- Ludwig Center at Harvard, Harvard Medical School, Boston, MA, USA
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
| | - Yang Dai
- Laboratory of Systems Pharmacology, Department of Systems Biology, Harvard Medical School, Boston, MA, USA
- Ludwig Center at Harvard, Harvard Medical School, Boston, MA, USA
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
| | - Ana Verma
- Laboratory of Systems Pharmacology, Department of Systems Biology, Harvard Medical School, Boston, MA, USA
- Ludwig Center at Harvard, Harvard Medical School, Boston, MA, USA
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
| | - Shu Wang
- Laboratory of Systems Pharmacology, Department of Systems Biology, Harvard Medical School, Boston, MA, USA
- Harvard Graduate Program in Biophysics, Harvard University, Cambridge, MA, USA
| | - Caitlin E Mills
- Laboratory of Systems Pharmacology, Department of Systems Biology, Harvard Medical School, Boston, MA, USA
| | - Mirra Chung
- Laboratory of Systems Pharmacology, Department of Systems Biology, Harvard Medical School, Boston, MA, USA
| | - Johann S Bergholz
- Department of Cancer Biology, Dana Farber Cancer Institute, Boston, MA, USA
| | - Shannon Coy
- Laboratory of Systems Pharmacology, Department of Systems Biology, Harvard Medical School, Boston, MA, USA
- Ludwig Center at Harvard, Harvard Medical School, Boston, MA, USA
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
| | - Jia-Ren Lin
- Laboratory of Systems Pharmacology, Department of Systems Biology, Harvard Medical School, Boston, MA, USA
- Ludwig Center at Harvard, Harvard Medical School, Boston, MA, USA
| | - Rinath Jeselsohn
- Department of Medical Oncology, Dana Farber Cancer Institute, Boston, MA, USA
- Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
| | - Otto Metzger
- Department of Medical Oncology, Dana Farber Cancer Institute, Boston, MA, USA
| | - Eric P Winer
- Department of Medical Oncology, Dana Farber Cancer Institute, Boston, MA, USA
| | - Deborah A Dillon
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
| | - Jean J Zhao
- Department of Cancer Biology, Dana Farber Cancer Institute, Boston, MA, USA
| | - Peter K Sorger
- Laboratory of Systems Pharmacology, Department of Systems Biology, Harvard Medical School, Boston, MA, USA
- Ludwig Center at Harvard, Harvard Medical School, Boston, MA, USA
| | - Sandro Santagata
- Laboratory of Systems Pharmacology, Department of Systems Biology, Harvard Medical School, Boston, MA, USA.
- Ludwig Center at Harvard, Harvard Medical School, Boston, MA, USA.
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.
- Department of Pathology, Dana Farber Cancer Institute, Boston, MA, USA.
| |
Collapse
|
12
|
Schapiro D, Yapp C, Sokolov A, Reynolds SM, Chen YA, Sudar D, Xie Y, Muhlich J, Arias-Camison R, Arena S, Taylor AJ, Nikolov M, Tyler M, Lin JR, Burlingame EA, Chang YH, Farhi SL, Thorsson V, Venkatamohan N, Drewes JL, Pe'er D, Gutman DA, Herrmann MD, Gehlenborg N, Bankhead P, Roland JT, Herndon JM, Snyder MP, Angelo M, Nolan G, Swedlow JR, Schultz N, Merrick DT, Mazzili SA, Cerami E, Rodig SJ, Santagata S, Sorger PK. MITI minimum information guidelines for highly multiplexed tissue images. Nat Methods 2022; 19:262-267. [PMID: 35277708 PMCID: PMC9009186 DOI: 10.1038/s41592-022-01415-4] [Citation(s) in RCA: 23] [Impact Index Per Article: 11.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/25/2023]
Abstract
The imminent release of tissue atlases combining multi-channel microscopy with single cell sequencing and other omics data from normal and diseased specimens creates an urgent need for data and metadata standards that guide data deposition, curation and release. We describe a Minimum Information about highly multiplexed Tissue Imaging (MITI) standard that applies best practices developed for genomics and other microscopy data to highly multiplexed tissue images and traditional histology.
Collapse
Affiliation(s)
- Denis Schapiro
- Laboratory of Systems Pharmacology, Ludwig Center for Cancer Research at Harvard, Harvard Medical School, Boston, MA, USA
- Klarman Cell Observatory, Broad Institute of MIT and Harvard, Cambridge, MA, USA
- Institute for Computational Biomedicine, Faculty of Medicine, Heidelberg University Hospital and Heidelberg University, Heidelberg, Germany
- Institute of Pathology, Heidelberg University Hospital, Heidelberg, Germany
| | - Clarence Yapp
- Laboratory of Systems Pharmacology, Ludwig Center for Cancer Research at Harvard, Harvard Medical School, Boston, MA, USA
- Image and Data Analysis Core, Harvard Medical School, Boston, MA, USA
| | - Artem Sokolov
- Laboratory of Systems Pharmacology, Ludwig Center for Cancer Research at Harvard, Harvard Medical School, Boston, MA, USA
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | | | - Yu-An Chen
- Laboratory of Systems Pharmacology, Ludwig Center for Cancer Research at Harvard, Harvard Medical School, Boston, MA, USA
| | - Damir Sudar
- Quantitative Imaging Systems LLC, Portland, OR, USA
| | - Yubin Xie
- Program in Computational and Systems Biology, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Jeremy Muhlich
- Laboratory of Systems Pharmacology, Ludwig Center for Cancer Research at Harvard, Harvard Medical School, Boston, MA, USA
| | - Raquel Arias-Camison
- Laboratory of Systems Pharmacology, Ludwig Center for Cancer Research at Harvard, Harvard Medical School, Boston, MA, USA
| | - Sarah Arena
- Laboratory of Systems Pharmacology, Ludwig Center for Cancer Research at Harvard, Harvard Medical School, Boston, MA, USA
| | | | | | - Madison Tyler
- Laboratory of Systems Pharmacology, Ludwig Center for Cancer Research at Harvard, Harvard Medical School, Boston, MA, USA
| | - Jia-Ren Lin
- Laboratory of Systems Pharmacology, Ludwig Center for Cancer Research at Harvard, Harvard Medical School, Boston, MA, USA
| | - Erik A Burlingame
- Oregon Health and Science University, Portland, OR, USA
- Indica Labs, Albuquerque, NM, USA
| | - Young H Chang
- Oregon Health and Science University, Portland, OR, USA
| | - Samouil L Farhi
- Klarman Cell Observatory, Broad Institute of MIT and Harvard, Cambridge, MA, USA
| | | | | | - Julia L Drewes
- Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Dana Pe'er
- Program in Computational and Systems Biology, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | | | - Markus D Herrmann
- Department of Pathology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, USA
| | - Nils Gehlenborg
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Peter Bankhead
- Edinburgh Pathology, Institute of Genetics and Cancer, University of Edinburgh, Edinburgh, UK
| | - Joseph T Roland
- Vanderbilt University School of Medicine, Nashville, TN, USA
| | - John M Herndon
- Department of Surgery, Washington University School of Medicine, St. Louis, MO, USA
| | | | - Michael Angelo
- School of Medicine, Stanford University, Stanford, CA, USA
| | - Garry Nolan
- School of Medicine, Stanford University, Stanford, CA, USA
| | - Jason R Swedlow
- Division of Computational Biology and Centre for Gene Regulation and Expression, University of Dundee, Dundee, UK
| | - Nikolaus Schultz
- Department of Epidemiology & Biostatistics, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | | | | | | | - Scott J Rodig
- Department of Pathology, Brigham and Women's Hospital, Boston, MA, USA
| | - Sandro Santagata
- Laboratory of Systems Pharmacology, Ludwig Center for Cancer Research at Harvard, Harvard Medical School, Boston, MA, USA.
- Department of Pathology, Brigham and Women's Hospital, Boston, MA, USA.
| | - Peter K Sorger
- Laboratory of Systems Pharmacology, Ludwig Center for Cancer Research at Harvard, Harvard Medical School, Boston, MA, USA.
- Department of Systems Biology, Harvard Medical School, Boston, MA, USA.
| |
Collapse
|