26
|
Pandey A, Kagiyama N, Yanamala N, Segar MW, Cho JS, Tokodi M, Sengupta PP. Deep-Learning Models for the Echocardiographic Assessment of Diastolic Dysfunction. JACC Cardiovasc Imaging 2021; 14:1887-1900. [PMID: 34023263 DOI: 10.1016/j.jcmg.2021.04.010] [Citation(s) in RCA: 44] [Impact Index Per Article: 14.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/05/2021] [Revised: 03/03/2021] [Accepted: 04/01/2021] [Indexed: 12/17/2022]
Abstract
OBJECTIVES The authors explored a deep neural network (DeepNN) model that integrates multidimensional echocardiographic data to identify distinct patient subgroups with heart failure with preserved ejection fraction (HFpEF). BACKGROUND The clinical algorithms for phenotyping the severity of diastolic dysfunction in HFpEF remain imprecise. METHODS The authors developed a DeepNN model to predict high- and low-risk phenogroups in a derivation cohort (n = 1,242). Model performance was first validated in 2 external cohorts to identify elevated left ventricular filling pressure (n = 84) and assess its prognostic value (n = 219) in patients with varying degrees of systolic and diastolic dysfunction. In 3 National Heart, Lung, and Blood Institute-funded HFpEF trials, the clinical significance of the model was further validated by assessing the relationships of the phenogroups with adverse clinical outcomes (TOPCAT [Aldosterone Antagonist Therapy for Adults With Heart Failure and Preserved Systolic Function] trial, n = 518), cardiac biomarkers, and exercise parameters (NEAT-HFpEF [Nitrate's Effect on Activity Tolerance in Heart Failure With Preserved Ejection Fraction] and RELAX-HF [Evaluating the Effectiveness of Sildenafil at Improving Health Outcomes and Exercise Ability in People With Diastolic Heart Failure] pooled cohort, n = 346). RESULTS The DeepNN model showed higher area under the receiver-operating characteristic curve than 2016 American Society of Echocardiography guideline grades for predicting elevated left ventricular filling pressure (0.88 vs. 0.67; p = 0.01). The high-risk (vs. low-risk) phenogroup showed higher rates of heart failure hospitalization and/or death, even after adjusting for global left ventricular and atrial longitudinal strain (hazard ratio [HR]: 3.96; 95% confidence interval [CI]: 1.24 to 12.67; p = 0.021). Similarly, in the TOPCAT cohort, the high-risk (vs. low-risk) phenogroup showed higher rates of heart failure hospitalization or cardiac death (HR: 1.92; 95% CI: 1.16 to 3.22; p = 0.01) and higher event-free survival with spironolactone therapy (HR: 0.65; 95% CI: 0.46 to 0.90; p = 0.01). In the pooled RELAX-HF/NEAT-HFpEF cohort, the high-risk (vs. low-risk) phenogroup had a higher burden of chronic myocardial injury (p < 0.001), neurohormonal activation (p < 0.001), and lower exercise capacity (p = 0.001). CONCLUSIONS This publicly available DeepNN classifier can characterize the severity of diastolic dysfunction and identify a specific subgroup of patients with HFpEF who have elevated left ventricular filling pressures, biomarkers of myocardial injury and stress, and adverse events and those who are more likely to respond to spironolactone.
Collapse
|
27
|
Sengupta PP, Shrestha S, Kagiyama N, Hamirani Y, Kulkarni H, Yanamala N, Bing R, Chin CWL, Pawade TA, Messika-Zeitoun D, Tastet L, Shen M, Newby DE, Clavel MA, Pibarot P, Dweck MR. A Machine-Learning Framework to Identify Distinct Phenotypes of Aortic Stenosis Severity. JACC Cardiovasc Imaging 2021; 14:1707-1720. [PMID: 34023273 DOI: 10.1016/j.jcmg.2021.03.020] [Citation(s) in RCA: 36] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/19/2020] [Revised: 03/02/2021] [Accepted: 03/16/2021] [Indexed: 12/20/2022]
Abstract
OBJECTIVES The authors explored the development and validation of machine-learning models for augmenting the echocardiographic grading of aortic stenosis (AS) severity. BACKGROUND In AS, symptoms and adverse events develop secondarily to valvular obstruction and left ventricular decompensation. The current echocardiographic grading of AS severity focuses on the valve and is limited by diagnostic uncertainty. METHODS Using echocardiography (ECHO) measurements (ECHO cohort, n = 1,052), we performed patient similarity analysis to derive high-severity and low-severity phenogroups of AS. We subsequently developed a supervised machine-learning classifier and validated its performance with independent markers of disease severity obtained using computed tomography (CT) (CT cohort, n = 752) and cardiovascular magnetic resonance (CMR) imaging (CMR cohort, n = 160). The classifier's prognostic value was further validated using clinical outcomes (aortic valve replacement [AVR] and death) observed in the ECHO and CMR cohorts. RESULTS In 1,964 patients from the 3 multi-institutional cohorts, 1,346 (68%) subjects had either nonsevere or discordant AS severity. Machine learning identified 1,117 (57%) patients as having high-severity and 847 (43%) as having low-severity AS. High-severity patients in CT and CMR cohorts had higher valve calcium scores and left ventricular mass and fibrosis, respectively than the low-severity group. In the ECHO cohort, progression to AVR and progression to death in patients who did not receive AVR was faster in the high-severity group. Compared with the conventional classification of disease severity, machine-learning-based severity classification improved discrimination (integrated discrimination improvement: 0.07; 95% confidence interval: 0.02 to 0.12) and reclassification (net reclassification improvement: 0.17; 95% confidence interval: 0.11 to 0.23) for the outcome of AVR at 5 years. For both ECHO and CMR cohorts, we observed prognostic value of the machine-learning classifications for subgroups with asymptomatic, nonsevere or discordant AS. CONCLUSIONS Machine learning can integrate ECHO measurements to augment the classification of disease severity in most patients with AS, with major potential to optimize the timing of AVR.
Collapse
|
28
|
Hathaway Q, Yanamala N, Budoff M, Sengupta P, Zeby I. CARDIOVASCULAR RISK STRATIFICATION THROUGH DEEP NEURAL SURVIVAL NETWORKS - THE MULTI-ETHNIC STUDY OF ATHEROSCLEROSIS (MESA). J Am Coll Cardiol 2021. [DOI: 10.1016/s0735-1097(21)01920-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
|
29
|
Yanamala N, Krishna NH, Hathaway Q, Radhakrishnan A, Sunkara S, Patel H, Farjo P, Patel B, Sengupta P. MACHINE-LEARNING CHARACTERIZATION OF SUBTLE HEMODYNAMIC INSTABILITY DIFFERENTIATES COVID-19 VERSUS SEASONAL INFLUENZA IN HOSPITALIZED PATIENTS. J Am Coll Cardiol 2021. [PMCID: PMC8091217 DOI: 10.1016/s0735-1097(21)04442-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
|
30
|
Patel H, Yanamala N, Tokodi M, Kagiyama N, Piccirilli M, Shrestha S, Farjo P, Casaclang-Verzosa G, Tarhuni W, Nezarat N, Budoff M, Narula J, Sengupta P. SURFACE ECG-BASED MACHINE LEARNING MODEL FOR PREDICTING PATIENT SUBGROUP AT A HIGH RISK FOR MAJOR ADVERSE CARDIAC EVENTS. J Am Coll Cardiol 2021. [DOI: 10.1016/s0735-1097(21)04582-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
|
31
|
Karunakaran KB, Yanamala N, Boyce G, Becich MJ, Ganapathiraju MK. Malignant Pleural Mesothelioma Interactome with 364 Novel Protein-Protein Interactions. Cancers (Basel) 2021; 13:1660. [PMID: 33916178 PMCID: PMC8037232 DOI: 10.3390/cancers13071660] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2021] [Revised: 03/19/2021] [Accepted: 03/22/2021] [Indexed: 12/20/2022] Open
Abstract
Malignant pleural mesothelioma (MPM) is an aggressive cancer affecting the outer lining of the lung, with a median survival of less than one year. We constructed an 'MPM interactome' with over 300 computationally predicted protein-protein interactions (PPIs) and over 2400 known PPIs of 62 literature-curated genes whose activity affects MPM. Known PPIs of the 62 MPM associated genes were derived from Biological General Repository for Interaction Datasets (BioGRID) and Human Protein Reference Database (HPRD). Novel PPIs were predicted by applying the HiPPIP algorithm, which computes features of protein pairs such as cellular localization, molecular function, biological process membership, genomic location of the gene, and gene expression in microarray experiments, and classifies the pairwise features as interacting or non-interacting based on a random forest model. We validated five novel predicted PPIs experimentally. The interactome is significantly enriched with genes differentially ex-pressed in MPM tumors compared with normal pleura and with other thoracic tumors, genes whose high expression has been correlated with unfavorable prognosis in lung cancer, genes differentially expressed on crocidolite exposure, and exosome-derived proteins identified from malignant mesothelioma cell lines. 28 of the interactors of MPM proteins are targets of 147 U.S. Food and Drug Administration (FDA)-approved drugs. By comparing disease-associated versus drug-induced differential expression profiles, we identified five potentially repurposable drugs, namely cabazitaxel, primaquine, pyrimethamine, trimethoprim and gliclazide. Preclinical studies may be con-ducted in vitro to validate these computational results. Interactome analysis of disease-associated genes is a powerful approach with high translational impact. It shows how MPM-associated genes identified by various high throughput studies are functionally linked, leading to clinically translatable results such as repurposed drugs. The PPIs are made available on a webserver with interactive user interface, visualization and advanced search capabilities.
Collapse
|
32
|
Yanamala N, Krishna NH, Hathaway QA, Radhakrishnan A, Sunkara S, Patel H, Farjo P, Patel B, Sengupta PP. A Vital Sign-based Prediction Algorithm for Differentiating COVID-19 Versus Seasonal Influenza in Hospitalized Patients. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2021:2021.01.13.21249540. [PMID: 33469602 PMCID: PMC7814848 DOI: 10.1101/2021.01.13.21249540] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
Patients with influenza and SARS-CoV2/Coronavirus disease 2019 (COVID-19) infections have different clinical course and outcomes. We developed and validated a supervised machine learning pipeline to distinguish the two viral infections using the available vital signs and demographic dataset from the first hospital/emergency room encounters of 3,883 patients who had confirmed diagnoses of influenza A/B, COVID-19 or negative laboratory test results. The models were able to achieve an area under the receiver operating characteristic curve (ROC AUC) of at least 97% using our multiclass classifier. The predictive models were externally validated on 15,697 encounters in 3,125 patients available on TrinetX database that contains patient-level data from different healthcare organizations. The influenza vs. COVID-19-positive model had an AUC of 98%, and 92% on the internal and external test sets, respectively. Our study illustrates the potentials of machine-learning models for accurately distinguishing the two viral infections. The code is made available at https://github.com/ynaveena/COVID-19-vs-Influenza and may be have utility as a frontline diagnostic tool to aid healthcare workers in triaging patients once the two viral infections start cocirculating in the communities.
Collapse
|
33
|
Brito D, Meester S, Yanamala N, Patel HB, Balcik BJ, Casaclang-Verzosa G, Seetharam K, Riveros D, Beto RJ, Balla S, Monseau AJ, Sengupta PP. High Prevalence of Pericardial Involvement in College Student Athletes Recovering From COVID-19. JACC Cardiovasc Imaging 2021; 14:541-555. [PMID: 33223496 PMCID: PMC7641597 DOI: 10.1016/j.jcmg.2020.10.023] [Citation(s) in RCA: 134] [Impact Index Per Article: 44.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/26/2020] [Revised: 10/27/2020] [Accepted: 10/29/2020] [Indexed: 02/06/2023]
Abstract
OBJECTIVES This study sought to explore the spectrum of cardiac abnormalities in student athletes who returned to university campus in July 2020 with uncomplicated coronavirus disease 2019 (COVID-19). BACKGROUND There is limited information on cardiovascular involvement in young individuals with mild or asymptomatic COVID-19. METHODS Screening echocardiograms were performed in 54 consecutive student athletes (mean age 19 years; 85% male) who had positive results of reverse transcription polymerase chain reaction nasal swab testing of the upper respiratory tract or immunoglobulin G antibodies against severe acute respiratory syndrome coronavirus type 2. Sequential cardiac magnetic resonance imaging was performed in 48 (89%) subjects. RESULTS A total of 16 (30%) athletes were asymptomatic, whereas 36 (66%) and 2 (4%) athletes reported mild and moderate COVID-19 related symptoms, respectively. For the 48 athletes completing both imaging studies, abnormal findings were identified in 27 (56.3%) individuals. This included 19 (39.5%) athletes with pericardial late enhancements with associated pericardial effusion. Of the individuals with pericardial enhancements, 6 (12.5%) had reduced global longitudinal strain and/or an increased native T1. One patient showed myocardial enhancement, and reduced left ventricular ejection fraction or reduced global longitudinal strain with or without increased native T1 values was also identified in an additional 7 (14.6%) individuals. Native T2 findings were normal in all subjects, and no specific imaging features of myocardial inflammation were identified. Hierarchical clustering of left ventricular regional strain identified 3 unique myopericardial phenotypes that showed significant association with the cardiac magnetic resonance findings (p = 0.03). CONCLUSIONS More than 1 in 3 previously healthy college athletes recovering from COVID-19 infection showed imaging features of a resolving pericardial inflammation. Although subtle changes in myocardial structure and function were identified, no athlete showed specific imaging features to suggest an ongoing myocarditis. Further studies are needed to understand the clinical implications and long-term evolution of these abnormalities in uncomplicated COVID-19.
Collapse
|
34
|
Kagiyama N, Piccirilli M, Yanamala N, Shrestha S, Farjo PD, Casaclang-Verzosa G, Tarhuni WM, Nezarat N, Budoff MJ, Narula J, Sengupta PP. Machine Learning Assessment of Left Ventricular Diastolic Function Based on Electrocardiographic Features. J Am Coll Cardiol 2021; 76:930-941. [PMID: 32819467 DOI: 10.1016/j.jacc.2020.06.061] [Citation(s) in RCA: 39] [Impact Index Per Article: 13.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/04/2020] [Accepted: 06/25/2020] [Indexed: 01/14/2023]
Abstract
BACKGROUND Left ventricular (LV) diastolic dysfunction is recognized as playing a major role in the pathophysiology of heart failure; however, clinical tools for identifying diastolic dysfunction before echocardiography remain imprecise. OBJECTIVES This study sought to develop machine-learning models that quantitatively estimate myocardial relaxation using clinical and electrocardiography (ECG) variables as a first step in the detection of LV diastolic dysfunction. METHODS A multicenter prospective study was conducted at 4 institutions in North America enrolling a total of 1,202 subjects. Patients from 3 institutions (n = 814) formed an internal cohort and were randomly divided into training and internal test sets (80:20). Machine-learning models were developed using signal-processed ECG, traditional ECG, and clinical features and were tested using the test set. Data from the fourth institution was reserved as an external test set (n = 388) to evaluate the model generalizability. RESULTS Despite diversity in subjects, the machine-learning model predicted the quantitative values of the LV relaxation velocities (e') measured by echocardiography in both internal and external test sets (mean absolute error: 1.46 and 1.93 cm/s; adjusted R2 = 0.57 and 0.46, respectively). Analysis of the area under the receiver operating characteristic curve (AUC) revealed that the estimated e' discriminated the guideline-recommended thresholds for abnormal myocardial relaxation and diastolic and systolic dysfunction (LV ejection fraction) the internal (area under the curve [AUC]: 0.83, 0.76, and 0.75) and external test sets (0.84, 0.80, and 0.81), respectively. Moreover, the estimated e' allowed prediction of LV diastolic dysfunction based on multiple age- and sex-adjusted reference limits (AUC: 0.88 and 0.94 in the internal and external sets, respectively). CONCLUSIONS A quantitative prediction of myocardial relaxation can be performed using easily obtained clinical and ECG features. This cost-effective strategy may be a valuable first clinical step for assessing the presence of LV dysfunction and may potentially aid in the early diagnosis and management of heart failure patients.
Collapse
|
35
|
Khaliullin TO, Kisin ER, Guppi S, Yanamala N, Zhernovkov V, Shvedova AA. Differential responses of murine alveolar macrophages to elongate mineral particles of asbestiform and non-asbestiform varieties: Cytotoxicity, cytokine secretion and transcriptional changes. Toxicol Appl Pharmacol 2020; 409:115302. [PMID: 33148505 DOI: 10.1016/j.taap.2020.115302] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2020] [Revised: 10/17/2020] [Accepted: 10/21/2020] [Indexed: 01/19/2023]
Abstract
Human exposures to asbestiform elongate mineral particles (EMP) may lead to diffuse fibrosis, lung cancer, malignant mesothelioma and autoimmune diseases. Cleavage fragments (CF) are chemically identical to asbestiform varieties (or habits) of the parent mineral, but no consensus exists on whether to treat them as asbestos from toxicological and regulatory standpoints. Alveolar macrophages (AM) are the first responders to inhaled particulates, participating in clearance and activating other resident and recruited immunocompetent cells, impacting the long-term outcomes. In this study we address how EMP of asbestiform versus non-asbestiform habit affect AM responses. Max Planck Institute (MPI) cells, a non-transformed mouse line that has an AM phenotype and genotype, were treated with mass-, surface area- (s.a.), and particle number- (p.n.) equivalent concentrations of respirable asbestiform and non-asbestiform riebeckite/tremolite EMP for 24 h. Cytotoxicity, cytokines secretion and transcriptional changes were evaluated. At the equal mass, asbestiform EMP were more cytotoxic, however EMP of both habits induced similar LDH leakage and decrease in viability at s.a. and p.n. equivalent doses. DNA damage assessment and cell cycle analysis revealed differences in the modes of cell death between asbestos and respective CF. There was an increase in chemokines, but not pro-inflammatory cytokines after all EMP treatments. Principal component analysis of the cytokine secretion showed close clustering for the s.a. and p.n. equivalent treatments. There were mineral- and habit-specific patterns of gene expression dysregulation at s.a. equivalent doses. Our study reveals the critical nature of EMP morphometric parameters for exposure assessment and dosing approaches used in toxicity studies.
Collapse
|
36
|
Fraser K, Kodali V, Yanamala N, Birch ME, Cena L, Casuccio G, Bunker K, Lersch TL, Evans DE, Stefaniak A, Hammer MA, Kashon ML, Boots T, Eye T, Hubczak J, Friend SA, Dahm M, Schubauer-Berigan MK, Siegrist K, Lowry D, Bauer AK, Sargent LM, Erdely A. Physicochemical characterization and genotoxicity of the broad class of carbon nanotubes and nanofibers used or produced in U.S. facilities. Part Fibre Toxicol 2020; 17:62. [PMID: 33287860 PMCID: PMC7720492 DOI: 10.1186/s12989-020-00392-w] [Citation(s) in RCA: 30] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2020] [Accepted: 11/18/2020] [Indexed: 12/23/2022] Open
Abstract
BACKGROUND Carbon nanotubes and nanofibers (CNT/F) have known toxicity but simultaneous comparative studies of the broad material class, especially those with a larger diameter, with computational analyses linking toxicity to their fundamental material characteristics was lacking. It was unclear if all CNT/F confer similar toxicity, in particular, genotoxicity. Nine CNT/F (MW #1-7 and CNF #1-2), commonly found in exposure assessment studies of U.S. facilities, were evaluated with reported diameters ranging from 6 to 150 nm. All materials were extensively characterized to include distributions of physical dimensions and prevalence of bundled agglomerates. Human bronchial epithelial cells were exposed to the nine CNT/F (0-24 μg/ml) to determine cell viability, inflammation, cellular oxidative stress, micronuclei formation, and DNA double-strand breakage. Computational modeling was used to understand various permutations of physicochemical characteristics and toxicity outcomes. RESULTS Analyses of the CNT/F physicochemical characteristics illustrate that using detailed distributions of physical dimensions provided a more consistent grouping of CNT/F compared to using particle dimension means alone. In fact, analysis of binning of nominal tube physical dimensions alone produced a similar grouping as all characterization parameters together. All materials induced epithelial cell toxicity and micronuclei formation within the dose range tested. Cellular oxidative stress, DNA double strand breaks, and micronuclei formation consistently clustered together and with larger physical CNT/F dimensions and agglomerate characteristics but were distinct from inflammatory protein changes. Larger nominal tube diameters, greater lengths, and bundled agglomerate characteristics were associated with greater severity of effect. The portion of tubes with greater nominal length and larger diameters within a sample was not the majority in number, meaning a smaller percentage of tubes with these characteristics was sufficient to increase toxicity. Many of the traditional physicochemical characteristics including surface area, density, impurities, and dustiness did not cluster with the toxicity outcomes. CONCLUSION Distributions of physical dimensions provided more consistent grouping of CNT/F with respect to toxicity outcomes compared to means only. All CNT/F induced some level of genotoxicity in human epithelial cells. The severity of toxicity was dependent on the sample containing a proportion of tubes with greater nominal lengths and diameters.
Collapse
|
37
|
Sager TM, Umbright CM, Mustafa GM, Yanamala N, Leonard HD, McKinney WG, Kashon ML, Joseph P. Tobacco Smoke Exposure Exacerbated Crystalline Silica-Induced Lung Toxicity in Rats. Toxicol Sci 2020; 178:375-390. [PMID: 32976597 PMCID: PMC7825013 DOI: 10.1093/toxsci/kfaa146] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022] Open
Abstract
Smoking may modify the lung response to silica exposure including cancer and silicosis. Nevertheless, the precise role of exposure to tobacco smoke (TS) on the lung response to crystalline silica (CS) exposure and the underlying mechanisms need further clarification. The objectives of the present study were to determine the role of TS on lung response to CS exposure and the underlying mechanism(s). Male Fischer 344 rats were exposed by inhalation to air, CS (15 mg/m3, 6 h/day, 5 days), TS (80 mg/m3, 3 h/day, twice weekly, 6 months), or CS (15 mg/m3, 6 h/day, 5 days) followed by TS (80 mg/m3, 3 h/day, twice weekly, 6 months). The rats were euthanized 6 months and 3 weeks following initiation of the first exposure and the lung response was assessed. Silica exposure resulted in significant lung toxicity as evidenced by lung histological changes, enhanced neutrophil infiltration, increased lactate dehydrogenase levels, enhanced oxidant production, and increased cytokine levels. The TS exposure alone had only a minimal effect on these toxicity parameters. However, the combined exposure to TS and CS exacerbated the lung response, compared with TS or CS exposure alone. Global gene expression changes in the lungs correlated with the lung toxicity severity. Bioinformatic analysis of the gene expression data demonstrated significant enrichment in functions, pathways, and networks relevant to the response to CS exposure which correlated with the lung toxicity detected. Collectively our data demonstrated an exacerbation of CS-induced lung toxicity by TS exposure and the molecular mechanisms underlying the exacerbated toxicity.
Collapse
|
38
|
Farjo PD, Yanamala N, Kagiyama N, Patel HB, Casaclang-Verzosa G, Nezarat N, Budoff MJ, Sengupta PP. Prediction of coronary artery calcium scoring from surface electrocardiogram in atherosclerotic cardiovascular disease: a pilot study. ACTA ACUST UNITED AC 2020; 1:51-61. [PMID: 37056293 PMCID: PMC10087019 DOI: 10.1093/ehjdh/ztaa008] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2020] [Revised: 10/06/2020] [Accepted: 10/13/2020] [Indexed: 02/07/2023]
Abstract
Abstract
Aims
Coronary artery calcium (CAC) scoring is an established tool for cardiovascular risk stratification. However, the lack of widespread availability and concerns about radiation exposure have limited the universal clinical utilization of CAC. In this study, we sought to explore whether machine learning (ML) approaches can aid cardiovascular risk stratification by predicting guideline recommended CAC score categories from clinical features and surface electrocardiograms.
Methods and results
In this substudy of a prospective, multicentre trial, a total of 534 subjects referred for CAC scores and electrocardiographic data were split into 80% training and 20% testing sets. Two binary outcome ML logistic regression models were developed for prediction of CAC scores equal to 0 and ≥400. Both CAC = 0 and CAC ≥400 models yielded values for the area under the curve, sensitivity, specificity, and accuracy of 84%, 92%, 70%, and 75%, and 87%, 91%, 75%, and 81%, respectively. We further tested the CAC ≥400 model to risk stratify a cohort of 87 subjects referred for invasive coronary angiography. Using an intermediate or higher pretest probability (≥15%) to predict CAC ≥400, the model predicted the presence of significant coronary artery stenosis (P = 0.025), the need for revascularization (P < 0.001), notably bypass surgery (P = 0.021), and major adverse cardiovascular events (P = 0.023) during a median follow-up period of 2 years.
Conclusion
ML techniques can extract information from electrocardiographic data and clinical variables to predict CAC score categories and similarly risk-stratify patients with suspected coronary artery disease.
Collapse
|
39
|
Sengupta PP, Shrestha S, Berthon B, Messas E, Donal E, Tison GH, Min JK, D'hooge J, Voigt JU, Dudley J, Verjans JW, Shameer K, Johnson K, Lovstakken L, Tabassian M, Piccirilli M, Pernot M, Yanamala N, Duchateau N, Kagiyama N, Bernard O, Slomka P, Deo R, Arnaout R. Proposed Requirements for Cardiovascular Imaging-Related Machine Learning Evaluation (PRIME): A Checklist: Reviewed by the American College of Cardiology Healthcare Innovation Council. JACC Cardiovasc Imaging 2020; 13:2017-2035. [PMID: 32912474 PMCID: PMC7953597 DOI: 10.1016/j.jcmg.2020.07.015] [Citation(s) in RCA: 105] [Impact Index Per Article: 26.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/18/2019] [Revised: 07/15/2020] [Accepted: 07/16/2020] [Indexed: 12/20/2022]
Abstract
Machine learning (ML) has been increasingly used within cardiology, particularly in the domain of cardiovascular imaging. Due to the inherent complexity and flexibility of ML algorithms, inconsistencies in the model performance and interpretation may occur. Several review articles have been recently published that introduce the fundamental principles and clinical application of ML for cardiologists. This paper builds on these introductory principles and outlines a more comprehensive list of crucial responsibilities that need to be completed when developing ML models. This paper aims to serve as a scientific foundation to aid investigators, data scientists, authors, editors, and reviewers involved in machine learning research with the intent of uniform reporting of ML investigations. An independent multidisciplinary panel of ML experts, clinicians, and statisticians worked together to review the theoretical rationale underlying 7 sets of requirements that may reduce algorithmic errors and biases. Finally, the paper summarizes a list of reporting items as an itemized checklist that highlights steps for ensuring correct application of ML models and the consistent reporting of model specifications and results. It is expected that the rapid pace of research and development and the increased availability of real-world evidence may require periodic updates to the checklist.
Collapse
|
40
|
Ramgopal S, Horvat CM, Yanamala N, Alpern ER. Machine Learning To Predict Serious Bacterial Infections in Young Febrile Infants. Pediatrics 2020; 146:e20194096. [PMID: 32855349 PMCID: PMC7461239 DOI: 10.1542/peds.2019-4096] [Citation(s) in RCA: 29] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 06/17/2020] [Indexed: 11/24/2022] Open
Abstract
BACKGROUND Recent decision rules for the management of febrile infants support the identification of infants at higher risk of serious bacterial infections (SBIs) without the performance of routine lumbar puncture. We derive and validate a model to identify febrile infants ≤60 days of age at low risk for SBIs using supervised machine learning approaches. METHODS We conducted a secondary analysis of a multicenter prospective study performed between December 2008 and May 2013 of febrile infants. Our outcome was SBI, (culture-positive urinary tract infection, bacteremia, and/or bacterial meningitis). We developed and validated 4 supervised learning models: logistic regression, random forest, support vector machine, and a single-hidden layer neural network. RESULTS A total of 1470 patients were included (1014 >28 days old). One hundred thirty-eight (9.3%) had SBIs (122 urinary tract infections, 20 bacteremia, and 8 meningitis; 11 with concurrent SBIs). Using 4 features (urinalysis, white blood cell count, absolute neutrophil count, and procalcitonin), we demonstrated with the random forest model the highest specificity (74.9, 95% confidence interval: 71.5%-78.2%) with a sensitivity of 98.6% (95% confidence interval: 92.2%-100.0%) in the validation cohort. One patient with bacteremia was misclassified. Among 1240 patients who received a lumbar puncture, this model could have prevented 849 (68.5%) such procedures. CONCLUSIONS We derived and internally validated a supervised learning model for the risk-stratification of febrile infants. Although computationally complex, lacking parameter cutoffs, and in need of external validation, this strategy may allow for reductions in unnecessary procedures, hospitalizations, and antibiotics while maintaining excellent sensitivity.
Collapse
|
41
|
Tarasov SA, Gorbunov EA, Don ES, Emelyanova AG, Kovalchuk AL, Yanamala N, Schleker ASS, Klein-Seetharaman J, Groenestein R, Tafani JP, van der Meide P, Epstein OI. Insights into the Mechanism of Action of Highly Diluted Biologics. THE JOURNAL OF IMMUNOLOGY 2020; 205:1345-1354. [PMID: 32727888 DOI: 10.4049/jimmunol.2000098] [Citation(s) in RCA: 29] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/29/2020] [Accepted: 07/04/2020] [Indexed: 12/12/2022]
Abstract
The therapeutic use of Abs in cancer, autoimmunity, transplantation, and other fields is among the major biopharmaceutical advances of the 20th century. Broader use of Ab-based drugs is constrained because of their high production costs and frequent side effects. One promising approach to overcome these limitations is the use of highly diluted Abs, which are produced by gradual reduction of an Ab concentration to an extremely low level. This technology was used to create a group of drugs for the treatment of various diseases, depending on the specificity of the used Abs. Highly diluted Abs to IFN-γ (hd-anti-IFN-γ) have been demonstrated to be efficacious against influenza and other respiratory infections in a variety of preclinical and clinical studies. In the current study, we provide evidence for a possible mechanism of action of hd-anti-IFN-γ. Using high-resolution solution nuclear magnetic resonance spectroscopy, we show that the drug induced conformational changes in the IFN-γ molecule. Chemical shift changes occurred in the amino acids located primarily at the dimer interface and at the C-terminal region of IFN-γ. These molecular changes could be crucial for the function of the protein, as evidenced by an observed hd-anti-IFN-γ-induced increase in the specific binding of IFN-γ to its receptor in U937 cells, enhanced induced production of IFN-γ in human PBMC culture, and increased survival of influenza A-infected mice.
Collapse
|
42
|
Kisin ER, Yanamala N, Rodin D, Menas A, Farcas M, Russo M, Guppi S, Khaliullin TO, Iavicoli I, Harper M, Star A, Kagan VE, Shvedova AA. Enhanced morphological transformation of human lung epithelial cells by continuous exposure to cellulose nanocrystals. CHEMOSPHERE 2020; 250:126170. [PMID: 32114335 PMCID: PMC7750788 DOI: 10.1016/j.chemosphere.2020.126170] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/15/2019] [Revised: 01/30/2020] [Accepted: 02/09/2020] [Indexed: 05/06/2023]
Abstract
Cellulose nanocrystals (CNC), also known as nanowhiskers, have recently gained much attention due to their biodegradable nature, advantageous chemical and mechanical properties, economic value and renewability thus making them attractive for a wide range of applications. However, before these materials can be considered for potential uses, investigation of their toxicity is prudent. Although CNC exposures are associated with pulmonary inflammation and damage as well as oxidative stress responses and genotoxicity in vivo, studies evaluating cell transformation or tumorigenic potential of CNC's were not previously conducted. In this study, we aimed to assess the neoplastic-like transformation potential of two forms of CNC derived from wood (powder and gel) in human pulmonary epithelial cells (BEAS-2B) in comparison to fibrous tremolite (TF), known to induce lung cancer. Short-term exposure to CNC or TF induced intracellular ROS increase and DNA damage while long-term exposure resulted in neoplastic-like transformation demonstrated by increased cell proliferation, anchorage-independent growth, migration and invasion. The increased proliferative responses were also in-agreement with observed levels of pro-inflammatory cytokines. Based on the hierarchical clustering analysis (HCA) of the inflammatory cytokine responses, CNC powder was segregated from the control and CNC-gel samples. This suggests that CNC may have the ability to influence neoplastic-like transformation events in pulmonary epithelial cells and that such effects are dependent on the type/form of CNC. Further studies focusing on determining and understanding molecular mechanisms underlying potential CNC cell transformation events and their likelihood to induce tumorigenic effects in vivo are highly warranted.
Collapse
|
43
|
Khaliullin TO, Yanamala N, Newman MS, Kisin ER, Fatkhutdinova LM, Shvedova AA. Comparative analysis of lung and blood transcriptomes in mice exposed to multi-walled carbon nanotubes. Toxicol Appl Pharmacol 2020; 390:114898. [PMID: 31978390 DOI: 10.1016/j.taap.2020.114898] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2019] [Revised: 01/16/2020] [Accepted: 01/21/2020] [Indexed: 12/16/2022]
Abstract
Pulmonary exposure to multi-walled carbon nanotubes (MWCNT) causes inflammation, fibroproliferation, immunotoxicity, and systemic responses in rodents. However, the search for representative biomarkers of exposure is an ongoing endeavor. Whole blood gene expression profiling is a promising new approach for the identification of novel disease biomarkers. We asked if the whole blood transcriptome reflects pathology-specific changes in lung gene expression caused by MWCNT. To answer this question, we performed mRNA sequencing analysis of the whole blood and lung in mice administered MWCNT or vehicle solution via pharyngeal aspiration and sacrificed 56 days later. The pattern of lung mRNA expression as determined using Ingenuity Pathway Analysis (IPA) was indicative of continued inflammation, immune cell trafficking, phagocytosis, and adaptive immune responses. Simultaneously, innate immunity-related transcripts (Plunc, Bpifb1, Reg3g) and cancer-related pathways were downregulated. IPA analysis of the differentially expressed genes in the whole blood suggested increased hematopoiesis, predicted activation of cancer/tumor development pathways, and atopy. There were several common upregulated genes between whole blood and lungs, important for adaptive immune responses: Cxcr1, Cd72, Sharpin, and Slc11a1. Trim24, important for TH2 cell effector function, was downregulated in both datasets. Hla-dqa1 mRNA was upregulated in the lungs and downregulated in the blood, as was Lilrb4, which controls the reactivity of immune response. "Cancer" disease category had opposing activation status in the two datasets, while the only commonality was "Hypersensitivity". Transcriptome changes occurring in the lungs did not produce a completely replicable pattern in whole blood; however, specific systemic responses may be shared between transcriptomic profiles.
Collapse
|
44
|
Mitchell J, Balem F, Tirupula K, Man D, Dhiman HK, Yanamala N, Ollesch J, Planas-Iglesias J, Jennings BJ, Gerwert K, Iannaccone A, Klein-Seetharaman J. Correction: Comparison of the molecular properties of retinitis pigmentosa P23H and N15S amino acid replacements in rhodopsin. PLoS One 2019; 14:e0225153. [PMID: 31697785 PMCID: PMC6837281 DOI: 10.1371/journal.pone.0225153] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022] Open
|
45
|
Yanamala N, Desai IC, Miller W, Kodali VK, Syamlal G, Roberts JR, Erdely AD. Grouping of carbonaceous nanomaterials based on association of patterns of inflammatory markers in BAL fluid with adverse outcomes in lungs. Nanotoxicology 2019; 13:1102-1116. [DOI: 10.1080/17435390.2019.1640911] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/07/2023]
|
46
|
Mitchell J, Balem F, Tirupula K, Man D, Dhiman HK, Yanamala N, Ollesch J, Planas-Iglesias J, Jennings BJ, Gerwert K, Iannaccone A, Klein-Seetharaman J. Comparison of the molecular properties of retinitis pigmentosa P23H and N15S amino acid replacements in rhodopsin. PLoS One 2019; 14:e0214639. [PMID: 31100078 PMCID: PMC6524802 DOI: 10.1371/journal.pone.0214639] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/06/2018] [Accepted: 03/19/2019] [Indexed: 12/16/2022] Open
Abstract
Mutations in the RHO gene encoding for the visual pigment protein, rhodopsin, are among the most common cause of autosomal dominant retinitis pigmentosa (ADRP). Previous studies of ADRP mutations in different domains of rhodopsin have indicated that changes that lead to more instability in rhodopsin structure are responsible for more severe disease in patients. Here, we further test this hypothesis by comparing side-by-side and therefore quantitatively two RHO mutations, N15S and P23H, both located in the N-terminal intradiscal domain. The in vitro biochemical properties of these two rhodopsin proteins, expressed in stably transfected tetracycline-inducible HEK293S cells, their UV-visible absorption, their Fourier transform infrared, circular dichroism and Metarhodopsin II fluorescence spectroscopy properties were characterized. As compared to the severely impaired P23H molecular function, N15S is only slightly defective in structure and stability. We propose that the molecular basis for these structural differences lies in the greater distance of the N15 residue as compared to P23 with respect to the predicted rhodopsin folding core. As described previously for WT rhodopsin, addition of the cytoplasmic allosteric modulator chlorin e6 stabilizes especially the P23H protein, suggesting that chlorin e6 may be generally beneficial in the rescue of those ADRP rhodopsin proteins whose stability is affected by amino acid replacement.
Collapse
|
47
|
Mitchell J, Yanamala N, Tan YL, Gardner EE, Tirupula KC, Balem F, Sheves M, Nietlispach D, Klein‐Seetharaman J. Structural and Functional Consequences of the Weak Binding of Chlorin e6 to Bovine Rhodopsin. Photochem Photobiol 2019; 95:787-802. [DOI: 10.1111/php.13074] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/09/2018] [Accepted: 12/07/2018] [Indexed: 12/14/2022]
|
48
|
Khaliullin TO, Kisin ER, Yanamala N, Guppi S, Harper M, Lee T, Shvedova AA. Comparative cytotoxicity of respirable surface-treated/untreated calcium carbonate rock dust particles in vitro. Toxicol Appl Pharmacol 2018; 362:67-76. [PMID: 30393145 DOI: 10.1016/j.taap.2018.10.023] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2018] [Revised: 10/20/2018] [Accepted: 10/23/2018] [Indexed: 01/17/2023]
Abstract
Calcium carbonate rock dust (RD) is used in mining to reduce the explosivity of aerosolized coal. During the dusting procedures, potential for human exposure occurs, raising health concerns. To improve RD aerosolization, several types of anti-caking surface treatments exist. The aim of the study was to evaluate cytotoxicity of four respirable RD samples: untreated/treated limestone (UL/TL), untreated/treated marble (UM/TM), and crystalline silica (SiO2) as a positive control in A549 and THP-1 transformed human cell lines. Respirable fractions were generated and collected using FSP10 high flow-rate cyclone samplers. THP-1 cells were differentiated with phorbol-12-myristate-13-acetate (20 ng/ml, 48 h). Cells were exposed to seven different concentrations of RD and SiO2 (0-0.2 mg/ml). RD caused a slight decrease in viability at 24 or 72 h post-exposure and were able to induce inflammatory cytokine production in A549 cells, however, with considerably less potency than SiO2. In THP-1 cells at 24 h, there was significant dose-dependent lactate dehydrogenase, inflammatory cytokine and chemokine release. Caspase-1 activity was increased in SiO2- and, on a lesser scale, in TM- exposed cells. To test if the increased toxicity of TM was uptake-related, THP-1 cells were pretreated with Cytochalasin D (CytD) or Bafilomycin A (BafA), followed by exposure to RD or SiO2 for 6 h. CytD blocked the uptake and significantly decreased cytotoxicity of all particles, while BafA prevented caspase-1 activation but not cytotoxic effects of TM. Only TM was able to induce an inflammatory response in THP-1 cells, however it was much less pronounced compared to silica.
Collapse
|
49
|
Park EJ, Khaliullin TO, Shurin MR, Kisin ER, Yanamala N, Fadeel B, Chang J, Shvedova AA. Fibrous nanocellulose, crystalline nanocellulose, carbon nanotubes, and crocidolite asbestos elicit disparate immune responses upon pharyngeal aspiration in mice. J Immunotoxicol 2018; 15:12-23. [PMID: 29237319 DOI: 10.1080/1547691x.2017.1414339] [Citation(s) in RCA: 33] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/20/2022] Open
Abstract
With the rapid development of synthetic alternatives to mineral fibers, their possible effects on the environment and human health have become recognized as important issues worldwide. This study investigated effects of four fibrous materials, i.e. nanofibrillar/nanocrystalline celluloses (NCF and CNC), single-walled carbon nanotubes (CNTs), and crocidolite asbestos (ASB), on pulmonary inflammation and immune responses found in the lungs, as well as the effects on spleen and peripheral blood immune cell subsets. BALB/c mice were given NCF, CNC, CNT, and ASB on Day 1 by oropharyngeal aspiration. At 14 days post-exposure, the animals were evaluated. Total cell number, mononuclear phagocytes, polymorphonuclear leukocytes, lymphocytes, and LDH levels were significantly increased in ASB and CNT-exposed mice. Expression of cytokines and chemokines in bronchoalveolar lavage (BAL) was quite different in mice exposed to four particle types, as well as expression of antigen presentation-related surface proteins on BAL cells. The results revealed that pulmonary exposure to fibrous materials led to discrete local immune cell polarization patterns with a TH2-like response caused by ASB and TH1-like immune reaction to NCF, while CNT and CNC caused non-classical or non-uniform responses. These alterations in immune response following pulmonary exposure should be taken into account when testing the applicability of new nanosized materials with fibrous morphology.
Collapse
|
50
|
Khaliullin TO, Kisin ER, Murray AR, Yanamala N, Shurin MR, Gutkin DW, Fatkhutdinova LM, Kagan VE, Shvedova AA. Mediation of the single-walled carbon nanotubes induced pulmonary fibrogenic response by osteopontin and TGF-β1. Exp Lung Res 2018; 43:311-326. [PMID: 29140132 DOI: 10.1080/01902148.2017.1377783] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/13/2023]
Abstract
PURPOSE OF THE STUDY A number of in vivo studies have shown that pulmonary exposure to carbon nanotubes (CNTs) may lead to an acute local inflammatory response, pulmonary fibrosis, and granulomatous lesions. Among the factors that play direct roles in initiation and progression of fibrotic processes are epithelial-mesenchymal transition and myofibroblasts recruitment/differentiation, both mediated by transforming growth factor-β1 (TGF-β1). Yet, other contributors to TGF-β1 associated signaling, such as osteopontin (OPN) has not been fully investigated. MATERIALS AND METHODS OPN-knockout female mice (OPN-KO) along with their wild-type (WT) counterparts were exposed to single-walled carbon nanotubes (SWCNT) (40 µg/mouse) via pharyngeal aspiration and fibrotic response was assessed 1, 7, and 28 days post-exposure. Simultaneously, RAW 264.7 and MLE-15 cells were treated with SWCNT (24 hours, 6 µg/cm2 to 48 µg/cm2) or bleomycin (0.1 µg/ml) in the presence of OPN-blocking antibody or isotype control, and TGF-β1 was measured in supernatants. RESULTS AND CONCLUSIONS Diminished lactate dehydrogenase activity at all time points, along with less pronounced neutrophil influx 24 h post-exposure, were measured in broncho-alveolar lavage (BAL) of OPN-KO mice compared to WT. Pro-inflammatory cytokine release (IL-6, TNF-α, MCP-1) was reduced. A significant two-fold increase of TGF-β1 was found in BAL of WT mice at 7 days, while TGF-β1 levels in OPN-KO animals remained unaltered. Histological examination revealed marked decrease in granuloma formation and less collagen deposition in the lungs of OPN-KO mice compared to WT. RAW 264.7 but not MLE-15 cells exposed to SWCNT and bleomycin had significantly less TGF-β1 released in the presence of OPN-blocking antibody. We believe that OPN is important in initiating the cellular mechanisms that produce an overall pathological response to SWCNT and it may act upstream of TGF-β1. Further investigation to understand the mechanistic details of such interactions is critical to predict outcomes of pulmonary exposure to CNT.
Collapse
|