1
|
Gu X, Myung Y, Rodrigues CHM, Ascher DB. EFG-CS: Predicting chemical shifts from amino acid sequences with protein structure prediction using machine learning and deep learning models. Protein Sci 2024; 33:e5096. [PMID: 38979954 PMCID: PMC11232051 DOI: 10.1002/pro.5096] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/17/2023] [Revised: 05/06/2024] [Accepted: 06/15/2024] [Indexed: 07/10/2024]
Abstract
Nuclear magnetic resonance (NMR) crystallography is one of the main methods in structural biology for analyzing protein stereochemistry and structure. The chemical shift of the resonance frequency reflects the effect of the protons in a molecule producing distinct NMR signals in different chemical environments. Apprehending chemical shifts from NMR signals can be challenging since having an NMR structure does not necessarily provide all the required chemical shift information, making predictive models essential for accurately deducing chemical shifts, either from protein structures or, more ideally, directly from amino acid sequences. Here, we present EFG-CS, a web server that specializes in chemical shift prediction. EFG-CS employs a machine learning-based transfer prediction model for backbone atom chemical shift prediction, using ESMFold-predicted protein structures. Additionally, ESG-CS incorporates a graph neural network-based model to provide comprehensive side-chain atom chemical shift predictions. Our method demonstrated reliable performance in backbone atom prediction, achieving comparable accuracy levels with root mean square errors (RMSE) of 0.30 ppm for H, 0.22 ppm for Hα, 0.89 ppm for C, 0.89 ppm for Cα, 0.84 ppm for Cβ, and 1.69 ppm for N. Moreover, our approach also showed predictive capabilities in side-chain atom chemical shift prediction achieving RMSE values of 0.71 ppm for Hβ, 0.74-1.15 ppm for Hδ, and 0.58-0.94 ppm for Hγ, solely utilizing amino acid sequences without homology or feature curation. This work shows for the first time that generative AI protein models can predict NMR shifts nearly comparable to experimental models. This web server is freely available at https://biosig.lab.uq.edu.au/efg_cs, and the chemical shift prediction results can be downloaded in tabular format and visualized in 3D format.
Collapse
Affiliation(s)
- Xiaotong Gu
- The Australian Centre for Ecogenomics, School of Chemistry and Molecular BiosciencesUniversity of QueenslandBrisbaneQueenslandAustralia
- Computational Biology and Clinical InformaticsBaker Heart and Diabetes InstituteMelbourneVictoriaAustralia
| | - Yoochan Myung
- The Australian Centre for Ecogenomics, School of Chemistry and Molecular BiosciencesUniversity of QueenslandBrisbaneQueenslandAustralia
- Computational Biology and Clinical InformaticsBaker Heart and Diabetes InstituteMelbourneVictoriaAustralia
| | - Carlos H. M. Rodrigues
- Computational Biology and Clinical InformaticsBaker Heart and Diabetes InstituteMelbourneVictoriaAustralia
| | - David B. Ascher
- The Australian Centre for Ecogenomics, School of Chemistry and Molecular BiosciencesUniversity of QueenslandBrisbaneQueenslandAustralia
- Computational Biology and Clinical InformaticsBaker Heart and Diabetes InstituteMelbourneVictoriaAustralia
| |
Collapse
|
2
|
Somberg NH, Gelenter MD, Hong M. Comparative analysis of 13C chemical shifts of β-sheet amyloid proteins and outer membrane proteins. JOURNAL OF BIOMOLECULAR NMR 2021; 75:151-166. [PMID: 33844106 PMCID: PMC8828358 DOI: 10.1007/s10858-021-00364-y] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/08/2021] [Accepted: 03/24/2021] [Indexed: 05/19/2023]
Abstract
Cross-β amyloid fibrils and membrane-bound β-barrels are two important classes of β-sheet proteins. To investigate whether there are systematic differences in the backbone and sidechain conformations of these two families of proteins, here we analyze the 13C chemical shifts of 17 amyloid proteins and 7 β-barrel membrane proteins whose high-resolution structures have been determined by NMR. These 24 proteins contain 373 β-sheet residues in amyloid fibrils and 521 β-sheet residues in β-barrel membrane proteins. The 13C chemical shifts are shown in 2D 13C-13C correlation maps, and the amino acid residues are categorized by two criteria: (1) whether they occur in β-strand segments or in loops and turns; (2) whether they are water-exposed or dry, facing other residues or lipids. We also examine the abundance of each amino acid in amyloid proteins and β-barrels and compare the sidechain rotameric populations. The 13C chemical shifts indicate that hydrophobic methyl-rich residues and aromatic residues exhibit larger static sidechain conformational disorder in amyloid fibrils than in β-barrels. In comparison, hydroxyl- and amide-containing polar residues have more ordered sidechains and more ordered backbones in amyloid fibrils than in β-barrels. These trends can be explained by steric zipper interactions between β-sheet planes in cross-β fibrils, and by the interactions of β-barrel residues with lipid and water in the membrane. These conformational trends should be useful for structural analysis of amyloid fibrils and β-barrels based principally on NMR chemical shifts.
Collapse
Affiliation(s)
- Noah H Somberg
- Department of Chemistry, Massachusetts Institute of Technology, 170 Albany Street, Cambridge, MA, 02139, USA
| | - Martin D Gelenter
- Department of Chemistry, Massachusetts Institute of Technology, 170 Albany Street, Cambridge, MA, 02139, USA
| | - Mei Hong
- Department of Chemistry, Massachusetts Institute of Technology, 170 Albany Street, Cambridge, MA, 02139, USA.
| |
Collapse
|
3
|
Li J, Bennett KC, Liu Y, Martin MV, Head-Gordon T. Accurate prediction of chemical shifts for aqueous protein structure on "Real World" data. Chem Sci 2020; 11:3180-3191. [PMID: 34122823 PMCID: PMC8152569 DOI: 10.1039/c9sc06561j] [Citation(s) in RCA: 26] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/29/2019] [Accepted: 03/02/2020] [Indexed: 02/04/2023] Open
Abstract
Here we report a new machine learning algorithm for protein chemical shift prediction that outperforms existing chemical shift calculators on realistic data that is not heavily curated, nor eliminates test predictions ad hoc. Our UCBShift predictor implements two modules: a transfer prediction module that employs both sequence and structural alignment to select reference candidates for experimental chemical shift replication, and a redesigned machine learning module based on random forest regression which utilizes more, and more carefully curated, feature extracted data. When combined together, this new predictor achieves state-of-the-art accuracy for predicting chemical shifts on a randomly selected dataset without careful curation, with root-mean-square errors of 0.31 ppm for amide hydrogens, 0.19 ppm for Hα, 0.84 ppm for C', 0.81 ppm for Cα, 1.00 ppm for Cβ, and 1.81 ppm for N. When similar sequences or structurally related proteins are available, UCBShift shows superior native state selection from misfolded decoy sets compared to SPARTA+ and SHIFTX2, and even without homology we exceed current prediction accuracy of all other popular chemical shift predictors.
Collapse
Affiliation(s)
- Jie Li
- Pitzer Center for Theoretical Chemistry, University of California Berkeley CA 94720 USA
- Department of Chemistry, University of California Berkeley CA 94720 USA
| | - Kochise C Bennett
- Pitzer Center for Theoretical Chemistry, University of California Berkeley CA 94720 USA
- Department of Chemistry, University of California Berkeley CA 94720 USA
| | - Yuchen Liu
- Pitzer Center for Theoretical Chemistry, University of California Berkeley CA 94720 USA
- Department of Chemistry, University of California Berkeley CA 94720 USA
| | - Michael V Martin
- Department of Bioengineering, University of California Berkeley CA 94720 USA
| | - Teresa Head-Gordon
- Pitzer Center for Theoretical Chemistry, University of California Berkeley CA 94720 USA
- Department of Chemistry, University of California Berkeley CA 94720 USA
- Department of Bioengineering, University of California Berkeley CA 94720 USA
- Department of Chemical and Biomolecular Engineering, University of California Berkeley CA 94720 USA
| |
Collapse
|
4
|
Auto-encoding NMR chemical shifts from their native vector space to a residue-level biophysical index. Nat Commun 2019; 10:2511. [PMID: 31175284 PMCID: PMC6555786 DOI: 10.1038/s41467-019-10322-w] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2018] [Accepted: 05/01/2019] [Indexed: 11/26/2022] Open
Abstract
Chemical shifts (CS) are determined from NMR experiments and represent the resonance frequency of the spin of atoms in a magnetic field. They contain a mixture of information, encompassing the in-solution conformations a protein adopts, as well as the movements it performs. Due to their intrinsically multi-faceted nature, CS are difficult to interpret and visualize. Classical approaches for the analysis of CS aim to extract specific protein-related properties, thus discarding a large amount of information that cannot be directly linked to structural features of the protein. Here we propose an autoencoder-based method, called ShiftCrypt, that provides a way to analyze, compare and interpret CS in their native, multidimensional space. We show that ShiftCrypt conserves information about the most common structural features. In addition, it can be used to identify hidden similarities between diverse proteins and peptides, and differences between the same protein in two different binding states. NMR chemical shift information is highly valuable in the investigation of small molecule and protein structure. Here, the authors developed a neural network approach to unify protein chemical shifts and their changes in response to changes in protein sequence, structure, and dimerization interactions.
Collapse
|
5
|
Hafsa NE, Berjanskii MV, Arndt D, Wishart DS. Rapid and reliable protein structure determination via chemical shift threading. JOURNAL OF BIOMOLECULAR NMR 2018; 70:33-51. [PMID: 29196969 DOI: 10.1007/s10858-017-0154-1] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/15/2017] [Accepted: 11/14/2017] [Indexed: 06/07/2023]
Abstract
Protein structure determination using nuclear magnetic resonance (NMR) spectroscopy can be both time-consuming and labor intensive. Here we demonstrate how chemical shift threading can permit rapid, robust, and accurate protein structure determination using only chemical shift data. Threading is a relatively old bioinformatics technique that uses a combination of sequence information and predicted (or experimentally acquired) low-resolution structural data to generate high-resolution 3D protein structures. The key motivations behind using NMR chemical shifts for protein threading lie in the fact that they are easy to measure, they are available prior to 3D structure determination, and they contain vital structural information. The method we have developed uses not only sequence and chemical shift similarity but also chemical shift-derived secondary structure, shift-derived super-secondary structure, and shift-derived accessible surface area to generate a high quality protein structure regardless of the sequence similarity (or lack thereof) to a known structure already in the PDB. The method (called E-Thrifty) was found to be very fast (often < 10 min/structure) and to significantly outperform other shift-based or threading-based structure determination methods (in terms of top template model accuracy)-with an average TM-score performance of 0.68 (vs. 0.50-0.62 for other methods). Coupled with recent developments in chemical shift refinement, these results suggest that protein structure determination, using only NMR chemical shifts, is becoming increasingly practical and reliable. E-Thrifty is available as a web server at http://ethrifty.ca .
Collapse
Affiliation(s)
- Noor E Hafsa
- Department of Computing Science, University of Alberta, Edmonton, AB, T6G 2E8, Canada
| | - Mark V Berjanskii
- Department of Biological Sciences, University of Alberta, Edmonton, AB, T6G 2E9, Canada
| | - David Arndt
- Department of Biological Sciences, University of Alberta, Edmonton, AB, T6G 2E9, Canada
| | - David S Wishart
- Department of Computing Science, University of Alberta, Edmonton, AB, T6G 2E8, Canada.
- Department of Biological Sciences, University of Alberta, Edmonton, AB, T6G 2E9, Canada.
| |
Collapse
|
6
|
van den Boom J, Trusch F, Hoppstock L, Beuck C, Bayer P. Structural Characterization of the Loop at the Alpha-Subunit C-Terminus of the Mixed Lineage Leukemia Protein Activating Protease Taspase1. PLoS One 2016; 11:e0151431. [PMID: 26974973 PMCID: PMC4790943 DOI: 10.1371/journal.pone.0151431] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2016] [Accepted: 02/29/2016] [Indexed: 11/27/2022] Open
Abstract
Type 2 asparaginases, a subfamily of N-terminal nucleophile (Ntn) hydrolases, are activated by limited proteolysis. This activation yields a heterodimer and a loop region at the C-terminus of the α-subunit is released. Since this region is unresolved in all type 2 asparaginase crystal structures but is close to the active site residues, we explored this loop region in six members of the type 2 asparaginase family using homology modeling. As the loop model for the childhood cancer-relevant protease Taspase1 differed from the other members, Taspase1 activation as well as the conformation and dynamics of the 56 amino acids loop were investigated by CD and NMR spectroscopy. We propose a helix-turn-helix motif, which can be exploited as novel anticancer target to inhibit Taspase1 proteolytic activity.
Collapse
Affiliation(s)
- Johannes van den Boom
- Department of Structural and Medicinal Biochemistry, Centre for Medical Biotechnology (ZMB), University of Duisburg-Essen, Essen, Germany
| | - Franziska Trusch
- Aberdeen Oomycetes Laboratory, Institute of Medical Sciences, University of Aberdeen, Aberdeen, United Kingdom
| | - Lukas Hoppstock
- Department of Structural and Medicinal Biochemistry, Centre for Medical Biotechnology (ZMB), University of Duisburg-Essen, Essen, Germany
| | - Christine Beuck
- Department of Structural and Medicinal Biochemistry, Centre for Medical Biotechnology (ZMB), University of Duisburg-Essen, Essen, Germany
| | - Peter Bayer
- Department of Structural and Medicinal Biochemistry, Centre for Medical Biotechnology (ZMB), University of Duisburg-Essen, Essen, Germany
- * E-mail:
| |
Collapse
|
7
|
Sikdar S, Chakrabarti J, Ghosh M. Conformational thermodynamics guided structural reconstruction of biomolecular fragments. MOLECULAR BIOSYSTEMS 2016; 12:444-53. [DOI: 10.1039/c5mb00529a] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/21/2022]
Abstract
Conformational thermodynamics compares the modeling protocols to identify the conformation of the missing region leading to a suitable model for metal ion free (apo) skeletal muscle Troponin C.
Collapse
Affiliation(s)
- Samapan Sikdar
- Department of Chemical
- Biological and Macromolecular Sciences
- S. N. Bose National Centre for Basic Sciences
- Salt Lake
- India
| | - J. Chakrabarti
- Department of Chemical
- Biological and Macromolecular Sciences
- S. N. Bose National Centre for Basic Sciences
- Salt Lake
- India
| | - Mahua Ghosh
- Department of Chemical
- Biological and Macromolecular Sciences
- S. N. Bose National Centre for Basic Sciences
- Salt Lake
- India
| |
Collapse
|
8
|
Dashti H, Tonelli M, Lee W, Westler WM, Cornilescu G, Ulrich EL, Markley JL. Probabilistic validation of protein NMR chemical shift assignments. JOURNAL OF BIOMOLECULAR NMR 2016; 64:17-25. [PMID: 26724815 PMCID: PMC4744101 DOI: 10.1007/s10858-015-0007-8] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/06/2015] [Accepted: 12/20/2015] [Indexed: 05/05/2023]
Abstract
Data validation plays an important role in ensuring the reliability and reproducibility of studies. NMR investigations of the functional properties, dynamics, chemical kinetics, and structures of proteins depend critically on the correctness of chemical shift assignments. We present a novel probabilistic method named ARECA for validating chemical shift assignments that relies on the nuclear Overhauser effect data . ARECA has been evaluated through its application to 26 case studies and has been shown to be complementary to, and usually more reliable than, approaches based on chemical shift databases. ARECA is available online at http://areca.nmrfam.wisc.edu/.
Collapse
Affiliation(s)
- Hesam Dashti
- Graduate Program in Biophysics, Biochemistry Department, University of Wisconsin-Madison, Madison, WI, USA
- Biochemistry Department, National Magnetic Resonance Facility at Madison, University of Wisconsin-Madison, Madison, WI, USA
| | - Marco Tonelli
- Biochemistry Department, National Magnetic Resonance Facility at Madison, University of Wisconsin-Madison, Madison, WI, USA
| | - Woonghee Lee
- Biochemistry Department, National Magnetic Resonance Facility at Madison, University of Wisconsin-Madison, Madison, WI, USA
| | - William M Westler
- Biochemistry Department, National Magnetic Resonance Facility at Madison, University of Wisconsin-Madison, Madison, WI, USA
| | - Gabriel Cornilescu
- Biochemistry Department, National Magnetic Resonance Facility at Madison, University of Wisconsin-Madison, Madison, WI, USA
| | - Eldon L Ulrich
- BioMagResBank, Biochemistry Department, University of Wisconsin-Madison, Madison, WI, USA
| | - John L Markley
- Biochemistry Department, National Magnetic Resonance Facility at Madison, University of Wisconsin-Madison, Madison, WI, USA.
- BioMagResBank, Biochemistry Department, University of Wisconsin-Madison, Madison, WI, USA.
| |
Collapse
|
9
|
Dashti H, Lee W, Tonelli M, Cornilescu CC, Cornilescu G, Assadi-Porter FM, Westler WM, Eghbalnia HR, Markley JL. NMRFAM-SDF: a protein structure determination framework. JOURNAL OF BIOMOLECULAR NMR 2015; 62:481-95. [PMID: 25900069 PMCID: PMC4569665 DOI: 10.1007/s10858-015-9933-8] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/16/2015] [Accepted: 04/15/2015] [Indexed: 05/21/2023]
Abstract
The computationally demanding nature of automated NMR structure determination necessitates a delicate balancing of factors that include the time complexity of data collection, the computational complexity of chemical shift assignments, and selection of proper optimization steps. During the past two decades the computational and algorithmic aspects of several discrete steps of the process have been addressed. Although no single comprehensive solution has emerged, the incorporation of a validation protocol has gained recognition as a necessary step for a robust automated approach. The need for validation becomes even more pronounced in cases of proteins with higher structural complexity, where potentially larger errors generated at each step can propagate and accumulate in the process of structure calculation, thereby significantly degrading the efficacy of any software framework. This paper introduces a complete framework for protein structure determination with NMR--from data acquisition to the structure determination. The aim is twofold: to simplify the structure determination process for non-NMR experts whenever feasible, while maintaining flexibility by providing a set of modules that validate each step, and to enable the assessment of error propagations. This framework, called NMRFAM-SDF (NMRFAM-Structure Determination Framework), and its various components are available for download from the NMRFAM website (http://nmrfam.wisc.edu/software.htm).
Collapse
Affiliation(s)
- Hesam Dashti
- National Magnetic Resonance Facility at Madison, Biochemistry Department, University of Wisconsin-Madison, 433 Babcock Drive, Madison, WI, USA
| | - Woonghee Lee
- National Magnetic Resonance Facility at Madison, Biochemistry Department, University of Wisconsin-Madison, 433 Babcock Drive, Madison, WI, USA
| | - Marco Tonelli
- National Magnetic Resonance Facility at Madison, Biochemistry Department, University of Wisconsin-Madison, 433 Babcock Drive, Madison, WI, USA
| | - Claudia C Cornilescu
- National Magnetic Resonance Facility at Madison, Biochemistry Department, University of Wisconsin-Madison, 433 Babcock Drive, Madison, WI, USA
| | - Gabriel Cornilescu
- National Magnetic Resonance Facility at Madison, Biochemistry Department, University of Wisconsin-Madison, 433 Babcock Drive, Madison, WI, USA
| | - Fariba M Assadi-Porter
- National Magnetic Resonance Facility at Madison, Biochemistry Department, University of Wisconsin-Madison, 433 Babcock Drive, Madison, WI, USA
| | - William M Westler
- National Magnetic Resonance Facility at Madison, Biochemistry Department, University of Wisconsin-Madison, 433 Babcock Drive, Madison, WI, USA
| | - Hamid R Eghbalnia
- National Magnetic Resonance Facility at Madison, Biochemistry Department, University of Wisconsin-Madison, 433 Babcock Drive, Madison, WI, USA
| | - John L Markley
- National Magnetic Resonance Facility at Madison, Biochemistry Department, University of Wisconsin-Madison, 433 Babcock Drive, Madison, WI, USA.
| |
Collapse
|
10
|
Hafsa NE, Arndt D, Wishart DS. Accessible surface area from NMR chemical shifts. JOURNAL OF BIOMOLECULAR NMR 2015; 62:387-401. [PMID: 26078090 DOI: 10.1007/s10858-015-9957-0] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/03/2015] [Accepted: 06/09/2015] [Indexed: 06/04/2023]
Abstract
Accessible surface area (ASA) is the surface area of an atom, amino acid or biomolecule that is exposed to solvent. The calculation of a molecule's ASA requires three-dimensional coordinate data and the use of a "rolling ball" algorithm to both define and calculate the ASA. For polymers such as proteins, the ASA for individual amino acids is closely related to the hydrophobicity of the amino acid as well as its local secondary and tertiary structure. For proteins, ASA is a structural descriptor that can often be as informative as secondary structure. Consequently there has been considerable effort over the past two decades to try to predict ASA from protein sequence data and to use ASA information (derived from chemical modification studies) as a structure constraint. Recently it has become evident that protein chemical shifts are also sensitive to ASA. Given the potential utility of ASA estimates as structural constraints for NMR we decided to explore this relationship further. Using machine learning techniques (specifically a boosted tree regression model) we developed an algorithm called "ShiftASA" that combines chemical-shift and sequence derived features to accurately estimate per-residue fractional ASA values of water-soluble proteins. This method showed a correlation coefficient between predicted and experimental values of 0.79 when evaluated on a set of 65 independent test proteins, which was an 8.2 % improvement over the next best performing (sequence-only) method. On a separate test set of 92 proteins, ShiftASA reported a mean correlation coefficient of 0.82, which was 12.3 % better than the next best performing method. ShiftASA is available as a web server ( http://shiftasa.wishartlab.com ) for submitting input queries for fractional ASA calculation.
Collapse
Affiliation(s)
- Noor E Hafsa
- Department of Computing Science, University of Alberta, Edmonton, Canada
| | | | | |
Collapse
|
11
|
van der Schot G, Zhang Z, Vernon R, Shen Y, Vranken WF, Baker D, Bonvin AMJJ, Lange OF. Improving 3D structure prediction from chemical shift data. JOURNAL OF BIOMOLECULAR NMR 2013; 57:27-35. [PMID: 23912841 DOI: 10.1007/s10858-013-9762-6] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/23/2013] [Accepted: 07/16/2013] [Indexed: 05/22/2023]
Abstract
We report advances in the calculation of protein structures from chemical shift nuclear magnetic resonance data alone. Our previously developed method, CS-Rosetta, assembles structures from a library of short protein fragments picked from a large library of protein structures using chemical shifts and sequence information. Here we demonstrate that combination of a new and improved fragment picker and the iterative sampling algorithm RASREC yield significant improvements in convergence and accuracy. Moreover, we introduce improved criteria for assessing the accuracy of the models produced by the method. The method was tested on 39 proteins in the 50-100 residue size range and yields reliable structures in 70 % of the cases. All structures that passed the reliability filter were accurate (<2 Å RMSD from the reference).
Collapse
Affiliation(s)
- Gijs van der Schot
- Computational Structural Biology, Bijvoet Center for Biomolecular Research, Faculty of Science-Chemistry, Utrecht University, Padualaan 8, 3584 CH, Utrecht, The Netherlands
| | | | | | | | | | | | | | | |
Collapse
|
12
|
Gutmanas A, Oldfield TJ, Patwardhan A, Sen S, Velankar S, Kleywegt GJ. The role of structural bioinformatics resources in the era of integrative structural biology. ACTA CRYSTALLOGRAPHICA. SECTION D, BIOLOGICAL CRYSTALLOGRAPHY 2013; 69:710-21. [PMID: 23633580 PMCID: PMC3640467 DOI: 10.1107/s0907444913001157] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/06/2012] [Accepted: 01/11/2013] [Indexed: 11/10/2022]
Abstract
The history and the current state of the PDB and EMDB archives is briefly described, as well as some of the challenges that they face. It seems natural that the role of structural biology archives will change from being a pure repository of historic data into becoming an indispensable resource for the wider biomedical community. As part of this transformation, it will be necessary to validate the biomacromolecular structure data and ensure the highest possible quality for the archive holdings, to combine structural data from different spatial scales into a unified resource and to integrate structural data with functional, genetic and taxonomic data as well as other information available in bioinformatics resources. Some recent developments and plans to address these challenges at PDBe are presented.
Collapse
Affiliation(s)
- Aleksandras Gutmanas
- Protein Data Bank in Europe, EMBL–EBI, Wellcome Trust Genome Campus, Hinxton, Cambridge CB10 1SD, England
| | - Thomas J. Oldfield
- Protein Data Bank in Europe, EMBL–EBI, Wellcome Trust Genome Campus, Hinxton, Cambridge CB10 1SD, England
| | - Ardan Patwardhan
- Protein Data Bank in Europe, EMBL–EBI, Wellcome Trust Genome Campus, Hinxton, Cambridge CB10 1SD, England
| | - Sanchayita Sen
- Protein Data Bank in Europe, EMBL–EBI, Wellcome Trust Genome Campus, Hinxton, Cambridge CB10 1SD, England
| | - Sameer Velankar
- Protein Data Bank in Europe, EMBL–EBI, Wellcome Trust Genome Campus, Hinxton, Cambridge CB10 1SD, England
| | - Gerard J. Kleywegt
- Protein Data Bank in Europe, EMBL–EBI, Wellcome Trust Genome Campus, Hinxton, Cambridge CB10 1SD, England
| |
Collapse
|
13
|
Sahakyan AB. Computational studies of dielectric permittivity effects on chemical shifts of alanine dipeptide. Chem Phys Lett 2012. [DOI: 10.1016/j.cplett.2012.07.069] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/30/2022]
|
14
|
Nielsen JT, Eghbalnia HR, Nielsen NC. Chemical shift prediction for protein structure calculation and quality assessment using an optimally parameterized force field. PROGRESS IN NUCLEAR MAGNETIC RESONANCE SPECTROSCOPY 2012; 60:1-28. [PMID: 22293396 PMCID: PMC3270304 DOI: 10.1016/j.pnmrs.2011.05.002] [Citation(s) in RCA: 18] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/25/2010] [Accepted: 05/09/2011] [Indexed: 05/05/2023]
Abstract
The exquisite sensitivity of chemical shifts as reporters of structural information, and the ability to measure them routinely and accurately, gives great import to formulations that elucidate the structure-chemical-shift relationship. Here we present a new and highly accurate, precise, and robust formulation for the prediction of NMR chemical shifts from protein structures. Our approach, shAIC (shift prediction guided by Akaikes Information Criterion), capitalizes on mathematical ideas and an information-theoretic principle, to represent the functional form of the relationship between structure and chemical shift as a parsimonious sum of smooth analytical potentials which optimally takes into account short-, medium-, and long-range parameters in a nuclei-specific manner to capture potential chemical shift perturbations caused by distant nuclei. shAIC outperforms the state-of-the-art methods that use analytical formulations. Moreover, for structures derived by NMR or structures with novel folds, shAIC delivers better overall results; even when it is compared to sophisticated machine learning approaches. shAIC provides for a computationally lightweight implementation that is unimpeded by molecular size, making it an ideal for use as a force field.
Collapse
Affiliation(s)
- Jakob T. Nielsen
- Center for Insoluble Protein Structures (inSPIN), Interdisciplinary Nanoscience Center (iNANO) and Department of Chemistry, Aarhus University, DK-8000 Aarhus C, Denmark
| | - Hamid R. Eghbalnia
- Department of Molecular and Cellular Physiology, University of Cincinnati, 231 Albert B. Sabin Way, Cincinnati, OH 45267-0576, United States
| | - Niels Chr. Nielsen
- Center for Insoluble Protein Structures (inSPIN), Interdisciplinary Nanoscience Center (iNANO) and Department of Chemistry, Aarhus University, DK-8000 Aarhus C, Denmark
| |
Collapse
|
15
|
Kjaergaard M, Poulsen FM. Disordered proteins studied by chemical shifts. PROGRESS IN NUCLEAR MAGNETIC RESONANCE SPECTROSCOPY 2012; 60:42-51. [PMID: 22293398 DOI: 10.1016/j.pnmrs.2011.10.001] [Citation(s) in RCA: 44] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/19/2011] [Accepted: 10/06/2011] [Indexed: 05/31/2023]
Affiliation(s)
- Magnus Kjaergaard
- Department of Biology, University of Copenhagen, Ole Maaløes Vej 5, DK-2200 København N, Denmark
| | | |
Collapse
|
16
|
Insaidoo FK, Borbulevych OY, Hossain M, Santhanagopolan SM, Baxter TK, Baker BM. Loss of T cell antigen recognition arising from changes in peptide and major histocompatibility complex protein flexibility: implications for vaccine design. J Biol Chem 2011; 286:40163-73. [PMID: 21937447 DOI: 10.1074/jbc.m111.283564] [Citation(s) in RCA: 43] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022] Open
Abstract
Modification of the primary anchor positions of antigenic peptides to improve binding to major histocompatibility complex (MHC) proteins is a commonly used strategy for engineering peptide-based vaccine candidates. However, such peptide modifications do not always improve antigenicity, complicating efforts to design effective vaccines for cancer and infectious disease. Here we investigated the MART-1(27-35) tumor antigen, for which anchor modification (replacement of the position two alanine with leucine) dramatically reduces or ablates antigenicity with a wide range of T cell clones despite significantly improving peptide binding to MHC. We found that anchor modification in the MART-1(27-35) antigen enhances the flexibility of both the peptide and the HLA-A*0201 molecule. Although the resulting entropic effects contribute to the improved binding of the peptide to MHC, they also negatively impact T cell receptor binding to the peptide·MHC complex. These results help explain how the "anchor-fixing" strategy fails to improve antigenicity in this case, and more generally, may be relevant for understanding the high specificity characteristic of the T cell repertoire. In addition to impacting vaccine design, modulation of peptide and MHC flexibility through changes to antigenic peptides may present an evolutionary strategy for the escape of pathogens from immune destruction.
Collapse
Affiliation(s)
- Francis K Insaidoo
- Department of Chemistry and Biochemistry and the Harper Cancer Research Institute, University of Notre Dame, Notre Dame, Indiana 46556, USA
| | | | | | | | | | | |
Collapse
|
17
|
Sahakyan AB, Vranken WF, Cavalli A, Vendruscolo M. Structure-based prediction of methyl chemical shifts in proteins. JOURNAL OF BIOMOLECULAR NMR 2011; 50:331-46. [PMID: 21748266 DOI: 10.1007/s10858-011-9524-2] [Citation(s) in RCA: 40] [Impact Index Per Article: 3.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/18/2011] [Accepted: 05/17/2011] [Indexed: 05/07/2023]
Abstract
Protein methyl groups have recently been the subject of much attention in NMR spectroscopy because of the opportunities that they provide to obtain information about the structure and dynamics of proteins and protein complexes. With the advent of selective labeling schemes, methyl groups are particularly interesting in the context of chemical shift based protein structure determination, an approach that to date has exploited primarily the mapping between protein structures and backbone chemical shifts. In order to extend the scope of chemical shifts for structure determination, we present here the CH3Shift method of performing structure-based predictions of methyl chemical shifts. The terms considered in the predictions take account of ring current, magnetic anisotropy, electric field, rotameric type, and dihedral angle effects, which are considered in conjunction with polynomial functions of interatomic distances. We show that the CH3Shift method achieves an accuracy in the predictions that ranges from 0.133 to 0.198 ppm for (1)H chemical shifts for Ala, Thr, Val, Leu and Ile methyl groups. We illustrate the use of the method by assessing the accuracy of side-chain structures in structural ensembles representing the dynamics of proteins.
Collapse
Affiliation(s)
- Aleksandr B Sahakyan
- Department of Chemistry, University of Cambridge, Lensfield Road, Cambridge, CB2 1EW, UK
| | | | | | | |
Collapse
|
18
|
Han B, Liu Y, Ginzinger SW, Wishart DS. SHIFTX2: significantly improved protein chemical shift prediction. JOURNAL OF BIOMOLECULAR NMR 2011; 50:43-57. [PMID: 21448735 PMCID: PMC3085061 DOI: 10.1007/s10858-011-9478-4] [Citation(s) in RCA: 498] [Impact Index Per Article: 38.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/22/2010] [Accepted: 01/28/2011] [Indexed: 05/03/2023]
Abstract
A new computer program, called SHIFTX2, is described which is capable of rapidly and accurately calculating diamagnetic (1)H, (13)C and (15)N chemical shifts from protein coordinate data. Compared to its predecessor (SHIFTX) and to other existing protein chemical shift prediction programs, SHIFTX2 is substantially more accurate (up to 26% better by correlation coefficient with an RMS error that is up to 3.3× smaller) than the next best performing program. It also provides significantly more coverage (up to 10% more), is significantly faster (up to 8.5×) and capable of calculating a wider variety of backbone and side chain chemical shifts (up to 6×) than many other shift predictors. In particular, SHIFTX2 is able to attain correlation coefficients between experimentally observed and predicted backbone chemical shifts of 0.9800 ((15)N), 0.9959 ((13)Cα), 0.9992 ((13)Cβ), 0.9676 ((13)C'), 0.9714 ((1)HN), 0.9744 ((1)Hα) and RMS errors of 1.1169, 0.4412, 0.5163, 0.5330, 0.1711, and 0.1231 ppm, respectively. The correlation between SHIFTX2's predicted and observed side chain chemical shifts is 0.9787 ((13)C) and 0.9482 ((1)H) with RMS errors of 0.9754 and 0.1723 ppm, respectively. SHIFTX2 is able to achieve such a high level of accuracy by using a large, high quality database of training proteins (>190), by utilizing advanced machine learning techniques, by incorporating many more features (χ(2) and χ(3) angles, solvent accessibility, H-bond geometry, pH, temperature), and by combining sequence-based with structure-based chemical shift prediction techniques. With this substantial improvement in accuracy we believe that SHIFTX2 will open the door to many long-anticipated applications of chemical shift prediction to protein structure determination, refinement and validation. SHIFTX2 is available both as a standalone program and as a web server ( http://www.shiftx2.ca ).
Collapse
Affiliation(s)
- Beomsoo Han
- Department of Computing Science, University of Alberta, Edmonton, AB Canada
| | - Yifeng Liu
- Department of Computing Science, University of Alberta, Edmonton, AB Canada
| | - Simon W. Ginzinger
- Department of Molecular Biology, Division of Bioinformatics, Center of Applied Molecular Engineering, University of Salzburg, Hellbrunnerstr. 34/3.OG, 5020 Salzburg, Austria
| | - David S. Wishart
- Department of Computing Science, University of Alberta, Edmonton, AB Canada
- Department of Biological Sciences, University of Alberta, Edmonton, AB Canada
- National Research Council, National Institute for Nanotechnology (NINT), Edmonton, AB T6G 2E8 Canada
| |
Collapse
|
19
|
Velankar S, Kleywegt GJ. The Protein Data Bank in Europe (PDBe): bringing structure to biology. ACTA CRYSTALLOGRAPHICA SECTION D: BIOLOGICAL CRYSTALLOGRAPHY 2011; 67:324-30. [PMID: 21460450 PMCID: PMC3069747 DOI: 10.1107/s090744491004117x] [Citation(s) in RCA: 24] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/30/2010] [Accepted: 10/13/2010] [Indexed: 12/30/2022]
Abstract
The Protein Data Bank in Europe (PDBe) is the European partner in the Worldwide PDB and as such handles depositions of X-ray, NMR and EM data and structure models. PDBe also provides advanced bioinformatics services based on data from the PDB and related resources. Some of the challenges facing the PDB and its guardians are discussed, as well as some of the areas on which PDBe activities will focus in the future (advanced services, ligands, integration, validation and experimental data). Finally, some recent developments at PDBe are described.
Collapse
Affiliation(s)
- Sameer Velankar
- Protein Data Bank in Europe (PDBe), EMBL-EBI, Wellcome Trust Genome Campus, Hinxton, Cambridge, England
| | | |
Collapse
|
20
|
Wishart DS. Interpreting protein chemical shift data. PROGRESS IN NUCLEAR MAGNETIC RESONANCE SPECTROSCOPY 2011; 58:62-87. [PMID: 21241884 DOI: 10.1016/j.pnmrs.2010.07.004] [Citation(s) in RCA: 184] [Impact Index Per Article: 14.2] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/14/2010] [Accepted: 07/29/2010] [Indexed: 05/12/2023]
Affiliation(s)
- David S Wishart
- Department of Biological Sciences, National Institute for Nanotechnology (NINT), Edmonton, AB, Canada T6G 2E8.
| |
Collapse
|
21
|
Abstract
The public archives containing protein information in the form of NMR chemical shift data at the BioMagResBank (BMRB) and of 3D structure coordinates at the Protein Data Bank are continuously expanding. The quality of the data contained in these archives, however, varies. The main issue for chemical shift values is that they are determined relative to a reference frequency. When this reference frequency is set incorrectly, all related chemical shift values are systematically offset. Such wrongly referenced chemical shift values, as well as other problems such as chemical shift values that are assigned to the wrong atom, are not easily distinguished from correct values and effectively reduce the usefulness of the archive. We describe a new method to correct and validate protein chemical shift values in relation to their 3D structure coordinates. This method classifies atoms using two parameters: the per-atom solvent accessible surface area (as calculated from the coordinates) and the secondary structure of the parent amino acid. Through the use of Gaussian statistics based on a large database of 3220 BMRB entries, we obtain per-entry chemical shift corrections as well as Z scores for the individual chemical shift values. In addition, information on the error of the correction value itself is available, and the method can retain only dependable correction values. We provide an online resource with chemical shift, atom exposure, and secondary structure information for all relevant BMRB entries (http://www.ebi.ac.uk/pdbe/nmr/vasco) and hope this data will aid the development of new chemical shift-based methods in NMR. Proteins 2010. © 2010 Wiley-Liss, Inc.
Collapse
Affiliation(s)
- Wolfgang Rieping
- Department of Biochemistry, University of Cambridge, Cambridge CB2 1GA, United Kingdom
| | | |
Collapse
|
22
|
Penkett CJ, van Ginkel G, Velankar S, Swaminathan J, Ulrich EL, Mading S, Stevens TJ, Fogh RH, Gutmanas A, Kleywegt GJ, Henrick K, Vranken WF. Straightforward and complete deposition of NMR data to the PDBe. JOURNAL OF BIOMOLECULAR NMR 2010; 48:85-92. [PMID: 20680401 PMCID: PMC2950272 DOI: 10.1007/s10858-010-9439-3] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/04/2010] [Accepted: 07/19/2010] [Indexed: 05/29/2023]
Abstract
We present a suite of software for the complete and easy deposition of NMR data to the PDB and BMRB. This suite uses the CCPN framework and introduces a freely downloadable, graphical desktop application called CcpNmr Entry Completion Interface (ECI) for the secure editing of experimental information and associated datasets through the lifetime of an NMR project. CCPN projects can be created within the CcpNmr Analysis software or by importing existing NMR data files using the CcpNmr FormatConverter. After further data entry and checking with the ECI, the project can then be rapidly deposited to the PDBe using AutoDep, or exported as a complete deposition NMR-STAR file. In full CCPN projects created with ECI, it is straightforward to select chemical shift lists, restraint data sets, structural ensembles and all relevant associated experimental collection details, which all are or will become mandatory when depositing to the PDB. Instructions and download information for the ECI are available from the PDBe web site at http://www.ebi.ac.uk/pdbe/nmr/deposition/eci.html .
Collapse
Affiliation(s)
- Christopher J. Penkett
- Protein Data Bank in Europe, European Bioinformatics Institute, Wellcome Trust Genome Campus, Hinxton, Cambridge, CB10 1SD UK
| | - Glen van Ginkel
- Protein Data Bank in Europe, European Bioinformatics Institute, Wellcome Trust Genome Campus, Hinxton, Cambridge, CB10 1SD UK
| | - Sameer Velankar
- Protein Data Bank in Europe, European Bioinformatics Institute, Wellcome Trust Genome Campus, Hinxton, Cambridge, CB10 1SD UK
| | - Jawahar Swaminathan
- Protein Data Bank in Europe, European Bioinformatics Institute, Wellcome Trust Genome Campus, Hinxton, Cambridge, CB10 1SD UK
| | - Eldon L. Ulrich
- BioMagResBank, Department of Biochemistry, University of Wisconsin Madison, 433 Babcock Dr., Madison, WI 53706 USA
| | - Steve Mading
- BioMagResBank, Department of Biochemistry, University of Wisconsin Madison, 433 Babcock Dr., Madison, WI 53706 USA
| | - Tim J. Stevens
- Department of Biochemistry, University of Cambridge, 80 Tennis Court Road, Cambridge, CB2 1GA UK
| | - Rasmus H. Fogh
- Department of Biochemistry, University of Cambridge, 80 Tennis Court Road, Cambridge, CB2 1GA UK
| | - Aleksandras Gutmanas
- Protein Data Bank in Europe, European Bioinformatics Institute, Wellcome Trust Genome Campus, Hinxton, Cambridge, CB10 1SD UK
| | - Gerard J. Kleywegt
- Protein Data Bank in Europe, European Bioinformatics Institute, Wellcome Trust Genome Campus, Hinxton, Cambridge, CB10 1SD UK
| | - Kim Henrick
- Protein Data Bank in Europe, European Bioinformatics Institute, Wellcome Trust Genome Campus, Hinxton, Cambridge, CB10 1SD UK
| | - Wim F. Vranken
- Protein Data Bank in Europe, European Bioinformatics Institute, Wellcome Trust Genome Campus, Hinxton, Cambridge, CB10 1SD UK
| |
Collapse
|
23
|
Lehtivarjo J, Hassinen T, Korhonen SP, Peräkylä M, Laatikainen R. 4D prediction of protein (1)H chemical shifts. JOURNAL OF BIOMOLECULAR NMR 2009; 45:413-26. [PMID: 19876601 DOI: 10.1007/s10858-009-9384-1] [Citation(s) in RCA: 19] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/03/2009] [Accepted: 10/09/2009] [Indexed: 05/11/2023]
Abstract
A 4D approach for protein (1)H chemical shift prediction was explored. The 4th dimension is the molecular flexibility, mapped using molecular dynamics simulations. The chemical shifts were predicted with a principal component model based on atom coordinates from a database of 40 protein structures. When compared to the corresponding non-dynamic (3D) model, the 4th dimension improved prediction by 6-7%. The prediction method achieved RMS errors of 0.29 and 0.50 ppm for Halpha and HN shifts, respectively. However, for individual proteins the RMS errors were 0.17-0.34 and 0.34-0.65 ppm for the Halpha and HN shifts, respectively. X-ray structures gave better predictions than the corresponding NMR structures, indicating that chemical shifts contain invaluable information about local structures. The (1)H chemical shift prediction tool 4DSPOT is available from http://www.uku.fi/kemia/4dspot .
Collapse
Affiliation(s)
- Juuso Lehtivarjo
- Department of Biosciences, University of Kuopio, Kuopio, Finland.
| | | | | | | | | |
Collapse
|
24
|
Velankar S, Best C, Beuth B, Boutselakis CH, Cobley N, Sousa Da Silva AW, Dimitropoulos D, Golovin A, Hirshberg M, John M, Krissinel EB, Newman R, Oldfield T, Pajon A, Penkett CJ, Pineda-Castillo J, Sahni G, Sen S, Slowley R, Suarez-Uruena A, Swaminathan J, van Ginkel G, Vranken WF, Henrick K, Kleywegt GJ. PDBe: Protein Data Bank in Europe. Nucleic Acids Res 2009; 38:D308-17. [PMID: 19858099 PMCID: PMC2808887 DOI: 10.1093/nar/gkp916] [Citation(s) in RCA: 85] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022] Open
Abstract
The Protein Data Bank in Europe (PDBe) (http://www.ebi.ac.uk/pdbe/) is actively working with its Worldwide Protein Data Bank partners to enhance the quality and consistency of the international archive of bio-macromolecular structure data, the Protein Data Bank (PDB). PDBe also works closely with its collaborators at the European Bioinformatics Institute and the scientific community around the world to enhance its databases and services by adding curated and actively maintained derived data to the existing structural data in the PDB. We have developed a new database infrastructure based on the remediated PDB archive data and a specially designed database for storing information on interactions between proteins and bound molecules. The group has developed new services that allow users to carry out simple textual queries or more complex 3D structure-based queries. The newly designed 'PDBeView Atlas pages' provide an overview of an individual PDB entry in a user-friendly layout and serve as a starting point to further explore the information available in the PDBe database. PDBe's active involvement with the X-ray crystallography, Nuclear Magnetic Resonance spectroscopy and cryo-Electron Microscopy communities have resulted in improved tools for structure deposition and analysis.
Collapse
Affiliation(s)
- S Velankar
- Protein Databank in Europe, EMBL-EBI, Wellcome Trust Genome Campus, Hinxton, Cambridge CB10 1SD, UK.
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
Collapse
|