Novel machine learning approaches revolutionize protein knowledge

Breakthrough methods in machine learning (ML), protein structure prediction, and novel ultrafast structural aligners are revolutionizing structural biology. Obtaining accurate models of proteins and annotating their functions on a large scale is no longer limited by time and resources. The most recent method to be top ranked by the Critical Assessment of Structure Prediction (CASP) assessment, AlphaFold 2 (AF2), is capable of building structural models with an accuracy comparable to that of experimental structures. Annotations of 3D models are keeping pace with the deposition of the structures due to advancements in protein language models (pLMs) and structural aligners that help validate these transferred annotations. In this review we describe how recent developments in ML for protein science are making large-scale structural bioinformatics available to the general scientific community.

LambdaPP: Fast and accessible protein-specific phenotype predictions

The availability of accurate and fast artificial intelligence (AI) solutions predicting aspects of proteins are revolutionizing experimental and computational molecular biology. The webserver LambdaPP aspires to supersede PredictProtein, the first internet server making AI protein predictions available in 1992. Given a protein sequence as input, LambdaPP provides easily accessible visualizations of protein 3D structure, along with predictions at the protein level (GeneOntology, subcellular location), and the residue level (binding to metal ions, small molecules, and nucleotides; conservation; intrinsic disorder; secondary structure; alpha-helical and beta-barrel transmembrane segments; signal-peptides; variant effect) in seconds. The structure prediction provided by LambdaPP-leveraging ColabFold and computed in minutes-is based on MMseqs2 multiple sequence alignments. All other feature prediction methods are based on the pLM ProtT5. Queried by a protein sequence, LambdaPP computes protein and residue predictions almost instantly for various phenotypes, including 3D structure and aspects of protein function. LambdaPP is freely available for everyone to use under embed.predictprotein.org, the interactive results for the case study can be found under https://embed.predictprotein.org/o/Q9NZC2. The frontend of LambdaPP can be found on GitHub (github.com/sacdallago/embed.predictprotein.org), and can be freely used and distributed under the academic free use license (AFL-2). For high-throughput applications, all methods can be executed locally via the bio-embeddings (bioembeddings.com) python package, or docker image at ghcr.io/bioembeddings/bio_embeddings, which also includes the backend of LambdaPP.

CATHe: detection of remote homologues for CATH superfamilies using embeddings from protein language models

MOTIVATION

CATH is a protein domain classification resource that exploits an automated workflow of structure and sequence comparison alongside expert manual curation to construct a hierarchical classification of evolutionary and structural relationships. The aim of this study was to develop algorithms for detecting remote homologues missed by state-of-the-art hidden Markov model (HMM)-based approaches. The method developed (CATHe) combines a neural network with sequence representations obtained from protein language models. It was assessed using a dataset of remote homologues having less than 20% sequence identity to any domain in the training set.

RESULTS

The CATHe models trained on 1773 largest and 50 largest CATH superfamilies had an accuracy of 85.6 ± 0.4% and 98.2 ± 0.3%, respectively. As a further test of the power of CATHe to detect more remote homologues missed by HMMs derived from CATH domains, we used a dataset consisting of protein domains that had annotations in Pfam, but not in CATH. By using highly reliable CATHe predictions (expected error rate <0.5%), we were able to provide CATH annotations for 4.62 million Pfam domains. For a subset of these domains from Homo sapiens, we structurally validated 90.86% of the predictions by comparing their corresponding AlphaFold2 structures with structures from the CATH superfamilies to which they were assigned.

AVAILABILITY AND IMPLEMENTATION

The code for the developed models is available on https://github.com/vam-sin/CATHe, and the datasets developed in this study can be accessed on https://zenodo.org/record/6327572.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

Contrastive learning on protein embeddings enlightens midnight zone

Experimental structures are leveraged through multiple sequence alignments, or more generally through homology-based inference (HBI), facilitating the transfer of information from a protein with known annotation to a query without any annotation. A recent alternative expands the concept of HBI from sequence-distance lookup to embedding-based annotation transfer (EAT). These embeddings are derived from protein Language Models (pLMs). Here, we introduce using single protein representations from pLMs for contrastive learning. This learning procedure creates a new set of embeddings that optimizes constraints captured by hierarchical classifications of protein 3D structures defined by the CATH resource. The approach, dubbed ProtTucker, has an improved ability to recognize distant homologous relationships than more traditional techniques such as threading or fold recognition. Thus, these embeddings have allowed sequence comparison to step into the 'midnight zone' of protein similarity, i.e. the region in which distantly related sequences have a seemingly random pairwise sequence similarity. The novelty of this work is in the particular combination of tools and sampling techniques that ascertained good performance comparable or better to existing state-of-the-art sequence comparison methods. Additionally, since this method does not need to generate alignments it is also orders of magnitudes faster. The code is available at https://github.com/Rostlab/EAT.

Protein embeddings and deep learning predict binding residues for various ligand classes

One important aspect of protein function is the binding of proteins to ligands, including small molecules, metal ions, and macromolecules such as DNA or RNA. Despite decades of experimental progress many binding sites remain obscure. Here, we proposed bindEmbed21, a method predicting whether a protein residue binds to metal ions, nucleic acids, or small molecules. The Artificial Intelligence (AI)-based method exclusively uses embeddings from the Transformer-based protein Language Model (pLM) ProtT5 as input. Using only single sequences without creating multiple sequence alignments (MSAs), bindEmbed21DL outperformed MSA-based predictions. Combination with homology-based inference increased performance to F1 = 48 ± 3% (95% CI) and MCC = 0.46 ± 0.04 when merging all three ligand classes into one. All results were confirmed by three independent data sets. Focusing on very reliably predicted residues could complement experimental evidence: For the 25% most strongly predicted binding residues, at least 73% were correctly predicted even when ignoring the problem of missing experimental annotations. The new method bindEmbed21 is fast, simple, and broadly applicable-neither using structure nor MSAs. Thereby, it found binding residues in over 42% of all human proteins not otherwise implied in binding and predicted about 6% of all residues as binding to metal ions, nucleic acids, or small molecules.

PredictProtein - Predicting Protein Structure and Function for 29 Years

Since 1992 PredictProtein (https://predictprotein.org) is a one-stop online resource for protein sequence analysis with its main site hosted at the Luxembourg Centre for Systems Biomedicine (LCSB) and queried monthly by over 3,000 users in 2020. PredictProtein was the first Internet server for protein predictions. It pioneered combining evolutionary information and machine learning. Given a protein sequence as input, the server outputs multiple sequence alignments, predictions of protein structure in 1D and 2D (secondary structure, solvent accessibility, transmembrane segments, disordered regions, protein flexibility, and disulfide bridges) and predictions of protein function (functional effects of sequence variation or point mutations, Gene Ontology (GO) terms, subcellular localization, and protein-, RNA-, and DNA binding). PredictProtein's infrastructure has moved to the LCSB increasing throughput; the use of MMseqs2 sequence search reduced runtime five-fold (apparently without lowering performance of prediction methods); user interface elements improved usability, and new prediction methods were added. PredictProtein recently included predictions from deep learning embeddings (GO and secondary structure) and a method for the prediction of proteins and residues binding DNA, RNA, or other proteins. PredictProtein.org aspires to provide reliable predictions to computational and experimental biologists alike. All scripts and methods are freely available for offline execution in high-throughput settings.

Learned Embeddings from Deep Learning to Visualize and Predict Protein Sets

Models from machine learning (ML) or artificial intelligence (AI) increasingly assist in guiding experimental design and decision making in molecular biology and medicine. Recently, Language Models (LMs) have been adapted from Natural Language Processing (NLP) to encode the implicit language written in protein sequences. Protein LMs show enormous potential in generating descriptive representations (embeddings) for proteins from just their sequences, in a fraction of the time with respect to previous approaches, yet with comparable or improved predictive ability. Researchers have trained a variety of protein LMs that are likely to illuminate different angles of the protein language. By leveraging the bio_embeddings pipeline and modules, simple and reproducible workflows can be laid out to generate protein embeddings and rich visualizations. Embeddings can then be leveraged as input features through machine learning libraries to develop methods predicting particular aspects of protein function and structure. Beyond the workflows included here, embeddings have been leveraged as proxies to traditional homology-based inference and even to align similar protein sequences. A wealth of possibilities remain for researchers to harness through the tools provided in the following protocols. © 2021 The Authors. Current Protocols published by Wiley Periodicals LLC. The following protocols are included in this manuscript: Basic Protocol 1: Generic use of the bio_embeddings pipeline to plot protein sequences and annotations Basic Protocol 2: Generate embeddings from protein sequences using the bio_embeddings pipeline Basic Protocol 3: Overlay sequence annotations onto a protein space visualization Basic Protocol 4: Train a machine learning classifier on protein embeddings Alternate Protocol 1: Generate 3D instead of 2D visualizations Alternate Protocol 2: Visualize protein solubility instead of protein subcellular localization Support Protocol: Join embedding generation and sequence space visualization in a pipeline.

Clustering FunFams using sequence embeddings improves EC purity

Motivation

Classifying proteins into functional families can improve our understanding of protein function and can allow transferring annotations within one family. For this, functional families need to be ‘pure’, i.e., contain only proteins with identical function. Functional Families (FunFams) cluster proteins within CATH superfamilies into such groups of proteins sharing function. 11% of all FunFams (22 830 of 203 639) contain EC annotations and of those, 7% (1526 of 22 830) have inconsistent functional annotations.

Results

We propose an approach to further cluster FunFams into functionally more consistent sub-families by encoding their sequences through embeddings. These embeddings originate from language models transferring knowledge gained from predicting missing amino acids in a sequence (ProtBERT) and have been further optimized to distinguish between proteins belonging to the same or a different CATH superfamily (PB-Tucker). Using distances between embeddings and DBSCAN to cluster FunFams and identify outliers, doubled the number of pure clusters per FunFam compared to random clustering. Our approach was not limited to FunFams but also succeeded on families created using sequence similarity alone. Complementing EC annotations, we observed similar results for binding annotations. Thus, we expect an increased purity also for other aspects of function. Our results can help generating FunFams; the resulting clusters with improved functional consistency allow more reliable inference of annotations. We expect this approach to succeed equally for any other grouping of proteins by their phenotypes.

Availability and implementation

Code and embeddings are available via GitHub: https://github.com/Rostlab/FunFamsClustering.

Supplementary information

Supplementary data are available at Bioinformatics online.

Embeddings from deep learning transfer GO annotations beyond homology

Knowing protein function is crucial to advance molecular and medical biology, yet experimental function annotations through the Gene Ontology (GO) exist for fewer than 0.5% of all known proteins. Computational methods bridge this sequence-annotation gap typically through homology-based annotation transfer by identifying sequence-similar proteins with known function or through prediction methods using evolutionary information. Here, we propose predicting GO terms through annotation transfer based on proximity of proteins in the SeqVec embedding rather than in sequence space. These embeddings originate from deep learned language models (LMs) for protein sequences (SeqVec) transferring the knowledge gained from predicting the next amino acid in 33 million protein sequences. Replicating the conditions of CAFA3, our method reaches an Fmax of 37 ± 2%, 50 ± 3%, and 57 ± 2% for BPO, MFO, and CCO, respectively. Numerically, this appears close to the top ten CAFA3 methods. When restricting the annotation transfer to proteins with < 20% pairwise sequence identity to the query, performance drops (Fmax BPO 33 ± 2%, MFO 43 ± 3%, CCO 53 ± 2%); this still outperforms naïve sequence-based transfer. Preliminary results from CAFA4 appear to confirm these findings. Overall, this new concept is likely to change the annotation of proteins, in particular for proteins from smaller families or proteins with intrinsically disordered regions.

FunFam protein families improve residue level molecular function prediction

BACKGROUND

The CATH database provides a hierarchical classification of protein domain structures including a sub-classification of superfamilies into functional families (FunFams). We analyzed the similarity of binding site annotations in these FunFams and incorporated FunFams into the prediction of protein binding residues.

RESULTS

FunFam members agreed, on average, in 36.9 ± 0.6% of their binding residue annotations. This constituted a 6.7-fold increase over randomly grouped proteins and a 1.2-fold increase (1.1-fold on the same dataset) over proteins with the same enzymatic function (identical Enzyme Commission, EC, number). Mapping de novo binding residue prediction methods (BindPredict-CCS, BindPredict-CC) onto FunFam resulted in consensus predictions for those residues that were aligned and predicted alike (binding/non-binding) within a FunFam. This simple consensus increased the F1-score (for binding) 1.5-fold over the original prediction method. Variation of the threshold for how many proteins in the consensus prediction had to agree provided a convenient control of accuracy/precision and coverage/recall, e.g. reaching a precision as high as 60.8 ± 0.4% for a stringent threshold.

CONCLUSIONS

The FunFams outperformed even the carefully curated EC numbers in terms of agreement of binding site residues. Additionally, we assume that our proof-of-principle through the prediction of protein binding residues will be relevant for many other solutions profiting from FunFams to infer functional information at the residue level.

Detailed prediction of protein sub-nuclear localization

Background

Sub-nuclear structures or locations are associated with various nuclear processes. Proteins localized in these substructures are important to understand the interior nuclear mechanisms. Despite advances in high-throughput methods, experimental protein annotations remain limited. Predictions of cellular compartments have become very accurate, largely at the expense of leaving out substructures inside the nucleus making a fine-grained analysis impossible.

Results

Here, we present a new method (LocNuclei) that predicts nuclear substructures from sequence alone. LocNuclei used a string-based Profile Kernel with Support Vector Machines (SVMs). It distinguishes sub-nuclear localization in 13 distinct substructures and distinguishes between nuclear proteins confined to the nucleus and those that are also native to other compartments (traveler proteins). High performance was achieved by implicitly leveraging a large biological knowledge-base in creating predictions by homology-based inference through BLAST. Using this approach, the performance reached AUC = 0.70–0.74 and Q13 = 59–65%. Travelling proteins (nucleus and other) were identified at Q2 = 70–74%. A Gene Ontology (GO) analysis of the enrichment of biological processes revealed that the predicted sub-nuclear compartments matched the expected functionality. Analysis of protein-protein interactions (PPI) show that formation of compartments and functionality of proteins in these compartments highly rely on interactions between proteins. This suggested that the LocNuclei predictions carry important information about function. The source code and data sets are available through GitHub: https://github.com/Rostlab/LocNuclei.

Conclusions

LocNuclei predicts subnuclear compartments and traveler proteins accurately. These predictions carry important information about functionality and PPIs.

Electronic supplementary material

The online version of this article (10.1186/s12859-019-2790-9) contains supplementary material, which is available to authorized users.

Latent class analysis of diagnostic tests for adenovirus, Bordetella pertussis and influenza virus infections in German adults with longer lasting coughs

Laboratory tests in adult outpatients with longer lasting coughs to identify a potential causal pathogen are rarely performed, and there is no gold standard for these diagnostic tests. While the diagnostic validity of serological tests for pertussis is well established their potential contribution for diagnosing adenovirus and influenza virus A and B infections is unclear. A sentinel study into the population-based incidence of longer lasting coughs in adults was done in Rostock (former East Germany) and Krefeld (former West Germany). A total of 971 outpatients who consulted general practitioners or internists were included. Inclusion criteria were coughing for ⩾1 week and no chronic respiratory diseases. We evaluated the performance of polymerase chain reaction (PCR) as well as IgG and IgA serology, applying a latent class model for diagnosing infections with adenovirus, B. pertussis, and influenza virus A and B. The adult outpatients first sought medical attention when they had been coughing for a median of 3 weeks. In this situation, direct detection of infectious agents by PCR had a low sensitivity. Modelling showed that additional serological tests equally improved sensitivity and specificity for diagnosis for adenovirus, B. pertussis and influenza virus A and B infections. The combination of serology and PCR may improve the overall performance of diagnostic tests for B. pertussis and also for adenovirus, and influenza virus A and B infections.

[Pertussis: incidence, symptoms and costs]

BACKGROUND AND OBJECTIVE

A prospective sentinel study into the population-based incidence of pertussis in adults was done between 2001 and 2004 in Rostock (former East Germany) and Krefeld (former West Germany).

PATIENTS AND METHODS

971 outpatients, who consulted general practitioners or internists, were included. Clinical inclusion criteria were coughing for one week or more and no chronic respiratory diseases. Bordetella infection was diagnosed by PCR on nasopharyngeal swabs and ELISA for serology (IgG-anti-PT, IgA-anti-PT, IgG-anti-FHA, IgA-anti-FHA).

RESULTS

We found a total of 97 cases of pertussis in this cohort. The main symptom was coughing for a median of 7-8 weeks. Population-based incidence was estimated in Krefeld at 169 cases/100000 population per year, and in Rostock at 160/100000 per year. Resource use was 120 EUR of direct medical cost and 434 euro of indirect medical cost, not including hospitalization in this study.

CONCLUSIONS

Pertussis is a frequent cause of longer lasting cough in German adults, and it causes significant morbidity and costs.

Ein Fall einer autochthonen Frühsommermeningoenzephalitis (FSME) in Mecklenburg-Vorpommern

HISTORY

A 61-year-old man was bitten by a tick at Lake Woblitz, near the town of Neustrelitz in former East Germany. Nine days later he saw his general practitioner because of fever and headache. Three weeks after the tick bite he was hospitalized with fever (39.2 degrees C) and mental confusion. Because he had taken a Nile cruise six months earlier, malaria was considered and he was transferred to the department of tropical medicine and infectious diseases of the University of Rostock.

INVESTIGATIONS

The patient was somnolent, his speech was slurred, and he had amnesic aphasia, as well as impaired fine motor control, but no meningism, focal signs, pyramidal tract or sensory impairment. Cerebrospinal fluid (CSF) showed mild lymphocytosis (9,400 leukocytes per microL; 89% lymphocytes) and elevated protein concentration (1322 mg/L) with blood brain barrier impairment and intrathecal IgM synthesis. Anti-tick-bite encephalitis (TBE) antibodies (ELISA: IgG and IgM) were present in serum and CSF, and serum immunofluorescence showed an eight-fold titer increase within two weeks. These findings confirm the diagnosis of TBE. Other infections (including those with cross-reacting flaviviruses) were excluded by appropriate antibody testing.

THERAPY AND CLINICAL COURSE

There is no specific antiviral treatment for TBE, but on symptomatic therapy the patient recovered fully within four weeks.

CONCLUSION

The site of the patient's infection is located 10 km to the west of an old TBE focus, but no TBE virus had been detected there after 1975. The case demonstrates that TBE should be included in the differential diagnosis of meningoencephalitis, even if the patient has not been in an acknowledged TBE endemic area.

[Autoimmune pancreatitis (AIMP) - a clinical entity of its own?]

HISTORY AND ADMISSION FINDINGS

A 34-year-old man with chronic inflammation of both tear glands for 10 years was admitted because of severe upper abdominal pain and jaundice.

INVESTIGATIONS

Transaminase and cholestasis parameters were raised, as were gamma-globulins on electrophoresis. Ultrasound revealed an echo-poor diffusely enlarged pancreas with hypervascularity on colour-Doppler. Endoscopic retrograde cholangiopancreatography (ERCP) indicated chronically changed pancreatic ducts of varying diameter and stenosis, as well as a filiform distal stenosis of the common bile duct. A pancreatic biopsy showed chronic sclerosing pancreatitis with broad periductal lymphocytic infiltration.

TREATMENT AND COURSE

ERCP and a guide-wire papillotomy were performed, followed by placement of a nasobiliary catheter. All findings markedly improved on oral prednisone.

CONCLUSION

This case supports the supposition that in idiopathic chronic pancreatitis autoimmune processes may be responsible for the changes in the pancreatic ducts. Immunosuppressive treatment is thus recommended in these cases.

[Improved ultrasound imaging of the pancreas with the transsplenic view and Tissue harmonic imaging]

The conventional ultrasound examination of the pancreas from ventral is - especially concerning the leftsided parts of the organ - limited by interposition of stomach, flexura duodenojejunalis or colon. The transsplenic view to the cauda pancreatis is less known, but allows quite often a good possibility to examine this part of the retroperitoneal organ, even when examination from ventral is impossible.

AIM

Comparison of ventral and transsplenic ultrasound examination - with and without tissue harmonic imaging (THI) - in regard to the diagnostic accessability to the cauda pancreatis.

METHOD

Ultrasound examination of 135 patients from both positions with and without THI and comparison of the accessability by experienced examiners.

RESULTS

As examination from the ventral position allows a proper view to the cauda only in 58 %, the transsplenic position allows an adequate view in 98 %. The examiners regarded THI as a useful improvement in examining the pancreas in 85 %.

CONCLUSION

The transsplenic view to the pancreas is less known than the examination from the classical ventral transducer position. It enables the examinator quite often to examine the pancreatic cauda even when examination from ventral is impossible. Therefore it should be enclosed in the routine procedure of abdominal ultrasound examination. THI is a further possibility to improve the ultrasound access to the pancreas, especially when conditions for a good view and proper examination are limited.

[Epidemiologic and serologic studies of pneumococcal infections with reference to the new STIKO recommendations]

Pneumococcal diseases play an important role especially for babies and toddlers (otitis media) and for older persons (pneumonia). 28% of the 481 reported cases of bacterial meningitides (without meningococcus) in Mecklenburg-Vorpommern were caused by pneumoniae streptococcus. A pneumococcus antibody study by the land register confirms the high contamination in older people. Therefore STIKO recommends since March 1998 to effect pneumococcal vaccination with every person from the age of 60 and dove as well as for children, adolescents and adults with higher risk due to a primary disease.

[Ultrasound guided forceps biopsy of the pleura]

AIM

Between cytology of pleural effusion and thorascopy there is a gap for another non invasive biopsy method to diagnose pleural diseases, especially since Adam's and Ramel's blind pleural biopsy is uncommon. Therefore it is suggestive to test feasibility and usefulness of pleural forceps biopsy.

METHOD

It is possible to take biopsies under ultrasound guidance with the help of a biopsy-forceps from the diaphragmatic pleura or pleural appositions through a 2.5 mm canula with a stop cock and a rubber vent. The specimen can be used for histological or immunohistochemical examinations. The procedure is coducted in a closed system to avoid pneumothorax. The puncture was done in 12 patients with a puncturable pleural effusion.

RESULTS

In 11 of 12 patients it was possible to get the final diagnosis. In one of three cases of mesotheliomas a rebiopsy was necessary. In 9 cases a malign tumor was diagnosed, effusion cytology was negative in 4 of 7 tumors. In 5 patients with a history of a former tumor pleural carcinosis was related three times correctly to the former cancer and twice to a secondary cancer. In one case a fibrous plaque was found. There were two patients with pleuritis, in one case a pulmonary tuberculosis was found 8 weeks later. In one patient with a mesothelioma inoculated metastasis were present in the sites of the punctures. In all pleural forceps punctures we got enough biopsy material for histological and immunohistochemical diagnosis.

CONCLUSION

The ultrasound guided forceps biopsy of the pleura is a very promising less invasive method to diagnose pleural tumors. Additional improvements of the equipment are possible. Definitive evaluation of the procedure especially in infectious pleural diseases requires a higher number of cases.