MicrographCleaner: A python package for cryo-EM micrograph cleaning using deep learning

Miffi: Improving the accuracy of CNN-based cryo-EM micrograph filtering with fine-tuning and Fourier space information

Efficient and high-accuracy filtering of cryo-electron microscopy (cryo-EM) micrographs is an emerging challenge with the growing speed of data collection and sizes of datasets. Convolutional neural networks (CNNs) are machine learning models that have been proven successful in many computer vision tasks, and have been previously applied to cryo-EM micrograph filtering. In this work, we demonstrate that two strategies, fine-tuning models from pretrained weights and including the power spectrum of micrographs as input, can greatly improve the attainable prediction accuracy of CNN models. The resulting software package, Miffi, is open-source and freely available for public use (https://github.com/ando-lab/miffi).

Miffi: Improving the accuracy of CNN-based cryo-EM micrograph filtering with fine-tuning and Fourier space information

Efficient and high-accuracy filtering of cryo-electron microscopy (cryo-EM) micrographs is an emerging challenge with the growing speed of data collection and sizes of datasets. Convolutional neural networks (CNNs) are machine learning models that have been proven successful in many computer vision tasks, and have been previously applied to cryo-EM micrograph filtering. In this work, we demonstrate that two strategies, fine-tuning models from pretrained weights and including the power spectrum of micrographs as input, can greatly improve the attainable prediction accuracy of CNN models. The resulting software package, Miffi, is open-source and freely available for public use (https://github.com/ando-lab/miffi).

TMEM106B is a receptor mediating ACE2-independent SARS-CoV-2 cell entry

SARS-CoV-2 is associated with broad tissue tropism, a characteristic often determined by the availability of entry receptors on host cells. Here, we show that TMEM106B, a lysosomal transmembrane protein, can serve as an alternative receptor for SARS-CoV-2 entry into angiotensin-converting enzyme 2 (ACE2)-negative cells. Spike substitution E484D increased TMEM106B binding, thereby enhancing TMEM106B-mediated entry. TMEM106B-specific monoclonal antibodies blocked SARS-CoV-2 infection, demonstrating a role of TMEM106B in viral entry. Using X-ray crystallography, cryogenic electron microscopy (cryo-EM), and hydrogen-deuterium exchange mass spectrometry (HDX-MS), we show that the luminal domain (LD) of TMEM106B engages the receptor-binding motif of SARS-CoV-2 spike. Finally, we show that TMEM106B promotes spike-mediated syncytium formation, suggesting a role of TMEM106B in viral fusion. Together, our findings identify an ACE2-independent SARS-CoV-2 infection mechanism that involves cooperative interactions with the receptors heparan sulfate and TMEM106B.

Chirality Analysis of Complex Microparticles using Deep Learning on Realistic Sets of Microscopy Images

Nanoscale chirality is an actively growing research field spurred by the giant chiroptical activity, enantioselective biological activity, and asymmetric catalytic activity of chiral nanostructures. Compared to chiral molecules, the handedness of chiral nano- and microstructures can be directly established via electron microscopy, which can be utilized for the automatic analysis of chiral nanostructures and prediction of their properties. However, chirality in complex materials may have multiple geometric forms and scales. Computational identification of chirality from electron microscopy images rather than optical measurements is convenient but is fundamentally challenging, too, because (1) image features differentiating left- and right-handed particles can be ambiguous and (2) three-dimensional structure essential for chirality is 'flattened' into two-dimensional projections. Here, we show that deep learning algorithms can identify twisted bowtie-shaped microparticles with nearly 100% accuracy and classify them as left- and right-handed with as high as 99% accuracy. Importantly, such accuracy was achieved with as few as 30 original electron microscopy images of bowties. Furthermore, after training on bowtie particles with complex nanostructured features, the model can recognize other chiral shapes with different geometries without retraining for their specific chiral geometry with 93% accuracy, indicating the true learning abilities of the employed neural networks. These findings indicate that our algorithm trained on a practically feasible set of experimental data enables automated analysis of microscopy data for the accelerated discovery of chiral particles and their complex systems for multiple applications.

High-throughput cryo-ET structural pattern mining by unsupervised deep iterative subtomogram clustering

Cryoelectron tomography directly visualizes heterogeneous macromolecular structures in their native and complex cellular environments. However, existing computer-assisted structure sorting approaches are low throughput or inherently limited due to their dependency on available templates and manual labels. Here, we introduce a high-throughput template-and-label-free deep learning approach, Deep Iterative Subtomogram Clustering Approach (DISCA), that automatically detects subsets of homogeneous structures by learning and modeling 3D structural features and their distributions. Evaluation on five experimental cryo-ET datasets shows that an unsupervised deep learning based method can detect diverse structures with a wide range of molecular sizes. This unsupervised detection paves the way for systematic unbiased recognition of macromolecular complexes in situ.

Learning to automate cryo-electron microscopy data collection with Ptolemy

Over the past decade, cryo-electron microscopy (cryoEM) has emerged as an important method for determining near-native, near-atomic resolution 3D structures of biological macromolecules. To meet the increasing demand for cryoEM, automated methods that improve throughput and efficiency of microscope operation are needed. Currently, the targeting algorithms provided by most data-collection software require time-consuming manual tuning of parameters for each grid, and, in some cases, operators must select targets completely manually. However, the development of fully automated targeting algorithms is non-trivial, because images often have low signal-to-noise ratios and optimal targeting strategies depend on a range of experimental parameters and macromolecule behaviors that vary between projects and collection sessions. To address this, Ptolemy provides a pipeline to automate low- and medium-magnification targeting using a suite of purpose-built computer vision and machine-learning algorithms, including mixture models, convolutional neural networks and U-Nets. Learned models in this pipeline are trained on a large set of images from real-world cryoEM data-collection sessions, labeled with locations selected by human operators. These models accurately detect and classify regions of interest in low- and medium-magnification images, and generalize to unseen sessions, as well as to images collected on different microscopes at another facility. This open-source, modular pipeline can be integrated with existing microscope control software to enable automation of cryoEM data collection and can serve as a foundation for future cryoEM automation software.

Image processing tools for the validation of CryoEM maps

The number of maps deposited in public databases (Electron Microscopy Data Bank, EMDB) determined by cryo-electron microscopy has quickly grown in recent years. With this rapid growth, it is critical to guarantee their quality. So far, map validation has primarily focused on the agreement between maps and models. From the image processing perspective, the validation has been mostly restricted to using two half-maps and the measurement of their internal consistency. In this article, we suggest that map validation can be taken much further from the point of view of image processing if 2D classes, particles, angles, coordinates, defoci, and micrographs are also provided. We present a progressive validation scheme that qualifies a result validation status from 0 to 5 and offers three optional qualifiers (A, W, and O) that can be added. The simplest validation state is 0, while the most complete would be 5AWO. This scheme has been implemented in a website https://biocomp.cnb.csic.es/EMValidationService/ to which reconstructed maps and their ESI can be uploaded.

Emerging Themes in CryoEM─Single Particle Analysis Image Processing

Cryo-electron microscopy (CryoEM) has become a vital technique in structural biology. It is an interdisciplinary field that takes advantage of advances in biochemistry, physics, and image processing, among other disciplines. Innovations in these three basic pillars have contributed to the boosting of CryoEM in the past decade. This work reviews the main contributions in image processing to the current reconstruction workflow of single particle analysis (SPA) by CryoEM. Our review emphasizes the time evolution of the algorithms across the different steps of the workflow differentiating between two groups of approaches: analytical methods and deep learning algorithms. We present an analysis of the current state of the art. Finally, we discuss the emerging problems and challenges still to be addressed in the evolution of CryoEM image processing methods in SPA.

On bias, variance, overfitting, gold standard and consensus in single-particle analysis by cryo-electron microscopy

Cryo-electron microscopy (cryoEM) has become a well established technique to elucidate the 3D structures of biological macromolecules. Projection images from thousands of macromolecules that are assumed to be structurally identical are combined into a single 3D map representing the Coulomb potential of the macromolecule under study. This article discusses possible caveats along the image-processing path and how to avoid them to obtain a reliable 3D structure. Some of these problems are very well known in the community. These may be referred to as sample-related (such as specimen denaturation at interfaces or non-uniform projection geometry leading to underrepresented projection directions). The rest are related to the algorithms used. While some have been discussed in depth in the literature, such as the use of an incorrect initial volume, others have received much less attention. However, they are fundamental in any data-analysis approach. Chiefly among them, instabilities in estimating many of the key parameters that are required for a correct 3D reconstruction that occur all along the processing workflow are referred to, which may significantly affect the reliability of the whole process. In the field, the term overfitting has been coined to refer to some particular kinds of artifacts. It is argued that overfitting is a statistical bias in key parameter-estimation steps in the 3D reconstruction process, including intrinsic algorithmic bias. It is also shown that common tools (Fourier shell correlation) and strategies (gold standard) that are normally used to detect or prevent overfitting do not fully protect against it. Alternatively, it is proposed that detecting the bias that leads to overfitting is much easier when addressed at the level of parameter estimation, rather than detecting it once the particle images have been combined into a 3D map. Comparing the results from multiple algorithms (or at least, independent executions of the same algorithm) can detect parameter bias. These multiple executions could then be averaged to give a lower variance estimate of the underlying parameters.

Applications of deep learning in electron microscopy

We review the growing use of machine learning in electron microscopy (EM) driven in part by the availability of fast detectors operating at kiloHertz frame rates leading to large data sets that cannot be processed using manually implemented algorithms. We summarize the various network architectures and error metrics that have been applied to a range of EM-related problems including denoising and inpainting. We then provide a review of the application of these in both physical and life sciences, highlighting how conventional networks and training data have been specifically modified for EM.

Machine Learning for Structure Determination in Single-Particle Cryo-Electron Microscopy: A Systematic Review

Recently, single-particle cryo-electron microscopy (cryo-EM) has become an indispensable method for determining macromolecular structures at high resolution to deeply explore the relevant molecular mechanism. Its recent breakthrough is mainly because of the rapid advances in hardware and image processing algorithms, especially machine learning. As an essential support of single-particle cryo-EM, machine learning has powered many aspects of structure determination and greatly promoted its development. In this article, we provide a systematic review of the applications of machine learning in this field. Our review begins with a brief introduction of single-particle cryo-EM, followed by the specific tasks and challenges of its image processing. Then, focusing on the workflow of structure determination, we describe relevant machine learning algorithms and applications at different steps, including particle picking, 2-D clustering, 3-D reconstruction, and other steps. As different tasks exhibit distinct characteristics, we introduce the evaluation metrics for each task and summarize their dynamics of technology development. Finally, we discuss the open issues and potential trends in this promising field.

Advances in Xmipp for Cryo-Electron Microscopy: From Xmipp to Scipion

Xmipp is an open-source software package consisting of multiple programs for processing data originating from electron microscopy and electron tomography, designed and managed by the Biocomputing Unit of the Spanish National Center for Biotechnology, although with contributions from many other developers over the world. During its 25 years of existence, Xmipp underwent multiple changes and updates. While there were many publications related to new programs and functionality added to Xmipp, there is no single publication on the Xmipp as a package since 2013. In this article, we give an overview of the changes and new work since 2013, describe technologies and techniques used during the development, and take a peek at the future of the package.

Deep Learning-Based Advances in Protein Structure Prediction

Obtaining an accurate description of protein structure is a fundamental step toward understanding the underpinning of biology. Although recent advances in experimental approaches have greatly enhanced our capabilities to experimentally determine protein structures, the gap between the number of protein sequences and known protein structures is ever increasing. Computational protein structure prediction is one of the ways to fill this gap. Recently, the protein structure prediction field has witnessed a lot of advances due to Deep Learning (DL)-based approaches as evidenced by the success of AlphaFold2 in the most recent Critical Assessment of protein Structure Prediction (CASP14). In this article, we highlight important milestones and progresses in the field of protein structure prediction due to DL-based methods as observed in CASP experiments. We describe advances in various steps of protein structure prediction pipeline viz. protein contact map prediction, protein distogram prediction, protein real-valued distance prediction, and Quality Assessment/refinement. We also highlight some end-to-end DL-based approaches for protein structure prediction approaches. Additionally, as there have been some recent DL-based advances in protein structure determination using Cryo-Electron (Cryo-EM) microscopy based, we also highlight some of the important progress in the field. Finally, we provide an outlook and possible future research directions for DL-based approaches in the protein structure prediction arena.

Detecting Protein Communities in Native Cell Extracts by Machine Learning: A Structural Biologist's Perspective

Native cell extracts hold great promise for understanding the molecular structure of ordered biological systems at high resolution. This is because higher-order biomolecular interactions, dubbed as protein communities, may be retained in their (near-)native state, in contrast to extensively purifying or artificially overexpressing the proteins of interest. The distinct machine-learning approaches are applied to discover protein-protein interactions within cell extracts, reconstruct dedicated biological networks, and report on protein community members from various organisms. Their validation is also important, e.g., by the cross-linking mass spectrometry or cell biology methods. In addition, the cell extracts are amenable to structural analysis by cryo-electron microscopy (cryo-EM), but due to their inherent complexity, sorting structural signatures of protein communities derived by cryo-EM comprises a formidable task. The application of image-processing workflows inspired by machine-learning techniques would provide improvements in distinguishing structural signatures, correlating proteomic and network data to structural signatures and subsequently reconstructed cryo-EM maps, and, ultimately, characterizing unidentified protein communities at high resolution. In this review article, we summarize recent literature in detecting protein communities from native cell extracts and identify the remaining challenges and opportunities. We argue that the progress in, and the integration of, machine learning, cryo-EM, and complementary structural proteomics approaches would provide the basis for a multi-scale molecular description of protein communities within native cell extracts.

Automated picking of amyloid fibrils from cryo-EM images for helical reconstruction with RELION

Cryogenic electron microscopy (cryo-EM) is an important tool for determining the molecular structure of proteins and protein assemblies, including helical assemblies such as amyloid fibrils. In reconstruction of amyloid fibril structures from cryo-EM images, an important early step is the selection of fibril locations. This fibril picking step is typically done by hand, a tedious process when thousands of images need to be analyzed. Here we present a computer program called FibrilFinder that identifies the locations and directions of fibril segments in cryo-EM images, by using the properties that the fibrils should be linear objects and have widths within a specified range. The program outputs the fibril locations in text files compatible with the RELION density reconstruction program. After RELION is used to extract the particle image boxes contained in the fibril segments identified by FibrilFinder, a second program called FibrilFixer removes boxes that contain more than one fibril, for instance because two fibrils cross each other. As concrete and realistic examples, we describe the application of the two programs to cryo-EM images of two different amyloid fibrils, namely 40-residue amyloid-β fibrils derived from human brain tissue by seeded growth and fibrils formed by the C-terminal half of the low-complexity domain of the RNA-binding protein FUS. Both examples of amyloid fibrils can be picked from cryo-EM images using the same set of FibrilFinder and FibrilFixer parameters, showing that this software does not require re-optimization for each sample. A set of 1337 cryo-EM images was analyzed in 17 min with one multi-core computer. The new fibril picking software should enable the rapid analysis and comparison of more helical structures using cryo-EM, and perhaps serve as part of the greater automation of the entire structure determination process.

Deep learning in electron microscopy

Deep learning is transforming most areas of science and technology, including electron microscopy. This review paper offers a practical perspective aimed at developers with limited familiarity. For context, we review popular applications of deep learning in electron microscopy. Following, we discuss hardware and software needed to get started with deep learning and interface with electron microscopes. We then review neural network components, popular architectures, and their optimization. Finally, we discuss future directions of deep learning in electron microscopy.

Continuous flexibility analysis of SARS-CoV-2 spike prefusion structures

Using a new consensus-based image-processing approach together with principal component analysis, the flexibility and conformational dynamics of the SARS-CoV-2 spike in the prefusion state have been analysed. These studies revealed concerted motions involving the receptor-binding domain (RBD), N-terminal domain, and subdomains 1 and 2 around the previously characterized 1-RBD-up state, which have been modeled as elastic deformations. It is shown that in this data set there are not well defined, stable spike conformations, but virtually a continuum of states. An ensemble map was obtained with minimum bias, from which the extremes of the change along the direction of maximal variance were modeled by flexible fitting. The results provide a warning of the potential image-processing classification instability of these complicated data sets, which has a direct impact on the interpretability of the results.

Continuous flexibility analysis of SARS-CoV-2 Spike prefusion structures

With the help of novel processing workflows and algorithms, we have obtained a better understanding of the flexibility and conformational dynamics of the SARS-CoV-2 spike in the prefusion state. We have re-analyzed previous cryo-EM data combining 3D clustering approaches with ways to explore a continuous flexibility space based on 3D Principal Component Analysis. These advanced analyses revealed a concerted motion involving the receptor-binding domain (RBD), N-terminal domain (NTD), and subdomain 1 and 2 (SD1 & SD2) around the previously characterized 1-RBD-up state, which have been modeled as elastic deformations. We show that in this dataset there are not well-defined, stable, spike conformations, but virtually a continuum of states moving in a concerted fashion. We obtained an improved resolution ensemble map with minimum bias, from which we model by flexible fitting the extremes of the change along the direction of maximal variance. Moreover, a high-resolution structure of a recently described biochemically stabilized form of the spike is shown to greatly reduce the dynamics observed for the wild-type spike. Our results provide new detailed avenues to potentially restrain the spike dynamics for structure-based drug and vaccine design and at the same time give a warning of the potential image processing classification instability of these complicated datasets, having a direct impact on the interpretability of the results.

Two particle-picking procedures for filamentous proteins: SPHIRE-crYOLO filament mode and SPHIRE-STRIPER

Two approaches for the selection of filaments from cryo-EM micrographs are described.

Structure determination of filamentous molecular complexes involves the selection of filaments from cryo-EM micrographs. The automatic selection of helical specimens is particularly difficult, and thus many challenging samples with issues such as contamination or aggregation are still manually picked. Here, two approaches for selecting filamentous complexes are presented: one uses a trained deep neural network to identify the filaments and is integrated in SPHIRE-crYOLO, while the other, called SPHIRE-STRIPER, is based on a classical line-detection approach. The advantage of the crYOLO-based procedure is that it performs accurately on very challenging data sets and selects filaments with high accuracy. Although STRIPER is less precise, the user benefits from less intervention, since in contrast to crYOLO, STRIPER does not require training. The performance of both procedures on Tobacco mosaic virus and filamentous F-actin data sets is described to demonstrate the robustness of each method.