Review: automatic particle detection in electron microscopy

Accurate size-based protein localization from cryo-ET tomograms

Cryo-electron tomography (cryo-ET) combined with sub-tomogram averaging (STA) allows the determination of protein structures imaged within the native context of the cell at near-atomic resolution. Particle picking is an essential step in the cryo-ET/STA image analysis pipeline that consists in locating the position of proteins within crowded cellular tomograms so that they can be aligned and averaged in 3D to improve resolution. While extensive work in 2D particle picking has been done in the context of single-particle cryo-EM, comparatively fewer strategies have been proposed to pick particles from 3D tomograms, in part due to the challenges associated with working with noisy 3D volumes affected by the missing wedge. While strategies based on 3D template-matching and deep learning are commonly used, these methods are computationally expensive and require either an external template or manual labelling which can bias the results and limit their applicability. Here, we propose a size-based method to pick particles from tomograms that is fast, accurate, and does not require external templates or user provided labels. We compare the performance of our approach against a commonly used algorithm based on deep learning, crYOLO, and show that our method: i) has higher detection accuracy, ii) does not require user input for labeling or time-consuming training, and iii) runs efficiently on non-specialized CPU hardware. We demonstrate the effectiveness of our approach by automatically detecting particles from tomograms representing different types of samples and using these particles to determine the high-resolution structures of ribosomes imaged in vitro and in situ.

UTILE-Gen: Automated Image Analysis in Nanoscience Using Synthetic Dataset Generator and Deep Learning

This work presents the development and implementation of a deep learning-based workflow for autonomous image analysis in nanoscience. A versatile, agnostic, and configurable tool was developed to generate instance-segmented imaging datasets of nanoparticles. The synthetic generator tool employs domain randomization to expand the image/mask pairs dataset for training supervised deep learning models. The approach eliminates tedious manual annotation and allows training of high-performance models for microscopy image analysis based on convolutional neural networks. We demonstrate how the expanded training set can significantly improve the performance of the classification and instance segmentation models for a variety of nanoparticle shapes, ranging from spherical-, cubic-, to rod-shaped nanoparticles. Finally, the trained models were deployed in a cloud-based analytics platform for the autonomous particle analysis of microscopy images.

A Novel Plenoptic Camera-Based Measurement System for the Investigation into Flight and Combustion Behavior of Refuse-Derived Fuel Particles

In the past several decades, refuse-derived fuels (RDFs) have been widely applied in industrial combustion processes, for instance, in cement production. Since RDF is composed of various waste fractions with complex shapes, its flight and combustion behaviors can be relatively complicated. In this paper, we present a novel plenoptic camera-based spatial measurement system that uses image processing approaches to determine the dwell time, the space-sliced velocity in the depth direction, and the ignition time of various applied RDF fractions based on the obtained images. The image processing approach follows the concept of tracking-by-detection and includes a novel combined detection method, a 2.5D multiple particle tracking algorithm, and a postprocessing framework to tackle the issues in the initial tracking results. The thereby obtained complete spatial fuel trajectories enable the analysis of the flight behaviors elaborated in the paper. The acquired particles' properties (duration, velocity, and ignition time) reversely prove the availability and applicability of the developed measurement system. The adequacy and accuracy of the proposed novel measurement system are validated by the experiments of detecting and tracking burning and nonburning fuel particles in a rotary kiln. This new measurement system and the provided experimental results can benefit a better understanding of the RDF's combustion for future research.

Two- and Three-Dimensional Benchmarks for Particle Detection from an Industrial Rotary Kiln Combustion Chamber Based on Light-Field-Camera Recording

This paper describes a benchmark dataset for the detection of fuel particles in 2D and 3D image data in a rotary kiln combustion chamber. The specific challenges of detecting the small particles under demanding environmental conditions allows for the performance of existing and new particle detection techniques to be evaluated. The data set includes a classification of burning and non-burning particles, which can be in the air but also on the rotary kiln wall. The light-field camera used for data generation offers the potential to develop and objectively evaluate new advanced particle detection methods due to the additional 3D information. Besides explanations of the data set and the contained ground truth, an evaluation procedure of the particle detection based on the ground truth and results for an own particle detection procedure for the data set are presented.

Trainable Segmentation for Transmission Electron Microscope Images of Inorganic Nanoparticles

We present a trainable segmentation method implemented within the python package ParticleSpy. The method takes user labelled pixels, which are used to train a classifier and segment images of inorganic nanoparticles from transmission electron microscope images. This implementation is based on the trainable Waikato Environment for Knowledge Analysis (WEKA) segmentation, but is written in python, allowing a large degree of flexibility and meaning it can be easily expanded using other python packages. We find that trainable segmentation offers better accuracy than global or local thresholding methods and requires as few as 100 user-labelled pixels to produce an accurate segmentation. Trainable segmentation presents a balance of accuracy and training time between global/local thresholding and neural networks, when used on transmission electron microscope images of nanoparticles. We also quantitatively investigate the effectiveness of the components of trainable segmentation, its filter kernels and classifiers, in order to demonstrate the use cases for the different filter kernels in ParticleSpy and the most accurate classifiers for different data types. A set of filter kernels is identified that are effective in distinguishing particles from background but that retain dissimilar features. In terms of classifiers, we find that different classifiers perform optimally for different image contrast; specifically, a Random Forest classifier performs best for high-contrast ADF images, but that QDA and Gaussian Naïve Bayes classifiers perform better for low-contrast TEM images. This article is protected by copyright. All rights reserved.

Machine Learning for Structure Determination in Single-Particle Cryo-Electron Microscopy: A Systematic Review

Recently, single-particle cryo-electron microscopy (cryo-EM) has become an indispensable method for determining macromolecular structures at high resolution to deeply explore the relevant molecular mechanism. Its recent breakthrough is mainly because of the rapid advances in hardware and image processing algorithms, especially machine learning. As an essential support of single-particle cryo-EM, machine learning has powered many aspects of structure determination and greatly promoted its development. In this article, we provide a systematic review of the applications of machine learning in this field. Our review begins with a brief introduction of single-particle cryo-EM, followed by the specific tasks and challenges of its image processing. Then, focusing on the workflow of structure determination, we describe relevant machine learning algorithms and applications at different steps, including particle picking, 2-D clustering, 3-D reconstruction, and other steps. As different tasks exhibit distinct characteristics, we introduce the evaluation metrics for each task and summarize their dynamics of technology development. Finally, we discuss the open issues and potential trends in this promising field.

A deep learning approach using synthetic images for segmenting and estimating 3D orientation of nanoparticles in EM images

BACKGROUND AND OBJECTIVE

Nanoparticles present properties that can be applied to a wide range of fields such as biomedicine, electronics or optics. The type of properties depends on several characteristics, being some of them related with the particle structure. A proper characterization of nanoparticles is crucial since it could affect their applications. To characterize a particle shape and size, the nanotechnologists employ Electron Microscopy (EM) to obtain images of nanoparticles and perform measures over them. This task could be tedious, repetitive and slow, we present a Deep Learning method based on Convolutional Neural Networks (CNNs) to detect, segment, infer orientations and reconstruct microscope images of nanoparticles. Since machine learning algorithms depend on annotated data and there is a lack of annotated datasets of nanoparticles, our work makes use of artificial datasets of images resembling real nanoparticles photographs.

METHODS

Our work is divided into three tasks. Firstly, a method to create annotated datasets of artificial images resembling Scanning Electron Microscope (SEM). Secondly, two models of convolutional neural networks are trained using the artificial datasets previously generated, the first one is in charge of the detection and segmentation of the nanoparticles while the second one will infer the nanoparticle orientation. Finally, the 3D reconstruction module will recreate in a 3D scene the set of detected particles.

RESULTS

We have tested our method with five different shapes of basic nanoparticles: spheres, cubes, ellipsoids, hexagonal discs and octahedrons. An analysis of the reconstructions was conducted by manually comparing each of them with the real images. The results obtained have been promising, the particles are segmented and reconstructed accordingly to their shapes and orientations.

CONCLUSIONS

We have developed a method for nanoparticle detection and segmentation in microscope images. Moreover, we can also infer an approximation of the 3D orientation of the particles and, in conjunction with the detections, create a 3D reconstruction of the photographs. The novelty of our approximation lies in the dataset used. Instead of using annotated images, we have created the datasets simulating the microscope images by using basic geometrical objects that imitate real nanoparticles.

Nanowire Detection in AFM Images Using Deep Learning

Atomic force microscope (AFM) based nanomanipulation has been proved to be a possible way to assemble various nanoparticles into complex patterns and devices. To achieve an efficient and full-automatic nanomanipulation, the nanoparticles as well as other features on the substrate must be quickly identified by the computer. This work focuses on an autodetection method for flexible nanowires based on deep learning. The You Only Look Once (YOLO) network is applied to find all movable nanowires in the AFM images. A series of morphology transformation algorithms, including an adaptive threshold edge detection, are applied to refine the skeletons of the nanowires. The bidirectional long short-term memory model with conditional random field layer (BI-LSTM-CRF) is proposed to precisely determine the posture and position of the detected nanowires. Benefiting from these algorithms, our detecting program is able to automatically detect the nanowires of different morphology with nanometer resolution and with over 80% reliability in the testing set. The detecting results are less affected by the image quality, which demonstrates good robustness of this algorithm.

Deep-learning with synthetic data enables automated picking of cryo-EM particle images of biological macromolecules

MOTIVATION

Single-particle cryo-electron microscopy (cryo-EM) has become a powerful technique for determining 3D structures of biological macromolecules at near-atomic resolution. However, this approach requires picking huge numbers of macromolecular particle images from thousands of low-contrast, high-noisy electron micrographs. Although machine-learning methods were developed to get rid of this bottleneck, it still lacks universal methods that could automatically picking the noisy cryo-EM particles of various macromolecules.

RESULTS

Here, we present a deep-learning segmentation model that employs fully convolutional networks trained with synthetic data of known 3D structures, called PARSED (PARticle SEgmentation Detector). Without using any experimental information, PARSED could automatically segment the cryo-EM particles in a whole micrograph at a time, enabling faster particle picking than previous template/feature-matching and particle-classification methods. Applications to six large public cryo-EM datasets clearly validated its universal ability to pick macromolecular particles of various sizes. Thus, our deep-learning method could break the particle-picking bottleneck in the single-particle analysis, and thereby accelerates the high-resolution structure determination by cryo-EM.

AVAILABILITY AND IMPLEMENTATION

The PARSED package and user manual for noncommercial use are available as Supplementary Material (in the compressed file: parsed_v1.zip).

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

Electron microscopy-based semi-automated characterization of aggregation in monoclonal antibody products

•

Size-based quantification of small heterogeneous proteins using electron microscopy.

•

Electron microscopy as an orthogonal tool for characterizing protein aggregates.

•

Quick assessment of small heterogeneous proteins via softEM, a GUI-based algorithm.

Aggregation is a critical parameter for protein-based therapeutics, due to its impact on the immunogenicity of the product. The traditional approach towards characterization of such products is to use a collection of orthogonal tools. However, the fact that none of these tools is able to completely classify the distribution and physical characteristics of aggregates, implies that there exists a need for additional analytical methods. We report one such method for characterization of heterogeneous population of proteins using transmission electron microscopy. The method involves semi-automated, size-based clustering of different protein species from micrographs. This method can be utilized for quantitative characterization of heterogeneous populations of antibody/protein aggregates from TEM images of proteins, and may also be applicable towards other instances of protein aggregation.

Two particle-picking procedures for filamentous proteins: SPHIRE-crYOLO filament mode and SPHIRE-STRIPER

Two approaches for the selection of filaments from cryo-EM micrographs are described.

Structure determination of filamentous molecular complexes involves the selection of filaments from cryo-EM micrographs. The automatic selection of helical specimens is particularly difficult, and thus many challenging samples with issues such as contamination or aggregation are still manually picked. Here, two approaches for selecting filamentous complexes are presented: one uses a trained deep neural network to identify the filaments and is integrated in SPHIRE-crYOLO, while the other, called SPHIRE-STRIPER, is based on a classical line-detection approach. The advantage of the crYOLO-based procedure is that it performs accurately on very challenging data sets and selects filaments with high accuracy. Although STRIPER is less precise, the user benefits from less intervention, since in contrast to crYOLO, STRIPER does not require training. The performance of both procedures on Tobacco mosaic virus and filamentous F-actin data sets is described to demonstrate the robustness of each method.

Research journey of respirasome

Respirasome, as a vital part of the oxidative phosphorylation system, undertakes the task of transferring electrons from the electron donors to oxygen and produces a proton concentration gradient across the inner mitochondrial membrane through the coupled translocation of protons. Copious research has been carried out on this lynchpin of respiration. From the discovery of individual respiratory complexes to the report of the high-resolution structure of mammalian respiratory supercomplex I1III2IV1, scientists have gradually uncovered the mysterious veil of the electron transport chain (ETC). With the discovery of the mammalian respiratory mega complex I2III2IV2, a new perspective emerges in the research field of the ETC. Behind these advances glitters the light of the revolution in both theory and technology. Here, we give a short review about how scientists 'see' the structure and the mechanism of respirasome from the macroscopic scale to the atomic scale during the past decades.

A Super-Clustering Approach for Fully Automated Single Particle Picking in Cryo-EM

Structure determination of proteins and macromolecular complexes by single-particle cryo-electron microscopy (cryo-EM) is poised to revolutionize structural biology. An early challenging step in the cryo-EM pipeline is the detection and selection of particles from two-dimensional micrographs (particle picking). Most existing particle-picking methods require human intervention to deal with complex (irregular) particle shapes and extremely low signal-to-noise ratio (SNR) in cryo-EM images. Here, we design a fully automated super-clustering approach for single particle picking (SuperCryoEMPicker) in cryo-EM micrographs, which focuses on identifying, detecting, and picking particles of the complex and irregular shapes in micrographs with extremely low signal-to-noise ratio (SNR). Our method first applies advanced image processing procedures to improve the quality of the cryo-EM images. The binary mask image-highlighting protein particles are then generated from each individual cryo-EM image using the super-clustering (SP) method, which improves upon base clustering methods (i.e., k-means, fuzzy c-means (FCM), and intensity-based cluster (IBC) algorithm) via a super-pixel algorithm. SuperCryoEMPicker is tested and evaluated on micrographs of β-galactosidase and 80S ribosomes, which are examples of cryo-EM data exhibiting complex and irregular particle shapes. The results show that the super-particle clustering method provides a more robust detection of particles than the base clustering methods, such as k-means, FCM, and IBC. SuperCryoEMPicker automatically and effectively identifies very complex particles from cryo-EM images of extremely low SNR. As a fully automated particle detection method, it has the potential to relieve researchers from laborious, manual particle-labeling work and therefore is a useful tool for cryo-EM protein structure determination.

SPHIRE-crYOLO is a fast and accurate fully automated particle picker for cryo-EM

Selecting particles from digital micrographs is an essential step in single-particle electron cryomicroscopy (cryo-EM). As manual selection of complete datasets-typically comprising thousands of particles-is a tedious and time-consuming process, numerous automatic particle pickers have been developed. However, non-ideal datasets pose a challenge to particle picking. Here we present the particle picking software crYOLO which is based on the deep-learning object detection system You Only Look Once (YOLO). After training the network with 200-2500 particles per dataset it automatically recognizes particles with high recall and precision while reaching a speed of up to five micrographs per second. Further, we present a general crYOLO network able to pick from previously unseen datasets, allowing for completely automated on-the-fly cryo-EM data preprocessing during data acquisition. crYOLO is available as a standalone program under http://sphire.mpg.de/ and is distributed as part of the image processing workflow in SPHIRE.

AutoCryoPicker: an unsupervised learning approach for fully automated single particle picking in Cryo-EM images

Background

An important task of macromolecular structure determination by cryo-electron microscopy (cryo-EM) is the identification of single particles in micrographs (particle picking). Due to the necessity of human involvement in the process, current particle picking techniques are time consuming and often result in many false positives and negatives. Adjusting the parameters to eliminate false positives often excludes true particles in certain orientations. The supervised machine learning (e.g. deep learning) methods for particle picking often need a large training dataset, which requires extensive manual annotation. Other reference-dependent methods rely on low-resolution templates for particle detection, matching and picking, and therefore, are not fully automated. These issues motivate us to develop a fully automated, unbiased framework for particle picking.

Results

We design a fully automated, unsupervised approach for single particle picking in cryo-EM micrographs. Our approach consists of three stages: image preprocessing, particle clustering, and particle picking. The image preprocessing is based on multiple techniques including: image averaging, normalization, cryo-EM image contrast enhancement correction (CEC), histogram equalization, restoration, adaptive histogram equalization, guided image filtering, and morphological operations. Image preprocessing significantly improves the quality of original cryo-EM images. Our particle clustering method is based on an intensity distribution model which is much faster and more accurate than traditional K-means and Fuzzy C-Means (FCM) algorithms for single particle clustering. Our particle picking method, based on image cleaning and shape detection with a modified Circular Hough Transform algorithm, effectively detects the shape and the center of each particle and creates a bounding box encapsulating the particles.

Conclusions

AutoCryoPicker can automatically and effectively recognize particle-like objects from noisy cryo-EM micrographs without the need of labeled training data or human intervention making it a useful tool for cryo-EM protein structure determination.

Electronic supplementary material

The online version of this article (10.1186/s12859-019-2926-y) contains supplementary material, which is available to authorized users.

Robustness of signal detection in cryo-electron microscopy via a bi-objective-function approach

Background

The detection of weak signals and selection of single particles from low-contrast micrographs of frozen hydrated biomolecules by cryo-electron microscopy (cryo-EM) represents a major practical bottleneck in cryo-EM data analysis. Template-based particle picking by an objective function using fast local correlation (FLC) allows computational extraction of a large number of candidate particles from micrographs. Another independent objective function based on maximum likelihood estimates (MLE) can be used to align the images and verify the presence of a signal in the selected particles. Despite the widespread applications of the two objective functions, an optimal combination of their utilities has not been exploited. Here we propose a bi-objective function (BOF) approach that combines both FLC and MLE and explore the potential advantages and limitations of BOF in signal detection from cryo-EM data.

Results

The robustness of the BOF strategy in particle selection and verification was systematically examined with both simulated and experimental cryo-EM data. We investigated how the performance of the BOF approach is quantitatively affected by the signal-to-noise ratio (SNR) of cryo-EM data and by the choice of initialization for FLC and MLE. We quantitatively pinpointed the critical SNR (~ 0.005), at which the BOF approach starts losing its ability to select and verify particles reliably. We found that the use of a Gaussian model to initialize the MLE suppresses the adverse effects of reference dependency in the FLC function used for template-matching.

Conclusion

The BOF approach, which combines two distinct objective functions, provides a sensitive way to verify particles for downstream cryo-EM structure analysis. Importantly, reference dependency of the FLC does not necessarily transfer to the MLE, enabling the robust detection of weak signals. Our insights into the numerical behavior of the BOF approach can be used to improve automation efficiency in the cryo-EM data processing pipeline for high-resolution structural determination.

Electronic supplementary material

The online version of this article (10.1186/s12859-019-2714-8) contains supplementary material, which is available to authorized users.

Automatic detection, localization and segmentation of nano-particles with deep learning in microscopy images

With the growing amount of high resolution microscopy images automatic nano-particle detection, shape analysis and size determination have gained importance for providing quantitative support that gives important information for the evaluation of the material. In this paper, we present a new method for detection of nano-particles and determination of their shapes and sizes simultaneously with deep learning. The proposed method employs multiple output convolutional neural networks (MO-CNN) and has two outputs: first is the detection output that gives the locations of the particles and the other one is the segmentation output for providing the boundaries of the nano-particles. The final sizes of particles are determined with the modified Hough algorithm that runs on the segmentation output. The proposed method is tested and evaluated on a dataset containing 17 TEM images of Fe₃O₄ and silica coated nano-particles. Also, we compared these results with U-net algorithm which is a popular deep learning method. The experiments showed that the proposed method has 98.23% accuracy for detection and 96.59% accuracy for segmentation of nano-particles.

Differentiating Polymorphs in Molybdenum Disulfide via Electron Microscopy

The presence of rich polymorphs and stacking polytypes in molybdenum disulfide (MoS₂ ) endows it with a diverse range of electrical, catalytic, optical, and magnetic properties. This has stimulated a lot of interest in the unique properties associated with each polymorph. Most techniques used for polymorph identification in MoS₂ are macroscopic techniques that sample averaged properties due to their limited spatial resolution. A reliable way of differentiating the atomic structure of different polymorphs is needed in order to understand their growth dynamics and establish the correlation between structure and properties. Herein, the use of electron microscopy for identifying the atomic structures of several important polymorphs in MoS₂ , some of which are the subjects of mistaken assignment in the literature, is discussed. In particular, scanning transmission electron microscopy-annular dark field imaging has emerged as the most effective and reliable approach for identifying the different phases in MoS₂ and other 2D materials because its images can be directly correlated to the atomic structures. Examples of the identification of polymorphs grown under different conditions in molecular beam epitaxy or chemical vapor deposition, for example, 3R, 1T, 1T'-phases, and 1T'-edges, are presented, including their atomic structures, fascinating properties, growth methods, and corresponding thermodynamic stabilities.

APPLE picker: Automatic particle picking, a low-effort cryo-EM framework

Particle picking is a crucial first step in the computational pipeline of single-particle cryo-electron microscopy (cryo-EM). Selecting particles from the micrographs is difficult especially for small particles with low contrast. As high-resolution reconstruction typically requires hundreds of thousands of particles, manually picking that many particles is often too time-consuming. While template-based particle picking is currently a popular approach, it may suffer from introducing manual bias into the selection process. In addition, this approach is still somewhat time-consuming. This paper presents the APPLE (Automatic Particle Picking with Low user Effort) picker, a simple and novel approach for fast, accurate, and template-free particle picking. This approach is evaluated on publicly available datasets containing micrographs of β-galactosidase, T20S proteasome, 70S ribosome and keyhole limpet hemocyanin projections.

Virus Particle Detection by Convolutional Neural Network in Transmission Electron Microscopy Images

A new computational method for the detection of virus particles in transmission electron microscopy (TEM) images is presented. Our approach is to use a convolutional neural network that transforms a TEM image to a probabilistic map that indicates where virus particles exist in the image. Our proposed approach automatically and simultaneously learns both discriminative features and classifier for virus particle detection by machine learning, in contrast to existing methods that are based on handcrafted features that yield many false positives and require several postprocessing steps. The detection performance of the proposed method was assessed against a dataset of TEM images containing feline calicivirus particles and compared with several existing detection methods, and the state-of-the-art performance of the developed method for detecting virus was demonstrated. Since our method is based on supervised learning that requires both the input images and their corresponding annotations, it is basically used for detection of already-known viruses. However, the method is highly flexible, and the convolutional networks can adapt themselves to any virus particles by learning automatically from an annotated dataset.

Automated tracing of helical assemblies from electron cryo-micrographs

Structure determination of helical specimens commonly requires datasets from thousands of micrographs often obtained by automated cryo-EM data acquisition. Interactive tracing of helical assemblies from such a number of micrographs is labor-intense and time-consuming. Here, we introduce an automated tracing tool MicHelixTrace that precisely locates helix traces from micrographs of rigid as well as very flexible helical assemblies with small numbers of false positives. The computer program is fast and has low computational requirements. In addition to helix coordinates required for a subsequent helical reconstruction work-flow, we determine the persistence length of the polymer ensemble. This information provides a useful measure to characterize mechanical properties of helical assemblies and to evaluate the potential for high-resolution structure determination.

DeepPicker: A deep learning approach for fully automated particle picking in cryo-EM

Particle picking is a time-consuming step in single-particle analysis and often requires significant interventions from users, which has become a bottleneck for future automated electron cryo-microscopy (cryo-EM). Here we report a deep learning framework, called DeepPicker, to address this problem and fill the current gaps toward a fully automated cryo-EM pipeline. DeepPicker employs a novel cross-molecule training strategy to capture common features of particles from previously-analyzed micrographs, and thus does not require any human intervention during particle picking. Tests on the recently-published cryo-EM data of three complexes have demonstrated that our deep learning based scheme can successfully accomplish the human-level particle picking process and identify a sufficient number of particles that are comparable to those picked manually by human experts. These results indicate that DeepPicker can provide a practically useful tool to significantly reduce the time and manual effort spent in single-particle analysis and thus greatly facilitate high-resolution cryo-EM structure determination. DeepPicker is released as an open-source program, which can be downloaded from https://github.com/nejyeah/DeepPicker-python.

Robust w-Estimators for Cryo-EM Class Means

A critical step in cryogenic electron microscopy (cryo-EM) image analysis is to calculate the average of all images aligned to a projection direction. This average, called the class mean, improves the signal-to-noise ratio in single-particle reconstruction. The averaging step is often compromised because of the outlier images of ice, contaminants, and particle fragments. Outlier detection and rejection in the majority of current cryo-EM methods are done using cross-correlation with a manually determined threshold. Empirical assessment shows that the performance of these methods is very sensitive to the threshold. This paper proposes an alternative: a w-estimator of the average image, which is robust to outliers and which does not use a threshold. Various properties of the estimator, such as consistency and influence function are investigated. An extension of the estimator to images with different contrast transfer functions is also provided. Experiments with simulated and real cryo-EM images show that the proposed estimator performs quite well in the presence of outliers.

Single Nanoparticle Detection Using Far-field Emission of Photonic Molecule around the Exceptional Point

Highly sensitive, label-free detection methods have important applications in fundamental research and healthcare diagnostics. To date, the detection of single nanoparticles has remained largely dependent on extremely precise spectral measurement, which relies on high-cost equipment. Here, we demonstrate a simple but very nontrivial mechanism for the label-free sizing of nanoparticles using the far-field emission of a photonic molecule (PM) around an exceptional point (EP). By attaching a nanoparticle to a PM around an EP, the main resonant behaviors are strongly disturbed. In addition to typical mode splitting, we find that the far-field pattern of the PM is significantly changed. Taking a heteronuclear diatomic PM as an example, we demonstrate that a single nanoparticle, whose radius is as small as 1 nm to 7 nm, can be simply monitored through the variation of the far-field pattern. Compared with conventional methods, our approach is much easier and does not rely on high-cost equipment. In addition, this research will illuminate new advances in single nanoparticle detection.

Robust estimation for class averaging in cryo-EM Single Particle Reconstruction

Single Particle Reconstruction (SPR) for Cryogenic Electron Microscopy (cryo-EM) aligns and averages the images extracted from micrographs to improve the Signal-to-Noise ratio (SNR). Outliers compromise the fidelity of the averaging. We propose a robust cross-correlation-like w-estimator for combating the effect of outliers on the average images in cryo-EM. The estimator accounts for the natural variation of signal contrast among the images and eliminates the need for a threshold for outlier rejection. We show that the influence function of our estimator is asymptotically bounded. Evaluations of the estimator on simulated and real cryo-EM images show good performance in the presence of outliers.

Semi-automated selection of cryo-EM particles in RELION-1.3

The selection of particles suitable for high-resolution cryo-EM structure determination from noisy micrographs may represent a tedious and time-consuming step. Here, a semi-automated particle selection procedure is presented that has been implemented within the open-source software RELION. At the heart of the procedure lies a fully CTF-corrected template-based picking algorithm, which is supplemented by a fast sorting algorithm and reference-free 2D class averaging to remove false positives. With only limited user-interaction, the proposed procedure yields results that are comparable to manual particle selection. Together with an improved graphical user interface, these developments further contribute to turning RELION from a stand-alone refinement program into a convenient image processing pipeline for the entire single-particle approach.

Progress and outlook in structural biology of large viral RNAs

The field of viral molecular biology has reached a precipice for which pioneering studies on the structure of viral RNAs are beginning to bridge the gap. It has become clear that viral genomic RNAs are not simply carriers of hereditary information, but rather are active players in many critical stages during replication. Indeed, functions such as cap-independent translation initiation mechanisms are, in some cases, primarily driven by RNA structural determinants. Other stages including reverse transcription initiation in retroviruses, nuclear export and viral packaging are specifically dependent on the proper 3-dimensional folding of multiple RNA domains to recruit necessary viral and host factors required for activity. Furthermore, a large-scale conformational change within the 5'-untranslated region of HIV-1 has been proposed to regulate the temporal switch between viral protein synthesis and packaging. These RNA-dependent functions are necessary for replication of many human disease-causing viruses such as severe acute respiratory syndrome (SARS)-associated coronavirus, West Nile virus, and HIV-1. The potential for antiviral development is currently hindered by a poor understanding of RNA-driven molecular mechanisms, resulting from a lack of structural information on large RNAs and ribonucleoprotein complexes. Herein, we describe the recent progress that has been made on characterizing these large RNAs and provide brief descriptions of the techniques that will be at the forefront of future advances. Ongoing and future work will contribute to a more complete understanding of the lifecycles of retroviruses and RNA viruses and potentially lead to novel antiviral strategies.

gEMpicker: a highly parallel GPU-accelerated particle picking tool for cryo-electron microscopy

Background

Picking images of particles in cryo-electron micrographs is an important step in solving the 3D structures of large macromolecular assemblies. However, in order to achieve sub-nanometre resolution it is often necessary to capture and process many thousands or even several millions of 2D particle images. Thus, a computational bottleneck in reaching high resolution is the accurate and automatic picking of particles from raw cryo-electron micrographs.

Results

We have developed “gEMpicker”, a highly parallel correlation-based particle picking tool. To our knowledge, gEMpicker is the first particle picking program to use multiple graphics processor units (GPUs) to accelerate the calculation. When tested on the publicly available keyhole limpet hemocyanin dataset, we find that gEMpicker gives similar results to the FindEM program. However, compared to calculating correlations on one core of a contemporary central processor unit (CPU), running gEMpicker on a modern GPU gives a speed-up of about 27 ×. To achieve even higher processing speeds, the basic correlation calculations are accelerated considerably by using a hierarchy of parallel programming techniques to distribute the calculation over multiple GPUs and CPU cores attached to multiple nodes of a computer cluster. By using a theoretically optimal reduction algorithm to collect and combine the cluster calculation results, the speed of the overall calculation scales almost linearly with the number of cluster nodes available.

Conclusions

The very high picking throughput that is now possible using GPU-powered workstations or computer clusters will help experimentalists to achieve higher resolution 3D reconstructions more rapidly than before.

Texture indicators for segmentation of polyomavirus particles in transmission electron microscopy images

A fully automatic approach to locate polyomavirus particles in transmission electron microscopy images is presented that can localize intact particles, many damaged capsids, and an acceptable percentage of superposed ones. Performance of the approach is quantified in 25 electron micrographs containing nearly 390 particles and compared with the interpretation of the micrographs by two independent electron microscopy experts. All parameterization is based on the particle expected dimensions. This approach uses indicators calculated from the local co-occurrence matrix of gray levels to assess the textured pattern typical of polyomavirus and prune the initial set of candidates. In more complicated backgrounds, about 2-10% of the elements survive. A restricted set of the accepted points is used to evaluate the typical average and variance and to reduce the set of survivors accordingly. These intermediate points are evaluated using (i) a statistical index concerning the radiometric distribution of a circular neighborhood around the centroid of each candidate and (ii) a structural index resuming the expected morphological characteristics of eight radial intensity profiles encompassing the area of the possible particle. This hierarchical approach attains 90% efficiency in the detection of entire virus particles, tolerating a certain lack of differentiation in the borders and a certain amount of shape alterations.

Automatic virus particle selection--the entropy approach

This paper describes a fully automatic approach to locate icosahedral virus particles in transmission electron microscopy images. The initial detection of the particles takes place through automatic segmentation of the entropy-proportion image; this image is computed in particular regions of interest defined by two concentric structuring elements contained in a small overlapping window running over all the image. Morphological features help to select the candidates, as the threshold is kept low enough to avoid false negatives. The candidate points are subject to a credibility test based on features extracted from eight radial intensity profiles in each point from a texture image. A candidate is accepted if these features meet the set of acceptance conditions describing the typical intensity profiles of these kinds of particles. The set of points accepted is subjected to a last validation in a three-parameter space using a discrimination plan that is a function of the input image to separate possible outliers.

Multi-reference-based multiple alignment statistics enables accurate protein-particle pickup from noisy images

Data mining from noisy data/images is one of the most important themes in modern science and technology. Statistical image processing is a promising technique for analysing such data. Automation of particle pickup from noisy electron micrographs is essential, especially when improvement of the resolution of single particle analysis requires a huge number of particle images. For such a purpose, reference-based matching using primary three-dimensional (3D) model projections is mainly adopted. In the matching, however, the highest peaks of the correlation may not accurately indicate particles when the image is very noisy. In contrast, the density and the heights of the peaks should reflect the probability distribution of the particles. To statistically determine the particle positions from the peak distributions, we have developed a density-based peak search followed by a peak selection based on average peak height, using multi-reference alignment (MRA). Its extension, using multi-reference multiple alignment (MRMA), was found to enable particle pickup at higher accuracy even from extremely noisy images with a signal-to-noise ratio of 0.001. We refer to these new methods as stochastic pickup with MRA (MRA-StoPICK) or with MRMA (MRMA-StoPICK). MRMA-StoPICK has a higher pickup accuracy and furthermore, is almost independent of parameter settings. They were successfully applied to cryo-electron micrographs of Rice dwarf virus. Because current computational resources and parallel data processing environments allow somewhat CPU-intensive MRA-StoPICK and MRMA-StoPICK to be performed in a short period, these methods are expected to allow high-resolution analysis of the 3D structure of particles.

Reference-free particle selection enhanced with semi-supervised machine learning for cryo-electron microscopy

Reference-based methods have dominated the approaches to the particle selection problem, proving fast, and accurate on even the most challenging micrographs. A reference volume, however, is not always available and compiling a set of reference projections from the micrographs themselves requires significant effort to attain the same level of accuracy. We propose a reference-free method to quickly extract particles from the micrograph. The method is augmented with a new semi-supervised machine-learning algorithm to accurately discriminate particles from contaminants and noise.

Computational approaches for automatic structural analysis of large biomolecular complexes

We present computational solutions to two problems of macromolecular structure interpretation from reconstructed three-dimensional electron microscopy (3D-EM) maps of large bio-molecular complexes at intermediate resolution (5A-15 A). The two problems addressed are: 1) 3D structural alignment (matching) between identified and segmented 3D maps of structure units (e.g. trimeric configuration of proteins), and 2) the secondary structure identification of a segmented protein 3D map (i.e.locations of alpha-helices, beta-sheets). For problem 1, we present an efficient algorithm to correlate spatially (and structurally) two 3D maps of structure units. Besides providing a similarity score between structure units, the algorithm yields an effective technique for resolution refinement of repeated structure units, by 3D alignment and averaging. For problem 2, we present an efficient algorithm to compute eigenvalues and link eigenvectors of a Gaussian convoluted structure tensor derived from the protein 3D Map, thereby identifying and locating secondary structural motifs of proteins. The efficiency and performance of our approach is demonstrated on several experimentally reconstructed 3D maps of virus capsid shells from single-particle cryo-electron microscopy (cryo-EM), as well as computationally simulated protein structure density 3D maps generated from protein model entries in the Protein Data Bank.

Evaluation of denoising algorithms for biological electron tomography

Tomograms of biological specimens derived using transmission electron microscopy can be intrinsically noisy due to the use of low electron doses, the presence of a "missing wedge" in most data collection schemes, and inaccuracies arising during 3D volume reconstruction. Before tomograms can be interpreted reliably, for example, by 3D segmentation, it is essential that the data be suitably denoised using procedures that can be individually optimized for specific data sets. Here, we implement a systematic procedure to compare various nonlinear denoising techniques on tomograms recorded at room temperature and at cryogenic temperatures, and establish quantitative criteria to select a denoising approach that is most relevant for a given tomogram. We demonstrate that using an appropriate denoising algorithm facilitates robust segmentation of tomograms of HIV-infected macrophages and Bdellovibrio bacteria obtained from specimens at room and cryogenic temperatures, respectively. We validate this strategy of automated segmentation of optimally denoised tomograms by comparing its performance with manual extraction of key features from the same tomograms.

Automated analysis of the architecture of receptors, imaged by atomic force microscopy

Fast neurotransmission involves the operation of ionotropic receptors, which are multi-subunit proteins that respond to activation by opening an integral ion channel. Examples of such channels include the GABA(A) receptor, the 5-HT(3) receptor and the P2X receptor for ATP. These receptors contain more than one type of subunit, although the exact subunit stoichiometry and arrangement around the receptor rosette is often unknown. We are using atomic force microscopy (AFM) of purified receptors to address these issues. Measurement of the molecular volume of the receptor permits the determination of the number of subunits that it contains. Furthermore, analysis of the geometry of complexes between receptors and subunit-specific antibodies reveals the subunit arrangement. Our AFM-based approach has so far been dependent on manual data processing, which is both time-consuming and prone to operator bias. In this study, we set out to develop a novel method capable of automatic segmentation and quantitative analysis of both single receptor particles and receptor-antibody complexes. The method was validated using images of wild type and mutant forms of the P2X(6) receptor. We suggest that the automated method will greatly facilitate further progress in the use of AFM for the determination of receptor and multi-protein architecture.

3D electron microscopy of biological nanomachines: principles and applications

Transmission electron microscopy is a powerful technique for studying the three-dimensional (3D) structure of a wide range of biological specimens. Knowledge of this structure is crucial for fully understanding complex relationships among macromolecular complexes and organelles in living cells. In this paper, we present the principles and main application domains of 3D transmission electron microscopy in structural biology. Moreover, we survey current developments needed in this field, and discuss the close relationship of 3D transmission electron microscopy with other experimental techniques aimed at obtaining structural and dynamical information from the scale of whole living cells to atomic structure of macromolecular complexes.

SCA: Symmetry-based center assignment of 2D projections of symmetric 3D objects

A method for finding the center of cryo-EM images which correspond to the projections of a symmetric 3D structure, based on mathematical properties of symmetry adapted functions and the Fourier-Bessel transform, is presented. It is a model independent one-step procedure with no parameters to be chosen by the user. The proposed method is tested in one synthetic tetrahedral case with different noise levels and in two real cases with D7 and icosahedral symmetries.