A convolutional autoencoder approach for mining features in cellular electron cryo-tomograms and weakly supervised coarse segmentation

AITom: AI-guided cryo-electron tomography image analyses toolkit

Cryo-electron tomography (cryo-ET) is an essential tool in structural biology, uniquely capable of visualizing three-dimensional macromolecular complexes within their native cellular environments, thereby providing profound molecular-level insights. Despite its significant promise, cryo-ET faces persistent challenges in the systematic localization, identification, segmentation, and structural recovery of three-dimensional subcellular components, necessitating the development of efficient and accurate large-scale image analysis methods. In response to these complexities, this paper introduces AITom, an open-source artificial intelligence platform tailored for cryo-ET researchers. AITom integrates a comprehensive suite of public and proprietary algorithms, supporting both traditional template-based and template-free approaches, alongside state-of-the-art deep learning methodologies for cryo-ET data analysis. By incorporating diverse computational strategies, AITom enables researchers to more effectively tackle the complexities inherent in cryo-ET, facilitating precise analysis and interpretation of complex biological structures. Furthermore, AITom provides extensive tutorials for each analysis module, offering valuable guidance to users in utilizing its comprehensive functionalities.

High-throughput cryo-ET structural pattern mining by unsupervised deep iterative subtomogram clustering

Cryoelectron tomography directly visualizes heterogeneous macromolecular structures in their native and complex cellular environments. However, existing computer-assisted structure sorting approaches are low throughput or inherently limited due to their dependency on available templates and manual labels. Here, we introduce a high-throughput template-and-label-free deep learning approach, Deep Iterative Subtomogram Clustering Approach (DISCA), that automatically detects subsets of homogeneous structures by learning and modeling 3D structural features and their distributions. Evaluation on five experimental cryo-ET datasets shows that an unsupervised deep learning based method can detect diverse structures with a wide range of molecular sizes. This unsupervised detection paves the way for systematic unbiased recognition of macromolecular complexes in situ.

Cryomicroscopy in situ: what is the smallest molecule that can be directly identified without labels in a cell?

Electron cryomicroscopy (cryoEM) has made great strides in the last decade, such that the atomic structure of most biological macromolecules can, at least in principle, be determined. Major technological advances - in electron imaging hardware, data analysis software, and cryogenic specimen preparation technology - continue at pace and contribute to the exponential growth in the number of atomic structures determined by cryoEM. It is now conceivable that within the next decade we will have structures for hundreds of thousands of unique protein and nucleic acid molecular complexes. But the answers to many important questions in biology would become obvious if we could identify these structures precisely inside cells with quantifiable error. In the context of an abundance of known structures, it is appropriate to consider the current state of electron cryomicroscopy for frozen specimens prepared directly from cells, and try to answer to the question of the title, both now and in the foreseeable future.

Self-supervised learning for macromolecular structure classification based on cryo-electron tomograms

Macromolecular structure classification from cryo-electron tomography (cryo-ET) data is important for understanding macro-molecular dynamics. It has a wide range of applications and is essential in enhancing our knowledge of the sub-cellular environment. However, a major limitation has been insufficient labelled cryo-ET data. In this work, we use Contrastive Self-supervised Learning (CSSL) to improve the previous approaches for macromolecular structure classification from cryo-ET data with limited labels. We first pretrain an encoder with unlabelled data using CSSL and then fine-tune the pretrained weights on the downstream classification task. To this end, we design a cryo-ET domain-specific data-augmentation pipeline. The benefit of augmenting cryo-ET datasets is most prominent when the original dataset is limited in size. Overall, extensive experiments performed on real and simulated cryo-ET data in the semi-supervised learning setting demonstrate the effectiveness of our approach in macromolecular labeling and classification.

Spaghetti Tracer: A Framework for Tracing Semiregular Filamentous Densities in 3D Tomograms

Within cells, cytoskeletal filaments are often arranged into loosely aligned bundles. These fibrous bundles are dense enough to exhibit a certain regularity and mean direction, however, their packing is not sufficient to impose a symmetry between-or specific shape on-individual filaments. This intermediate regularity is computationally difficult to handle because individual filaments have a certain directional freedom, however, the filament densities are not well segmented from each other (especially in the presence of noise, such as in cryo-electron tomography). In this paper, we develop a dynamic programming-based framework, Spaghetti Tracer, to characterizing the structural arrangement of filaments in the challenging 3D maps of subcellular components. Assuming that the tomogram can be rotated such that the filaments are oriented in a mean direction, the proposed framework first identifies local seed points for candidate filament segments, which are then grown from the seeds using a dynamic programming algorithm. We validate various algorithmic variations of our framework on simulated tomograms that closely mimic the noise and appearance of experimental maps. As we know the ground truth in the simulated tomograms, the statistical analysis consisting of precision, recall, and F1 scores allows us to optimize the performance of this new approach. We find that a bipyramidal accumulation scheme for path density is superior to straight-line accumulation. In addition, the multiplication of forward and backward path densities provides for an efficient filter that lifts the filament density above the noise level. Resulting from our tests is a robust method that can be expected to perform well (F1 scores 0.86-0.95) under experimental noise conditions.

Better, Faster, Cheaper: Recent Advances in Cryo-Electron Microscopy

Cryo-electron microscopy (cryo-EM) continues its remarkable growth as a method for visualizing biological objects, which has been driven by advances across the entire pipeline. Developments in both single-particle analysis and in situ tomography have enabled more structures to be imaged and determined to better resolutions, at faster speeds, and with more scientists having improved access. This review highlights recent advances at each stageof the cryo-EM pipeline and provides examples of how these techniques have been used to investigate real-world problems, including antibody development against the SARS-CoV-2 spike during the recent COVID-19 pandemic.

CryoETGAN: Cryo-Electron Tomography Image Synthesis via Unpaired Image Translation

Cryo-electron tomography (Cryo-ET) has been regarded as a revolution in structural biology and can reveal molecular sociology. Its unprecedented quality enables it to visualize cellular organelles and macromolecular complexes at nanometer resolution with native conformations. Motivated by developments in nanotechnology and machine learning, establishing machine learning approaches such as classification, detection and averaging for Cryo-ET image analysis has inspired broad interest. Yet, deep learning-based methods for biomedical imaging typically require large labeled datasets for good results, which can be a great challenge due to the expense of obtaining and labeling training data. To deal with this problem, we propose a generative model to simulate Cryo-ET images efficiently and reliably: CryoETGAN. This cycle-consistent and Wasserstein generative adversarial network (GAN) is able to generate images with an appearance similar to the original experimental data. Quantitative and visual grading results on generated images are provided to show that the results of our proposed method achieve better performance compared to the previous state-of-the-art simulation methods. Moreover, CryoETGAN is stable to train and capable of generating plausibly diverse image samples.

Air-Writing Character Recognition with Ultrasonic Transceivers

The interfaces between users and systems are evolving into a more natural communication, including user gestures as part of the interaction, where air-writing is an emerging application for this purpose. The aim of this work is to propose a new air-writing system based on only one array of ultrasonic transceivers. This track will be obtained based on the pairwise distance of the hand marker with each transceiver. After acquiring the track, different deep learning algorithms, such as long short-term memory (LSTM), convolutional neural networks (CNN), convolutional autoencoder (ConvAutoencoder), and convolutional LSTM have been evaluated for character recognition. It has been shown how these algorithms provide high accuracy, where the best result is extracted from the ConvLSTM, with 99.51% accuracy and 71.01 milliseconds of latency. Real data were used in this work to evaluate the proposed system in a real scenario to demonstrate its high performance regarding data acquisition and classification.

Weakly Supervised 3D Semantic Segmentation Using Cross-Image Consensus and Inter-Voxel Affinity Relations

We propose a novel weakly supervised approach for 3D semantic segmentation on volumetric images. Unlike most existing methods that require voxel-wise densely labeled training data, our weakly-supervised CIVA-Net is the first model that only needs image-level class labels as guidance to learn accurate volumetric segmentation. Our model learns from cross-image co-occurrence for integral region generation, and explores inter-voxel affinity relations to predict segmentation with accurate boundaries. We empirically validate our model on both simulated and real cryo-ET datasets. Our experiments show that CIVA-Net achieves comparable performance to the state-of-the-art models trained with stronger supervision.

The promise and the challenges of cryo-electron tomography

Structural biologists have traditionally approached cellular complexity in a reductionist manner in which the cellular molecular components are fractionated and purified before being studied individually. This 'divide and conquer' approach has been highly successful. However, awareness has grown in recent years that biological functions can rarely be attributed to individual macromolecules. Most cellular functions arise from their concerted action, and there is thus a need for methods enabling structural studies performed in situ, ideally in unperturbed cellular environments. Cryo-electron tomography (Cryo-ET) combines the power of 3D molecular-level imaging with the best structural preservation that is physically possible to achieve. Thus, it has a unique potential to reveal the supramolecular architecture or 'molecular sociology' of cells and to discover the unexpected. Here, we review state-of-the-art Cryo-ET workflows, provide examples of biological applications, and discuss what is needed to realize the full potential of Cryo-ET.

A unified framework for packing deformable and non-deformable subcellular structures in crowded cryo-electron tomogram simulation

BACKGROUND

Cryo-electron tomography is an important and powerful technique to explore the structure, abundance, and location of ultrastructure in a near-native state. It contains detailed information of all macromolecular complexes in a sample cell. However, due to the compact and crowded status, the missing edge effect, and low signal to noise ratio (SNR), it is extremely challenging to recover such information with existing image processing methods. Cryo-electron tomogram simulation is an effective solution to test and optimize the performance of the above image processing methods. The simulated images could be regarded as the labeled data which covers a wide range of macromolecular complexes and ultrastructure. To approximate the crowded cellular environment, it is very important to pack these heterogeneous structures as tightly as possible. Besides, simulating non-deformable and deformable components under a unified framework also need to be achieved.

RESULT

In this paper, we proposed a unified framework for simulating crowded cryo-electron tomogram images including non-deformable macromolecular complexes and deformable ultrastructures. A macromolecule was approximated using multiple balls with fixed relative positions to reduce the vacuum volume. A ultrastructure, such as membrane and filament, was approximated using multiple balls with flexible relative positions so that this structure could deform under force field. In the experiment, 400 macromolecules of 20 representative types were packed into simulated cytoplasm by our framework, and numerical verification proved that our method has a smaller volume and higher compression ratio than the baseline single-ball model. We also packed filaments, membranes and macromolecules together, to obtain a simulated cryo-electron tomogram image with deformable structures. The simulated results are closer to the real Cryo-ET, making the analysis more difficult. The DOG particle picking method and the image segmentation method are tested on our simulation data, and the experimental results show that these methods still have much room for improvement.

CONCLUSION

The proposed multi-ball model can achieve more crowded packaging results and contains richer elements with different properties to obtain more realistic cryo-electron tomogram simulation. This enables users to simulate cryo-electron tomogram images with non-deformable macromolecular complexes and deformable ultrastructures under a unified framework. To illustrate the advantages of our framework in improving the compression ratio, we calculated the volume of simulated macromolecular under our multi-ball method and traditional single-ball method. We also performed the packing experiment of filaments and membranes to demonstrate the simulation ability of deformable structures. Our method can be used to do a benchmark by generating large labeled cryo-ET dataset and evaluating existing image processing methods. Since the content of the simulated cryo-ET is more complex and crowded compared with previous ones, it will pose a greater challenge to existing image processing methods.

Looking back and looking forward: contributions of electron microscopy to the structural cell biology of gametes and fertilization

Mammalian gametes-the sperm and the egg-represent opposite extremes of cellular organization and scale. Studying the ultrastructure of gametes is crucial to understanding their interactions, and how to manipulate them in order to either encourage or prevent their union. Here, we survey the prominent electron microscopy (EM) techniques, with an emphasis on considerations for applying them to study mammalian gametes. We review how conventional EM has provided significant insight into gamete ultrastructure, but also how the harsh sample preparation methods required preclude understanding at a truly molecular level. We present recent advancements in cryo-electron tomography that provide an opportunity to image cells in a near-native state and at unprecedented levels of detail. New and emerging cellular EM techniques are poised to rekindle exploration of fundamental questions in mammalian reproduction, especially phenomena that involve complex membrane remodelling and protein reorganization. These methods will also allow novel lines of enquiry into problems of practical significance, such as investigating unexplained causes of human infertility and improving assisted reproductive technologies for biodiversity conservation.

Quality Assessment of the Neural Algorithms on the Example of EIT-UST Hybrid Tomography

The paper presents the results of research on the hybrid industrial tomograph electrical impedance tomography (EIT) and ultrasonic tomography (UST) (EIT-UST), operating on the basis of electrical and ultrasonic data. The emphasis of the research was placed on the algorithmic domain. However, it should be emphasized that all hardware components of the hybrid tomograph, including electronics, sensors and transducers, have been designed and mostly made in the Netrix S.A. laboratory. The test object was a tank filled with water with several dozen percent concentration. As part of the study, the original multiple neural networks system was trained, the characteristic feature of which is the generation of each of the individual pixels of the tomographic image, using an independent artificial neural network (ANN), with the input vector for all ANNs being the same. Despite the same measurement vector, each of the ANNs generates its own independent output value for a given tomogram pixel, because, during training, the networks get their respective weights and biases. During the tests, the results of three tomographic methods were compared: EIT, UST and EIT-UST hybrid. The results confirm that the use of heterogeneous tomographic systems (hybrids) increases the reliability of reconstruction in various measuring cases, which is used to solve quality problems in managing production processes.

Investigating eukaryotic cells with cryo-ET

The interior of eukaryotic cells is mysterious. How do the large communities of macromolecular machines interact with each other? How do the structures and positions of these nanoscopic entities respond to new stimuli? Questions like these can now be answered with the help of a method called electron cryotomography (cryo-ET). Cryo-ET will ultimately reveal the inner workings of a cell at the protein, secondary structure, and perhaps even side-chain levels. Combined with genetic or pharmacological perturbation, cryo-ET will allow us to answer previously unimaginable questions, such as how structure, biochemistry, and forces are related in situ. Because it bridges structural biology and cell biology, cryo-ET is indispensable for structural cell biology-the study of the 3-D macromolecular structure of cells. Here we discuss some of the key ideas, strategies, auxiliary techniques, and innovations that an aspiring structural cell biologist will consider when planning to ask bold questions.

An integration of fast alignment and maximum-likelihood methods for electron subtomogram averaging and classification

Motivation

Cellular Electron CryoTomography (CECT) is an emerging 3D imaging technique that visualizes subcellular organization of single cells at sub-molecular resolution and in near-native state. CECT captures large numbers of macromolecular complexes of highly diverse structures and abundances. However, the structural complexity and imaging limits complicate the systematic de novo structural recovery and recognition of these macromolecular complexes. Efficient and accurate reference-free subtomogram averaging and classification represent the most critical tasks for such analysis. Existing subtomogram alignment based methods are prone to the missing wedge effects and low signal-to-noise ratio (SNR). Moreover, existing maximum-likelihood based methods rely on integration operations, which are in principle computationally infeasible for accurate calculation.

Results

Built on existing works, we propose an integrated method, Fast Alignment Maximum Likelihood method (FAML), which uses fast subtomogram alignment to sample sub-optimal rigid transformations. The transformations are then used to approximate integrals for maximum-likelihood update of subtomogram averages through expectation–maximization algorithm. Our tests on simulated and experimental subtomograms showed that, compared to our previously developed fast alignment method (FA), FAML is significantly more robust to noise and missing wedge effects with moderate increases of computation cost. Besides, FAML performs well with significantly fewer input subtomograms when the FA method fails. Therefore, FAML can serve as a key component for improved construction of initial structural models from macromolecules captured by CECT.

Availability and implementation

http://www.cs.cmu.edu/mxu1

Adversarial domain adaptation for cross data source macromolecule in situ structural classification in cellular electron cryo-tomograms

MOTIVATION

Since 2017, an increasing amount of attention has been paid to the supervised deep learning-based macromolecule in situ structural classification (i.e. subtomogram classification) in cellular electron cryo-tomography (CECT) due to the substantially higher scalability of deep learning. However, the success of such supervised approach relies heavily on the availability of large amounts of labeled training data. For CECT, creating valid training data from the same data source as prediction data is usually laborious and computationally intensive. It would be beneficial to have training data from a separate data source where the annotation is readily available or can be performed in a high-throughput fashion. However, the cross data source prediction is often biased due to the different image intensity distributions (a.k.a. domain shift).

RESULTS

We adapt a deep learning-based adversarial domain adaptation (3D-ADA) method to timely address the domain shift problem in CECT data analysis. 3D-ADA first uses a source domain feature extractor to extract discriminative features from the training data as the input to a classifier. Then it adversarially trains a target domain feature extractor to reduce the distribution differences of the extracted features between training and prediction data. As a result, the same classifier can be directly applied to the prediction data. We tested 3D-ADA on both experimental and realistically simulated subtomogram datasets under different imaging conditions. 3D-ADA stably improved the cross data source prediction, as well as outperformed two popular domain adaptation methods. Furthermore, we demonstrate that 3D-ADA can improve cross data source recovery of novel macromolecular structures.

AVAILABILITY AND IMPLEMENTATION

https://github.com/xulabs/projects.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

A collection of yeast cellular electron cryotomography data

BACKGROUND

Cells are powered by a large set of macromolecular complexes, which work together in a crowded environment. The in situ mechanisms of these complexes are unclear because their 3D distribution, organization, and interactions are largely unknown. Electron cryotomography (cryo-ET) can address these knowledge gaps because it produces cryotomograms-3D images that reveal biological structure at ∼4-nm resolution. Cryo-ET uses no fixation, dehydration, staining, or plastic embedment, so cellular features are visualized in a life-like, frozen-hydrated state. To study chromatin and mitotic machinery in situ, we subjected yeast cells to genetic and chemical perturbations, cryosectioned them, and then imaged the cells by cryo-ET.

FINDINGS

Here we share >1,000 cryo-ET raw datasets of cryosectioned budding yeast Saccharomyces cerevisiaecollected as part of previously published studies. These data will be valuable to cell biologists who are interested in the nanoscale organization of yeasts and of eukaryotic cells in general. All the unpublished tilt series and a subset of corresponding cryotomograms have been deposited in the EMPIAR resource for the community to use freely. To improve tilt series discoverability, we have uploaded metadata and preliminary notes to publicly accessible Google Sheets, EMPIAR, and GigaDB.

CONCLUSIONS

Cellular cryo-ET data can be mined to obtain new cell-biological, structural, and 3D statistical insights in situ. These data contain structures not visible in traditional electron-microscopy data. Template matching and subtomogram averaging of known macromolecular complexes can reveal their 3D distributions and low-resolution structures. Furthermore, these data can serve as testbeds for high-throughput image-analysis pipelines, as training sets for feature-recognition software, for feasibility analysis when planning new structural-cell-biology projects, and as practice data for students.

Securing the future of research computing in the biosciences

Improvements in technology often drive scientific discovery. Therefore, research requires sustained investment in the latest equipment and training for the researchers who are going to use it. Prioritising and administering infrastructure investment is challenging because future needs are difficult to predict. In the past, highly computationally demanding research was associated primarily with particle physics and astronomy experiments. However, as biology becomes more quantitative and bioscientists generate more and more data, their computational requirements may ultimately exceed those of physical scientists. Computation has always been central to bioinformatics, but now imaging experiments have rapidly growing data processing and storage requirements. There is also an urgent need for new modelling and simulation tools to provide insight and understanding of these biophysical experiments. Bioscience communities must work together to provide the software and skills training needed in their areas. Research-active institutions need to recognise that computation is now vital in many more areas of discovery and create an environment where it can be embraced. The public must also become aware of both the power and limitations of computing, particularly with respect to their health and personal data.

Automatic localization and identification of mitochondria in cellular electron cryo-tomography using faster-RCNN

BACKGROUND

Cryo-electron tomography (cryo-ET) enables the 3D visualization of cellular organization in near-native state which plays important roles in the field of structural cell biology. However, due to the low signal-to-noise ratio (SNR), large volume and high content complexity within cells, it remains difficult and time-consuming to localize and identify different components in cellular cryo-ET. To automatically localize and recognize in situ cellular structures of interest captured by cryo-ET, we proposed a simple yet effective automatic image analysis approach based on Faster-RCNN.

RESULTS

Our experimental results were validated using in situ cyro-ET-imaged mitochondria data. Our experimental results show that our algorithm can accurately localize and identify important cellular structures on both the 2D tilt images and the reconstructed 2D slices of cryo-ET. When ran on the mitochondria cryo-ET dataset, our algorithm achieved Average Precision >0.95. Moreover, our study demonstrated that our customized pre-processing steps can further improve the robustness of our model performance.

CONCLUSIONS

In this paper, we proposed an automatic Cryo-ET image analysis algorithm for localization and identification of different structure of interest in cells, which is the first Faster-RCNN based method for localizing an cellular organelle in Cryo-ET images and demonstrated the high accuracy and robustness of detection and classification tasks of intracellular mitochondria. Furthermore, our approach can be easily applied to detection tasks of other cellular structures as well.

Feature Decomposition Based Saliency Detection in Electron Cryo-Tomograms

Electron Cryo-Tomography (ECT) allows 3D visualization of subcellular structures at the submolecular resolution in close to the native state. However, due to the high degree of structural complexity and imaging limits, the automatic segmentation of cellular components from ECT images is very difficult. To complement and speed up existing segmentation methods, it is desirable to develop a generic cell component segmentation method that is 1) not specific to particular types of cellular components, 2) able to segment unknown cellular components, 3) fully unsupervised and does not rely on the availability of training data. As an important step towards this goal, in this paper, we propose a saliency detection method that computes the likelihood that a subregion in a tomogram stands out from the background. Our method consists of four steps: supervoxel over-segmentation, feature extraction, feature matrix decomposition, and computation of saliency. The method produces a distribution map that represents the regions' saliency in tomograms. Our experiments show that our method can successfully label most salient regions detected by a human observer, and able to filter out regions not containing cellular components. Therefore, our method can remove the majority of the background region, and significantly speed up the subsequent processing of segmentation and recognition of cellular components captured by ECT.

Improved deep learning-based macromolecules structure classification from electron cryo-tomograms

Cellular processes are governed by macromolecular complexes inside the cell. Study of the native structures of macromolecular complexes has been extremely difficult due to lack of data. With recent breakthroughs in Cellular Electron Cryo-Tomography (CECT) 3D imaging technology, it is now possible for researchers to gain accesses to fully study and understand the macro-molecular structures single cells. However, systematic recovery of macromolecular structures from CECT is very difficult due to high degree of structural complexity and practical imaging limitations. Specifically, we proposed a deep learning-based image classification approach for large-scale systematic macromolecular structure separation from CECT data. However, our previous work was only a very initial step toward exploration of the full potential of deep learning-based macromolecule separation. In this paper, we focus on improving classification performance by proposing three newly designed individual CNN models: an extended version of (Deep Small Receptive Field) DSRF3D, donated as DSRF3D-v2, a 3D residual block-based neural network, named as RB3D, and a convolutional 3D (C3D)-based model, CB3D. We compare them with our previously developed model (DSRF3D) on 12 datasets with different SNRs and tilt angle ranges. The experiments show that our new models achieved significantly higher classification accuracies. The accuracies are not only higher than 0.9 on normal datasets, but also demonstrate potentials to operate on datasets with high levels of noises and missing wedge effects presented.

Unravelling molecular complexity in structural cell biology

Structural and cell biology have traditionally been separate disciplines and employed techniques that were well defined within the realm of either one or the other. Recent technological breakthroughs propelled electron microscopy of frozen hydrated specimens (cryo-EM) followed by single-particle analysis (SPA) to become a widely applied approach for obtaining near-atomic resolution structures of purified macromolecules. In parallel, ongoing developments on sample preparation are increasingly successful in bringing molecular views into cell biology. Cryo-electron tomography (cryo-ET) has so far served as the main imaging modality employed in these efforts towards obtaining three-dimensional (3D) volumes of heterogeneous molecular assemblies. We review the state-of-the-art in cryo-ET and computational processing and describe the current opportunities and frontiers for in-cell applications.

Multi-task Learning for Macromolecule Classification, Segmentation and Coarse Structural Recovery in Cryo-Tomography

Cellular Electron Cryo-Tomography (CECT) is a powerful 3D imaging tool for studying the native structure and organization of macromolecules inside single cells. For systematic recognition and recovery of macromolecular structures captured by CECT, methods for several important tasks such as subtomogram classification and semantic segmentation have been developed. However, the recognition and recovery of macromolecular structures are still very difficult due to high molecular structural diversity, crowding molecular environment, and the imaging limitations of CECT. In this paper, we propose a novel multi-task 3D convolutional neural network model for simultaneous classification, segmentation, and coarse structural recovery of macromolecules of interest in subtomograms. In our model, the learned image features of one task are shared and thereby mutually reinforce the learning of other tasks. Evaluated on realistically simulated and experimental CECT data, our multi-task learning model outperformed all single-task learning methods for classification and segmentation. In addition, we demonstrate that our model can generalize to discover, segment and recover novel structures that do not exist in the training data.

Fine details in complex environments: the power of cryo-electron tomography

Cryo-electron tomography (CET) is uniquely suited to obtain structural information from a wide range of biological scales, integrating and bridging knowledge from molecules to cells. In particular, CET can be used to visualise molecular structures in their native environment. Depending on the experiment, a varying degree of resolutions can be achieved, with the first near-atomic molecular structures becoming recently available. The power of CET has increased significantly in the last 5 years, in parallel with improvements in cryo-EM hardware and software that have also benefited single-particle reconstruction techniques. In this review, we cover the typical CET pipeline, starting from sample preparation, to data collection and processing, and highlight in particular the recent developments that support structural biology in situ. We provide some examples that highlight the importance of structure determination of molecules embedded within their native environment, and propose future directions to improve CET performance and accessibility.

Big data in cryoEM: automated collection, processing and accessibility of EM data

The scope and complexity of cryogenic electron microscopy (cryoEM) data has greatly increased, and will continue to do so, due to recent and ongoing technical breakthroughs that have led to much improved resolutions for macromolecular structures solved using this method. This big data explosion includes single particle data as well as tomographic tilt series, both generally acquired as direct detector movies of ∼10-100 frames per image or per tilt-series. We provide a brief survey of the developments leading to the current status, and describe existing cryoEM pipelines, with an emphasis on the scope of data acquisition, methods for automation, and use of cloud storage and computing.