Quantifying noise limitations of neural network segmentations in high-resolution transmission electron microscopy

Interpretability of high-resolution transmission electron microscopy images

High-resolution electron microscopy is a well-suited tool for characterizing the nanoscale structure of materials. However, the interaction of the sample and the high-energy electrons of the beam can often have a detrimental impact on the sample structure. This effect can only be alleviated by decreasing the number of electrons to which the sample is exposed but will come at the cost of a decreased signal-to-noise ratio in the resulting image. Images with low signal to noise ratios are often challenging to interpret as parts of the sample with a low interaction with the electron beam are reproduced with very low contrast. Here we suggest simple measures as alternatives to the conventional signal-to-noise ratio and investigate how these can be used to predict the interpretability of the electron microscopy images. We test the models on a sample consisting of gold nanoparticles supported on a cerium dioxide substrate. The models are evaluated based on series of images acquired at varying electron dose.

Generalization Across Experimental Parameters in Neural Network Analysis of High-Resolution Transmission Electron Microscopy Datasets

Neural networks are promising tools for high-throughput and accurate transmission electron microscopy (TEM) analysis of nanomaterials, but are known to generalize poorly on data that is "out-of-distribution" from their training data. Given the limited set of image features typically seen in high-resolution TEM imaging, it is unclear which images are considered out-of-distribution from others. Here, we investigate how the choice of metadata features in the training dataset influences neural network performance, focusing on the example task of nanoparticle segmentation. We train and validate neural networks across curated, experimentally collected high-resolution TEM image datasets of nanoparticles under various imaging and material parameters, including magnification, dosage, nanoparticle diameter, and nanoparticle material. Overall, we find that our neural networks are not robust across microscope parameters, but do generalize across certain sample parameters. Additionally, data preprocessing can have unintended consequences on neural network generalization. Our results highlight the need to understand how dataset features affect deployment of data-driven algorithms.