1
|
Escobar Díaz Guerrero R, Carvalho L, Bocklitz T, Popp J, Oliveira JL. A Data Augmentation Methodology to Reduce the Class Imbalance in Histopathology Images. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024; 37:1767-1782. [PMID: 38485898 PMCID: PMC11300732 DOI: 10.1007/s10278-024-01018-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/05/2023] [Revised: 12/26/2023] [Accepted: 12/28/2023] [Indexed: 08/07/2024]
Abstract
Deep learning techniques have recently yielded remarkable results across various fields. However, the quality of these results depends heavily on the quality and quantity of data used during the training phase. One common issue in multi-class and multi-label classification is class imbalance, where one or several classes make up a substantial portion of the total instances. This imbalance causes the neural network to prioritize features of the majority classes during training, as their detection leads to higher scores. In the context of object detection, two types of imbalance can be identified: (1) an imbalance between the space occupied by the foreground and background and (2) an imbalance in the number of instances for each class. This paper aims to address the second type of imbalance without exacerbating the first. To achieve this, we propose a modification of the copy-paste data augmentation technique, combined with weight-balancing methods in the loss function. This strategy was specifically tailored to improve the performance in datasets with a high instance density, where instance overlap could be detrimental. To validate our methodology, we applied it to a highly unbalanced dataset focused on nuclei detection. The results show that this hybrid approach improves the classification of minority classes without significantly compromising the performance of majority classes.
Collapse
Affiliation(s)
- Rodrigo Escobar Díaz Guerrero
- BMD Software, PCI - Creative Science Park, 3830-352, Ilhavo, Portugal.
- DETI/IEETA, University of Aveiro, 3810-193, Aveiro, Portugal.
- Leibniz Institute of Photonic Technology Jena, Member of Leibniz Research Alliance 'Health Technologies', Albert-Einstein-Straße 9, 07745, Jena, Germany.
| | - Lina Carvalho
- Institute of Anatomical and Molecular Pathology, Faculty of Medicine, University of Coimbra, 3004-504, Coimbra, Portugal
| | - Thomas Bocklitz
- Leibniz Institute of Photonic Technology Jena, Member of Leibniz Research Alliance 'Health Technologies', Albert-Einstein-Straße 9, 07745, Jena, Germany
- Institute of Physical Chemistry and Abbe Center of Photonics (IPC), Friedrich-Schiller-University, Jena, Germany
- Institute of Computer Science, Faculty of Mathematics, Physics & Computer Science, Bayreuth, Germany
| | - Juergen Popp
- Leibniz Institute of Photonic Technology Jena, Member of Leibniz Research Alliance 'Health Technologies', Albert-Einstein-Straße 9, 07745, Jena, Germany
- Institute of Physical Chemistry and Abbe Center of Photonics (IPC), Friedrich-Schiller-University, Jena, Germany
| | | |
Collapse
|
2
|
Matveev AV, Nartova AV, Sankova NN, Okunev AG. DLgram cloud service for deep-learning analysis of microscopy images. Microsc Res Tech 2024; 87:991-998. [PMID: 38186233 DOI: 10.1002/jemt.24480] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/23/2023] [Accepted: 12/12/2023] [Indexed: 01/09/2024]
Abstract
To analyze images in various fields of science and technology, it is often necessary to count observed objects and determine their parameters. This can be quite labor-intensive and time-consuming. This article presents DLgram, a universal, user-friendly cloud service that is developed for this purpose. It is based on deep learning technologies and does not require programming skills. The user labels several objects in the image and uploads it to the cloud where the neural network is trained to recognize the objects being studied. The user receives recognition results, which if necessary, can be corrected, errors removed, or missing objects added. In addition, it is possible to carry out mathematical processing of the data obtained to get information about the sizes, areas, and coordinates of the observed objects. The article describes the service features and discusses examples of its application. The DLgram service allows to reduce significantly the time spent on quantitative image analysis, reduce subjective factor influence, and increase the accuracy of analysis. RESEARCH HIGHLIGHTS: DLgram automatically recognizes and counts the number of objects in images and their parameters. DLgram is a universal service, which was created on the basis of the latest deep learning developments and does not require programming skills.
Collapse
Affiliation(s)
- Andrey V Matveev
- Institute of Intellectual Robototechnics, Novosibirsk State University, Novosibirsk, Russia
| | - Anna V Nartova
- Institute of Intellectual Robototechnics, Novosibirsk State University, Novosibirsk, Russia
- Department of Physico-Chemical Research Methods, Boreskov Institute of Catalysis SB RAS, Novosibirsk, Russia
| | - Natalya N Sankova
- Institute of Intellectual Robototechnics, Novosibirsk State University, Novosibirsk, Russia
- Department of Non-Traditional Catalytic Processes, Boreskov Institute of Catalysis SB RAS, Novosibirsk, Russia
| | - Alexey G Okunev
- Institute of Intellectual Robototechnics, Novosibirsk State University, Novosibirsk, Russia
| |
Collapse
|
3
|
Mohsin ASM, Choudhury SH. Label-free quantification of gold nanoparticles at the single-cell level using a multi-column convolutional neural network (MC-CNN). Analyst 2024; 149:2412-2419. [PMID: 38487894 DOI: 10.1039/d3an01982a] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/16/2024]
Abstract
Gold nanoparticles (AuNPs) are extensively used in cellular imaging, single-particle tracking, disease diagnosis, studying membrane protein interaction, and drug delivery. Understanding the dynamics of AuNP uptake in live cells is crucial for optimizing their efficacy and safety. Traditional manual methods for quantifying AuNP uptake are time-consuming and subjective, limiting their scalability and accuracy. The available fluorescence-based techniques are limited to photobleaching and photoblinking. Optical microscopy techniques are limited by diffraction limits. Electron microscopy-based imaging techniques are destructive and unsuitable for live cell imaging. Furthermore, the resulting images may contain hundreds of particles with varied intensities, blurring, and substantial occlusion, making it difficult to manually quantify AuNP uptake. To overcome this issue and measure AuNP uptake by live cells, we annotated a dataset of dark-field images of 50 nanometer-radius AuNPs at different incubation durations. Then, to count the number of particles present in a cell, we created a customized multi-column convolutional neural network (MC-CNN). The customized MC-CNN outperformed typical particle counting architectures when compared to spectroscopy-based counting. This will allow researchers to gain a better understanding of AuNP behavior and interactions with cells, paving the way for advancements in nanomedicine, drug delivery, and biomedical research. The code for this paper is available at the following link: https://github.com/Namerlight/LabelFree_AuNP_Quantification.
Collapse
Affiliation(s)
- Abu S M Mohsin
- Nanotechnology, IoT and Applied Machine Learning Research Group, Brac University, Dhaka, Bangladesh.
| | - Shadab H Choudhury
- Nanotechnology, IoT and Applied Machine Learning Research Group, Brac University, Dhaka, Bangladesh.
| |
Collapse
|
4
|
Fedorov AY, Bukhtiyarov AV, Panafidin MA, Prosvirin IP, Zubavichus YV, Bukhtiyarov VI. Thermally Induced Surface Structure and Morphology Evolution in Bimetallic Pt-Au/HOPG Nanoparticles as Probed Using XPS and STM. NANOMATERIALS (BASEL, SWITZERLAND) 2023; 14:57. [PMID: 38202512 PMCID: PMC10780797 DOI: 10.3390/nano14010057] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/09/2023] [Revised: 12/18/2023] [Accepted: 12/21/2023] [Indexed: 01/12/2024]
Abstract
Bimetallic nanoparticles expand the possibilities of catalyst design, providing an extra degree of freedom for tailoring the catalyst structure in comparison to purely monometallic systems. The distribution mode of two metal species defines the structure of surface catalytic sites, and current research efforts are focused on the development of methods for their controlled tuning. In light of this, a comprehensive investigation of the factors which influence the changes in the morphology of bimetallic nanoparticles, including the elemental redistribution, are mandatory for each particular bimetallic system. Here we present the combined XPS/STM study of the surface structure and morphology of bimetallic Pt-Au/HOPG nanoparticles prepared by thermal vacuum deposition and show that thermal annealing up to 350 °C induces the alloying process between the two bulk-immiscible metal components. Increasing the treatment temperature enhances the extent of Pt-Au alloying. However, the sintering of nanoparticles starts to occur above 500 °C. The approach implemented in this work includes the theoretical simulation of XPS signal intensities for a more meticulous analysis of the compositional distribution and can be helpful from a methodological perspective for other XPS/STM studies of bimetallic nanoparticles on planar supports.
Collapse
Affiliation(s)
| | - Andrey V. Bukhtiyarov
- Boreskov Institute of Catalysis SB RAS, Novosibirsk 630090, Russia (M.A.P.); (I.P.P.); (Y.V.Z.); (V.I.B.)
| | | | | | | | | |
Collapse
|
5
|
Mishra A, Sarbapalli D, Rodríguez O, Rodríguez-López J. Electrochemical Imaging of Interfaces in Energy Storage via Scanning Probe Methods: Techniques, Applications, and Prospects. ANNUAL REVIEW OF ANALYTICAL CHEMISTRY (PALO ALTO, CALIF.) 2023; 16:93-115. [PMID: 37068746 DOI: 10.1146/annurev-anchem-091422-110703] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Developing a deeper understanding of dynamic chemical, electronic, and morphological changes at interfaces is key to solving practical issues in electrochemical energy storage systems (EESSs). To unravel this complexity, an assortment of tools with distinct capabilities and spatiotemporal resolutions have been used to creatively visualize interfacial processes as they occur. This review highlights how electrochemical scanning probe techniques (ESPTs) such as electrochemical atomic force microscopy, scanning electrochemical microscopy, scanning ion conductance microscopy, and scanning electrochemical cell microscopy are uniquely positioned to address these challenges in EESSs. We describe the operating principles of ESPTs, focusing on the inspection of interfacial structure and chemical processes involved in Li-ion batteries and beyond. We discuss current examples, performance limitations, and complementary ESPTs. Finally, we discuss prospects for imaging improvements and deep learning for automation. We foresee that ESPTs will play an enabling role in advancing EESSs as we transition to renewable energies.
Collapse
Affiliation(s)
- Abhiroop Mishra
- Department of Chemistry, University of Illinois Urbana-Champaign, Urbana, Illinois, USA;
| | - Dipobrato Sarbapalli
- Department of Chemistry, University of Illinois Urbana-Champaign, Urbana, Illinois, USA;
| | - Oliver Rodríguez
- Department of Chemistry, University of Illinois Urbana-Champaign, Urbana, Illinois, USA;
| | | |
Collapse
|
6
|
Visheratina A, Visheratin A, Kumar P, Veksler M, Kotov NA. Chirality Analysis of Complex Microparticles using Deep Learning on Realistic Sets of Microscopy Images. ACS NANO 2023; 17:7431-7442. [PMID: 37058327 DOI: 10.1021/acsnano.2c12056] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/19/2023]
Abstract
Nanoscale chirality is an actively growing research field spurred by the giant chiroptical activity, enantioselective biological activity, and asymmetric catalytic activity of chiral nanostructures. Compared to chiral molecules, the handedness of chiral nano- and microstructures can be directly established via electron microscopy, which can be utilized for the automatic analysis of chiral nanostructures and prediction of their properties. However, chirality in complex materials may have multiple geometric forms and scales. Computational identification of chirality from electron microscopy images rather than optical measurements is convenient but is fundamentally challenging, too, because (1) image features differentiating left- and right-handed particles can be ambiguous and (2) three-dimensional structure essential for chirality is 'flattened' into two-dimensional projections. Here, we show that deep learning algorithms can identify twisted bowtie-shaped microparticles with nearly 100% accuracy and classify them as left- and right-handed with as high as 99% accuracy. Importantly, such accuracy was achieved with as few as 30 original electron microscopy images of bowties. Furthermore, after training on bowtie particles with complex nanostructured features, the model can recognize other chiral shapes with different geometries without retraining for their specific chiral geometry with 93% accuracy, indicating the true learning abilities of the employed neural networks. These findings indicate that our algorithm trained on a practically feasible set of experimental data enables automated analysis of microscopy data for the accelerated discovery of chiral particles and their complex systems for multiple applications.
Collapse
Affiliation(s)
- Anastasia Visheratina
- Department of Chemical Engineering and Biointerfaces Institute, University of Michigan, Ann Arbor, Michigan 48109, United States
| | | | - Prashant Kumar
- Department of Chemical Engineering and Biointerfaces Institute, University of Michigan, Ann Arbor, Michigan 48109, United States
| | - Michael Veksler
- Department of Chemical Engineering and Biointerfaces Institute, University of Michigan, Ann Arbor, Michigan 48109, United States
| | - Nicholas A Kotov
- Department of Chemical Engineering and Biointerfaces Institute, University of Michigan, Ann Arbor, Michigan 48109, United States
- Department of Aeronautics, Faculty of Engineering, Imperial College London, South Kensington Campus London, SW7 2AZ, United Kingdom
| |
Collapse
|
7
|
Boiko DA, Sulimova VV, Kurbakov MY, Kopylov AV, Seredin OS, Cherepanova VA, Pentsak EO, Ananikov VP. Automated Recognition of Nanoparticles in Electron Microscopy Images of Nanoscale Palladium Catalysts. NANOMATERIALS (BASEL, SWITZERLAND) 2022; 12:3914. [PMID: 36364691 PMCID: PMC9657885 DOI: 10.3390/nano12213914] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/13/2022] [Revised: 10/28/2022] [Accepted: 11/01/2022] [Indexed: 06/16/2023]
Abstract
Automated computational analysis of nanoparticles is the key approach urgently required to achieve further progress in catalysis, the development of new nanoscale materials, and applications. Analysis of nanoscale objects on the surface relies heavily on scanning electron microscopy (SEM) as the experimental analytic method, allowing direct observation of nanoscale structures and morphology. One of the important examples of such objects is palladium on carbon catalysts, allowing access to various chemical reactions in laboratories and industry. SEM images of Pd/C catalysts show a large number of nanoparticles that are usually analyzed manually. Manual analysis of a statistically significant number of nanoparticles is a tedious and highly time-consuming task that is impossible to perform in a reasonable amount of time for practically needed large amounts of samples. This work provides a comprehensive comparison of various computer vision methods for the detection of metal nanoparticles. In addition, multiple new types of data representations were developed, and their applicability in practice was assessed.
Collapse
Affiliation(s)
- Daniil A. Boiko
- Zelinsky Institute of Organic Chemistry, Russian Academy of Sciences, 119991 Moscow, Russia
| | | | | | | | | | - Vera A. Cherepanova
- Zelinsky Institute of Organic Chemistry, Russian Academy of Sciences, 119991 Moscow, Russia
| | - Evgeniy O. Pentsak
- Zelinsky Institute of Organic Chemistry, Russian Academy of Sciences, 119991 Moscow, Russia
| | - Valentine P. Ananikov
- Zelinsky Institute of Organic Chemistry, Russian Academy of Sciences, 119991 Moscow, Russia
| |
Collapse
|
8
|
Konstantopoulos G, Koumoulos EP, Charitidis CA. Digital Innovation Enabled Nanomaterial Manufacturing; Machine Learning Strategies and Green Perspectives. NANOMATERIALS 2022; 12:nano12152646. [PMID: 35957077 PMCID: PMC9370746 DOI: 10.3390/nano12152646] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/30/2022] [Revised: 07/28/2022] [Accepted: 07/29/2022] [Indexed: 02/05/2023]
Abstract
Machine learning has been an emerging scientific field serving the modern multidisciplinary needs in the Materials Science and Manufacturing sector. The taxonomy and mapping of nanomaterial properties based on data analytics is going to ensure safe and green manufacturing with consciousness raised on effective resource management. The utilization of predictive modelling tools empowered with artificial intelligence (AI) has proposed novel paths in materials discovery and optimization, while it can further stimulate the cutting-edge and data-driven design of a tailored behavioral profile of nanomaterials to serve the special needs of application environments. The previous knowledge of the physics and mathematical representation of material behaviors, as well as the utilization of already generated testing data, received specific attention by scientists. However, the exploration of available information is not always manageable, and machine intelligence can efficiently (computational resources, time) meet this challenge via high-throughput multidimensional search exploration capabilities. Moreover, the modelling of bio-chemical interactions with the environment and living organisms has been demonstrated to connect chemical structure with acute or tolerable effects upon exposure. Thus, in this review, a summary of recent computational developments is provided with the aim to cover excelling research and present challenges towards unbiased, decentralized, and data-driven decision-making, in relation to increased impact in the field of advanced nanomaterials manufacturing and nanoinformatics, and to indicate the steps required to realize rapid, safe, and circular-by-design nanomaterials.
Collapse
Affiliation(s)
- Georgios Konstantopoulos
- RNANO Lab—Research Unit of Advanced, Composite, Nano Materials & Nanotechnology, School of Chemical Engineering, National Technical University of Athens, GR15773 Athens, Greece; (G.K.); (C.A.C.)
| | - Elias P. Koumoulos
- Innovation in Research & Engineering Solutions (IRES), Boulevard Edmond Machtens 79/22, 1080 Brussels, Belgium
- Correspondence:
| | - Costas A. Charitidis
- RNANO Lab—Research Unit of Advanced, Composite, Nano Materials & Nanotechnology, School of Chemical Engineering, National Technical University of Athens, GR15773 Athens, Greece; (G.K.); (C.A.C.)
| |
Collapse
|
9
|
López Gutiérrez JD, Abundez Barrera IM, Torres Gómez N. Nanoparticle Detection on SEM Images Using a Neural Network and Semi-Synthetic Training Data. NANOMATERIALS 2022; 12:nano12111818. [PMID: 35683674 PMCID: PMC9182493 DOI: 10.3390/nano12111818] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/20/2022] [Revised: 05/20/2022] [Accepted: 05/20/2022] [Indexed: 01/25/2023]
Abstract
Processing images represents a necessary step in the process of analysing the information gathered about nanoparticles after characteristic material samples have been scanned with electron microscopy, which often requires the use of image processing techniques or general purpose image manipulation software to carry out tasks such as nanoparticle detection and measurement. In recent years, the use of networks has been successfully implemented to detect and classify electron microscopy images as well as the objects within them. In this work, we present four detection models using two versions of the YOLO neural network architectures trained to detect cubical and quasi-spherical particles in SEM images; the training datasets are a mixture of real images and synthetic ones generated by a semi-arbitrary method. The resulting models were capable of detecting nanoparticles in images different than the ones used for training and identifying them in some cases as the close proximity between nanoparticles proved a challenge for the neural networks in most situations.
Collapse
|
10
|
Sankova N, Vyvdenko D, Luzina E, Shestakova D, Babina K, Malakhova Y, Yakush E, Parkhomchuk E. Polymer particle growth and morphology evolution during dispersion polymerization through optical microscopy. Colloid Polym Sci 2022. [DOI: 10.1007/s00396-022-04972-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/03/2022]
|
11
|
Particle Recognition on Transmission Electron Microscopy Images Using Computer Vision and Deep Learning for Catalytic Applications. Catalysts 2022. [DOI: 10.3390/catal12020135] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/04/2023] Open
Abstract
Recognition and measuring particles on microscopy images is an important part of many scientific studies, including catalytic investigations. In this paper, we present the results of the application of deep learning to the automated recognition of nanoparticles deposited on porous supports (heterogeneous catalysts) on images obtained by transmission electron microscopy (TEM). The Cascade Mask-RCNN neural network was used. During the training, two types of objects were labeled on raw TEM images of ‘real’ catalysts: visible particles and overlapping particle projections. The trained neural network recognized nanoparticles in the test dataset with 0.71 precision and 0.72 recall for both classes of objects and 0.84 precision and 0.79 recall for visible particles. The developed model is integrated into the open-access web service ‘ParticlesNN’, which can be used by any researcher in the world. Instead of hours, TEM data processing per one image analysis is reduced to a maximum of a couple of minutes and the divergence of mean particle size determination is approximately 2% compared to manual analysis. The proposed tool encourages accelerating catalytic research and improving the objectivity and accuracy of analysis.
Collapse
|
12
|
AI-based atomic force microscopy image analysis allows to predict electrochemical impedance spectra of defects in tethered bilayer membranes. Sci Rep 2022; 12:1127. [PMID: 35064137 PMCID: PMC8783026 DOI: 10.1038/s41598-022-04853-4] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/22/2021] [Accepted: 12/24/2021] [Indexed: 01/08/2023] Open
Abstract
Atomic force microscopy (AFM) image analysis of supported bilayers, such as tethered bilayer membranes (tBLMs) can reveal the nature of the membrane damage by pore-forming proteins and predict the electrochemical impedance spectroscopy (EIS) response of such objects. However, automated analysis involving pore detection in such images is often non-trivial and can require AI-based object detection techniques. The specific object-detection algorithm we used to determine the defect coordinates in real AFM images was a convolutional neural network (CNN). Defect coordinates allow to predict the EIS response of tBLMs populated by the pore-forming toxins using finite element analysis (FEA) modeling. We tested if the accuracy of the CNN algorithm affected the EIS spectral features sensitive to defect densities and other physical parameters of tBLMs. We found that the EIS spectra can be predicted sufficiently well, however, systematic errors of characteristic spectral points were observed and need to be taken into account. Importantly, the comparison of predicted EIS curves with experimental ones allowed to estimate important physical parameters of tBLMs such as the specific resistance of submembrane reservoir. This reservoir separates phospholipid bilayer from the solid support. We found that the specific resistance of the reservoir amounts to \documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$10^{4.25 \pm 0.10}$$\end{document}104.25±0.10\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$\Omega \cdot cm$$\end{document}Ω·cm which is approximately two orders of a magnitude higher compared to the specific resistance of the buffer bathing tBLMs studied in this work. We hypothesize that such effect may be related in part due to decreased concentration of ionic carriers in the submembrane due to decreased relative dielectric permittivity in this region.
Collapse
|
13
|
Zhao H, Ni B, Jin X, Zhang H, Hou JJ, Hou L, Marsh JH, Dong L, Li S, Gao XW, Shi D, Liu X, Xiong J. Gold-viral particle identification by deep learning in wide-field photon scattering parametric images. APPLIED OPTICS 2022; 61:546-553. [PMID: 35200896 DOI: 10.1364/ao.445953] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/13/2021] [Accepted: 12/06/2021] [Indexed: 06/14/2023]
Abstract
The ability to identify virus particles is important for research and clinical applications. Because of the optical diffraction limit, conventional optical microscopes are generally not suitable for virus particle detection, and higher resolution instruments such as transmission electron microscopy (TEM) and scanning electron microscopy (SEM) are required. In this paper, we propose a new method for identifying virus particles based on polarization parametric indirect microscopic imaging (PIMI) and deep learning techniques. By introducing an abrupt change of refractivity at the virus particle using antibody-conjugated gold nanoparticles (AuNPs), the strength of the photon scattering signal can be magnified. After acquiring the PIMI images, a deep learning method was applied to identify discriminating features and classify the virus particles, using electron microscopy (EM) images as the ground truth. Experimental results confirm that gold-virus particles can be identified in PIMI images with a high level of confidence.
Collapse
|
14
|
Panafidin MA, Bukhtiyarov AV, Prosvirin IP, Chetyrin IA, Klyushin AY, Knop-Gericke A, Smirnova NS, Markov PV, Mashkovsky IS, Zubavichus YV, Stakheev AY, Bukhtiyarov VI. SRPES and STM data for the model bimetallic Pd-In/HOPG catalysts: Effects of mild post-synthesis oxidative treatments. Data Brief 2021; 39:107626. [PMID: 34877389 PMCID: PMC8633860 DOI: 10.1016/j.dib.2021.107626] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2021] [Revised: 11/09/2021] [Accepted: 11/18/2021] [Indexed: 11/28/2022] Open
Abstract
Post-synthesis treatment of bimetallic catalysts in different gas phases resulting in the adsorption-induced segregation is among promising approaches to enhance their activity not compromising selectivity towards a number of low-temperature reactions. Our recently published paper (M.A. Panafidin, A.V. Bukhtiyarov, I.P. Prosvirin, I.A. Chetyrin, A.Yu. Klyushin, A. Knop-Gericke, N.S. Smirnova, P.V. Markov, I.S. Mashkovsky, Y.V. Zubavichus, A.Yu. Stakheev, V.I. Bukhtiyarov, A mild post-synthesis oxidative treatment of Pd-In/HOPG bimetallic catalysts as a tool of their surface structure fine tuning. Appl. Surf. Sci.) reports on Pd-In intermetallic formation regularities and their evolution after storage in air as well as during treatment in oxygen at submillibar pressures. The current paper gives an extended representation of experimental ex situ/in situ synchrotron-based photoelectron spectroscopy (SRPES) and scanning tunnelling microscopy (STM) data used to derive scientific conclusions in the paper quoted above.
Collapse
Affiliation(s)
- M A Panafidin
- G. K. Boreskov Institute of Catalysis, Siberian Branch of the Russian Academy of Sciences, 630090 Novosibirsk, Russian Federation
| | - A V Bukhtiyarov
- G. K. Boreskov Institute of Catalysis, Siberian Branch of the Russian Academy of Sciences, 630090 Novosibirsk, Russian Federation
| | - I P Prosvirin
- G. K. Boreskov Institute of Catalysis, Siberian Branch of the Russian Academy of Sciences, 630090 Novosibirsk, Russian Federation
| | - I A Chetyrin
- G. K. Boreskov Institute of Catalysis, Siberian Branch of the Russian Academy of Sciences, 630090 Novosibirsk, Russian Federation
| | - A Yu Klyushin
- Fritz Haber Institute of the Max Planck Society, 14195 Berlin, Germany.,Helmholtz Center for Materials and Energy, 12489 Berlin, Germany
| | - A Knop-Gericke
- Fritz Haber Institute of the Max Planck Society, 14195 Berlin, Germany.,Max Planck Institute for Chemical Energy Conversion, Department of Heterogeneous Reactions,45470 Mülheim an der Ruhr, Germany
| | - N S Smirnova
- G. K. Boreskov Institute of Catalysis, Siberian Branch of the Russian Academy of Sciences, 630090 Novosibirsk, Russian Federation.,N. D. Zelinsky Institute of Organic Chemistry, Russian Academy of Sciences, 119991 Moscow, Russian Federation
| | - P V Markov
- G. K. Boreskov Institute of Catalysis, Siberian Branch of the Russian Academy of Sciences, 630090 Novosibirsk, Russian Federation.,N. D. Zelinsky Institute of Organic Chemistry, Russian Academy of Sciences, 119991 Moscow, Russian Federation
| | - I S Mashkovsky
- N. D. Zelinsky Institute of Organic Chemistry, Russian Academy of Sciences, 119991 Moscow, Russian Federation
| | - Y V Zubavichus
- G. K. Boreskov Institute of Catalysis, Siberian Branch of the Russian Academy of Sciences, 630090 Novosibirsk, Russian Federation
| | - A Yu Stakheev
- N. D. Zelinsky Institute of Organic Chemistry, Russian Academy of Sciences, 119991 Moscow, Russian Federation
| | - V I Bukhtiyarov
- G. K. Boreskov Institute of Catalysis, Siberian Branch of the Russian Academy of Sciences, 630090 Novosibirsk, Russian Federation
| |
Collapse
|
15
|
Bukhtiyarov AV, Prosvirin IP, Panafidin MA, Fedorov AY, Klyushin AY, Knop-Gericke A, Zubavichus YV, Bukhtiyarov VI. Near-Ambient Pressure XPS and MS Study of CO Oxidation over Model Pd-Au/HOPG Catalysts: The Effect of the Metal Ratio. NANOMATERIALS 2021; 11:nano11123292. [PMID: 34947641 PMCID: PMC8703420 DOI: 10.3390/nano11123292] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/29/2021] [Revised: 12/01/2021] [Accepted: 12/02/2021] [Indexed: 11/16/2022]
Abstract
In this study, the dependence of the catalytic activity of highly oriented pyrolytic graphite (HOPG)-supported bimetallic Pd-Au catalysts towards the CO oxidation based on the Pd/Au atomic ratio was investigated. The activities of two model catalysts differing from each other in the initial Pd/Au atomic ratios appeared as distinctly different in terms of their ignition temperatures. More specifically, the PdAu-2 sample with a lower Pd/Au surface ratio (~0.75) was already active at temperatures less than 150 °C, while the PdAu-1 sample with a higher Pd/Au surface ratio (~1.0) became active only at temperatures above 200 °C. NAP XPS revealed that the exposure of the catalysts to a reaction mixture at RT induces the palladium surface segregation accompanied by an enrichment of the near-surface regions of the two-component Pd-Au alloy nanoparticles with Pd due to adsorption of CO on palladium atoms. The segregation extent depends on the initial Pd/Au surface ratio. The difference in activity between these two catalysts is determined by the presence or higher concentration of specific active Pd sites on the surface of bimetallic particles, i.e., by the ensemble effect. Upon cooling the sample down to room temperature, the reverse redistribution of the atomic composition within near-surface regions occurs, which switches the catalyst back into inactive state. This observation strongly suggests that the optimum active sites emerge under reaction conditions exclusively, involving both high temperature and a reactive atmosphere.
Collapse
Affiliation(s)
- Andrey V. Bukhtiyarov
- Synchrotron Radiation Facility SKIF, Boreskov Institute of Catalysis SB RAS, 630559 Kol’tsovo, Russia; (M.A.P.); (Y.V.Z.)
- Correspondence: ; Tel.: +7-383-326-97-87
| | - Igor P. Prosvirin
- Department of Physicochemical Techniques, Boreskov Institute of Catalysis SB RAS, 630090 Novosibirsk, Russia; (I.P.P.); (A.Y.F.); (V.I.B.)
| | - Maxim A. Panafidin
- Synchrotron Radiation Facility SKIF, Boreskov Institute of Catalysis SB RAS, 630559 Kol’tsovo, Russia; (M.A.P.); (Y.V.Z.)
| | - Alexey Yu. Fedorov
- Department of Physicochemical Techniques, Boreskov Institute of Catalysis SB RAS, 630090 Novosibirsk, Russia; (I.P.P.); (A.Y.F.); (V.I.B.)
- Department of Natural Sciences, Novosibirsk State University, 630090 Novosibirsk, Russia
| | - Alexander Yu. Klyushin
- Inorganic Chemistry Department, Fritz Haber Institute of the Max Planck Society, 14195 Berlin, Germany; (A.Y.K.); (A.K.-G.)
- BESSY II, Helmholtz Center for Materials and Energy, 12489 Berlin, Germany
| | - Axel Knop-Gericke
- Inorganic Chemistry Department, Fritz Haber Institute of the Max Planck Society, 14195 Berlin, Germany; (A.Y.K.); (A.K.-G.)
- Department of Heterogeneous Reactions, Max Planck Institute for Chemical Energy Conversion, 45470 Mülheim an der Ruhr, Germany
| | - Yan V. Zubavichus
- Synchrotron Radiation Facility SKIF, Boreskov Institute of Catalysis SB RAS, 630559 Kol’tsovo, Russia; (M.A.P.); (Y.V.Z.)
| | - Valery I. Bukhtiyarov
- Department of Physicochemical Techniques, Boreskov Institute of Catalysis SB RAS, 630090 Novosibirsk, Russia; (I.P.P.); (A.Y.F.); (V.I.B.)
| |
Collapse
|
16
|
Larichev YV. Dynamic Light Scattering for Studying Supported Metal Catalysts. KINETICS AND CATALYSIS 2021. [DOI: 10.1134/s0023158421040066] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/09/2023]
|
17
|
Mill L, Wolff D, Gerrits N, Philipp P, Kling L, Vollnhals F, Ignatenko A, Jaremenko C, Huang Y, De Castro O, Audinot JN, Nelissen I, Wirtz T, Maier A, Christiansen S. Synthetic Image Rendering Solves Annotation Problem in Deep Learning Nanoparticle Segmentation. SMALL METHODS 2021; 5:e2100223. [PMID: 34927995 DOI: 10.1002/smtd.202100223] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/25/2021] [Revised: 04/17/2021] [Indexed: 05/14/2023]
Abstract
Nanoparticles occur in various environments as a consequence of man-made processes, which raises concerns about their impact on the environment and human health. To allow for proper risk assessment, a precise and statistically relevant analysis of particle characteristics (such as size, shape, and composition) is required that would greatly benefit from automated image analysis procedures. While deep learning shows impressive results in object detection tasks, its applicability is limited by the amount of representative, experimentally collected and manually annotated training data. Here, an elegant, flexible, and versatile method to bypass this costly and tedious data acquisition process is presented. It shows that using a rendering software allows to generate realistic, synthetic training data to train a state-of-the art deep neural network. Using this approach, a segmentation accuracy can be derived that is comparable to man-made annotations for toxicologically relevant metal-oxide nanoparticle ensembles which were chosen as examples. The presented study paves the way toward the use of deep learning for automated, high-throughput particle detection in a variety of imaging techniques such as in microscopies and spectroscopies, for a wide range of applications, including the detection of micro- and nanoplastic particles in water and tissue samples.
Collapse
Affiliation(s)
- Leonid Mill
- Pattern Recognition Lab, Friedrich-Alexander-University Erlangen-Nuremberg, 91058, Erlangen, Germany
- Institute of Optics, Information and Photonics, Friedrich-Alexander-University Erlangen-Nuremberg, 91058, Erlangen, Germany
| | - David Wolff
- Institut für Nanotechnologie und korrelative Mikroskopie, 91301, Forchheim, Germany
| | - Nele Gerrits
- Health Unit, Flemish Institute for Technological Research, Mol, 2400, Belgium
| | - Patrick Philipp
- Advanced Instrumentation for Ion Nano-Analytics, Materials Research and Technology Department, Luxembourg Institute of Science and Technology, Belvaux, L-4422, Luxembourg
| | - Lasse Kling
- Institut für Nanotechnologie und korrelative Mikroskopie, 91301, Forchheim, Germany
| | - Florian Vollnhals
- Institute of Optics, Information and Photonics, Friedrich-Alexander-University Erlangen-Nuremberg, 91058, Erlangen, Germany
- Institut für Nanotechnologie und korrelative Mikroskopie, 91301, Forchheim, Germany
| | - Andrew Ignatenko
- Advanced Instrumentation for Ion Nano-Analytics, Materials Research and Technology Department, Luxembourg Institute of Science and Technology, Belvaux, L-4422, Luxembourg
| | - Christian Jaremenko
- Pattern Recognition Lab, Friedrich-Alexander-University Erlangen-Nuremberg, 91058, Erlangen, Germany
- Institut für Nanotechnologie und korrelative Mikroskopie, 91301, Forchheim, Germany
| | - Yixing Huang
- Pattern Recognition Lab, Friedrich-Alexander-University Erlangen-Nuremberg, 91058, Erlangen, Germany
- Institut für Nanotechnologie und korrelative Mikroskopie, 91301, Forchheim, Germany
| | - Olivier De Castro
- Advanced Instrumentation for Ion Nano-Analytics, Materials Research and Technology Department, Luxembourg Institute of Science and Technology, Belvaux, L-4422, Luxembourg
| | - Jean-Nicolas Audinot
- Advanced Instrumentation for Ion Nano-Analytics, Materials Research and Technology Department, Luxembourg Institute of Science and Technology, Belvaux, L-4422, Luxembourg
| | - Inge Nelissen
- Health Unit, Flemish Institute for Technological Research, Mol, 2400, Belgium
| | - Tom Wirtz
- Advanced Instrumentation for Ion Nano-Analytics, Materials Research and Technology Department, Luxembourg Institute of Science and Technology, Belvaux, L-4422, Luxembourg
| | - Andreas Maier
- Pattern Recognition Lab, Friedrich-Alexander-University Erlangen-Nuremberg, 91058, Erlangen, Germany
| | - Silke Christiansen
- Institute of Optics, Information and Photonics, Friedrich-Alexander-University Erlangen-Nuremberg, 91058, Erlangen, Germany
- Physics Department, Free University, 14195, Berlin, Germany
- Correlative Microscopy and Material Data Department, Fraunhofer Institute for Ceramic Technologies and Systems, 01277, Dresden, Germany
| |
Collapse
|
18
|
Boiko DA, Pentsak EO, Cherepanova VA, Gordeev EG, Ananikov VP. Deep neural network analysis of nanoparticle ordering to identify defects in layered carbon materials. Chem Sci 2021; 12:7428-7441. [PMID: 34163833 PMCID: PMC8171319 DOI: 10.1039/d0sc05696k] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2020] [Accepted: 04/05/2021] [Indexed: 11/21/2022] Open
Abstract
Smoothness/defectiveness of the carbon material surface is a key issue for many applications, spanning from electronics to reinforced materials, adsorbents and catalysis. Several surface defects cannot be observed with conventional analytic techniques, thus requiring the development of a new imaging approach. Here, we evaluate a convenient method for mapping such "hidden" defects on the surface of carbon materials using 1-5 nm metal nanoparticles as markers. A direct relationship between the presence of defects and the ordering of nanoparticles was studied experimentally and modeled using quantum chemistry calculations and Monte Carlo simulations. An automated pipeline for analyzing microscopic images is described: the degree of smoothness of experimental images was determined by a classification neural network, and then the images were searched for specific types of defects using a segmentation neural network. An informative set of features was generated from both networks: high-dimensional embeddings of image patches and statics of defect distribution.
Collapse
Affiliation(s)
- Daniil A Boiko
- Zelinsky Institute of Organic Chemistry, Russian Academy of Sciences Leninsky Pr. 47 Moscow 119991 Russia
| | - Evgeniy O Pentsak
- Zelinsky Institute of Organic Chemistry, Russian Academy of Sciences Leninsky Pr. 47 Moscow 119991 Russia
| | - Vera A Cherepanova
- Zelinsky Institute of Organic Chemistry, Russian Academy of Sciences Leninsky Pr. 47 Moscow 119991 Russia
| | - Evgeniy G Gordeev
- Zelinsky Institute of Organic Chemistry, Russian Academy of Sciences Leninsky Pr. 47 Moscow 119991 Russia
| | - Valentine P Ananikov
- Zelinsky Institute of Organic Chemistry, Russian Academy of Sciences Leninsky Pr. 47 Moscow 119991 Russia
| |
Collapse
|
19
|
Monchot P, Coquelin L, Guerroudj K, Feltin N, Delvallée A, Crouzier L, Fischer N. Deep Learning Based Instance Segmentation of Titanium Dioxide Particles in the Form of Agglomerates in Scanning Electron Microscopy. NANOMATERIALS (BASEL, SWITZERLAND) 2021; 11:968. [PMID: 33918779 PMCID: PMC8068950 DOI: 10.3390/nano11040968] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/09/2021] [Revised: 03/29/2021] [Accepted: 04/03/2021] [Indexed: 11/17/2022]
Abstract
The size characterization of particles present in the form of agglomerates in images measured by scanning electron microscopy (SEM) requires a powerful image segmentation tool in order to properly define the boundaries of each particle. In this work, we propose to use an algorithm from the deep statistical learning community, the Mask-RCNN, coupled with transfer learning to overcome the problem of generalization of the commonly used image processing methods such as watershed or active contour. Indeed, the adjustment of the parameters of these algorithms is almost systematically necessary and slows down the automation of the processing chain. The Mask-RCNN is adapted here to the case study and we present results obtained on titanium dioxide samples (non-spherical particles) with a level of performance evaluated by different metrics such as the DICE coefficient, which reaches an average value of 0.95 on the test images.
Collapse
Affiliation(s)
- Paul Monchot
- Data Science and Uncertainty Department, National Laboratory of Metrology and Testing, 29 Avenue Roger Hennequin, 78197 Trappes, France; (K.G.); (N.F.)
| | - Loïc Coquelin
- Data Science and Uncertainty Department, National Laboratory of Metrology and Testing, 29 Avenue Roger Hennequin, 78197 Trappes, France; (K.G.); (N.F.)
| | - Khaled Guerroudj
- Data Science and Uncertainty Department, National Laboratory of Metrology and Testing, 29 Avenue Roger Hennequin, 78197 Trappes, France; (K.G.); (N.F.)
| | - Nicolas Feltin
- Department of Materials Science, National Laboratory of Metrology and Testing, 29 Avenue Roger Hennequin, 78197 Trappes, France; (N.F.); (A.D.); (L.C.)
| | - Alexandra Delvallée
- Department of Materials Science, National Laboratory of Metrology and Testing, 29 Avenue Roger Hennequin, 78197 Trappes, France; (N.F.); (A.D.); (L.C.)
| | - Loïc Crouzier
- Department of Materials Science, National Laboratory of Metrology and Testing, 29 Avenue Roger Hennequin, 78197 Trappes, France; (N.F.); (A.D.); (L.C.)
| | - Nicolas Fischer
- Data Science and Uncertainty Department, National Laboratory of Metrology and Testing, 29 Avenue Roger Hennequin, 78197 Trappes, France; (K.G.); (N.F.)
| |
Collapse
|
20
|
Qu EZ, Jimenez AM, Kumar SK, Zhang K. Quantifying Nanoparticle Assembly States in a Polymer Matrix through Deep Learning. Macromolecules 2021. [DOI: 10.1021/acs.macromol.0c02483] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/16/2023]
Affiliation(s)
- Eric Zhonghang Qu
- Division of Natural and Applied Sciences, Duke Kunshan University, Kunshan, Jiangsu 215300, China
| | - Andrew Matthew Jimenez
- Department of Chemical Engineering, Columbia University, New York, New York 10027, United States
| | - Sanat K. Kumar
- Department of Chemical Engineering, Columbia University, New York, New York 10027, United States
| | - Kai Zhang
- Division of Natural and Applied Sciences, Duke Kunshan University, Kunshan, Jiangsu 215300, China
| |
Collapse
|