1
|
Jin G, Hong S, Rich J, Xia J, Kim K, You L, Zhao C, Huang TJ. Intelligent nanoscope for rapid nanomaterial identification and classification. LAB ON A CHIP 2022; 22:2978-2985. [PMID: 35647808 PMCID: PMC9378457 DOI: 10.1039/d2lc00206j] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/25/2023]
Abstract
Machine learning image recognition and classification of particles and materials is a rapidly expanding field. However, nanomaterial identification and classification are dependent on the image resolution, the image field of view, and the processing time. Optical microscopes are one of the most widely utilized technologies in laboratories across the world, due to their nondestructive abilities to identify and classify critical micro-sized objects and processes, but identifying and classifying critical nano-sized objects and processes with a conventional microscope are outside of its capabilities, due to the diffraction limit of the optics and small field of view. To overcome these challenges of nanomaterial identification and classification, we developed an intelligent nanoscope that combines machine learning and microsphere array-based imaging to: (1) surpass the diffraction limit of the microscope objective with microsphere imaging to provide high-resolution images; (2) provide large field-of-view imaging without the sacrifice of resolution by utilizing a microsphere array; and (3) rapidly classify nanomaterials using a deep convolution neural network. The intelligent nanoscope delivers more than 46 magnified images from a single image frame so that we collected more than 1000 images within 2 seconds. Moreover, the intelligent nanoscope achieves a 95% nanomaterial classification accuracy using 1000 images of training sets, which is 45% more accurate than without the microsphere array. The intelligent nanoscope also achieves a 92% bacteria classification accuracy using 50 000 images of training sets, which is 35% more accurate than without the microsphere array. This platform accomplished rapid, accurate detection and classification of nanomaterials with miniscule size differences. The capabilities of this device wield the potential to further detect and classify smaller biological nanomaterial, such as viruses or extracellular vesicles.
Collapse
Affiliation(s)
- Geonsoo Jin
- Thomas Lord Department of Mechanical Engineering and Materials Science, Duke University, Durham, NC 27708, USA.
| | - Seongwoo Hong
- Office of Biomedical Graduate Education, Duke University School of Medicine, Durham, NC 27710, USA
| | - Joseph Rich
- Department of Biomedical Engineering, Duke University, Durham, NC 27708, USA.
| | - Jianping Xia
- Thomas Lord Department of Mechanical Engineering and Materials Science, Duke University, Durham, NC 27708, USA.
| | - Kyeri Kim
- Department of Biomedical Engineering, Duke University, Durham, NC 27708, USA.
| | - Lingchong You
- Department of Biomedical Engineering, Duke University, Durham, NC 27708, USA.
- Center for Genomic and Computational Biology, Duke University, Durham, NC 27708, USA
- Department of Molecular Genetics and Microbiology, Duke University School of Medicine, Durham, NC 27708, USA
| | - Chenglong Zhao
- Department of Physics, University of Dayton, 300 College Park, Dayton, Ohio 45469, USA.
- Department of Electro-Optics and Photonics, University of Dayton, 300 College Park, Dayton, Ohio 45469, USA
| | - Tony Jun Huang
- Thomas Lord Department of Mechanical Engineering and Materials Science, Duke University, Durham, NC 27708, USA.
| |
Collapse
|
2
|
Lv H, Chen X. Intelligent control of nanoparticle synthesis through machine learning. NANOSCALE 2022; 14:6688-6708. [PMID: 35450983 DOI: 10.1039/d2nr00124a] [Citation(s) in RCA: 14] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
The synthesis of nanoparticles is affected by many reaction conditions, and their properties are usually determined by factors such as their size, shape and surface chemistry. In order for the synthesized nanoparticles to have functions suitable for different fields (for example, optics, electronics, sensor applications and so on), precise control of their properties is essential. However, with the current technology of preparing nanoparticles on a microreactor, it is time-consuming and laborious to achieve precise synthesis. In order to improve the efficiency of synthesizing nanoparticles with the expected functionality, the application of machine learning-assisted synthesis is an intelligent choice. In this article, we mainly introduce the typical methods of preparing nanoparticles on microreactors, and explain the principles and procedures of machine learning, as well as the main ways of obtaining data sets. We have studied three types of representative nanoparticle preparation methods assisted by machine learning. Finally, the current problems in machine learning-assisted nanoparticle synthesis and future development prospects are discussed.
Collapse
Affiliation(s)
- Honglin Lv
- College of Transportation, Ludong University, Yantai, Shandong 264025, China.
| | - Xueye Chen
- College of Transportation, Ludong University, Yantai, Shandong 264025, China.
| |
Collapse
|
3
|
Asari Y, Terada S, Tanigaki T, Takahashi Y, Shinada H, Nakajima H, Kanie K, Murakami Y. Deep convolutional neural network image processing method providing improved signal-to-noise ratios in electron holography. Microscopy (Oxf) 2021; 70:442-449. [PMID: 33730158 DOI: 10.1093/jmicro/dfab012] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/18/2020] [Revised: 03/08/2021] [Accepted: 03/17/2021] [Indexed: 11/14/2022] Open
Abstract
An image identification method was developed with the aid of a deep convolutional neural network (CNN) and applied to the analysis of inorganic particles using electron holography. Despite significant variation in the shapes of α-Fe2O3 particles that were observed by transmission electron microscopy, this CNN-based method could be used to identify isolated, spindle-shaped particles that were distinct from other particles that had undergone pairing and/or agglomeration. The averaging of images of these isolated particles provided a significant improvement in the phase analysis precision of the electron holography observations. This method is expected to be helpful in the analysis of weak electromagnetic fields generated by nanoparticles showing only small phase shifts.
Collapse
Affiliation(s)
- Yusuke Asari
- Center for Technology Innovation, Hitachi Ltd, 7-1-1 Omika, Hitachi, Ibaraki 312-1292, Japan
| | - Shohei Terada
- Center for Technology Innovation, Hitachi Ltd, 7-1-1 Omika, Hitachi, Ibaraki 312-1292, Japan
| | - Toshiaki Tanigaki
- Center for Exploratory Research, Hitachi Ltd, Akanuma 2520, Hatoyama, Saitama 350-0395, Japan
| | - Yoshio Takahashi
- Center for Exploratory Research, Hitachi Ltd, Akanuma 2520, Hatoyama, Saitama 350-0395, Japan
| | - Hiroyuki Shinada
- Center for Exploratory Research, Hitachi Ltd, Akanuma 2520, Hatoyama, Saitama 350-0395, Japan
| | - Hiroshi Nakajima
- Department of Applied Quantum Physics and Nuclear Engineering, Kyushu University, 744 Motooka, Nishi-ku, Fukuoka, Fukuoka 819-0395, Japan
| | - Kiyoshi Kanie
- Institute of Multidisciplinary Research for Advanced Materials, Tohoku University, 2-1-1 Katahira, Aoba-ku, Sendai, Miyagi 980-8577, Japan
| | - Yasukazu Murakami
- Department of Applied Quantum Physics and Nuclear Engineering, Kyushu University, 744 Motooka, Nishi-ku, Fukuoka, Fukuoka 819-0395, Japan.,The Ultramicroscopy Research Center, Kyushu University, 744 Motooka, Nishi-ku, Fukuoka, Fukuoka 819-0395, Japan
| |
Collapse
|
4
|
Frydrych K, Karimi K, Pecelerowicz M, Alvarez R, Dominguez-Gutiérrez FJ, Rovaris F, Papanikolaou S. Materials Informatics for Mechanical Deformation: A Review of Applications and Challenges. MATERIALS (BASEL, SWITZERLAND) 2021; 14:5764. [PMID: 34640157 PMCID: PMC8510221 DOI: 10.3390/ma14195764] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 09/07/2021] [Revised: 09/24/2021] [Accepted: 09/27/2021] [Indexed: 11/23/2022]
Abstract
In the design and development of novel materials that have excellent mechanical properties, classification and regression methods have been diversely used across mechanical deformation simulations or experiments. The use of materials informatics methods on large data that originate in experiments or/and multiscale modeling simulations may accelerate materials' discovery or develop new understanding of materials' behavior. In this fast-growing field, we focus on reviewing advances at the intersection of data science with mechanical deformation simulations and experiments, with a particular focus on studies of metals and alloys. We discuss examples of applications, as well as identify challenges and prospects.
Collapse
Affiliation(s)
- Karol Frydrych
- NOMATEN Centre of Excellence, National Centre for Nuclear Research, ul. A. Sołtana 7, 05-400 Swierk-Otwock, Poland; (K.F.); (K.K.); (M.P.); (R.A.); (F.J.D.-G.); (F.R.)
| | - Kamran Karimi
- NOMATEN Centre of Excellence, National Centre for Nuclear Research, ul. A. Sołtana 7, 05-400 Swierk-Otwock, Poland; (K.F.); (K.K.); (M.P.); (R.A.); (F.J.D.-G.); (F.R.)
| | - Michal Pecelerowicz
- NOMATEN Centre of Excellence, National Centre for Nuclear Research, ul. A. Sołtana 7, 05-400 Swierk-Otwock, Poland; (K.F.); (K.K.); (M.P.); (R.A.); (F.J.D.-G.); (F.R.)
| | - Rene Alvarez
- NOMATEN Centre of Excellence, National Centre for Nuclear Research, ul. A. Sołtana 7, 05-400 Swierk-Otwock, Poland; (K.F.); (K.K.); (M.P.); (R.A.); (F.J.D.-G.); (F.R.)
| | - Francesco Javier Dominguez-Gutiérrez
- NOMATEN Centre of Excellence, National Centre for Nuclear Research, ul. A. Sołtana 7, 05-400 Swierk-Otwock, Poland; (K.F.); (K.K.); (M.P.); (R.A.); (F.J.D.-G.); (F.R.)
- Institute for Advanced Computational Science, Stony Brook University, Stony Brook, NY 11749, USA
| | - Fabrizio Rovaris
- NOMATEN Centre of Excellence, National Centre for Nuclear Research, ul. A. Sołtana 7, 05-400 Swierk-Otwock, Poland; (K.F.); (K.K.); (M.P.); (R.A.); (F.J.D.-G.); (F.R.)
| | - Stefanos Papanikolaou
- NOMATEN Centre of Excellence, National Centre for Nuclear Research, ul. A. Sołtana 7, 05-400 Swierk-Otwock, Poland; (K.F.); (K.K.); (M.P.); (R.A.); (F.J.D.-G.); (F.R.)
| |
Collapse
|
5
|
Mochalova EN, Kotov IA, Rozenberg JM, Nikitin MP. Precise Quantitative Analysis of Cell Targeting by Particle-Based Agents Using Imaging Flow Cytometry and Convolutional Neural Network. Cytometry A 2019; 97:279-287. [PMID: 31809002 DOI: 10.1002/cyto.a.23939] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2019] [Revised: 10/13/2019] [Accepted: 11/06/2019] [Indexed: 01/10/2023]
Abstract
Understanding the intricacies of particle-cell interactions is essential for many applications such as imaging, phototherapy, and drug/gene delivery, because it is the key to accurate control of the particle properties for the improvement of their therapeutic and diagnostic efficiency. Recently, high-throughput methods have emerged for the detailed investigation of these interactions. For example, imaging flow cytometry (IFC) collects up to 60,000 images of cells per second (in 12 optical channels) and provides information about morphology and organelle localization in combination with fluorescence and side scatter intensity data. However, analysis of IFC data is extremely difficult to perform using conventional methods that calculate integral parameters or use mask-based object recognition. Here, we show application of a convolutional neural network (CNN) for precise quantitative analysis of particle targeting of cells using IFC data. CNN provides high-throughput object detection with almost human precision but avoids the subjective choice of image processing parameters that often leads to incorrect data interpretation. The method allows accurate counting of cell-bound particles with reliable discrimination from the nonbound particles in the field of view. The proposed method expands capabilities of spot counting applications (such as organelle counting, quantification of cell-cell and cell-bacteria interactions) and is going to be useful not only for high-throughput analysis of IFC data but also for other imaging techniques. © 2019 International Society for Advancement of Cytometry.
Collapse
Affiliation(s)
- Elizaveta N Mochalova
- Moscow Institute of Physics and Technology, 1A Kerchenskaya St., 117303, Moscow, Russia.,Prokhorov General Physics Institute of the Russian Academy of Sciences, 38 Vavilov St., 119991, Moscow, Russia
| | - Ivan A Kotov
- Moscow Institute of Physics and Technology, 1A Kerchenskaya St., 117303, Moscow, Russia
| | - Julian M Rozenberg
- Moscow Institute of Physics and Technology, 1A Kerchenskaya St., 117303, Moscow, Russia
| | - Maxim P Nikitin
- Moscow Institute of Physics and Technology, 1A Kerchenskaya St., 117303, Moscow, Russia
| |
Collapse
|
6
|
Wahab N, Khan A, Lee YS. Transfer learning based deep CNN for segmentation and detection of mitoses in breast cancer histopathological images. Microscopy (Oxf) 2019; 68:216-233. [DOI: 10.1093/jmicro/dfz002] [Citation(s) in RCA: 56] [Impact Index Per Article: 11.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/16/2018] [Revised: 12/21/2018] [Accepted: 01/11/2019] [Indexed: 01/17/2023] Open
Affiliation(s)
- Noorul Wahab
- Pattern Recognition Lab, Department of Computer and Information Sciences, Pakistan Institute of Engineering and Applied Sciences, Nilore, Islamabad
| | - Asifullah Khan
- Pattern Recognition Lab, Department of Computer and Information Sciences, Pakistan Institute of Engineering and Applied Sciences, Nilore, Islamabad
- Deep Learning Lab, Centre for Mathematical Sciences, Pakistan Institute of Engineering and Applied Sciences, Nilore, Islamabad
| | - Yeon Soo Lee
- Department of Biomedical Engineering, College of Medical Science, Catholic University of Daegu, Gyoungsangbuk-do, Republic of Korea
| |
Collapse
|