1
|
Wang X, Ji Z, Yu Y, Pang Y, Han J. Model Attention Expansion for Few-Shot Class-Incremental Learning. IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY 2024; 33:4419-4431. [PMID: 39088502 DOI: 10.1109/tip.2024.3434475] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/03/2024]
Abstract
Few-Shot Class-Incremental Learning (FSCIL) aims at incrementally learning new knowledge from limited training examples without forgetting previous knowledge. However, we observe that existing methods face a challenge known as supervision collapse, where the model disproportionately emphasizes class-specific features of base classes at the detriment of novel class representations, leading to restricted cognitive capabilities. To alleviate this issue, we propose a new framework, Model aTtention Expansion for Few-Shot Class-Incremental Learning (MTE-FSCIL), aimed at expanding the model attention fields to improve transferability without compromising the discriminative capability for base classes. Specifically, the framework adopts a dual-stage training strategy, comprising pre-training and meta-training stages. In the pre-training stage, we present a new regularization technique, named the Reserver (RS) loss, to expand the global perception and reduce over-reliance on class-specific features by amplifying feature map activations. During the meta-training stage, we introduce the Repeller (RP) loss, a novel pair-based loss that promotes variation in representations and improves the model's recognition of sample uniqueness by scattering intra-class samples within the embedding space. Furthermore, we propose a Transformational Adaptation (TA) strategy to enable continuous incorporation of new knowledge from downstream tasks, thus facilitating cross-task knowledge transfer. Extensive experimental results on mini-ImageNet, CIFAR100, and CUB200 datasets demonstrate that our proposed framework consistently outperforms the state-of-the-art methods.
Collapse
|
2
|
Yuan T, Song X, Shi Y, Wei S, Han Y, Yang L, Zhang Y, Li X, Li Y, Shen L, Fan L. Perspectives on development of optoelectronic materials in artificial intelligence age. Chem Asian J 2024:e202301088. [PMID: 38317532 DOI: 10.1002/asia.202301088] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2023] [Revised: 01/28/2024] [Accepted: 02/05/2024] [Indexed: 02/07/2024]
Abstract
Optoelectronic devices, such as light-emitting diodes, have been demonstrated as one of the most demanded forthcoming display and lighting technologies because of their low cost, low power consumption, high brightness, and high contrast. The improvement of device performance relies on advances in precisely designing novelty functional materials, including light-emitting materials, hosts, hole/electron transport materials, and yet which is a time-consuming, laborious and resource-intensive task. Recently, machine learning (ML) has shown great prospects to accelerate material discovery and property enhancement. This review will summarize the workflow of ML in optoelectronic materials discovery, including data collection, feature engineering, model selection, model evaluation and model application. We highlight multiple recent applications of machine-learned potentials in various optoelectronic functional materials, ranging from semiconductor quantum dots (QDs) or perovskite QDs, organic molecules to carbon-based nanomaterials. We furthermore discuss the current challenges to fully realize the potential of ML-assisted materials design for optoelectronics applications. It is anticipated that this review will provide critical insights to inspire new exciting discoveries on ML-guided of high-performance optoelectronic devices with a combined effort from different disciplines.
Collapse
Affiliation(s)
- Ting Yuan
- College of Chemistry, Key Laboratory of Theoretical & Computational Photochemistry of Ministry of Education, Beijing Normal University, Beijing, 100875, China
| | - Xianzhi Song
- College of Chemistry, Key Laboratory of Theoretical & Computational Photochemistry of Ministry of Education, Beijing Normal University, Beijing, 100875, China
| | - Yuxin Shi
- College of Chemistry, Key Laboratory of Theoretical & Computational Photochemistry of Ministry of Education, Beijing Normal University, Beijing, 100875, China
| | - Shuyan Wei
- College of Chemistry, Key Laboratory of Theoretical & Computational Photochemistry of Ministry of Education, Beijing Normal University, Beijing, 100875, China
| | - Yuyi Han
- College of Chemistry, Key Laboratory of Theoretical & Computational Photochemistry of Ministry of Education, Beijing Normal University, Beijing, 100875, China
| | - Linjuan Yang
- College of Chemistry, Key Laboratory of Theoretical & Computational Photochemistry of Ministry of Education, Beijing Normal University, Beijing, 100875, China
| | - Yang Zhang
- College of Chemistry, Key Laboratory of Theoretical & Computational Photochemistry of Ministry of Education, Beijing Normal University, Beijing, 100875, China
| | - Xiaohong Li
- College of Chemistry, Key Laboratory of Theoretical & Computational Photochemistry of Ministry of Education, Beijing Normal University, Beijing, 100875, China
| | - Yunchao Li
- College of Chemistry, Key Laboratory of Theoretical & Computational Photochemistry of Ministry of Education, Beijing Normal University, Beijing, 100875, China
| | - Lin Shen
- College of Chemistry, Key Laboratory of Theoretical & Computational Photochemistry of Ministry of Education, Beijing Normal University, Beijing, 100875, China
| | - Louzhen Fan
- College of Chemistry, Key Laboratory of Theoretical & Computational Photochemistry of Ministry of Education, Beijing Normal University, Beijing, 100875, China
| |
Collapse
|
3
|
Schlender T, Viljanen M, van Rijn JN, Mohr F, Peijnenburg WJGM, Hoos HH, Rorije E, Wong A. The Bigger Fish: A Comparison of Meta-Learning QSAR Models on Low-Resourced Aquatic Toxicity Regression Tasks. ENVIRONMENTAL SCIENCE & TECHNOLOGY 2023; 57:17818-17830. [PMID: 37315216 PMCID: PMC10666535 DOI: 10.1021/acs.est.3c00334] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/12/2023] [Revised: 05/25/2023] [Accepted: 05/25/2023] [Indexed: 06/16/2023]
Abstract
Toxicological information as needed for risk assessments of chemical compounds is often sparse. Unfortunately, gathering new toxicological information experimentally often involves animal testing. Simulated alternatives, e.g., quantitative structure-activity relationship (QSAR) models, are preferred to infer the toxicity of new compounds. Aquatic toxicity data collections consist of many related tasks─each predicting the toxicity of new compounds on a given species. Since many of these tasks are inherently low-resource, i.e., involve few associated compounds, this is challenging. Meta-learning is a subfield of artificial intelligence that can lead to more accurate models by enabling the utilization of information across tasks. In our work, we benchmark various state-of-the-art meta-learning techniques for building QSAR models, focusing on knowledge sharing between species. Specifically, we employ and compare transformational machine learning, model-agnostic meta-learning, fine-tuning, and multi-task models. Our experiments show that established knowledge-sharing techniques outperform single-task approaches. We recommend the use of multi-task random forest models for aquatic toxicity modeling, which matched or exceeded the performance of other approaches and robustly produced good results in the low-resource settings we studied. This model functions on a species level, predicting toxicity for multiple species across various phyla, with flexible exposure duration and on a large chemical applicability domain.
Collapse
Affiliation(s)
- Thalea Schlender
- Leiden
Institute of Advanced Computer Science, Leiden University, Leiden 2333 CA, The Netherlands
- National
Institute for Public Health and the Environment (RIVM), Bilthoven 3720 BA, The Netherlands
| | - Markus Viljanen
- National
Institute for Public Health and the Environment (RIVM), Bilthoven 3720 BA, The Netherlands
| | - Jan N. van Rijn
- Leiden
Institute of Advanced Computer Science, Leiden University, Leiden 2333 CA, The Netherlands
| | - Felix Mohr
- Universidad
de La Sabana, Chía 250001, Colombia
| | - Willie JGM. Peijnenburg
- National
Institute for Public Health and the Environment (RIVM), Bilthoven 3720 BA, The Netherlands
- Institute
of Environmental Sciences, Leiden University, Leiden 2333 CC, The Netherlands
| | - Holger H. Hoos
- Leiden
Institute of Advanced Computer Science, Leiden University, Leiden 2333 CA, The Netherlands
- Chair
for AI Methodology, RWTH Aaachen University, Aachen 52056, Germany
- Department
of Computer Science, The University of British
Columbia, Vancouver V6T 1Z4, Canada
| | - Emiel Rorije
- National
Institute for Public Health and the Environment (RIVM), Bilthoven 3720 BA, The Netherlands
| | - Albert Wong
- National
Institute for Public Health and the Environment (RIVM), Bilthoven 3720 BA, The Netherlands
| |
Collapse
|
4
|
Garcia‐Quintas A, Roy A, Barbraud C, Demarcq H, Denis D, Lanco Bertrand S. Machine and deep learning approaches to understand and predict habitat suitability for seabird breeding. Ecol Evol 2023; 13:e10549. [PMID: 37727776 PMCID: PMC10505760 DOI: 10.1002/ece3.10549] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2023] [Revised: 08/31/2023] [Accepted: 09/05/2023] [Indexed: 09/21/2023] Open
Abstract
The way animals select their breeding habitat may have great impacts on individual fitness. This complex process depends on the integration of information on various environmental factors, over a wide range of spatiotemporal scales. For seabirds, breeding habitat selection integrates both land and sea features over several spatial scales. Seabirds explore these features prior to breeding, assessing habitats' quality. However, the information-gathering and decision-making process by seabirds when choosing a breeding habitat remains poorly understood. We compiled 49 historical records of larids colonies in Cuba from 1980 to 2020. Then, we predicted potentially suitable breeding sites for larids and assessed their breeding macrohabitat selection, using deep and machine learning algorithms respectively. Using a convolutional neural network and Landsat satellite images we predicted the suitability for nesting of non-monitored sites of this archipelago. Furthermore, we assessed the relative contribution of 18 land- and marine-based environmental covariates describing macrohabitats at three spatial scales (i.e. 10, 50 and 100 km) using random forests. Convolutional neural network exhibited good performance at training, validation and test (F1-scores >85%). Sites with higher habitat suitability (p > .75) covered 20.3% of the predicting area. Larids breeding macrohabitats were sites relatively close to main islands, featuring sparse vegetation cover and high chlorophyll-a concentration at sea in 50 and 100 km around colonies. Lower sea surface temperature at larger spatial scales was determinant to distinguish the breeding from non-breeding sites. A more comprehensive understanding of the seabird breeding macrohabitats selection can be reached from the complementary use of convolutional neural networks and random forest models. Our analysis provides crucial knowledge in tropical regions that lack complete and regular monitoring of seabirds' breeding sites.
Collapse
Affiliation(s)
- Antonio Garcia‐Quintas
- Institut de Recherche pour le Développement (IRD)MARBEC (Université de Montpellier, Ifremer, CNRS, IRD)SèteFrance
- Centro de Investigaciones de Ecosistemas Costeros (CIEC)Cayo CocoCuba
| | - Amédée Roy
- Institut de Recherche pour le Développement (IRD)MARBEC (Université de Montpellier, Ifremer, CNRS, IRD)SèteFrance
| | - Christophe Barbraud
- Centres d'Etudes Biologiques de Chizé UMR7372Centre National de la Recherche ScientifiqueVilliers en BoisFrance
| | - Hervé Demarcq
- Institut de Recherche pour le Développement (IRD)MARBEC (Université de Montpellier, Ifremer, CNRS, IRD)SèteFrance
| | | | - Sophie Lanco Bertrand
- Institut de Recherche pour le Développement (IRD)MARBEC (Université de Montpellier, Ifremer, CNRS, IRD)SèteFrance
| |
Collapse
|
5
|
Vinciguerra M, Olier I, Ortega-Martorell S, Lip GYH. New use for an old drug: Metformin and atrial fibrillation. Cell Rep Med 2022; 3:100875. [PMID: 36543101 PMCID: PMC9798075 DOI: 10.1016/j.xcrm.2022.100875] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
Abstract
Lal and colleagues1 reported an integrative approach-combining transcriptomics, iPSCs, and epidemiological evidence-to identify and repurpose metformin, a main first-line medication for the treatment of type 2 diabetes, as an effective risk reducer for atrial fibrillation.
Collapse
Affiliation(s)
- Manlio Vinciguerra
- Liverpool Centre for Cardiovascular Science (LCCS) at University of Liverpool, Liverpool Heart and Chest Hospital, Liverpool John Moores University, Liverpool, UK; Faculty of Health, Liverpool John Moores University, Liverpool, UK
| | - Ivan Olier
- Liverpool Centre for Cardiovascular Science (LCCS) at University of Liverpool, Liverpool Heart and Chest Hospital, Liverpool John Moores University, Liverpool, UK; School of Computer Science and Mathematics, Liverpool John Moores University, Liverpool, UK
| | - Sandra Ortega-Martorell
- Liverpool Centre for Cardiovascular Science (LCCS) at University of Liverpool, Liverpool Heart and Chest Hospital, Liverpool John Moores University, Liverpool, UK; School of Computer Science and Mathematics, Liverpool John Moores University, Liverpool, UK
| | - Gregory Y H Lip
- Liverpool Centre for Cardiovascular Science (LCCS) at University of Liverpool, Liverpool Heart and Chest Hospital, Liverpool John Moores University, Liverpool, UK; Department of Clinical Medicine, Aalborg University, Aalborg, Denmark.
| |
Collapse
|
6
|
Artificial intelligence in science: An emerging general method of invention. RESEARCH POLICY 2022. [DOI: 10.1016/j.respol.2022.104604] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
|
7
|
Mai H, Le TC, Chen D, Winkler DA, Caruso RA. Machine Learning for Electrocatalyst and Photocatalyst Design and Discovery. Chem Rev 2022; 122:13478-13515. [PMID: 35862246 DOI: 10.1021/acs.chemrev.2c00061] [Citation(s) in RCA: 61] [Impact Index Per Article: 30.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/17/2022]
Abstract
Electrocatalysts and photocatalysts are key to a sustainable future, generating clean fuels, reducing the impact of global warming, and providing solutions to environmental pollution. Improved processes for catalyst design and a better understanding of electro/photocatalytic processes are essential for improving catalyst effectiveness. Recent advances in data science and artificial intelligence have great potential to accelerate electrocatalysis and photocatalysis research, particularly the rapid exploration of large materials chemistry spaces through machine learning. Here a comprehensive introduction to, and critical review of, machine learning techniques used in electrocatalysis and photocatalysis research are provided. Sources of electro/photocatalyst data and current approaches to representing these materials by mathematical features are described, the most commonly used machine learning methods summarized, and the quality and utility of electro/photocatalyst models evaluated. Illustrations of how machine learning models are applied to novel electro/photocatalyst discovery and used to elucidate electrocatalytic or photocatalytic reaction mechanisms are provided. The review offers a guide for materials scientists on the selection of machine learning methods for electrocatalysis and photocatalysis research. The application of machine learning to catalysis science represents a paradigm shift in the way advanced, next-generation catalysts will be designed and synthesized.
Collapse
Affiliation(s)
- Haoxin Mai
- Applied Chemistry and Environmental Science, School of Science, STEM College, RMIT University, GPO Box 2476, Melbourne, Victoria 3001, Australia
| | - Tu C Le
- School of Engineering, STEM College, RMIT University, GPO Box 2476, Melbourne, Victoria 3001, Australia
| | - Dehong Chen
- Applied Chemistry and Environmental Science, School of Science, STEM College, RMIT University, GPO Box 2476, Melbourne, Victoria 3001, Australia
| | - David A Winkler
- Monash Institute of Pharmaceutical Sciences, Monash University, Parkville, Victoria 3052, Australia.,Biochemistry and Chemistry, La Trobe University, Kingsbury Drive, Bundoora, Victoria 3042, Australia.,School of Pharmacy, University of Nottingham, Nottingham NG7 2RD, United Kingdom
| | - Rachel A Caruso
- Applied Chemistry and Environmental Science, School of Science, STEM College, RMIT University, GPO Box 2476, Melbourne, Victoria 3001, Australia
| |
Collapse
|