1
|
Kohan A, Rietman EA, Siegelmann HT. Signal Propagation: The Framework for Learning and Inference in a Forward Pass. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2024; 35:8585-8596. [PMID: 37022224 DOI: 10.1109/tnnls.2022.3230914] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/19/2023]
Abstract
We propose a new learning framework, signal propagation (sigprop), for propagating a learning signal and updating neural network parameters via a forward pass, as an alternative to backpropagation (BP). In sigprop, there is only the forward path for inference and learning. So, there are no structural or computational constraints necessary for learning to take place, beyond the inference model itself, such as feedback connectivity, weight transport, or a backward pass, which exist under BP-based approaches. That is, sigprop enables global supervised learning with only a forward path. This is ideal for parallel training of layers or modules. In biology, this explains how neurons without feedback connections can still receive a global learning signal. In hardware, this provides an approach for global supervised learning without backward connectivity. Sigprop by construction has compatibility with models of learning in the brain and in hardware than BP, including alternative approaches relaxing learning constraints. We also demonstrate that sigprop is more efficient in time and memory than they are. To further explain the behavior of sigprop, we provide evidence that sigprop provides useful learning signals in context to BP. To further support relevance to biological and hardware learning, we use sigprop to train continuous time neural networks with the Hebbian updates and train spiking neural networks (SNNs) with only the voltage or with biologically and hardware-compatible surrogate functions.
Collapse
|
2
|
Duan X, Cao Z, Gao K, Yan W, Sun S, Zhou G, Wu Z, Ren F, Sun B. Memristor-Based Neuromorphic Chips. ADVANCED MATERIALS (DEERFIELD BEACH, FLA.) 2024; 36:e2310704. [PMID: 38168750 DOI: 10.1002/adma.202310704] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/14/2023] [Revised: 12/15/2023] [Indexed: 01/05/2024]
Abstract
In the era of information, characterized by an exponential growth in data volume and an escalating level of data abstraction, there has been a substantial focus on brain-like chips, which are known for their robust processing power and energy-efficient operation. Memristors are widely acknowledged as the optimal electronic devices for the realization of neuromorphic computing, due to their innate ability to emulate the interconnection and information transfer processes witnessed among neurons. This review paper focuses on memristor-based neuromorphic chips, which provide an extensive description of the working principle and characteristic features of memristors, along with their applications in the realm of neuromorphic chips. Subsequently, a thorough discussion of the memristor array, which serves as the pivotal component of the neuromorphic chip, as well as an examination of the present mainstream neural networks, is delved. Furthermore, the design of the neuromorphic chip is categorized into three crucial sections, including synapse-neuron cores, networks on chip (NoC), and neural network design. Finally, the key performance metrics of the chip is highlighted, as well as the key metrics related to the memristor devices are employed to realize both the synaptic and neuronal components.
Collapse
Affiliation(s)
- Xuegang Duan
- National Local Joint Engineering Research Center for Precision Surgery & Regenerative Medicine, the First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
- Department of hepatobiliary surgery, the First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
- Frontier Institute of Science and Technology (FIST), Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
- Micro-and Nano-technology Research Center, State Key Laboratory for Manufacturing Systems Engineering, Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
| | - Zelin Cao
- National Local Joint Engineering Research Center for Precision Surgery & Regenerative Medicine, the First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
- Department of hepatobiliary surgery, the First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
- Frontier Institute of Science and Technology (FIST), Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
- Micro-and Nano-technology Research Center, State Key Laboratory for Manufacturing Systems Engineering, Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
| | - Kaikai Gao
- National Local Joint Engineering Research Center for Precision Surgery & Regenerative Medicine, the First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
- Department of hepatobiliary surgery, the First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
- Frontier Institute of Science and Technology (FIST), Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
- Micro-and Nano-technology Research Center, State Key Laboratory for Manufacturing Systems Engineering, Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
| | - Wentao Yan
- National Local Joint Engineering Research Center for Precision Surgery & Regenerative Medicine, the First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
- Department of hepatobiliary surgery, the First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
- Frontier Institute of Science and Technology (FIST), Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
- Micro-and Nano-technology Research Center, State Key Laboratory for Manufacturing Systems Engineering, Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
| | - Siyu Sun
- Frontier Institute of Science and Technology (FIST), Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
- Micro-and Nano-technology Research Center, State Key Laboratory for Manufacturing Systems Engineering, Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
| | - Guangdong Zhou
- College of Artificial Intelligence, Brain-inspired Computing & Intelligent Control of Chongqing Key Lab, Southwest University, Chongqing, 400715, China
| | - Zhenhua Wu
- School of Mechanical Engineering, Shanghai Jiao Tong University, 800 DongChuan Rd, Shanghai, 200240, China
| | - Fenggang Ren
- National Local Joint Engineering Research Center for Precision Surgery & Regenerative Medicine, the First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
- Department of hepatobiliary surgery, the First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
| | - Bai Sun
- National Local Joint Engineering Research Center for Precision Surgery & Regenerative Medicine, the First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
- Department of hepatobiliary surgery, the First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
- Frontier Institute of Science and Technology (FIST), Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
- Micro-and Nano-technology Research Center, State Key Laboratory for Manufacturing Systems Engineering, Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
| |
Collapse
|
3
|
Guo Q, Xie F, Zhong F, Wen W, Zhang X, Yu X, Wang X, Huang B, Li L, Wang X. Application of interpretable machine learning algorithms to predict distant metastasis in ovarian clear cell carcinoma. Cancer Med 2024; 13:e7161. [PMID: 38613173 PMCID: PMC11015070 DOI: 10.1002/cam4.7161] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/17/2023] [Revised: 03/16/2024] [Accepted: 03/26/2024] [Indexed: 04/14/2024] Open
Abstract
BACKGROUND Ovarian clear cell carcinoma (OCCC) represents a subtype of ovarian epithelial carcinoma (OEC) known for its limited responsiveness to chemotherapy, and the onset of distant metastasis significantly impacts patient prognoses. This study aimed to identify potential risk factors contributing to the occurrence of distant metastasis in OCCC. METHODS Utilizing the Surveillance, Epidemiology, and End Results (SEER) database, we identified patients diagnosed with OCCC between 2004 and 2015. The most influential factors were selected through the application of Gaussian Naive Bayes (GNB) and Adaboost machine learning algorithms, employing a Venn test for further refinement. Subsequently, six machine learning (ML) techniques, namely XGBoost, LightGBM, Random Forest (RF), Adaptive Boosting (Adaboost), Support Vector Machine (SVM), and Multilayer Perceptron (MLP), were employed to construct predictive models for distant metastasis. Shapley Additive Interpretation (SHAP) analysis facilitated a visual interpretation for individual patient. Model validity was assessed using accuracy, sensitivity, specificity, positive predictive value, negative predictive value, F1 score, and the area under the receiver operating characteristic curve (AUC). RESULTS In the realm of predicting distant metastasis, the Random Forest (RF) model outperformed the other five machine learning algorithms. The RF model demonstrated accuracy, sensitivity, specificity, positive predictive value, negative predictive value, F1 score, and AUC (95% CI) values of 0.792 (0.762-0.823), 0.904 (0.835-0.973), 0.759 (0.731-0.787), 0.221 (0.186-0.256), 0.974 (0.967-0.982), 0.353 (0.306-0.399), and 0.834 (0.696-0.967), respectively, surpassing the performance of other models. Additionally, the calibration curve's Brier Score (95%) for the RF model reached the minimum value of 0.06256 (0.05753-0.06759). SHAP analysis provided independent explanations, reaffirming the critical clinical factors associated with the risk of metastasis in OCCC patients. CONCLUSIONS This study successfully established a precise predictive model for OCCC patient metastasis using machine learning techniques, offering valuable support to clinicians in making informed clinical decisions.
Collapse
Affiliation(s)
- Qin‐Hua Guo
- Jiangxi Province Key Laboratory of Laboratory Medicine, Jiangxi Provincial Clinical Research Center for Laboratory Medicine, Department of Clinical Laboratory, The Second Affiliated HospitalJiangxi Medical College, Nanchang UniversityNanchangJiangxiChina
- Department of Clinical LaboratoryThe First Hospital of Nanchang (The Third Affiliated Hospital of Nanchang University)NanchangJiangxiChina
- School of Public HealthNanchang UniversityNanchangJiangxiChina
| | - Feng‐Chun Xie
- Department of Clinical LaboratoryNanchang Renai Obstetrics and Gynecology HospitalNanchangJiangxiChina
| | - Fang‐Min Zhong
- Jiangxi Province Key Laboratory of Laboratory Medicine, Jiangxi Provincial Clinical Research Center for Laboratory Medicine, Department of Clinical Laboratory, The Second Affiliated HospitalJiangxi Medical College, Nanchang UniversityNanchangJiangxiChina
| | - Wen Wen
- Jiangxi Province Key Laboratory of Laboratory Medicine, Jiangxi Provincial Clinical Research Center for Laboratory Medicine, Department of Clinical Laboratory, The Second Affiliated HospitalJiangxi Medical College, Nanchang UniversityNanchangJiangxiChina
- School of Public HealthNanchang UniversityNanchangJiangxiChina
| | - Xue‐Ru Zhang
- Jiangxi Province Key Laboratory of Laboratory Medicine, Jiangxi Provincial Clinical Research Center for Laboratory Medicine, Department of Clinical Laboratory, The Second Affiliated HospitalJiangxi Medical College, Nanchang UniversityNanchangJiangxiChina
- School of Public HealthNanchang UniversityNanchangJiangxiChina
| | - Xia‐Jing Yu
- Jiangxi Province Key Laboratory of Laboratory Medicine, Jiangxi Provincial Clinical Research Center for Laboratory Medicine, Department of Clinical Laboratory, The Second Affiliated HospitalJiangxi Medical College, Nanchang UniversityNanchangJiangxiChina
- School of Public HealthNanchang UniversityNanchangJiangxiChina
| | - Xin‐Lu Wang
- Jiangxi Province Key Laboratory of Laboratory Medicine, Jiangxi Provincial Clinical Research Center for Laboratory Medicine, Department of Clinical Laboratory, The Second Affiliated HospitalJiangxi Medical College, Nanchang UniversityNanchangJiangxiChina
- School of Public HealthNanchang UniversityNanchangJiangxiChina
| | - Bo Huang
- Jiangxi Province Key Laboratory of Laboratory Medicine, Jiangxi Provincial Clinical Research Center for Laboratory Medicine, Department of Clinical Laboratory, The Second Affiliated HospitalJiangxi Medical College, Nanchang UniversityNanchangJiangxiChina
| | - Li‐Ping Li
- Department of Clinical LaboratoryThe First Hospital of Nanchang (The Third Affiliated Hospital of Nanchang University)NanchangJiangxiChina
| | - Xiao‐Zhong Wang
- Jiangxi Province Key Laboratory of Laboratory Medicine, Jiangxi Provincial Clinical Research Center for Laboratory Medicine, Department of Clinical Laboratory, The Second Affiliated HospitalJiangxi Medical College, Nanchang UniversityNanchangJiangxiChina
- Department of Clinical LaboratoryThe First Hospital of Nanchang (The Third Affiliated Hospital of Nanchang University)NanchangJiangxiChina
- School of Public HealthNanchang UniversityNanchangJiangxiChina
| |
Collapse
|