1
|
Sajid M, Sharma R, Beheshti I, Tanveer M. Decoding cognitive health using machine learning: A comprehensive evaluation for diagnosis of significant memory concern. WIRES DATA MINING AND KNOWLEDGE DISCOVERY 2024; 14. [DOI: 10.1002/widm.1546] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/26/2023] [Accepted: 04/29/2024] [Indexed: 01/03/2025]
Abstract
AbstractThe timely identification of significant memory concern (SMC) is crucial for proactive cognitive health management, especially in an aging population. Detecting SMC early enables timely intervention and personalized care, potentially slowing cognitive disorder progression. This study presents a state‐of‐the‐art review followed by a comprehensive evaluation of machine learning models within the randomized neural networks (RNNs) and hyperplane‐based classifiers (HbCs) family to investigate SMC diagnosis thoroughly. Utilizing the Alzheimer's Disease Neuroimaging Initiative 2 (ADNI2) dataset, 111 individuals with SMC and 111 healthy older adults are analyzed based on T1W magnetic resonance imaging (MRI) scans, extracting rich features. This analysis is based on baseline structural MRI (sMRI) scans, extracting rich features from gray matter (GM), white matter (WM), Jacobian determinant (JD), and cortical thickness (CT) measurements. In RNNs, deep random vector functional link (dRVFL) and ensemble dRVFL (edRVFL) emerge as the best classifiers in terms of performance metrics in the identification of SMC. In HbCs, Kernelized pinball general twin support vector machine (Pin‐GTSVM‐K) excels in CT and WM features, whereas Linear Pin‐GTSVM (Pin‐GTSVM‐L) and Linear intuitionistic fuzzy TSVM (IFTSVM‐L) performs well in the JD and GM features sets, respectively. This comprehensive evaluation emphasizes the critical role of feature selection, feature based‐interpretability and model choice in attaining an effective classifier for SMC diagnosis. The inclusion of statistical analyses further reinforces the credibility of the results, affirming the rigor of this analysis. The performance measures exhibit the suitability of this framework in aiding researchers with the automated and accurate assessment of SMC. The source codes of the algorithms and datasets used in this study are available at https://github.com/mtanveer1/SMC.This article is categorized under:
Technologies > Classification
Technologies > Machine Learning
Application Areas > Health Care
Collapse
Affiliation(s)
- M. Sajid
- Department of Mathematics Indian Institute of Technology Indore Indore India
| | - R. Sharma
- Department of Mathematics Indian Institute of Technology Indore Indore India
| | - I. Beheshti
- Department of Human Anatomy and Cell Science, Rady Faculty of Health Sciences University of Manitoba Winnipeg Manitoba Canada
- Neuroscience Research Program, Kleysen Institute for Advanced Medicine Health Sciences Centre Winnipeg Manitoba Canada
| | - M. Tanveer
- Department of Mathematics Indian Institute of Technology Indore Indore India
| | | |
Collapse
|
2
|
Gómez-Valadés A, Martínez R, Rincón M. Designing an effective semantic fluency test for early MCI diagnosis with machine learning. Comput Biol Med 2024; 180:108955. [PMID: 39153392 DOI: 10.1016/j.compbiomed.2024.108955] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2024] [Revised: 07/04/2024] [Accepted: 07/26/2024] [Indexed: 08/19/2024]
Abstract
Semantic fluency tests are one of the key tests used in batteries for the early detection of Mild Cognitive Impairment (MCI) as the impairment in speech and semantic memory are among the first symptoms, attracting the attention of a large number of studies. Several new semantic categories and variables capable of providing complementary information of clinical interest have been proposed to increase their effectiveness. However, this also extends the time required to complete all tests and get the overall diagnosis. Therefore, there is a need to reduce the number of tests in the batteries and thus the time spent on them while maintaining or increasing their effectiveness. This study used machine learning methods to determine the smallest and most efficient combination of semantic categories and variables to achieve this goal. We utilized a database containing 423 assessments from 141 subjects, with each subject having undergone three assessments spaced approximately one year apart. Subjects were categorized into three diagnostic groups: Healthy (if diagnosed as healthy in all three assessments), stable MCI (consistently diagnosed as MCI), and heterogeneous MCI (when exhibiting alternations between healthy and MCI diagnoses across assessments). We obtained that the most efficient combination to distinguish between these categories of semantic fluency tests included the animals and clothes semantic categories with the variables corrects, switching, clustering, and total clusters. This combination is ideal for scenarios that require a balance between time efficiency and diagnosis capability, such as population-based screenings.
Collapse
Affiliation(s)
- Alba Gómez-Valadés
- Universidad Nacional de Educación a Distancia, Madrid, 28040, Comunidad Autónoma de Madrid, Spain(1).
| | - Rafael Martínez
- Universidad Nacional de Educación a Distancia, Madrid, 28040, Comunidad Autónoma de Madrid, Spain(1).
| | - Mariano Rincón
- Universidad Nacional de Educación a Distancia, Madrid, 28040, Comunidad Autónoma de Madrid, Spain(1).
| |
Collapse
|
3
|
Maharjan J, Garikipati A, Singh NP, Cyrus L, Sharma M, Ciobanu M, Barnes G, Thapa R, Mao Q, Das R. OpenMedLM: prompt engineering can out-perform fine-tuning in medical question-answering with open-source large language models. Sci Rep 2024; 14:14156. [PMID: 38898116 PMCID: PMC11187169 DOI: 10.1038/s41598-024-64827-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2024] [Accepted: 06/13/2024] [Indexed: 06/21/2024] Open
Abstract
LLMs can accomplish specialized medical knowledge tasks, however, equitable access is hindered by the extensive fine-tuning, specialized medical data requirement, and limited access to proprietary models. Open-source (OS) medical LLMs show performance improvements and provide the transparency and compliance required in healthcare. We present OpenMedLM, a prompting platform delivering state-of-the-art (SOTA) performance for OS LLMs on medical benchmarks. We evaluated OS foundation LLMs (7B-70B) on medical benchmarks (MedQA, MedMCQA, PubMedQA, MMLU medical-subset) and selected Yi34B for developing OpenMedLM. Prompting strategies included zero-shot, few-shot, chain-of-thought, and ensemble/self-consistency voting. OpenMedLM delivered OS SOTA results on three medical LLM benchmarks, surpassing previous best-performing OS models that leveraged costly and extensive fine-tuning. OpenMedLM displays the first results to date demonstrating the ability of OS foundation models to optimize performance, absent specialized fine-tuning. The model achieved 72.6% accuracy on MedQA, outperforming the previous SOTA by 2.4%, and 81.7% accuracy on MMLU medical-subset, establishing itself as the first OS LLM to surpass 80% accuracy on this benchmark. Our results highlight medical-specific emergent properties in OS LLMs not documented elsewhere to date and validate the ability of OS models to accomplish healthcare tasks, highlighting the benefits of prompt engineering to improve performance of accessible LLMs for medical applications.
Collapse
Affiliation(s)
- Jenish Maharjan
- Montera, Inc. Dba Forta, 548 Market St., PMB 89605, San Francisco, CA, 94104-5401, USA
| | - Anurag Garikipati
- Montera, Inc. Dba Forta, 548 Market St., PMB 89605, San Francisco, CA, 94104-5401, USA
| | - Navan Preet Singh
- Montera, Inc. Dba Forta, 548 Market St., PMB 89605, San Francisco, CA, 94104-5401, USA
| | - Leo Cyrus
- Montera, Inc. Dba Forta, 548 Market St., PMB 89605, San Francisco, CA, 94104-5401, USA
| | - Mayank Sharma
- Montera, Inc. Dba Forta, 548 Market St., PMB 89605, San Francisco, CA, 94104-5401, USA
| | - Madalina Ciobanu
- Montera, Inc. Dba Forta, 548 Market St., PMB 89605, San Francisco, CA, 94104-5401, USA
| | - Gina Barnes
- Montera, Inc. Dba Forta, 548 Market St., PMB 89605, San Francisco, CA, 94104-5401, USA
| | - Rahul Thapa
- Montera, Inc. Dba Forta, 548 Market St., PMB 89605, San Francisco, CA, 94104-5401, USA
| | - Qingqing Mao
- Montera, Inc. Dba Forta, 548 Market St., PMB 89605, San Francisco, CA, 94104-5401, USA.
| | - Ritankar Das
- Montera, Inc. Dba Forta, 548 Market St., PMB 89605, San Francisco, CA, 94104-5401, USA
| |
Collapse
|
4
|
Park I, Lee SK, Choi HC, Ahn ME, Ryu OH, Jang D, Lee U, Kim YJ. Machine Learning Model for Mild Cognitive Impairment Stage Based on Gait and MRI Images. Brain Sci 2024; 14:480. [PMID: 38790458 PMCID: PMC11119859 DOI: 10.3390/brainsci14050480] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/13/2024] [Revised: 04/28/2024] [Accepted: 05/01/2024] [Indexed: 05/26/2024] Open
Abstract
In patients with mild cognitive impairment (MCI), a lower level of cognitive function is associated with a higher likelihood of progression to dementia. In addition, gait disturbances and structural changes on brain MRI scans reflect cognitive levels. Therefore, we aimed to classify MCI based on cognitive level using gait parameters and brain MRI data. Eighty patients diagnosed with MCI from three dementia centres in Gangwon-do, Korea, were recruited for this study. We defined MCI as a Clinical Dementia Rating global score of ≥0.5, with a memory domain score of ≥0.5. Patients were classified as early-stage or late-stage MCI based on their mini-mental status examination (MMSE) z-scores. We trained a machine learning model using gait and MRI data parameters. The convolutional neural network (CNN) resulted in the best classifier performance in separating late-stage MCI from early-stage MCI; its performance was maximised when feature patterns that included multimodal features (GAIT + white matter dataset) were used. The single support time was the strongest predictor. Machine learning that incorporated gait and white matter parameters achieved the highest accuracy in distinguishing between late-stage MCI and early-stage MCI.
Collapse
Affiliation(s)
- Ingyu Park
- Department of Electronic Engineering, Hallym University, Chuncheon 24252, Republic of Korea; (I.P.); (D.J.)
| | - Sang-Kyu Lee
- Department of Psychiatry, Hallym University-Chuncheon Sacred Heart Hospital, Hallym University College of Medicine, Chuncheon 24253, Republic of Korea;
| | - Hui-Chul Choi
- Department of Neurology, Hallym University-Chuncheon Sacred Heart Hospital, Hallym University College of Medicine, Chuncheon 24253, Republic of Korea;
| | - Moo-Eob Ahn
- Department of Emergency Medicine, Hallym University-Chuncheon Sacred Heart Hospital, Hallym University College of Medicine, Chuncheon 24253, Republic of Korea;
| | - Ohk-Hyun Ryu
- Division of Endocrinology and Metabolism, Department of Internal Medicine, Hallym University-Chuncheon Sacred Heart Hospital, Hallym University College of Medicine, Chuncheon 24253, Republic of Korea;
| | - Daehun Jang
- Department of Electronic Engineering, Hallym University, Chuncheon 24252, Republic of Korea; (I.P.); (D.J.)
| | - Unjoo Lee
- Division of Software, School of Information Science, Hallym University, Chuncheon 24252, Republic of Korea
| | - Yeo Jin Kim
- Department of Neurology, Kangdong Sacred Heart Hospital, Seoul 05355, Republic of Korea
| |
Collapse
|
5
|
Du X, Novoa-Laurentiev J, Plasaek JM, Chuang YW, Wang L, Marshall G, Mueller SK, Chang F, Datta S, Paek H, Lin B, Wei Q, Wang X, Wang J, Ding H, Manion FJ, Du J, Bates DW, Zhou L. Enhancing Early Detection of Cognitive Decline in the Elderly: A Comparative Study Utilizing Large Language Models in Clinical Notes. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2024:2024.04.03.24305298. [PMID: 38633810 PMCID: PMC11023645 DOI: 10.1101/2024.04.03.24305298] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/19/2024]
Abstract
Background Large language models (LLMs) have shown promising performance in various healthcare domains, but their effectiveness in identifying specific clinical conditions in real medical records is less explored. This study evaluates LLMs for detecting signs of cognitive decline in real electronic health record (EHR) clinical notes, comparing their error profiles with traditional models. The insights gained will inform strategies for performance enhancement. Methods This study, conducted at Mass General Brigham in Boston, MA, analyzed clinical notes from the four years prior to a 2019 diagnosis of mild cognitive impairment in patients aged 50 and older. We used a randomly annotated sample of 4,949 note sections, filtered with keywords related to cognitive functions, for model development. For testing, a random annotated sample of 1,996 note sections without keyword filtering was utilized. We developed prompts for two LLMs, Llama 2 and GPT-4, on HIPAA-compliant cloud-computing platforms using multiple approaches (e.g., both hard and soft prompting and error analysis-based instructions) to select the optimal LLM-based method. Baseline models included a hierarchical attention-based neural network and XGBoost. Subsequently, we constructed an ensemble of the three models using a majority vote approach. Results GPT-4 demonstrated superior accuracy and efficiency compared to Llama 2, but did not outperform traditional models. The ensemble model outperformed the individual models, achieving a precision of 90.3%, a recall of 94.2%, and an F1-score of 92.2%. Notably, the ensemble model showed a significant improvement in precision, increasing from a range of 70%-79% to above 90%, compared to the best-performing single model. Error analysis revealed that 63 samples were incorrectly predicted by at least one model; however, only 2 cases (3.2%) were mutual errors across all models, indicating diverse error profiles among them. Conclusions LLMs and traditional machine learning models trained using local EHR data exhibited diverse error profiles. The ensemble of these models was found to be complementary, enhancing diagnostic performance. Future research should investigate integrating LLMs with smaller, localized models and incorporating medical data and domain knowledge to enhance performance on specific tasks.
Collapse
Affiliation(s)
- Xinsong Du
- Division of General Internal Medicine and Primary Care, Brigham and Women’s Hospital, Boston, Massachusetts 02115
- Department of Medicine, Harvard Medical School, Boston, Massachusetts 02115
| | - John Novoa-Laurentiev
- Division of General Internal Medicine and Primary Care, Brigham and Women’s Hospital, Boston, Massachusetts 02115
| | - Joseph M. Plasaek
- Division of General Internal Medicine and Primary Care, Brigham and Women’s Hospital, Boston, Massachusetts 02115
- Department of Medicine, Harvard Medical School, Boston, Massachusetts 02115
| | - Ya-Wen Chuang
- Division of Nephrology, Taichung Veterans General Hospital, Taichung, Taiwan, 407219
| | - Liqin Wang
- Division of General Internal Medicine and Primary Care, Brigham and Women’s Hospital, Boston, Massachusetts 02115
- Department of Medicine, Harvard Medical School, Boston, Massachusetts 02115
| | - Gad Marshall
- Department of Medicine, Harvard Medical School, Boston, Massachusetts 02115
- Department of Neurology, Brigham and Women’s Hospital, Boston, Massachusetts 02115
| | - Stephanie K. Mueller
- Division of General Internal Medicine and Primary Care, Brigham and Women’s Hospital, Boston, Massachusetts 02115
- Department of Medicine, Harvard Medical School, Boston, Massachusetts 02115
| | - Frank Chang
- Division of General Internal Medicine and Primary Care, Brigham and Women’s Hospital, Boston, Massachusetts 02115
| | - Surabhi Datta
- Intelligent Medical Objects, Rosemont, Illinois, 60018
| | - Hunki Paek
- Intelligent Medical Objects, Rosemont, Illinois, 60018
| | - Bin Lin
- Intelligent Medical Objects, Rosemont, Illinois, 60018
| | - Qiang Wei
- Intelligent Medical Objects, Rosemont, Illinois, 60018
| | - Xiaoyan Wang
- Intelligent Medical Objects, Rosemont, Illinois, 60018
| | - Jingqi Wang
- Intelligent Medical Objects, Rosemont, Illinois, 60018
| | - Hao Ding
- Intelligent Medical Objects, Rosemont, Illinois, 60018
| | | | - Jingcheng Du
- Intelligent Medical Objects, Rosemont, Illinois, 60018
| | - David W. Bates
- Division of General Internal Medicine and Primary Care, Brigham and Women’s Hospital, Boston, Massachusetts 02115
- Department of Medicine, Harvard Medical School, Boston, Massachusetts 02115
| | - Li Zhou
- Division of General Internal Medicine and Primary Care, Brigham and Women’s Hospital, Boston, Massachusetts 02115
- Department of Medicine, Harvard Medical School, Boston, Massachusetts 02115
| |
Collapse
|
6
|
Khemka S, Sehar U, Manna PR, Kshirsagar S, Reddy PH. Cell-Free DNA As Peripheral Biomarker of Alzheimer's Disease. Aging Dis 2024; 16:787-803. [PMID: 38607732 PMCID: PMC11964419 DOI: 10.14336/ad.2024.0329] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2024] [Accepted: 03/29/2024] [Indexed: 04/14/2024] Open
Abstract
Alzheimer's disease (AD) and Alzheimer's disease-related disorders (ADRD) are progressive neurodegenerative diseases without cure. Alzheimer's disease occurs in 2 forms, early-onset familial AD and late-onset sporadic AD. Early-onset AD is a rare (~1%), autosomal dominant, caused by mutations in presenilin-1, presenilin-2, and amyloid precursor protein genes and the other is a late-onset, prevalent and is evolved due to age-associated complex interactions between environmental and genetic factors, in addition to apolipoprotein E4 polymorphism. Cellular senescence, promoting the impairment of physical and mental functions is constituted to be the main cause of aging, the primary risk factor for AD, which results in progressive loss of cognitive function, memory, and visual-spatial skills for an individual to live or act independently. Despite significant progress in the understanding of the biology and pathophysiology of AD, we continue to lack definitive early detectable biomarkers and/or drug targets that can be used to delay the development of AD and ADRD in elderly populations. However, recent developments in the studies of DNA double-strand breaks result in the release of fragmented DNA into the bloodstream and contribute to higher levels of cell-free DNA (cf-DNA). This fragmented cf-DNA can be released into the bloodstream from various cell types, including normal cells and cells undergoing apoptosis or necrosis and elevated levels of cf-DNA in the blood have the potential to serve as blood blood-based biomarker for early detection of AD and ADRD. The overall goal of our study is to discuss the latest developments in circulating cell-free DNA into the blood in the progression of AD and ADRD. Our article summarized the status of research on double-strand breaks and circulating cell-free DNA in both healthy and disease states and how these recent developments can be used to develop early detectable biomarkers for AD and ADRD. Our article also discussed the impact of lifestyle and epigenetic factors that are involved in DNA double-strand breaks and circulating cell-free DNA in AD and ADRD.
Collapse
Affiliation(s)
- Sachi Khemka
- Department of Internal Medicine, Texas Tech University Health Sciences Center, Lubbock, TX 79430, USA.
| | - Ujala Sehar
- Department of Internal Medicine, Texas Tech University Health Sciences Center, Lubbock, TX 79430, USA.
| | - Pulak R Manna
- Department of Internal Medicine, Texas Tech University Health Sciences Center, Lubbock, TX 79430, USA.
| | - Sudhir Kshirsagar
- Department of Internal Medicine, Texas Tech University Health Sciences Center, Lubbock, TX 79430, USA.
| | - P. Hemachandra Reddy
- Department of Internal Medicine, Texas Tech University Health Sciences Center, Lubbock, TX 79430, USA.
- Public Health Department, School of Population and Public Health, Texas Tech University Health Sciences Center, Lubbock, TX 79430, USA.
- Neurology, Departments of School of Medicine, Texas Tech University Health Sciences Center, Lubbock, TX 79430, USA.
- Department of Speech, Language and Hearing Sciences, School Health Professions, Texas Tech University Health Sciences Center, Lubbock, TX 79430, USA.
- Department of Pharmacology and Neuroscience, Texas Tech University Health Sciences Center, Lubbock, TX 79430, USA.
- Nutritional Sciences Department, College of Human Sciences, Texas Tech University, 1301 Akron Ave, Lubbock, TX 79409, USA.
| |
Collapse
|
7
|
Naderi N, Mohammadgholi A, Asghari Moghaddam N. Biosynthesis of Copper Oxide-Silver Nanoparticles from Ephedra Intermedia Extract and Study of Anticancer Effects in HepG2 Cell Line: Apoptosis-Related Genes Analysis and Nitric Oxide Level Investigations. INTERNATIONAL JOURNAL OF MOLECULAR AND CELLULAR MEDICINE 2024; 13:303-324. [PMID: 39493510 PMCID: PMC11530949 DOI: 10.22088/ijmcm.bums.13.3.303] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/29/2024] [Accepted: 08/26/2024] [Indexed: 11/05/2024]
Abstract
Liver cancer treatment faces significant obstacles such as resistance, recurrence, metastasis, and toxicity to healthy cells. Biometallic nanoparticles (NPs) have emerged as a promising approach to address these challenges. In this study, copper oxide-silver (Ag-doped CuO) NPs were prepared using a reduction method with Ephedra intermedia extract. The physicochemical properties of the NPs were evaluated using various techniques such as Field emission scanning electron microscopy (FESEM), Transmission Electron Microscope (TEM), X-ray diffraction (XRD), and Fourier-transform infrared spectroscopy (FTIR). Additionally, this study has evaluated nitric oxide levels (NO), reactive oxygen species (ROS) production, Bax, Bcl2, P53, and Caspase3 genes expression, as well as cell viability within 24 hours in liver cancer cell line HepG2. FESEM and TEM imaging confirmed the nanostructural nature of the synthesized particles with sizes ranging from 31.27 to 88.98 nanometers. XRD analysis confirmed the crystal structure of the NPs. Comparative analysis showed that the IC50 values of the Ag-doped CuO NPs were significantly lower than that of the plant extracts. Molecular studies showed significantly increased expression of Bax, Caspase3, and P53 genes, inducing apoptosis in cancer cells, and downregulation of Bcl2 as a pro-metastasis gene. Additionally, the presence of Ag-doped CuO NPs significantly increased NO activity enzyme and ROS generation compared to the plant extract. The biosynthesized Ag-doped CuO NPs demonstrated the ability to induce apoptosis, increase ROS production, and enhance NO enzyme activity in HepG2 cancer cells, suggesting their potential as a therapeutic agent for liver cancer.
Collapse
Affiliation(s)
| | - Azadeh Mohammadgholi
- Department of Biology, Central Tehran Branch, Islamic Azad University, Tehran, Iran.
| | | |
Collapse
|