Validation of an automatic tool for the rapid measurement of brain atrophy and white matter hyperintensity: QyScore®

Artificial Intelligence and Multiple Sclerosis

In this paper, we analyse the different advances in artificial intelligence (AI) approaches in multiple sclerosis (MS). AI applications in MS range across investigation of disease pathogenesis, diagnosis, treatment, and prognosis. A subset of AI, Machine learning (ML) models analyse various data sources, including magnetic resonance imaging (MRI), genetic, and clinical data, to distinguish MS from other conditions, predict disease progression, and personalize treatment strategies. Additionally, AI models have been extensively applied to lesion segmentation, identification of biomarkers, and prediction of outcomes, disease monitoring, and management. Despite the big promises of AI solutions, model interpretability and transparency remain critical for gaining clinician and patient trust in these methods. The future of AI in MS holds potential for open data initiatives that could feed ML models and increasing generalizability, the implementation of federated learning solutions for training the models addressing data sharing issues, and generative AI approaches to address challenges in model interpretability, and transparency. In conclusion, AI presents an opportunity to advance our understanding and management of MS. AI promises to aid clinicians in MS diagnosis and prognosis improving patient outcomes and quality of life, however ensuring the interpretability and transparency of AI-generated results is going to be key for facilitating the integration of AI into clinical practice.

Data-driven normative values based on generative manifold learning for quantitative MRI

In medicine, abnormalities in quantitative metrics such as the volume reduction of one brain region of an individual versus a control group are often provided as deviations from so-called normal values. These normative reference values are traditionally calculated based on the quantitative values from a control group, which can be adjusted for relevant clinical co-variables, such as age or sex. However, these average normative values do not take into account the globality of the available quantitative information. For example, quantitative analysis of T1-weighted magnetic resonance images based on anatomical structure segmentation frequently includes over 100 cerebral structures in the quantitative reports, and these tend to be analyzed separately. In this study, we propose a global approach to personalized normative values for each brain structure using an unsupervised Artificial Intelligence technique known as generative manifold learning. We test the potential benefit of these personalized normative values in comparison with the more traditional average normative values on a population of patients with drug-resistant epilepsy operated for focal cortical dysplasia, as well as on a supplementary healthy group and on patients with Alzheimer's disease.

FDA-approved machine learning algorithms in neuroradiology: A systematic review of the current evidence for approval

Over the past decade, machine learning (ML) and artificial intelligence (AI) have become increasingly prevalent in the medical field. In the United States, the Food and Drug Administration (FDA) is responsible for regulating AI algorithms as "medical devices" to ensure patient safety. However, recent work has shown that the FDA approval process may be deficient. In this study, we evaluate the evidence supporting FDA-approved neuroalgorithms, the subset of machine learning algorithms with applications in the central nervous system (CNS), through a systematic review of the primary literature. Articles covering the 53 FDA-approved algorithms with applications in the CNS published in PubMed, EMBASE, Google Scholar and Scopus between database inception and January 25, 2022 were queried. Initial searches identified 1505 studies, of which 92 articles met the criteria for extraction and inclusion. Studies were identified for 26 of the 53 neuroalgorithms, of which 10 algorithms had only a single peer-reviewed publication. Performance metrics were available for 15 algorithms, external validation studies were available for 24 algorithms, and studies exploring the use of algorithms in clinical practice were available for 7 algorithms. Papers studying the clinical utility of these algorithms focused on three domains: workflow efficiency, cost savings, and clinical outcomes. Our analysis suggests that there is a meaningful gap between the FDA approval of machine learning algorithms and their clinical utilization. There appears to be room for process improvement by implementation of the following recommendations: the provision of compelling evidence that algorithms perform as intended, mandating minimum sample sizes, reporting of a predefined set of performance metrics for all algorithms and clinical application of algorithms prior to widespread use. This work will serve as a baseline for future research into the ideal regulatory framework for AI applications worldwide.

How far MS lesion detection and segmentation are integrated into the clinical workflow? A systematic review

INTRODUCTION

Over the past few years, the deep learning community has developed and validated a plethora of tools for lesion detection and segmentation in Multiple Sclerosis (MS). However, there is an important gap between validating models technically and clinically. To this end, a six-step framework necessary for the development, validation, and integration of quantitative tools in the clinic was recently proposed under the name of the Quantitative Neuroradiology Initiative (QNI).

AIMS

Investigate to what extent automatic tools in MS fulfill the QNI framework necessary to integrate automated detection and segmentation into the clinical neuroradiology workflow.

METHODS

Adopting the systematic Cochrane literature review methodology, we screened and summarised published scientific articles that perform automatic MS lesions detection and segmentation. We categorised the retrieved studies based on their degree of fulfillment of QNI's six-steps, which include a tool's technical assessment, clinical validation, and integration.

RESULTS

We found 156 studies; 146/156 (94%) fullfilled the first QNI step, 155/156 (99%) the second, 8/156 (5%) the third, 3/156 (2%) the fourth, 5/156 (3%) the fifth and only one the sixth.

CONCLUSIONS

To date, little has been done to evaluate the clinical performance and the integration in the clinical workflow of available methods for MS lesion detection/segmentation. In addition, the socio-economic effects and the impact on patients' management of such tools remain almost unexplored.

Preliminary validation of a structural magnetic resonance imaging metric for tracking dementia-related neurodegeneration and future decline

Alzheimer's disease (AD) is a progressive neurodegenerative disease characterized by cognitive decline and atrophy in the medial temporal lobe (MTL) and subsequent brain regions. Structural magnetic resonance imaging (sMRI) has been widely used in research and clinical care for diagnosis and monitoring AD progression. However, atrophy patterns are complex and vary by patient. To address this issue, researchers have made efforts to develop more concise metrics that can summarize AD-specific atrophy. Many of these methods can be difficult to interpret clinically, hampering adoption. In this study, we introduce a novel index which we call an "AD-NeuroScore," that uses a modified Euclidean-inspired distance function to calculate differences between regional brain volumes associated with cognitive decline. The index is adjusted for intracranial volume (ICV), age, sex, and scanner model. We validated AD-NeuroScore using 929 older adults from the Alzheimer's Disease Neuroimaging Initiative (ADNI) study, with a mean age of 72.7 years (SD = 6.3; 55.1-91.5) and cognitively normal (CN), mild cognitive impairment (MCI), or AD diagnoses. Our validation results showed that AD-NeuroScore was significantly associated with diagnosis and disease severity scores (measured by MMSE, CDR-SB, and ADAS-11) at baseline. Furthermore, baseline AD-NeuroScore was associated with both changes in diagnosis and disease severity scores at all time points with available data. The performance of AD-NeuroScore was equivalent or superior to adjusted hippocampal volume (AHV), a widely used metric in AD research. Further, AD-NeuroScore typically performed as well as or sometimes better when compared to other existing sMRI-based metrics. In conclusion, we have introduced a new metric, AD-NeuroScore, which shows promising results in detecting AD, benchmarking disease severity, and predicting disease progression. AD-NeuroScore differentiates itself from other metrics by being clinically practical and interpretable.

Commercial volumetric MRI reporting tools in multiple sclerosis: a systematic review of the evidence

PURPOSE

MRI is integral to the diagnosis of multiple sclerosis (MS) and is important for clinical prognostication. Quantitative volumetric reporting tools (QReports) can improve the accuracy and objectivity of MRI-based assessments. Several QReports are commercially available; however, validation can be difficult to establish and does not currently follow a common pathway. To aid evidence-based clinical decision-making, we performed a systematic review of commercial QReports for use in MS including technical details and published reports of validation and in-use evaluation.

METHODS

We categorized studies into three types of testing: technical validation, for example, comparison to manual segmentation, clinical validation by clinicians or interpretation of results alongside clinician-rated variables, and in-use evaluation, such as health economic assessment.

RESULTS

We identified 10 companies, which provide MS lesion and brain segmentation and volume quantification, and 38 relevant publications. Tools received regulatory approval between 2006 and 2020, contextualize results to normative reference populations, ranging from 620 to 8000 subjects, and require T1- and T2-FLAIR-weighted input sequences for longitudinal assessment of whole-brain volume and lesions. In MS, six QReports provided evidence of technical validation, four companies have conducted clinical validation by correlating results with clinical variables, only one has tested their QReport by clinician end-users, and one has performed a simulated in-use socioeconomic evaluation.

CONCLUSION

We conclude that there is limited evidence in the literature regarding clinical validation and in-use evaluation of commercial MS QReports with a particular lack of clinician end-user testing. Our systematic review provides clinicians and institutions with the available evidence when considering adopting a quantitative reporting tool for MS.

Making Visible the Invisible: Automatically Measured Global and Regional Brain Volume Is Associated with Cognitive Impairment and Fatigue in Multiple Sclerosis

In multiple sclerosis (MS), the transition from relapsing-remitting to the secondary-progressive phase is characterized by a progression independent of relapse activity (PIRA), resulting in physical disability accumulation and invisible symptoms, i.e., fatigue and cognitive impairment (CI). These symptoms are related to neurodegenerative processes and have been correlated with MRI measures of brain atrophy only at a group level; however, the application in clinical practice of atrophy-based measurements for single-patient evaluation is yet to be fully investigated. In the present study, we aimed to evaluate the association between brain atrophy, measured with easy-to-use automatic software, and the "invisible" MS symptoms of cognition and fatigue. A total of 69 MS patients were included in the study; cognitive impairment and fatigue (FSS) (in addition to neurological disability, EDSS) were assessed and correlated with brain volumes calculated using the automated software QyScore^® which is validated for single-patient use in the clinical setting. Results showed that the cognitive status was accurately reflected by measures of atrophy, with a sensitivity of up to 90%. CI patients showed a lower volume compared to cognitively normal patients in the whole brain (p = 0.017), gray matter (p = 0.042), insula (p = 0.035), cerebellum (p = 0.008), and limbic lobe (p = 0.049). FSS was associated with temporal lobe (r = -0.37, p = 0.013) and insular (r = -0.36, p = 0.019) volumes. The volumes of the same regions were also associated with EDSS. The global/regional atrophy results, assessed with automatic and easy-to-use software, correlated with cognitive and fatigue symptoms, thus supporting the clinical application in routine patient management.