Validation of a machine learning approach to estimate expanded disability status scale scores for multiple sclerosis

Integrating large language models in care, research, and education in multiple sclerosis management

Use of techniques derived from generative artificial intelligence (AI), specifically large language models (LLMs), offer a transformative potential on the management of multiple sclerosis (MS). Recent LLMs have exhibited remarkable skills in producing and understanding human-like texts. The integration of AI in imaging applications and the deployment of foundation models for the classification and prognosis of disease course, including disability progression and even therapy response, have received considerable attention. However, the use of LLMs within the context of MS remains relatively underexplored. LLMs have the potential to support several activities related to MS management. Clinical decision support systems could help selecting proper disease-modifying therapies; AI-based tools could leverage unstructured real-world data for research or virtual tutors may provide adaptive education materials for neurologists and people with MS in the foreseeable future. In this focused review, we explore practical applications of LLMs across the continuum of MS management as an initial scope for future analyses, reflecting on regulatory hurdles and the indispensable role of human supervision.

AI and disability: A systematic scoping review

Background: Artificial intelligence (AI) can enhance life experiences and present challenges for people with disabilities. Objectives: This study aims to investigate the relationship between AI and disability, exploring the potential benefits and challenges of using AI for people with disabilities. Methods: A systematic scoping review was conducted using eight online databases; 45 scholarly articles from the last 5 years were identified and selected for thematic analysis. Results: The review's findings revealed AI's potential to enhance healthcare; however, it showed a high prevalence of a narrow medical model of disability and an ableist perspective in AI research. This raises concerns about the perpetuation of biases and discrimination against individuals with disabilities in the development and deployment of AI technologies. Conclusion: We recommend shifting towards a social model of disability, promoting interdisciplinary collaboration, addressing AI bias and discrimination, prioritizing privacy and security in AI development, focusing on accessibility and usability, investing in education and training, and advocating for robust policy and regulatory frameworks. The review emphasizes the urgent need for further research to ensure that AI benefits all members of society equitably and that future AI systems are designed with inclusivity and accessibility as core principles.

Natural language processing systems for extracting information from electronic health records about activities of daily living. A systematic review

Objective

Natural language processing (NLP) can enhance research on activities of daily living (ADL) by extracting structured information from unstructured electronic health records (EHRs) notes. This review aims to give insight into the state-of-the-art, usability, and performance of NLP systems to extract information on ADL from EHRs.

Materials and Methods

A systematic review was conducted based on searches in Pubmed, Embase, Cinahl, Web of Science, and Scopus. Studies published between 2017 and 2022 were selected based on predefined eligibility criteria.

Results

The review identified 22 studies. Most studies (65%) used NLP for classifying unstructured EHR data on 1 or 2 ADL. Deep learning, combined with a ruled-based method or machine learning, was the approach most commonly used. NLP systems varied widely in terms of the pre-processing and algorithms. Common performance evaluation methods were cross-validation and train/test datasets, with F1, precision, and sensitivity as the most frequently reported evaluation metrics. Most studies reported relativity high overall scores on the evaluation metrics.

Discussion

NLP systems are valuable for the extraction of unstructured EHR data on ADL. However, comparing the performance of NLP systems is difficult due to the diversity of the studies and challenges related to the dataset, including restricted access to EHR data, inadequate documentation, lack of granularity, and small datasets.

Conclusion

This systematic review indicates that NLP is promising for deriving information on ADL from unstructured EHR notes. However, what the best-performing NLP system is, depends on characteristics of the dataset, research question, and type of ADL.

Claims-based algorithm to estimate the Expanded Disability Status Scale for multiple sclerosis in a German health insurance fund: a validation study using patient medical records

Background

The Expanded Disability Status Scale (EDSS) quantifies disability and measures disease progression in multiple sclerosis (MS), however is not available in administrative claims databases.

Objectives

To develop a claims-based algorithm for deriving EDSS and validate it against a clinical dataset capturing true EDSS values from medical records.

Methods

We built a unique linked dataset combining claims data from the German AOK PLUS sickness fund and medical records from the Multiple Sclerosis Management System 3D (MSDS3D). Data were deterministically linked based on insurance numbers. We used 69 MS-related diagnostic indicators recorded with ICD-10-GM codes within 3 months before and after recorded true EDSS measures to estimate a claims-based EDSS proxy (pEDSS). Predictive performance of the pEDSS was assessed as an eight-fold (EDSS 1.0-7.0, ≥8.0), three-fold (EDSS 1.0-3.0, 4.0-5.0, ≥6.0), and binary classifier (EDSS <6.0, ≥6.0). For each classifier, predictive performance measures were determined, and overall performance was summarized using a macro F1-score. Finally, we implemented the algorithm to determine pEDSS among an overall cohort of patients with MS in AOK PLUS, who were alive and insured 12 months prior to and after index diagnosis.

Results

We recruited 100 people with MS insured by AOK PLUS who had ≥1 EDSS measure in MSDS3D between 01/10/2015 and 30/06/2019 (620 measurements overall). Patients had a mean rescaled EDSS of 3.2 and pEDSS of 3.0. The pEDSS deviated from the true EDSS by 1.2 points, resulting in a mean squared error of prediction of 2.6. For the eight-fold classifier, the macro F1-score of 0.25 indicated low overall predictive performance. Broader severity groupings were better performing, with the three-fold and binary classifiers for severe disability achieving a F1-score of 0.68 and 0.84, respectively. In the overall AOK PLUS cohort (3,756 patients, 71.9% female, mean 51.9 years), older patients, patients with progressive forms of MS and those with higher comorbidity burden showed higher pEDSS.

Conclusion

Generally, EDSS was underestimated by the algorithm as mild-to-moderate symptoms were poorly captured in claims across all functional systems. While the proxy-based approach using claims data may not allow for granular description of MS disability, broader severity groupings show good predictive performance.