Diagnostic support for selected neuromuscular diseases using answer-pattern recognition and data mining techniques: a proof of concept multicenter prospective trial

Patient-Oriented Questionnaires and Machine Learning for Rare Disease Diagnosis: A Systematic Review

Background: A major challenge faced by patients with rare diseases (RDs) often stems from delays in diagnosis, typically due to nonspecific clinical symptoms or doctors' limited experience in connecting symptoms to the underlying RD. Using patient-oriented questionnaires (POQs) as a data source for machine learning (ML) techniques can serve as a potential solution. These questionnaires enable patients to portray their day-to-day experiences living with their condition, irrespective of clinical symptoms. This systematic review-registered at PROSPERO with the Registration-ID: CRD42023490838-aims to present the current state of research in this domain by conducting a systematic literature search and identifying the potentials and limitations of this methodology. Methods: The review adheres to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines and was primarily funded by the German Federal Ministry of Education and Research under grant no. 16DHBKI056 (ki4all). The methodology involved a systematic search across the databases PubMed, Semantic Scholar and Google Scholar, covering articles published until June 2023. The inclusion criteria encompass examining the use of POQs in diagnosing rare and common diseases. Additionally, studies that focused on applying ML techniques to the resulting datasets were considered for inclusion. The primary objective was to include English as well as German research that involved the generation of predictions regarding the underlying disease based on the information gathered from POQs. Furthermore, studies exploring identifying predictive indicators associated with the underlying disease were also included in the literature review. The following data were extracted from the selected studies: year of publication, number of questions in the POQs, answer scale in the questionnaires, the ML algorithms used, the input data for the ML algorithms, the performance of these algorithms and how the performance was measured. In addition, information on the development of the questionnaires was recorded. Results: This search retrieved 421 results in total. After one superficial and two comprehensive screening runs performed by two authors independently, we ended up with 26 studies for further consideration. Sixteen of these studies deal with diseases and ML algorithms to analyse data; the other ten studies provide contributing research in this field. We discuss several potentials and limitations of the evaluated approach. Conclusions: Overall, the results show that the full potential has not yet been exploited and that further research in this direction is worthwhile, because the study results show that ML algorithms can achieve promising results on POQ data; however, their use in everyday medical practice has not yet been investigated.

The value of knowing: preferences for genetic testing to diagnose rare muscle diseases

BACKGROUND

Genetic testing can offer early diagnosis and subsequent treatment of rare neuromuscular diseases. Options for these tests could be improved by understanding the preferences of patients for the features of different genetic tests, especially features that increase information available to patients.

METHODS

We developed an online discrete-choice experiment using key attributes of currently available tests for Pompe disease with six test attributes: number of rare muscle diseases tested for with corresponding probability of diagnosis, treatment availability, time from testing to results, inclusion of secondary findings, necessity of a muscle biopsy, and average time until final diagnosis if the first test is negative. Respondents were presented a choice between two tests with different costs, with respondents randomly assigned to one of two costs. Data were analyzed using random-parameters logit.

RESULTS

A total of 600 online respondents, aged 18 to 50 years, were recruited from the U.S. general population and included in the final analysis. Tests that targeted more diseases, required less time from testing to results, included information about unrelated health risks, and were linked to shorter time to the final diagnosis were preferred and associated with diseases with available treatment. Men placed relatively more importance than women on tests for diseases with available treatments. Most of the respondents would be more willing to get a genetic test that might return unrelated health information, with women exhibiting a statistically significant preference. While respondents were sensitive to cost, 30% of the sample assigned to the highest cost was willing to pay $500 for a test that could offer a diagnosis almost 2 years earlier.

CONCLUSION

The results highlight the value people place on the information genetic tests can provide about their health, including faster diagnosis of rare, unexplained muscle weakness, but also the value of tests for multiple diseases, diseases without treatments, and incidental findings. An earlier time to diagnosis can provide faster access to treatment and an end to the diagnostic journey, which patients highly prefer.

[Short paths to diagnosis with artificial intelligence: systematic literature review on diagnostic decision support systems]

BACKGROUND

Rare diseases are often recognized late. Their diagnosis is particularly challenging due to the diversity, complexity and heterogeneity of clinical symptoms. Computer-aided diagnostic aids, often referred to as diagnostic decision support systems (DDSS), are promising tools for shortening the time to diagnosis. Despite initial positive evaluations, DDSS are not yet widely used, partly due to a lack of integration with existing clinical or practice information systems.

OBJECTIVE

This article provides an insight into currently existing diagnostic support systems that function without access to electronic patient records and only require information that is easily obtainable.

MATERIALS AND METHODS

A systematic literature search identified eight articles on DDSS that can assist in the diagnosis of rare diseases with no need for access to electronic patient records or other information systems in practices and hospitals. The main advantages and disadvantages of the identified rare disease diagnostic support systems were extracted and summarized.

RESULTS

Symptom checkers and DDSS based on portrait photos and pain drawings already exist. The degree of maturity of these applications varies.

CONCLUSION

DDSS currently still face a number of challenges, such as concerns about data protection and accuracy, and acceptance and awareness continue to be rather low. On the other hand, there is great potential for faster diagnosis, especially for rare diseases, which are easily overlooked due to their large number and the low awareness of them. The use of DDSS should therefore be carefully considered by doctors on a case-by-case basis.

[Diagnosis and Management of Late-Onset Pompe Disease]

Pompe disease is a lysosomal storage disorder, with onset between the first weeks after birth and adulthood, depending on its phenotype. It can affect multiple organ systems and presents itself with a wide variety of symptoms. Thus, recognizing Pompe disease is difficult. Especially since enzyme replacement therapy for Pompe disease was introduced (in Germany in 2006), early diagnosis by means of enzyme activity determination from dried blood spot analysis and genetic verification has become important for outcome and quality of life. When facing an obscure muscular disorder, it is crucial to consider Pompe disease. This article provides an overview about Pompe disease and focuses on the diagnosis of the late onset type. The most important aspects of interdiciplinary care for patients with Pompe disease are presented. Additionally, it contains a section focusing on psychosocial challenges for children with Pompe disease and their families, which may include mental disorders and social retreat, and gives advice on how to support parents of affected children.

Effects of machine learning-based clinical decision support systems on decision-making, care delivery, and patient outcomes: a scoping review

OBJECTIVE

This study aims to summarize the research literature evaluating machine learning (ML)-based clinical decision support (CDS) systems in healthcare settings.

MATERIALS AND METHODS

We conducted a review in accordance with the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta Analyses extension for Scoping Review). Four databases, including PubMed, Medline, Embase, and Scopus were searched for studies published from January 2016 to April 2021 evaluating the use of ML-based CDS in clinical settings. We extracted the study design, care setting, clinical task, CDS task, and ML method. The level of CDS autonomy was examined using a previously published 3-level classification based on the division of clinical tasks between the clinician and CDS; effects on decision-making, care delivery, and patient outcomes were summarized.

RESULTS

Thirty-two studies evaluating the use of ML-based CDS in clinical settings were identified. All were undertaken in developed countries and largely in secondary and tertiary care settings. The most common clinical tasks supported by ML-based CDS were image recognition and interpretation (n = 12) and risk assessment (n = 9). The majority of studies examined assistive CDS (n = 23) which required clinicians to confirm or approve CDS recommendations for risk assessment in sepsis and for interpreting cancerous lesions in colonoscopy. Effects on decision-making, care delivery, and patient outcomes were mixed.

CONCLUSION

ML-based CDS are being evaluated in many clinical areas. There remain many opportunities to apply and evaluate effects of ML-based CDS on decision-making, care delivery, and patient outcomes, particularly in resource-constrained settings.

Muscle MRI patterns for limb girdle muscle dystrophies: systematic review

Limb girdle muscle dystrophies (LGMDs) are a group of inherited neuromuscular disorders comprising more than 20 genes. There have been increasing efforts to characterize this group with Muscle MRI. However, due to the complexity and similarities, the interpretation of the MRI patterns is usually done by experts in the field. Here, we proposed a step-by-step image interpretation of Muscle MRI in LGDM by evaluating the variability of muscle pattern involvement reported in the literature. A systematic review with an open start date to November 2022 was conducted to describe all LGMDs' muscle MRI patterns. Eighty-eight studies were included in the final review. Data were found to describe muscle MRI patterns for 15 out of 17 LGMDs types. Although the diagnosis of LGMDs is challenging despite the advanced genetic testing and other diagnostic modalities, muscle MRI is shown to help in the diagnosis of LGMDs. To further increase the yield for muscle MRI in the neuromuscular field, larger cohorts of patients need to be conducted.

A diagnostic support system based on pain drawings: binary and k-disease classification of EDS, GBS, FSHD, PROMM, and a control group with Pain2D

BACKGROUND AND OBJECTIVE

The diagnosis of rare diseases (RDs) is often challenging due to their rarity, variability and the high number of individual RDs, resulting in a delay in diagnosis with adverse effects for patients and healthcare systems. The development of computer assisted diagnostic decision support systems could help to improve these problems by supporting differential diagnosis and by prompting physicians to initiate the right diagnostic tests. Towards this end, we developed, trained and tested a machine learning model implemented as part of the software called Pain2D to classify four rare diseases (EDS, GBS, FSHD and PROMM), as well as a control group of unspecific chronic pain, from pen-and-paper pain drawings filled in by patients.

METHODS

Pain drawings (PDs) were collected from patients suffering from one of the four RDs, or from unspecific chronic pain. The latter PDs were used as an outgroup in order to test how Pain2D handles more common pain causes. A total of 262 (59 EDS, 29 GBS, 35 FSHD, 89 PROMM, 50 unspecific chronic pain) PDs were collected and used to generate disease specific pain profiles. PDs were then classified by Pain2D in a leave-one-out-cross-validation approach.

RESULTS

Pain2D was able to classify the four rare diseases with an accuracy of 61-77% with its binary classifier. EDS, GBS and FSHD were classified correctly by the Pain2D k-disease classifier with sensitivities between 63 and 86% and specificities between 81 and 89%. For PROMM, the k-disease classifier achieved a sensitivity of 51% and specificity of 90%.

CONCLUSIONS

Pain2D is a scalable, open-source tool that could potentially be trained for all diseases presenting with pain.

Improving Recognition of Treatable Rare Neuromuscular Disorders in Primary Care: A Pilot Feasibility Study

Innovative targeted treatments for neuromuscular disorders (NMDs) can dramatically improve the course of illness. Diagnostic delay, however, is a major impediment. Here, we present a pilot project aimed at assessing the feasibility of a screening program to identify children at high risk for NMDs within the first 30 months of life. The Promoting Early Diagnosis for Neuromuscular Disorders (PEDINE) project implemented a three-step sequential screening in an area of about 300,000 people with (1) an assessment of the motor development milestones to identify “red flags” for NMDs by primary care pediatricians (PCPs) as part of the routine Health Status Check visits; (2) for the children who screened positive, a community neuropsychiatric assessment, with further referral of suspected NMD cases to (3) a hospital-based specialized tertiary care center. In the first-year feasibility study, a total of 10,032 PCP visits were conducted, and twenty children (0.2% of the total Health Status Check visits) screened positive and were referred to the community neuropsychiatrist. Of these, four had elevated creatine kinase (CK) serum levels. This pilot study shows that screening for NMDs in primary care settings is feasible and allows children at high risk for muscular disorder to be promptly identified.

Pain drawings as a diagnostic tool for the differentiation between two pain-associated rare diseases (Ehlers-Danlos-Syndrome, Guillain-Barré-Syndrome)

Background

The diagnosis of rare diseases poses a particular challenge to clinicians. This study analyzes whether patients’ pain drawings (PDs) help in the differentiation of two pain-associated rare diseases, Ehlers-Danlos Syndrome (EDS) and Guillain-Barré Syndrome (GBS).

Method

The study was designed as a prospective, observational, single-center study. The sample comprised 60 patients with EDS (3 male, 52 female, 5 without gender information; 39.2 ± 11.4 years) and 32 patients with GBS (10 male, 20 female, 2 without gender information; 50.5 ± 13.7 years). Patients marked areas afflicted by pain on a sketch of a human body with anterior, posterior, and lateral views. PDs were electronically scanned and processed. Each PD was classified based on the Ružička similarity to the EDS and the GBS averaged image (pain profile) in a leave-one-out cross validation approach. A receiver operating characteristic (ROC) curve was plotted.

Results

60–80% of EDS patients marked the vertebral column with the neck and the tailbone and the knee joints as pain areas, 40–50% the shoulder-region, the elbows and the thumb saddle joint. 60–70% of GBS patients marked the dorsal and plantar side of the feet as pain areas, 40–50% the palmar side of the fingertips, the dorsal side of the left palm and the tailbone. 86% of the EDS patients and 96% of the GBS patients were correctly identified by computing the Ružička similarity. The ROC curve yielded an excellent area under the curve value of 0.95.

Conclusion

PDs are a useful and economic tool to differentiate between GBS and EDS. Further studies should investigate its usefulness in the diagnosis of other pain-associated rare diseases. This study was registered in the German Clinical Trials Register, No. DRKS00014777 (Deutsches Register klinischer Studien, DRKS), on 01.06.2018.

Diagnosis of Rare Diseases: a scoping review of clinical decision support systems

BACKGROUND

Rare Diseases (RDs), which are defined as diseases affecting no more than 5 out of 10,000 people, are often severe, chronic and life-threatening. A main problem is the delay in diagnosing RDs. Clinical decision support systems (CDSSs) for RDs are software systems to support clinicians in the diagnosis of patients with RDs. Due to their clinical importance, we conducted a scoping review to determine which CDSSs are available to support the diagnosis of RDs patients, whether the CDSSs are available to be used by clinicians and which functionalities and data are used to provide decision support.

METHODS

We searched PubMed for CDSSs in RDs published between December 16, 2008 and December 16, 2018. Only English articles, original peer reviewed journals and conference papers describing a clinical prototype or a routine use of CDSSs were included. For data charting, we used the data items "Objective and background of the publication/project", "System or project name", "Functionality", "Type of clinical data", "Rare Diseases covered", "Development status", "System availability", "Data entry and integration", "Last software update" and "Clinical usage".

RESULTS

The search identified 636 articles. After title and abstracting screening, as well as assessing the eligibility criteria for full-text screening, 22 articles describing 19 different CDSSs were identified. Three types of CDSSs were classified: "Analysis or comparison of genetic and phenotypic data," "machine learning" and "information retrieval". Twelve of nineteen CDSSs use phenotypic and genetic data, followed by clinical data, literature databases and patient questionnaires. Fourteen of nineteen CDSSs are fully developed systems and therefore publicly available. Data can be entered or uploaded manually in six CDSSs, whereas for four CDSSs no information for data integration was available. Only seven CDSSs allow further ways of data integration. thirteen CDSS do not provide information about clinical usage.

CONCLUSIONS

Different CDSS for various purposes are available, yet clinicians have to determine which is best for their patient. To allow a more precise usage, future research has to focus on CDSSs RDs data integration, clinical usage and updating clinical knowledge. It remains interesting which of the CDSSs will be used and maintained in the future.

Diagnosis support systems for rare diseases: a scoping review

INTRODUCTION

Rare diseases affect approximately 350 million people worldwide. Delayed diagnosis is frequent due to lack of knowledge of most clinicians and a small number of expert centers. Consequently, computerized diagnosis support systems have been developed to address these issues, with many relying on rare disease expertise and taking advantage of the increasing volume of generated and accessible health-related data. Our objective is to perform a review of all initiatives aiming to support the diagnosis of rare diseases.

METHODS

A scoping review was conducted based on methods proposed by Arksey and O'Malley. A charting form for relevant study analysis was developed and used to categorize data.

RESULTS

Sixty-eight studies were retained at the end of the charting process. Diagnosis targets varied from 1 rare disease to all rare diseases. Material used for diagnosis support consisted mostly of phenotype concepts, images or fluids. Fifty-seven percent of the studies used expert knowledge. Two-thirds of the studies relied on machine learning algorithms, and one-third used simple similarities. Manual algorithms were encountered as well. Most of the studies presented satisfying performance of evaluation by comparison with references or with external validation. Fourteen studies provided online tools, most of which aimed to support the diagnosis of all rare diseases by considering queries based on phenotype concepts.

CONCLUSION

Numerous solutions relying on different materials and use of various methodologies are emerging with satisfying preliminary results. However, the variability of approaches and evaluation processes complicates the comparison of results. Efforts should be made to adequately validate these tools and guarantee reproducibility and explicability.

Common pre-diagnostic features in individuals with different rare diseases represent a key for diagnostic support with computerized pattern recognition?

BACKGROUND

Rare diseases (RD) result in a wide variety of clinical presentations, and this creates a significant diagnostic challenge for health care professionals. We hypothesized that there exist a set of consistent and shared phenomena among all individuals affected by (different) RD during the time before diagnosis is established.

OBJECTIVE

We aimed to identify commonalities between different RD and developed a machine learning diagnostic support tool for RD.

METHODS

20 interviews with affected individuals with different RD, focusing on the time period before their diagnosis, were performed and qualitatively analyzed. Out of these pre-diagnostic experiences, we distilled key phenomena and created a questionnaire which was then distributed among individuals with the established diagnosis of i.) RD, ii.) other common non-rare diseases (NRO) iii.) common chronic diseases (CD), iv.), or psychosomatic/somatoform disorders (PSY). Finally, four combined single machine learning methods and a fusion algorithm were used to distinguish the different answer patterns of the questionnaires.

RESULTS

The questionnaire contained 53 questions. A total sum of 1763 questionnaires (758 RD, 149 CD, 48 PSY, 200 NRO, 34 healthy individuals and 574 not evaluable questionnaires) were collected. Based on 3 independent data sets the 10-fold stratified cross-validation method for the answer-pattern recognition resulted in sensitivity values of 88.9% to detect the answer pattern of a RD, 86.6% for NRO, 87.7% for CD and 84.2% for PSY.

CONCLUSION

Despite the great diversity in presentation and pathogenesis of each RD, patients with RD share surprisingly similar pre-diagnosis experiences. Our questionnaire and data-mining based approach successfully detected unique patterns in groups of individuals affected by a broad range of different rare diseases. Therefore, these results indicate distinct patterns that may be used for diagnostic support in RD.

Model-Based and Model-Free Techniques for Amyotrophic Lateral Sclerosis Diagnostic Prediction and Patient Clustering

Amyotrophic lateral sclerosis (ALS) is a complex progressive neurodegenerative disorder with an estimated prevalence of about 5 per 100,000 people in the United States. In this study, the ALS disease progression is measured by the change of Amyotrophic Lateral Sclerosis Functional Rating Scale (ALSFRS) score over time. The study aims to provide clinical decision support for timely forecasting of the ALS trajectory as well as accurate and reproducible computable phenotypic clustering of participants. Patient data are extracted from DREAM-Phil Bowen ALS Prediction Prize4Life Challenge data, most of which are from the Pooled Resource Open-Access ALS Clinical Trials Database (PRO-ACT) archive. We employed model-based and model-free machine-learning methods to predict the change of the ALSFRS score over time. Using training and testing data we quantified and compared the performance of different techniques. We also used unsupervised machine learning methods to cluster the patients into separate computable phenotypes and interpret the derived subcohorts. Direct prediction of univariate clinical outcomes based on model-based (linear models) or model-free (machine learning based techniques - random forest and Bayesian adaptive regression trees) was only moderately successful. The correlation coefficients between clinically observed changes in ALSFRS scores relative to the model-based/model-free predicted counterparts were 0.427 (random forest) and 0.545(BART). The reliability of these results were assessed using internal statistical cross validation and well as external data validation. Unsupervised clustering generated very reliable and consistent partitions of the patient cohort into four computable phenotypic subgroups. These clusters were explicated by identifying specific salient clinical features included in the PRO-ACT archive that discriminate between the derived subcohorts. There are differences between alternative analytical methods in forecasting specific clinical phenotypes. Although predicting univariate clinical outcomes may be challenging, our results suggest that modern data science strategies are useful in clustering patients and generating evidence-based ALS hypotheses about complex interactions of multivariate factors. Predicting univariate clinical outcomes using the PRO-ACT data yields only marginal accuracy (about 70%). However, unsupervised clustering of participants into sub-groups generates stable, reliable and consistent (exceeding 95%) computable phenotypes whose explication requires interpretation of multivariate sets of features. HIGHLIGHTS: • Used a large ALS data archive of 8,000 patients consisting of 3 million records, including 200 clinical features tracked over 12 months. • Employed model-based and model-free methods to predict ALSFRS changes over time, cluster patients into cohorts, and derive computable phenotypes. • Research findings include stable, reliable, and consistent (95%) patient stratification into computable phenotypes. However, clinical explication of the results requires interpretation of multivariate information. Graphical Abstract ᅟ.

[Computer-assisted diagnosis of rare diseases]

To establish a comprehensive diagnosis is by far the most challenging task in a physician's daily routine. Especially rare diseases place high demands on differential diagnosis, caused by the high number of around 8000 diseases and their clinical variability. No clinician can be aware of all the different entities and memorizing them all is impossible and inefficient. Specific diagnostic decision-supported systems provide better results than standard search engines in this context. The systems FindZebra, Phenomizer, Orphanet, and Isabel are presented here concisely with their advantages and limitations. An outlook is given to social media usage and big data technologies. Due to the high number of initial misdiagnoses and long periods of time until a confirmatory diagnosis is reached, these tools might be promising in practice to improve the diagnosis of rare diseases.

Patient's Experience in Pediatric Primary Immunodeficiency Disorders: Computerized Classification of Questionnaires

Introduction

Primary immunodeficiency disorders (PIDs) are a heterogeneous group of more than 200 rare diseases. Timely diagnosis is of uttermost importance. Therefore, we aimed to develop a diagnostic questionnaire with computerized pattern-recognition in order to support physicians to identify suspicious patient histories.

Materials and methods

Standardized interviews were conducted with guardians of children with PID. The questionnaire based on parental observations was developed using Colaizzis’ framework for content analysis. Answers from 64 PID patients and 62 controls were analyzed by data mining methods in order to make a diagnostic prediction. Performance was evaluated by k-fold stratified cross-validation.

Results

The diagnostic support tool achieved a diagnostic sensitivity of up to 98%. The analysis of 12 interviews revealed 26 main phenomena observed by parents in the pre-diagnostic period. The questions were systematically phrased and selected resulting in a 36-item questionnaire. This was answered by 126 patients with or without PID to evaluate prediction. Item analysis revealed significant questions.

Discussion

Our approach proved suitable for recognizing patterns and thus differentiates between observations of PID patients and control groups. These findings provide the basis for developing a tool supporting physicians to consider a PID with a questionnaire. These data support the notion that patient’s experience is a cornerstone in the diagnostic process.