Automatic diagnosis of primary headaches by machine learning methods

Clinical decision support system using hierarchical fuzzy diagnosis model for migraine and tension-type headache based on International Classification of Headache Disorders, 3rd edition

Objective

To determine whether the diagnostic ability of the newly designed hierarchical fuzzy diagnosis method is consistent with that of headache experts for probable migraine (PM) and probable tension-type headache (PTTH).

Background

Clinical decision support systems (CDSS) are computer systems designed to help doctors to make clinician decisions by information technology, and have proven to be effective in improving headache diagnosis by making medical knowledge readily available to users in some studies. However, one serious drawback is that the CDSS lacks the ability to deal with some fuzzy boundaries of the headache features utilized in diagnostic criteria, which might be caused by patients' recall bias and subjective bias.

Methods

A hybrid mechanism of rule-based reasoning and hierarchical fuzzy diagnosis method based on International Classification of Headache Disorders, 3rd edition (ICHD-3) was designed and then validated by a retrospective study with 325 consecutive patients and a prospective study with 380 patients who were clinically diagnosed with migraine and TTH at the headache clinic of Chinese PLA General Hospital.

Results

The results of the diagnostic test in the retrospective study indicated that the fuzzy-based CDSS can be used in the diagnosis of migraine without aura (MO) (sensitivity 97.71%, specificity 100%), TTH (sensitivity 98.57%, specificity 100%), PM (sensitivity 91.25%, specificity 98.75%) and PTTH (sensitivity 90.91%, specificity 99.63%). While in the prospective study, the diagnostic performances were MO (sensitivity 91.62%, specificity 96.52%), TTH (sensitivity 92.17%, specificity 95.47%), PM (sensitivity 85.48%, specificity 98.11%) and PTTH (sensitivity 87.50%, specificity 98.60%). Cohen's kappa values for the consistency test were 0.984 ± 0.018 (MO), 0.991 ± 0.018 (TTH), 0.916 ± 0.051 (PM), 0.932 ± 0.059 (PTTH) in the retrospective study and 0.884 ± 0.047 (MO), 0.870 ± 0.055 (TTH), 0.853 ± 0.073 (PM), 0.827 ± 0.118 (PTTH) in the prospective study, which indicated good consistency with the fuzzy-based CDSS and the gold standard (p < 0.001).

Conclusion

We developed a fuzzy-based CDSS performs much more similarly to expert diagnosis and performs better than the routine CDSS method in the diagnosis of migraine and TTH, and it could promote the application of artificial intelligence in the area of headache diagnosis.

Migraine headache (MH) classification using machine learning methods with data augmentation

Migraine headache, a prevalent and intricate neurovascular disease, presents significant challenges in its clinical identification. Existing techniques that use subjective pain intensity measures are insufficiently accurate to make a reliable diagnosis. Even though headaches are a common condition with poor diagnostic specificity, they have a significant negative influence on the brain, body, and general human function. In this era of deeply intertwined health and technology, machine learning (ML) has emerged as a crucial force in transforming every aspect of healthcare, utilizing advanced facilities ML has shown groundbreaking achievements related to developing classification and automatic predictors. With this, deep learning models, in particular, have proven effective in solving complex problems spanning computer vision and data analytics. Consequently, the integration of ML in healthcare has become vital, especially in developing countries where limited medical resources and lack of awareness prevail, the urgent need to forecast and categorize migraines using artificial intelligence (AI) becomes even more crucial. By training these models on a publicly available dataset, with and without data augmentation. This study focuses on leveraging state-of-the-art ML algorithms, including support vector machine (SVM), K-nearest neighbors (KNN), random forest (RF), decision tree (DST), and deep neural networks (DNN), to predict and classify various types of migraines. The proposed models with data augmentations were trained to classify seven various types of migraine. The proposed models with data augmentations were trained to classify seven various types of migraine. The revealed results show that DNN, SVM, KNN, DST, and RF achieved an accuracy of 99.66%, 94.60%, 97.10%, 88.20%, and 98.50% respectively with data augmentation highlighting the transformative potential of AI in enhancing migraine diagnosis.

Using natural language processing to automatically classify written self-reported narratives by patients with migraine or cluster headache

Background

Headache medicine is largely based on detailed history taking by physicians analysing patients’ descriptions of headache. Natural language processing (NLP) structures and processes linguistic data into quantifiable units. In this study, we apply these digital techniques on self-reported narratives by patients with headache disorders to research the potential of analysing and automatically classifying human-generated text and information extraction in clinical contexts.

Methods

A prospective cross-sectional clinical trial collected self-reported narratives on headache disorders from participants with either migraine or cluster headache. NLP was applied for the analysis of lexical, semantic and thematic properties of the texts. Machine learning (ML) algorithms were applied to classify the descriptions of headache attacks from individual participants into their correct group (migraine versus cluster headache).

Results

One-hundred and twenty-one patients (81 participants with migraine and 40 participants with cluster headache) provided a self-reported narrative on their headache disorder. Lexical analysis of this text corpus resulted in several specific key words per diagnostic group (cluster headache: Dutch (nl): “oog” | English (en): “eye”, nl: “pijn” | en: “pain” and nl: “terug” | en: “back/to come back”; migraine: nl: “hoofdpijn” | en: “headache”, nl: “stress” | en: “stress” and nl: “misselijkheid” | en: “nausea”). Thematic and sentiment analysis of text revealed largely negative sentiment in texts by both patients with migraine and cluster headache. Logistic regression and support vector machine algorithms with different feature groups performed best for the classification of attack descriptions (with F1-scores for detecting cluster headache varying between 0.82 and 0.86) compared to naïve Bayes classifiers.

Conclusions

Differences in lexical choices between patients with migraine and cluster headache are detected with NLP and are congruent with domain expert knowledge of the disorders. Our research shows that ML algorithms have potential to classify patients’ self-reported narratives of migraine or cluster headache with good performance. NLP shows its capability to discern relevant linguistic aspects in narratives from patients with different headache disorders and demonstrates relevance in clinical information extraction. The potential benefits on the classification performance of larger datasets and neural NLP methods can be investigated in the future.

Trial registration

This study was registered with clinicaltrials.gov with ID NCT05377437.

Supplementary Information

The online version contains supplementary material available at 10.1186/s10194-022-01490-0.

Artificial Intelligence-Aided Headache Classification Based on a Set of Questionnaires: A Short Review

Wielding modern technology in the form of artificial intelligence (AI) or deep learning (DL) can utilize the best possible latest computer application in intricate decision-making and enigmatic problem-solving. It has been recommended in many fields. However, it is a long way from achieving an ambitious genuine intention when it comes to understanding and identifying any headache condition or classification, and using it error-free. No studies hitherto formalized any headache AI models to accurately classify headaches. 

A machine’s job can be arduous when incorporating an emotional dimension in decision making, re-challenging its own diagnosis by keeping a differential at all times, where even experienced neurologists or headache experts sometimes find it demanding to make a precise analysis and formulate a methodical plan. This could be because of spanning clinical presentation at a given moment of time or a change in clinical pattern over time which apparently could be due to intercrossing multiple pathophysiologies.

We did a short literature review on the role of artificial intelligence and machine learning in headache classification. This brings forth a minuscule insight into the vastness of headaches and the perpetual effort and exploration headache may demand from AI when trying to scrutinize its classification. Undoubtedly, AI or DL could better be utilized in identifying the red flags of headache, as it might help our patients at home or the primary care physicians/practicing doctors/non- neurologists in their clinic to triage the headache patients if they need an imperative higher center referral to a neurologist for advanced evaluation. This outlook can limit the burden on a handful of headache specialists by minimizing the referrals to a tertiary care setting.

Computerized migraine diagnostic tools: a systematic review

BACKGROUND

Computerized migraine diagnostic tools have been developed and validated since 1960. We conducted a systematic review to summarize and critically appraise the quality of all published studies involving computerized migraine diagnostic tools.

METHODS

We performed a systematic literature search using PubMed, Web of Science, Scopus, snowballing, and citation searching. Cutoff date for search was 1 June 2021. Published articles in English that evaluated a computerized/automated migraine diagnostic tool were included. The following summarized each study: publication year, digital tool name, development basis, sample size, sensitivity, specificity, reference diagnosis, strength, and limitations. The Quality Assessment of Diagnostic Accuracy Studies (QUADAS) tool was applied to evaluate the quality of included studies in terms of risk of bias and concern of applicability.

RESULTS

A total of 41 studies (median sample size: 288 participants, median age = 43 years; 77% women) were included. Most (60%) tools were developed based on International Classification of Headache Disorders criteria, half were self-administered, and 82% were evaluated using face-to-face interviews as reference diagnosis. Some of the automated algorithms and machine learning programs involved case-based reasoning, deep learning, classifier ensemble, ant-colony, artificial immune, random forest, white and black box combinations, and hybrid fuzzy expert systems. The median diagnostic accuracy was concordance = 89% [interquartile range (IQR) = 76-93%; range = 45-100%], sensitivity = 87% (IQR = 80-95%; range = 14-100%), and specificity = 90% (IQR = 77-96%; range = 65-100%). Lack of random patient sampling was observed in 95% of studies. Case-control designs were avoided in all studies. Most (76%) reference tests exhibited low risk of bias and low concern of applicability. Patient flow and timing showed low risk of bias in 83%.

CONCLUSION

Different computerized and automated migraine diagnostic tools are available with varying accuracies. Random patient sampling, head-to-head comparison among tools, and generalizability to other headache diagnoses may improve their utility.

A decision support system for primary headache developed through machine learning

BACKGROUND

Primary headache is a disorder with a high incidence and low diagnostic accuracy; the incidence of migraine and tension-type headache ranks first among primary headaches. Artificial intelligence (AI) decision support systems have shown great potential in the medical field. Therefore, we attempt to use machine learning to build a clinical decision-making system for primary headaches.

METHODS

The demographic data and headache characteristics of 173 patients were collected by questionnaires. Decision tree, random forest, gradient boosting algorithm and support vector machine (SVM) models were used to construct a discriminant model and a confusion matrix was used to calculate the evaluation indicators of the models. Furthermore, we have carried out feature selection through univariate statistical analysis and machine learning.

RESULTS

In the models, the accuracy, F1 score were calculated through the confusion matrix. The logistic regression model has the best discrimination effect, with the accuracy reaching 0.84 and the area under the ROC curve also being the largest at 0.90. Furthermore, we identified the most important factors for distinguishing the two disorders through statistical analysis and machine learning: nausea/vomiting and photophobia/phonophobia. These two factors represent potential independent factors for the identification of migraines and tension-type headaches, with the accuracy reaching 0.74 and the area under the ROC curve being at 0.74.

CONCLUSIONS

Applying machine learning to the decision-making system for primary headaches can achieve a high diagnostic accuracy. Among them, the discrimination effect obtained by the integrated algorithm is significantly better than that of a single learner. Second, nausea/vomiting, photophobia/phonophobia may be the most important factors for distinguishing migraine from tension-type headaches.

An Application of a Hybrid Intelligent System for Diagnosing Primary Headaches

(1) Background: Modern medicine generates a great deal of information that stored in medical databases. Simultaneously, extracting useful knowledge and making scientific decisions for diagnosis and treatment of diseases becomes increasingly necessary. Headache disorders are the most prevalent of all the neurological conditions. Headaches have not only medical but also great socioeconomic significance. The aim of this research is to develop an intelligent system for diagnosing primary headache disorders. (2) Methods: This research applied various mathematical, statistical and artificial intelligence techniques, among which the most important are: Calinski-Harabasz index, Analytical Hierarchy Process, and Weighted Fuzzy C-means Clustering Algorithm. These methods, techniques and methodologies are used to create a hybrid intelligent system for diagnosing primary headache disorders. The proposed intelligent diagnostic system is tested with original real-world data set with different metrics. (3) Results: First at all, nine of 20 attributes – features from International Headache Society (IHS) criteria are selected, and then only five most important attributes from IHS criteria are selected. The calculation result based on the Calinski–Harabasz index value (178) for the optimal number of clusters is three, and they present three classes of headaches: (i) migraine, (ii) tension-type headaches (TTHs), and (iii) other primary headaches (OPHs). The proposed hybrid intelligent system shows the following quality metrics: Accuracy 75%; Precision 67% for migraine, 74% for TTHs, 86% for OPHs, and Average Precision 77%; Recall 86% for migraine, 73% for TTHs, 67% for OPHs, Average Recall 75%; F₁ score 75% for migraine, 74% for TTHs, 75% for OPHs, and Average F₁ score 75%. (4) Conclusions: The hybrid intelligent system presents qualitative and respectable experimental results. The implementation of existing diagnostics systems and the development of new diagnostics systems in medicine is necessary in order to help physicians make quality diagnosis and decide the best treatments for the patients.

Impact of Individual Headache Types on the Work and Work Efficiency of Headache Sufferers

Background: Headaches have not only medical but also great socioeconomic significance, therefore, it is necessary to evaluate the overall impact of headaches on a patient’s life, including their work and work efficiency. The aim of this study was to determine the impact of individual headache types on work and work efficiency. Methods: This research was designed as a cross-sectional study performed by administering a questionnaire among employees. The questionnaire consisted of general questions, questions about headache features, and questions about the impact of headaches on work. Results: Monthly absence from work was mostly represented by migraine sufferers (7.1%), significantly more than with sufferers with tension-type headaches (2.23%; p = 0.019) and other headache types (2.15%; p = 0.025). Migraine sufferers (30.2%) worked in spite of a headache for more than 25 h, which was more frequent than with sufferers from tension-type and other-type headaches (13.4%). On average, headache sufferers reported work efficiency ranging from 66% to 90%. With regard to individual headache types, this range was significantly more frequent in subjects with tension-type headaches, whereas 91–100% efficiency was significantly more frequent in subjects with other headache types. Lower efficiency, i.e., 0–40% and 41–65%, was significantly more frequent with migraine sufferers. Conclusions: Headaches, especially migraines, significantly affect the work and work efficiency of headache sufferers by reducing their productivity. Loss is greater due to reduced efficiency than due to absenteeism.

Quantitative Cluster Headache Analysis for Neurological Diagnosis Support Using Statistical Classification

Cluster headache (CH) belongs to the group III of The International Classification of Headaches. It is characterized by attacks of severe pain in the ocular/periocular area accompanied by cranial autonomic signs, including parasympathetic activation and sympathetic hypofunction on the symptomatic side. Iris pigmentation occurs in the neonatal period and depends on the sympathetic tone in each eye. We hypothesized that the presence of visible or subtle color iris changes in both eyes could be used as a quantitative biomarker for screening and early detection of CH. This work scrutinizes the scope of an automatic diagnosis-support system for early detection of CH, by using as indicator the error rate provided by a statistical classifier designed to identify the eye (left vs. right) from iris pixels in color images. Systematic tests were performed on a database with images of 11 subjects (four with CH, four with other ophthalmic diseases affecting the iris pigmentation, and three control subjects). Several aspects were addressed to design the classifier, including: (a) the most convenient color space for the statistical classifier; (b) whether the use of features associated to several color spaces is convenient; (c) the robustness of the classifier to iris spatial subregions; (d) the contribution of the pixels neighborhood. Our results showed that a reduced value for the error rate (lower than 0.25) can be used as CH marker, whereas structural regions of the iris image need to be taken into account. The iris color feature analysis using statistical classification is a potentially useful technique to investigate disorders affecting the autonomous nervous system in CH.

Automatic migraine classification using artificial neural networks

Background: Previous studies of migraine classification have focused on the analysis of brain waves, leading to the development of complex tests that are not accessible to the majority of the population. In the early stages of this pathology, patients tend to go to the emergency services or outpatient department, where timely identification largely depends on the expertise of the physician and continuous monitoring of the patient. However, owing to the lack of time to make a proper diagnosis or the inexperience of the physician, migraines are often misdiagnosed either because they are wrongly classified or because the disease severity is underestimated or disparaged. Both cases can lead to inappropriate, unnecessary, or imprecise therapies, which can result in damage to patients' health. Methods: This study focuses on designing and testing an early classification system capable of distinguishing between seven types of migraines based on the patient's symptoms. The methodology proposed comprises four steps: data collection based on symptoms and diagnosis by the treating physician, selection of the most relevant variables, use of artificial neural network models for automatic classification, and selection of the best model based on the accuracy and precision of the diagnosis. Results: The artificial neural network models used provide an excellent classification performance, with accuracy and precision levels >97% and which exceed the classifications made using other model, such as logistic regression, support vector machines, nearest neighbor, and decision trees. Conclusions: The implementation of migraine classification through artificial neural networks is a powerful tool that reduces the time to obtain accurate, reliable, and timely clinical diagnoses.

Automatic migraine classification using artificial neural networks

Background: Previous studies of migraine classification have focused on the analysis of brain waves, leading to the development of complex tests that are not accessible to the majority of the population. In the early stages of this pathology, patients tend to go to the emergency services or outpatient department, where timely identification largely depends on the expertise of the physician and continuous monitoring of the patient. However, owing to the lack of time to make a proper diagnosis or the inexperience of the physician, migraines are often misdiagnosed either because they are wrongly classified or because the disease severity is underestimated or disparaged. Both cases can lead to inappropriate, unnecessary, or imprecise therapies, which can result in damage to patients' health. Methods: This study focuses on designing and testing an early classification system capable of distinguishing between seven types of migraines based on the patient's symptoms. The methodology proposed comprises four steps: data collection based on symptoms and diagnosis by the treating physician, selection of the most relevant variables, use of artificial neural network models for automatic classification, and selection of the best model based on the accuracy and precision of the diagnosis. Results: The neural network models used provide an excellent classification performance, with accuracy and precision levels >97% and which exceed the classifications made using other model, such as logistic regression, support vector machines, nearest neighbor, and decision trees. Conclusions: The implementation of migraine classification through neural networks is a powerful tool that reduces the time to obtain accurate, reliable, and timely clinical diagnoses.

A decision support system to follow up and diagnose primary headache patients using semantically enriched data

BACKGROUND

Headache disorders are an important health burden, having a large health-economic impact worldwide. Current treatment & follow-up processes are often archaic, creating opportunities for computer-aided and decision support systems to increase their efficiency. Existing systems are mostly completely data-driven, and the underlying models are a black-box, deteriorating interpretability and transparency, which are key factors in order to be deployed in a clinical setting.

METHODS

In this paper, a decision support system is proposed, composed of three components: (i) a cross-platform mobile application to capture the required data from patients to formulate a diagnosis, (ii) an automated diagnosis support module that generates an interpretable decision tree, based on data semantically annotated with expert knowledge, in order to support physicians in formulating the correct diagnosis and (iii) a web application such that the physician can efficiently interpret captured data and learned insights by means of visualizations.

RESULTS

We show that decision tree induction techniques achieve competitive accuracy rates, compared to other black- and white-box techniques, on a publicly available dataset, referred to as migbase. Migbase contains aggregated information of headache attacks from 849 patients. Each sample is labeled with one of three possible primary headache disorders. We demonstrate that we are able to reduce the classification error, statistically significant (ρ≤0.05), with more than 10% by balancing the dataset using prior expert knowledge. Furthermore, we achieve high accuracy rates by using features extracted using the Weisfeiler-Lehman kernel, which is completely unsupervised. This makes it an ideal approach to solve a potential cold start problem.

CONCLUSION

Decision trees are the perfect candidate for the automated diagnosis support module. They achieve predictive performances competitive to other techniques on the migbase dataset and are, foremost, completely interpretable. Moreover, the incorporation of prior knowledge increases both predictive performance as well as transparency of the resulting predictive model on the studied dataset.

Diagnostic Accuracy Comparison of Artificial Immune Algorithms for Primary Headaches

The present study evaluated the diagnostic accuracy of immune system algorithms with the aim of classifying the primary types of headache that are not related to any organic etiology. They are divided into four types: migraine, tension, cluster, and other primary headaches. After we took this main objective into consideration, three different neurologists were required to fill in the medical records of 850 patients into our web-based expert system hosted on our project web site. In the evaluation process, Artificial Immune Systems (AIS) were used as the classification algorithms. The AIS are classification algorithms that are inspired by the biological immune system mechanism that involves significant and distinct capabilities. These algorithms simulate the specialties of the immune system such as discrimination, learning, and the memorizing process in order to be used for classification, optimization, or pattern recognition. According to the results, the accuracy level of the classifier used in this study reached a success continuum ranging from 95% to 99%, except for the inconvenient one that yielded 71% accuracy.