Preliminary development of a deep learning-based automated primary headache diagnosis model using Japanese natural language processing of medical questionnaire

Preliminary External Validation Results of the Artificial Intelligence-Based Headache Diagnostic Model: A Multicenter Prospective Observational Study

The misdiagnosis of headache disorders is a serious issue, and AI-based headache model diagnoses with external validation are scarce. We previously developed an artificial intelligence (AI)-based headache diagnosis model using a database of 4000 patients' questionnaires in a headache-specializing clinic and herein performed external validation prospectively. The validation cohort of 59 headache patients was prospectively collected from August 2023 to February 2024 at our or collaborating multicenter institutions. The ground truth was specialists' diagnoses based on the initial questionnaire and at least a one-month headache diary after the initial consultation. The diagnostic performance of the AI model was evaluated. The mean age was 42.55 ± 12.74 years, and 51/59 (86.67%) of the patients were female. No missing values were reported. Of the 59 patients, 56 (89.83%) had migraines or medication-overuse headaches, and 3 (5.08%) had tension-type headaches. No one had trigeminal autonomic cephalalgias or other headaches. The models' overall accuracy and kappa for the ground truth were 94.92% and 0.65 (95%CI 0.21-1.00), respectively. The sensitivity, specificity, precision, and F values for migraines were 98.21%, 66.67%, 98.21%, and 98.21%, respectively. There was disagreement between the AI diagnosis and the ground truth by headache specialists in two patients. This is the first external validation of the AI headache diagnosis model. Further data collection and external validation are required to strengthen and improve its performance in real-world settings.

Using the Natural Language Processing System Medical Named Entity Recognition-Japanese to Analyze Pharmaceutical Care Records: Natural Language Processing Analysis

BACKGROUND

Large language models have propelled recent advances in artificial intelligence technology, facilitating the extraction of medical information from unstructured data such as medical records. Although named entity recognition (NER) is used to extract data from physicians' records, it has yet to be widely applied to pharmaceutical care records.

OBJECTIVE

In this study, we aimed to investigate the feasibility of automatic extraction of the information regarding patients' diseases and symptoms from pharmaceutical care records. The verification was performed using Medical Named Entity Recognition-Japanese (MedNER-J), a Japanese disease-extraction system designed for physicians' records.

METHODS

MedNER-J was applied to subjective, objective, assessment, and plan data from the care records of 49 patients who received cefazolin sodium injection at Keio University Hospital between April 2018 and March 2019. The performance of MedNER-J was evaluated in terms of precision, recall, and F1-score.

RESULTS

The F1-scores of NER for subjective, objective, assessment, and plan data were 0.46, 0.70, 0.76, and 0.35, respectively. In NER and positive-negative classification, the F1-scores were 0.28, 0.39, 0.64, and 0.077, respectively. The F1-scores of NER for objective (0.70) and assessment data (0.76) were higher than those for subjective and plan data, which supported the superiority of NER performance for objective and assessment data. This might be because objective and assessment data contained many technical terms, similar to the training data for MedNER-J. Meanwhile, the F1-score of NER and positive-negative classification was high for assessment data alone (F1-score=0.64), which was attributed to the similarity of its description format and contents to those of the training data.

CONCLUSIONS

MedNER-J successfully read pharmaceutical care records and showed the best performance for assessment data. However, challenges remain in analyzing records other than assessment data. Therefore, it will be necessary to reinforce the training data for subjective data in order to apply the system to pharmaceutical care records.

A Case of Headache Treated by Online Telemedicine in Collaboration With a Midwifery Home

Midwifery centers are places where midwives not only provide antenatal checkups and delivery care but also offer a wide range of health guidance to pregnant women, postpartum mothers, newborns, and older women. In recent years, midwives have also provided onsite and online health guidance. However, diagnosis and prescribing medication are impossible in midwifery centers because no doctor is present. If the midwife determines that the patient should consult doctors, the patient may have to go to a hospital and see doctors in person, which can be burdensome. Online telemedicine facilitates midwife-doctor collaboration and may solve this problem. We report a case of headache management by telemedicine that minimized the patient's travel burden by collaborating with a midwifery center that provides onsite, visiting, and online health guidance for patients who have difficulty visiting a hospital due to postpartum period, childcare, and breastfeeding. A 29-year-old woman and her husband were raising an infant in Sado City (a remote island across the sea), Niigata Prefecture. She developed acute back pain and was bedridden for several days due to immobility. She consulted a midwife because of stress and anxiety caused by childcare and acute back pain, as well as newly occurring headaches. The midwife visited her and provided on-site health guidance. The midwife decided that a doctor's diagnosis and treatment with painkillers were desirable for the headache and back pain, so she contacted a doctor based on the patient's request. The doctor provided online telemedicine across the sea, diagnosed her headache as a tension-type headache, and prescribed acetaminophen 500 mg as an abortive prescription. The prescription was faxed to a pharmacy on the island, and the original was sent by post. The midwife picked up the medication and delivered it to the patient. After taking the medication, the patient's back pain and headache went into remission. Collaboration between midwifery centers that provide onsite, visiting, and online health guidance and medical institutions that offer online telemedicine can potentially improve accessibility to medical care. It differs from conventional online telemedicine in the midwife's coordination practice by monitoring the patient's condition and requesting the physician based on the patient's request.

Identification of risk factors for the onset of delirium associated with COVID-19 by mining nursing records

COVID-19 has a range of complications, from no symptoms to severe pneumonia. It can also affect multiple organs including the nervous system. COVID-19 affects the brain, leading to neurological symptoms such as delirium. Delirium, a sudden change in consciousness, can increase the risk of death and prolong the hospital stay. However, research on delirium prediction in patients with COVID-19 is insufficient. This study aimed to identify new risk factors that could predict the onset of delirium in patients with COVID-19 using machine learning (ML) applied to nursing records. This retrospective cohort study used natural language processing and ML to develop a model for classifying the nursing records of patients with delirium. We extracted the features of each word from the model and grouped similar words. To evaluate the usefulness of word groups in predicting the occurrence of delirium in patients with COVID-19, we analyzed the temporal changes in the frequency of occurrence of these word groups before and after the onset of delirium. Moreover, the sensitivity, specificity, and odds ratios were calculated. We identified (1) elimination-related behaviors and conditions and (2) abnormal patient behavior and conditions as risk factors for delirium. Group 1 had the highest sensitivity (0.603), whereas group 2 had the highest specificity and odds ratio (0.938 and 6.903, respectively). These results suggest that these parameters may be useful in predicting delirium in these patients. The risk factors for COVID-19-associated delirium identified in this study were more specific but less sensitive than the ICDSC (Intensive Care Delirium Screening Checklist) and CAM-ICU (Confusion Assessment Method for the Intensive Care Unit). However, they are superior to the ICDSC and CAM-ICU because they can predict delirium without medical staff and at no cost.

Headache Education by E-Learning Through Social Networking Services (Social Media)

Introduction

Headache is a common public health problem, but its burden could be avoided by raising headache awareness and the appropriate use of acute medication and prophylactic medication. Few reports on raising headache awareness in the general public have been reported, and there are no reports on headache awareness campaigns through social networking services (SNS), or social media, in Japan. We prospectively performed a headache awareness campaign from March 2022 through 2 SNS, targeting nurse and wind instrumental musicians, because they are with high headache prevalence.

Methods

Through the 2 SNS, the article and video were distributed, respectively. The article and video described the 6 important topics for the general public about headaches, which were described in the Clinical Practice Guideline for Headache Disorders 2021. Just after reading or watching them as e-learning, we performed online questionnaire sheets to investigate the awareness of the 6 topics through the 2 SNS. The awareness of the 6 topics before and after the campaign was evaluated.

Results

In the SNS nurse-senka, we obtained 1191 responses. Women comprised 94.4%, and the median (range) age was 45 (20 to 71) years old. Headache sufferers were 63.8%, but only 35.1% had consulted doctors. In the SNS Creatone, we got the response from 134 professional musicians, with 77.3% of women. The largest number of respondents were in their 20s (range 18-60 years old). Headache sufferers were 87.9%. Of them, 36.4% had consulted doctors, 24.2% were medication-overuse headache. The ratios of individuals who were aware of the 6 topics significantly increased from 15.2%-47.0% to 80.4-98.7% after the online questionnaire in both SNS (p < 0.001, all).

Conclusion

E-learning and online survey via SNS can improve headache awareness.

Developing an artificial intelligence-based diagnostic model of headaches from a dataset of clinic patients' records

OBJECTIVE

We developed an artificial intelligence (AI)-based headache diagnosis model using a large questionnaire database from a headache-specializing clinic.

BACKGROUND

Misdiagnosis of headache disorders is a serious issue and AI-based headache diagnosis models are scarce.

METHODS

We developed an AI-based headache diagnosis model and conducted internal validation based on a retrospective investigation of 6058 patients (4240 training dataset for model development and 1818 test dataset for internal validation) diagnosed by a headache specialist. The ground truth was the diagnosis by the headache specialist. The diagnostic performance of the AI model was evaluated.

RESULTS

The dataset included 4829/6058 (79.7%) patients with migraine, 834/6058 (13.8%) with tension-type headache, 78/6058 (1.3%) with trigeminal autonomic cephalalgias, 38/6058 (0.6%) with other primary headache disorders, and 279/6058 (4.6%) with other headaches. The mean (standard deviation) age was 34.7 (14.5) years, and 3986/6058 (65.8%) were female. The model's micro-average accuracy, sensitivity (recall), specificity, precision, and F-values for the test dataset were 93.7%, 84.2%, 84.2%, 96.1%, and 84.2%, respectively. The diagnostic performance for migraine was high, with a sensitivity of 88.8% and c-statistics of 0.89 (95% confidence interval 0.87-0.91).

CONCLUSIONS

Our AI model demonstrated high diagnostic performance for migraine. If secondary headaches can be ruled out, the model can be a powerful tool for diagnosing migraine; however, further data collection and external validation are required to strengthen the performance, ensure the generalizability in other outpatients, and demonstrate its utility in real-world settings.

Developing an Artificial Intelligence-Based Pediatric and Adolescent Migraine Diagnostic Model

Introduction Misdiagnosis of pediatric and adolescent migraine is a significant problem. The first artificial intelligence (AI)-based pediatric migraine diagnosis model was made utilizing a database of questionnaires obtained from a previous epidemiological study, the Itoigawa Benizuwaigani Study. Methods The AI-based headache diagnosis model was created based on the internal validation based on a retrospective investigation of 909 patients (636 training dataset for model development and 273 test dataset for internal validation) aged six to 17 years diagnosed based on the International Classification of Headache Disorders 3rd edition. The diagnostic performance of the AI model was evaluated. Results The dataset included 234/909 (25.7%) pediatric or adolescent patients with migraine. The mean age was 11.3 (standard deviation 3.17) years. The model's accuracy, sensitivity (recall), specificity, precision, and F-values for the test dataset were 94.5%, 88.7%, 96.5%, 90.0%, and 89.4%, respectively. Conclusions The AI model exhibited high diagnostic performance for pediatric and adolescent migraine. It holds great potential as a powerful tool for diagnosing these conditions, especially when secondary headaches are ruled out. Nonetheless, further data collection and external validation are necessary to enhance the model's performance and ensure its applicability in real-world settings.

Headache education by leaflet distribution during COVID-19 vaccination and school-based on-demand e-learning: Itoigawa Geopark Headache Awareness Campaign

OBJECTIVE

We prospectively performed the Itoigawa Headache Awareness Campaign from August 2021 to June 2022, with two main interventions, and evaluated its effectiveness.

BACKGROUND

Headache is a common public health problem, but its burden could be reduced by raising awareness about headache and the appropriate use of acute and prophylactic medication. However, few studies on raising headache awareness in the general public have been reported.

METHODS

The target group was the general public aged 15-64. We performed two main interventions synergistically supported by other small interventions. Intervention 1 included leaflet distribution and a paper-based questionnaire about headache during COVID-19 vaccination, and intervention 2 included on-demand e-learning and online survey through schools. In these interventions, we emphasize the six important topics for the general public that were described in the Clinical Practice Guideline for Headache Disorders 2021. Each response among the two interventions' cohorts was collected on pre and post occasions. The awareness of the six topics before and after the campaign was evaluated.

RESULTS

We obtained 4016 valid responses from 6382 individuals who underwent vaccination in intervention 1 and 2577 from 594 students and 1983 parents in intervention 2; thus, 6593 of 20,458 (32.2%) of the overall working-age population in Itoigawa city experienced these interventions. The percentage of individuals' aware of the six topics significantly increased after the two main interventions ranging from 6.6% (39/594)-40.0% (1606/4016) to 64.1% (381/594)-92.6% (1836/1983) (p < 0.001, all).

CONCLUSIONS

We conducted this campaign through two main interventions with an improved percentage of individuals who know about headache. The two methods of community-based interventions could raise headache awareness effectively. Furthermore, we can achieve outstanding results by doing something to raise disease awareness during mass vaccination, when almost all residents gather in a certain place, and school-based e-learning without face-to-face instruction due to the COVID-19 pandemic.

Investigating the effects of weather on headache occurrence using a smartphone application and artificial intelligence: A retrospective observational cross-sectional study

OBJECTIVE

To investigate the relationship between weather and headache occurrence using big data from an electronic headache diary smartphone application with recent statistical and deep learning (DL)-based methods.

BACKGROUND

The relationship between weather and headache occurrence remains unknown.

METHODS

From a database of 1 million users, data from 4375 users with 336,951 hourly headache events and weather data from December 2020 to November 2021 were analyzed. We developed statistical and DL-based models to predict the number of hourly headache occurrences mainly from weather factors. Temporal validation was performed using data from December 2019 to November 2020. Apart from the user dataset used in this model development, the physician-diagnosed headache prevalence was gathered.

RESULTS

Of the 40,617 respondents, 15,127/40,617 (37.2%) users experienced physician-diagnosed migraine, and 2458/40,617 (6.1%) users had physician-diagnosed non-migraine headaches. The mean (standard deviation) age of the 4375 filtered users was 34 (11.2) years, and 89.2% were female (3902/4375). Lower barometric pressure (p < 0.001, gain = 3.9), higher humidity (p < 0.001, gain = 7.1), more rainfall (p < 0.001, gain = 3.1), a significant decrease in barometric pressure 6 h before (p < 0.001, gain = 11.7), higher barometric pressure at 6:00 a.m. on the day (p < 0.001, gain = 4.6), lower barometric pressure on the next day (p < 0.001, gain = 6.7), and raw time-series barometric type I (remaining low around headache attack, p < 0.001, gain = 10.1) and type II (decreasing around headache attack, p < 0.001, gain = 10.1) changes over 6 days, were significantly associated with headache occurrences in both the statistical and DL-based models. For temporal validation, the root mean squared error (RMSE) was 13.4, and the determination coefficient (R² ) was 52.9% for the statistical model. The RMSE was 10.2, and the R² was 53.7% for the DL-based model.

CONCLUSIONS

Using big data, we found that low barometric pressure, barometric pressure changes, higher humidity, and rainfall were associated with an increased number of headache occurrences.

Is artificial intelligence capable of generating hospital discharge summaries from inpatient records?

Medical professionals have been burdened by clerical work, and artificial intelligence may efficiently support physicians by generating clinical summaries. However, whether hospital discharge summaries can be generated automatically from inpatient records stored in electronic health records remains unclear. Therefore, this study investigated the sources of information in discharge summaries. First, the discharge summaries were automatically split into fine-grained segments, such as those representing medical expressions, using a machine learning model from a previous study. Second, these segments in the discharge summaries that did not originate from inpatient records were filtered out. This was performed by calculating the n-gram overlap between inpatient records and discharge summaries. The final source origin decision was made manually. Finally, to reveal the specific sources (e.g., referral documents, prescriptions, and physician's memory) from which the segments originated, they were manually classified by consulting medical professionals. For further and deeper analysis, this study designed and annotated clinical role labels that represent the subjectivity of the expressions and builds a machine learning model to assign them automatically. The analysis results revealed the following: First, 39% of the information in the discharge summary originated from external sources other than inpatient records. Second, patient's past clinical records constituted 43%, and patient referral documents constituted 18% of the expressions derived from external sources. Third, 11% of the missing information was not derived from any documents. These are possibly derived from physicians' memories or reasoning. According to these results, end-to-end summarization using machine learning is considered infeasible. Machine summarization with an assisted post-editing process is the best fit for this problem domain.

The First Case Series From Japan of Primary Headache Patients Treated by Completely Online Telemedicine

Background Since March 2020, the coronavirus disease 2019 pandemic has increased the need for telemedicine to avoid in-person consultations. Online clinics for most diseases officially started in Japan in April 2022. Here, we report the cases of eight Japanese headache patients treated by completely online telemedicine for three months from the first visit. Methodology From the medical records between July 2022 and October 2022, we retrospectively investigated eight consecutive first-visit primary headache patients who consulted our online headache clinic via telemedicine and continued to see us via telemedicine only. The Headache Impact Test-6 (HIT-6) score, monthly headache days (MHD), and monthly acute medication intake days (AMD) were investigated over the observation period. Results A total of eight women were included, and the median (interquartile range) age was 30 (24-51) years. The median HIT-6 scores before, one, and three months after treatment were 63 (58-64), 54 (53-62), and 52 (49-54), respectively. MHD before, one, and three months after treatment were 15 (9-28), 12 (3-17), and 2 (2-8), respectively. AMD before, one, and three months after treatment were 10 (3-13), 3 (1-8), and 2 (0-3), respectively. Significant reductions in HIT-6 and MDH were observed three months after the initial consultation (p = 0.007 and p = 0.042, respectively). AMD was not significantly decreased at three months (p = 0.447). Conclusions This is the first report of Japanese patients treated by completely online telemedicine for three months from the first visit. HIT-6 and MDH can be significantly decreased at three months by only telemedicine. Online telemedicine is expected to be widely used to resolve unmet needs in headache treatment.

Using natural language processing to automatically classify written self-reported narratives by patients with migraine or cluster headache

Background

Headache medicine is largely based on detailed history taking by physicians analysing patients’ descriptions of headache. Natural language processing (NLP) structures and processes linguistic data into quantifiable units. In this study, we apply these digital techniques on self-reported narratives by patients with headache disorders to research the potential of analysing and automatically classifying human-generated text and information extraction in clinical contexts.

Methods

A prospective cross-sectional clinical trial collected self-reported narratives on headache disorders from participants with either migraine or cluster headache. NLP was applied for the analysis of lexical, semantic and thematic properties of the texts. Machine learning (ML) algorithms were applied to classify the descriptions of headache attacks from individual participants into their correct group (migraine versus cluster headache).

Results

One-hundred and twenty-one patients (81 participants with migraine and 40 participants with cluster headache) provided a self-reported narrative on their headache disorder. Lexical analysis of this text corpus resulted in several specific key words per diagnostic group (cluster headache: Dutch (nl): “oog” | English (en): “eye”, nl: “pijn” | en: “pain” and nl: “terug” | en: “back/to come back”; migraine: nl: “hoofdpijn” | en: “headache”, nl: “stress” | en: “stress” and nl: “misselijkheid” | en: “nausea”). Thematic and sentiment analysis of text revealed largely negative sentiment in texts by both patients with migraine and cluster headache. Logistic regression and support vector machine algorithms with different feature groups performed best for the classification of attack descriptions (with F1-scores for detecting cluster headache varying between 0.82 and 0.86) compared to naïve Bayes classifiers.

Conclusions

Differences in lexical choices between patients with migraine and cluster headache are detected with NLP and are congruent with domain expert knowledge of the disorders. Our research shows that ML algorithms have potential to classify patients’ self-reported narratives of migraine or cluster headache with good performance. NLP shows its capability to discern relevant linguistic aspects in narratives from patients with different headache disorders and demonstrates relevance in clinical information extraction. The potential benefits on the classification performance of larger datasets and neural NLP methods can be investigated in the future.

Trial registration

This study was registered with clinicaltrials.gov with ID NCT05377437.

Supplementary Information

The online version contains supplementary material available at 10.1186/s10194-022-01490-0.

Artificial Intelligence-Aided Headache Classification Based on a Set of Questionnaires: A Short Review

Wielding modern technology in the form of artificial intelligence (AI) or deep learning (DL) can utilize the best possible latest computer application in intricate decision-making and enigmatic problem-solving. It has been recommended in many fields. However, it is a long way from achieving an ambitious genuine intention when it comes to understanding and identifying any headache condition or classification, and using it error-free. No studies hitherto formalized any headache AI models to accurately classify headaches. 

A machine’s job can be arduous when incorporating an emotional dimension in decision making, re-challenging its own diagnosis by keeping a differential at all times, where even experienced neurologists or headache experts sometimes find it demanding to make a precise analysis and formulate a methodical plan. This could be because of spanning clinical presentation at a given moment of time or a change in clinical pattern over time which apparently could be due to intercrossing multiple pathophysiologies.

We did a short literature review on the role of artificial intelligence and machine learning in headache classification. This brings forth a minuscule insight into the vastness of headaches and the perpetual effort and exploration headache may demand from AI when trying to scrutinize its classification. Undoubtedly, AI or DL could better be utilized in identifying the red flags of headache, as it might help our patients at home or the primary care physicians/practicing doctors/non- neurologists in their clinic to triage the headache patients if they need an imperative higher center referral to a neurologist for advanced evaluation. This outlook can limit the burden on a handful of headache specialists by minimizing the referrals to a tertiary care setting.

Exploring optimal granularity for extractive summarization of unstructured health records: Analysis of the largest multi-institutional archive of health records in Japan

Automated summarization of clinical texts can reduce the burden of medical professionals. "Discharge summaries" are one promising application of the summarization, because they can be generated from daily inpatient records. Our preliminary experiment suggests that 20-31% of the descriptions in discharge summaries overlap with the content of the inpatient records. However, it remains unclear how the summaries should be generated from the unstructured source. To decompose the physician's summarization process, this study aimed to identify the optimal granularity in summarization. We first defined three types of summarization units with different granularities to compare the performance of the discharge summary generation: whole sentences, clinical segments, and clauses. We defined clinical segments in this study, aiming to express the smallest medically meaningful concepts. To obtain the clinical segments, it was necessary to automatically split the texts in the first stage of the pipeline. Accordingly, we compared rule-based methods and a machine learning method, and the latter outperformed the formers with an F1 score of 0.846 in the splitting task. Next, we experimentally measured the accuracy of extractive summarization using the three types of units, based on the ROUGE-1 metric, on a multi-institutional national archive of health records in Japan. The measured accuracies of extractive summarization using whole sentences, clinical segments, and clauses were 31.91, 36.15, and 25.18, respectively. We found that the clinical segments yielded higher accuracy than sentences and clauses. This result indicates that summarization of inpatient records demands finer granularity than sentence-oriented processing. Although we used only Japanese health records, it can be interpreted as follows: physicians extract "concepts of medical significance" from patient records and recombine them in new contexts when summarizing chronological clinical records, rather than simply copying and pasting topic sentences. This observation suggests that a discharge summary is created by higher-order information processing over concepts on sub-sentence level, which may guide future research in this field.

Primary headaches increase the risk of dementias: An 8-year nationwide cohort study

Background

Headache, a highly prevalent neurological disorder, has consistently been linked with an elevated risk of dementia. However, most studies are focused on the relationship with migraine in limited age groups. Therefore, the objective of this research was to look at the link between various type of headaches and dementias based on longitudinal population-based data.

Methods and results

Participants diagnosed with headache from 2002 to 2005 were selected and major covariates were collected. The diagnoses of Alzheimer’s disease, vascular dementia, and other dementias were observed from 2006 until 2013. The adjusted hazard ratios (aHRs) and 95% confidence intervals (CIs) of dementias according to headache type were calculated by Cox proportional hazards regression. A number of 470,652 participants were observed for a mean of 7.6 years (standard deviation: 1.2), for approximately 3.6 million person-years. Both tension type headache (TTH) and migraine elevated the risk of all-cause dementias (TTH, aHR 1.18, 95% CI 1.13–2.24; migraine, aHR 1.18, 95% CI 1.13–2.24). Headaches had a greater influence in females and non-smokers as a risk factor of dementias. Patients with migraine who consumed alcohol had a higher risk of dementia, however this was not true with TTH patients. Among participants without comorbidities, TTH patients were more susceptible to dementia than migraine patients. Headache patients had a higher proportion of females regardless of headache type and approximately 1.5 times more individuals had three or more comorbidities compared to those without headache.

Conclusions

Headache could be an independent predictor for subsequent dementia risk. Future studies should focus on clarifying pathogenic pathways and possible dementia-related preventive measures in headache populations.

Computerized migraine diagnostic tools: a systematic review

BACKGROUND

Computerized migraine diagnostic tools have been developed and validated since 1960. We conducted a systematic review to summarize and critically appraise the quality of all published studies involving computerized migraine diagnostic tools.

METHODS

We performed a systematic literature search using PubMed, Web of Science, Scopus, snowballing, and citation searching. Cutoff date for search was 1 June 2021. Published articles in English that evaluated a computerized/automated migraine diagnostic tool were included. The following summarized each study: publication year, digital tool name, development basis, sample size, sensitivity, specificity, reference diagnosis, strength, and limitations. The Quality Assessment of Diagnostic Accuracy Studies (QUADAS) tool was applied to evaluate the quality of included studies in terms of risk of bias and concern of applicability.

RESULTS

A total of 41 studies (median sample size: 288 participants, median age = 43 years; 77% women) were included. Most (60%) tools were developed based on International Classification of Headache Disorders criteria, half were self-administered, and 82% were evaluated using face-to-face interviews as reference diagnosis. Some of the automated algorithms and machine learning programs involved case-based reasoning, deep learning, classifier ensemble, ant-colony, artificial immune, random forest, white and black box combinations, and hybrid fuzzy expert systems. The median diagnostic accuracy was concordance = 89% [interquartile range (IQR) = 76-93%; range = 45-100%], sensitivity = 87% (IQR = 80-95%; range = 14-100%), and specificity = 90% (IQR = 77-96%; range = 65-100%). Lack of random patient sampling was observed in 95% of studies. Case-control designs were avoided in all studies. Most (76%) reference tests exhibited low risk of bias and low concern of applicability. Patient flow and timing showed low risk of bias in 83%.

CONCLUSION

Different computerized and automated migraine diagnostic tools are available with varying accuracies. Random patient sampling, head-to-head comparison among tools, and generalizability to other headache diagnoses may improve their utility.

Natural language processing in clinical neuroscience and psychiatry: A review

Natural language processing (NLP) is rapidly becoming an important topic in the medical community. The ability to automatically analyze any type of medical document could be the key factor to fully exploit the data it contains. Cutting-edge artificial intelligence (AI) architectures, particularly machine learning and deep learning, have begun to be applied to this topic and have yielded promising results. We conducted a literature search for 1,024 papers that used NLP technology in neuroscience and psychiatry from 2010 to early 2022. After a selection process, 115 papers were evaluated. Each publication was classified into one of three categories: information extraction, classification, and data inference. Automated understanding of clinical reports in electronic health records has the potential to improve healthcare delivery. Overall, the performance of NLP applications is high, with an average F1-score and AUC above 85%. We also derived a composite measure in the form of Z-scores to better compare the performance of NLP models and their different classes as a whole. No statistical differences were found in the unbiased comparison. Strong asymmetry between English and non-English models, difficulty in obtaining high-quality annotated data, and train biases causing low generalizability are the main limitations. This review suggests that NLP could be an effective tool to help clinicians gain insights from medical reports, clinical research forms, and more, making NLP an effective tool to improve the quality of healthcare services.

Deep Learning-Based Functional Independence Measure Score Prediction After Stroke in Kaifukuki (Convalescent) Rehabilitation Ward Annexed to Acute Care Hospital

Introduction

Prediction models of functional independent measure (FIM) score after kaifukuki (convalescent) rehabilitation ward (KRW) are needed to decide the treatment strategies and save medical resources. Statistical models were reported, but their accuracies were not satisfactory. We made such prediction models using the deep learning (DL) framework, Prediction One (Sony Network Communications Inc., Tokyo, Japan).

Methods

Of the 559 consecutive stroke patients, 122 patients were transferred to our KRW. We divided our 122 patients’ data randomly into halves of training and validation datasets. Prediction One made three prediction models from the training dataset using (1) variables at the acute care ward admission, (2) those at the KRW admission, and (3) those combined (1) and (2). The models’ determination coefficients (R²), correlation coefficients (rs), and residuals were calculated using the validation dataset.

Results

Of the 122 patients, the median age was 71, length of stay (LOS) in acute care ward 23 (17-30) days, LOS in KRW 53 days, total FIM scores at the admission of KRW 85, those at discharge 108. The mean FIM gain and FIM efficiency were 19 and 0.417. All patients were discharged home. Model (1), (2), and (3)’s R² were 0.794, 0.970, and 0.972. Their mean residuals between the predicted and actual total FIM scores were -1.56±24.6, -4.49±17.1, and -2.69±15.7.

Conclusion

Our FIM gain and efficiency were better than national averages of FIM gain 17.1 and FIM efficiency 0.187. We made DL-based total FIM score prediction models, and their accuracies were superior to those of previous statistically calculated ones. The DL-based FIM score prediction models would save medical costs and perform efficient stroke and rehabilitation medicine.

Relationship Between Medical Questionnaire and Influenza Rapid Test Positivity: Subjective Pretest Probability, "I Think I Have Influenza," Contributes to the Positivity Rate

Introduction

Rapid influenza diagnostic tests (RIDTs) are considered essential for determining when to start influenza treatment using anti-influenza drugs, but their accuracy is about 70%. Under the COVID-19 pandemic, we hope to refrain from performing unnecessary RIDTs considering droplet infection of COVID-19 and influenza. We re-examined the medical questionnaire’s importance and its relationship to the positivity of RIDTs. Then we built a positivity prediction model for RIDTs using automated artificial intelligence (AI).

Methods

We retrospectively investigated 96 patients who underwent RIDTs at the outpatient department from December 2019 to March 2020. We used a questionnaire sheet with 24 items before conducting RIDTs. The factors associated with the positivity of RIDTs were statistically analyzed. We then used an automated AI framework to produce the positivity prediction model using the 24 items, sex, and age, with five-fold cross-validation.

Results

Of the 47 women and 49 men (median age was 39 years), 56 patients were RIDT positive with influenza A. The AI-based model using 26 variables had an area under the curve (AUC) of 0.980. The stronger variables are subjective pretest probability, which is a numerically described score ranging from 0% to 100% of “I think I have influenza,” cough, past hours after the onset, muscle pain, and maximum body temperature in order.

Conclusion

We easily built the RIDT positivity prediction model using automated AI. Its AUC was satisfactory, and it suggested the importance of a detailed medical interview. Both the univariate analysis and AI-based model suggested that subjective pretest probability, “I think I have influenza,” might be useful.

Medication Overuse Headache Successfully Treated by Three Types of Japanese Herbal Kampo Medicine

Medication overuse headache (MOH) usually resolves after the overuse is stopped and starting prophylactic medications. However, it can be challenging to prescribe common prophylactic medications when patients have a history of side effects. As an alternative therapy, traditional Japanese herbal kampo medicine can be used. We herein report a case of a MOH woman with a history of side effects by such common prophylactic medications. A 50-year-old woman presented with a severe migraine attack. She had suffered from migraines for 10 years. She had taken loxoprofen and sumatriptan every day for over eight years. As prophylactic medications, lomerizine, valproic acid, and amitriptyline had been prescribed in the past, but they were discontinued due to side effects. Therefore, she could continue only propranolol as prophylactic medication. She had severe pulsatile headaches and nausea every day. We diagnosed triptan- and non-steroidal anti-inflammatory drug-overuse headache (the International Classification of Headache Disorders 3rd edition code 8.2.2 and 8.2.3.2) and chronic migraine (code 1.3). She was admitted and stopped loxoprofen and sumatriptan. We prescribed three types of Japanese herbal kampo medicines - kakkonto (TJ-1), goreisan (TJ-17), and goshuyuto (TJ-31). Her headache was relieved on day 5, and she was discharged on day 7. In the 40 days after discharge, she had only three times mild headaches with a numeric rating scale (NRS) of 2/10. She did not need any triptans nor anti-inflammatory drugs. We herein presented the MOH woman who was successfully treated using three types of kampo medicine. We should pay attention to their side effects, but kampo medicine may be useful for MOH treatment as acute and prophylactic medications for primary headaches.

Easily Created Prediction Model Using Automated Artificial Intelligence Framework (Prediction One, Sony Network Communications Inc., Tokyo, Japan) for Subarachnoid Hemorrhage Outcomes Treated by Coiling and Delayed Cerebral Ischemia

Introduction Reliable prediction models of subarachnoid hemorrhage (SAH) outcomes and delayed cerebral ischemia (DCI) are needed to decide the treatment strategy. Automated artificial intelligence (AutoAI) is attractive, but there are few reports on AutoAI-based models for SAH functional outcomes and DCI. We herein made models using an AutoAI framework, Prediction One (Sony Network Communications Inc., Tokyo, Japan), and compared it to other previous statistical prediction scores. Methods We used an open dataset of 298 SAH patients, who were with non-severe neurological grade and treated by coiling. Modified Rankin Scale 0-3 at six months was defined as a favorable functional outcome and DCI occurrence as another outcome. We randomly divided them into a 248-patient training dataset and a 50-patient test dataset. Prediction One made the model using training dataset with 5-fold cross-validation. We evaluated the model using the test dataset and compared the area under the curves (AUCs) of the created models. Those of the modified SAFIRE score and the Fisher computed tomography (CT) scale to predict the outcomes. Results The AUCs of the AutoAI-based models for functional outcome in the training and test dataset were 0.994 and 0.801, and those for the DCI occurrence were 0.969 and 0.650. AUCs for functional outcome calculated using modified SAFIRE score were 0.844 and 0.892. Those for the DCI occurrence calculated using the Fisher CT scale were 0.577 and 0.544. Conclusions We easily and quickly made AutoAI-based prediction models. The models' AUCs were not inferior to the previous prediction models despite the easiness.

Postsurgical functional outcome prediction model using deep learning framework (Prediction One, Sony Network Communications Inc.) for hypertensive intracerebral hemorrhage

BACKGROUND

Reliable prediction models of intracerebral hemorrhage (ICH) outcomes are needed for decision-making of the treatment. Statistically making such prediction models needs a large number of samples and time-consuming statistical analysis. Deep learning (DL), one of the artificial intelligence, is attractive, but there were no reports on DL-based functional outcome prediction models for ICH outcomes after surgery. We herein made a functional outcome prediction model using DLframework, Prediction One (Sony Network Communications Inc., Tokyo, Japan), and compared it to original ICH score, ICH Grading Scale, and FUNC score.

METHODS

We used 140 consecutive hypertensive ICH patients' data in our hospital between 2012 and 2019. All patients were surgically treated. Modified Rankin Scale 0-3 at 6 months was defined as a favorable outcome. We randomly divided them into 100 patients training dataset and 40 patients validation dataset. Prediction One made the prediction model using the training dataset with 5-fold cross-validation. We calculated area under the curves (AUCs) regarding the outcome using the DL-based model, ICH score, ICH Grading Scale, and FUNC score. The AUCs were compared.

RESULTS

The model made by Prediction One using 64 variables had AUC of 0.997 in the training dataset and that of 0.884 in the validation dataset. These AUCs were superior to those derived from ICH score, ICH Grading Scale, and FUNC score.

CONCLUSION

We easily and quickly made prediction models using Prediction One, even with a small single-center dataset. The accuracy of the DL-based model was superior to those of previous statistically calculated models.

Prediction Model of Deep Learning for Ambulance Transports in Kesennuma City by Meteorological Data

PURPOSE

With the aging population in Japan, the prediction of ambulance transports is needed to save the limited medical resources. Some meteorological factors were risks of ambulance transports, but it is difficult to predict in a classically statistical way because Japan has 4 seasons. We tried to make prediction models for ambulance transports using the deep learning (DL) framework, Prediction One (Sony Network Communications Inc., Tokyo, Japan), with the meteorological and calendarial variables.

MATERIALS AND METHODS

We retrospectively investigated the daily ambulance transports and meteorological data between 2017 and 2019. First, to confirm their association, we performed classically statistical analysis. Second, to test the DL framework's utility for ambulance transports prediction, we made 3 prediction models for daily ambulance transports (total daily ambulance transports more than 5 or not, cardiopulmonary arrest (CPA), and trauma) using meteorological and calendarial factors and evaluated their accuracies by internal cross-validation.

RESULTS

During the 1095 days of 3 years, the total ambulance transports were 5948, including 240 CPAs and 337 traumas. Cardiogenic CPA accounted for 72.3%, according to the Utstein classification. The relation between ambulance transports and meteorological parameters by polynomial curves were statistically obtained, but their r2s were small. On the other hand, all DL-based prediction models obtained satisfactory accuracies in the internal cross-validation. The areas under the curves obtained from each model were all over 0.947.

CONCLUSION

We could statistically make polynomial curves between the meteorological variables and the number of ambulance transport. We also preliminarily made DL-based prediction models. The DL-based prediction for daily ambulance transports would be used in the future, leading to solving the lack of medical resources in Japan.