Free-Text Documentation of Dementia Symptoms in Home Healthcare: A Natural Language Processing Study

Using Natural Language Processing to Identify Home Health Care Patients at Risk for Diagnosis of Alzheimer's Disease and Related Dementias

This study aimed to: (1) validate a natural language processing (NLP) system developed for the home health care setting to identify signs and symptoms of Alzheimer's disease and related dementias (ADRD) documented in clinicians' free-text notes; (2) determine whether signs and symptoms detected via NLP help to identify patients at risk of a new ADRD diagnosis within four years after admission. This study applied NLP to a longitudinal dataset including medical record and Medicare claims data for 56,652 home health care patients and Cox proportional hazard models to the subset of 24,874 patients admitted without an ADRD diagnosis. Selected ADRD signs and symptoms were associated with increased risk of a new ADRD diagnosis during follow-up, including: motor issues; hoarding/cluttering; uncooperative behavior; delusions or hallucinations; mention of ADRD disease names; and caregiver stress. NLP can help to identify patients in need of ADRD-related evaluation and support services.

Enhancing readmission prediction models by integrating insights from home healthcare notes: Retrospective cohort study

BACKGROUND

Hospital readmission is an important indicator of inpatient care quality and a significant driver of increasing medical costs. Therefore, it is important to explore the effects of postdischarge information, particularly from home healthcare notes, on enhancing readmission prediction models. Despite the use of Natural Language Processing (NLP) and machine learning in prediction model development, current studies often overlook insights from home healthcare notes.

OBJECTIVE

This study aimed to develop prediction models for 30-day readmissions using home healthcare notes and structured data. In addition, it explored the development of 14- and 180-day prediction models using variables in the 30-day model.

DESIGN

A retrospective observational cohort study.

SETTING(S)

This study was conducted at Ajou University School of Medicine in South Korea.

PARTICIPANTS

Data from electronic health records, encompassing demographic characteristics of 1819 participants, along with information on conditions, drug, and home healthcare, were utilized.

METHODS

Two distinct models were developed for each prediction window (30-, 14-, 180-day): the traditional model, which utilized structured variables alone, and the common data model (CDM)-NLP model, which incorporated structured and topic variables extracted from home healthcare notes. BERTopic facilitated topic generation and risk probability, representing the likelihood of documents being assigned to specific topics. Feature selection involved experimenting with various algorithms. The best-performing algorithm, determined using the area under the receiver operating characteristic curve (AUROC), was used for model development. Model performance was assessed using various learning metrics including AUROC.

RESULTS

Among 1819 patients, 251 (13.80 %) experienced 30-day readmission. The least absolute shrinkage and selection operator was used for feature extraction and model development. The 15 structured features were used in the traditional model. Moreover, five additional topic variables from the home healthcare notes were applied in the CDM-NLP model. The AUROC of the traditional model was 0.739 (95 % CI: 0.672-0.807). The AUROC of the CDM-NLP model was high at 0.824 (95 % CI: 0.768-0.880), which indicated an outstanding performance. The topics in the CDM-NLP model included emotional distress, daily living functions, nutrition, postoperative status, and cardiorespiratory issues. In extended prediction model development for 14- and 180-day readmissions, the CDM-NLP consistently outperformed the traditional model.

CONCLUSIONS

This study developed effective prediction models using both structured and unstructured data, thereby emphasizing the significance of postdischarge information from home healthcare notes in readmission prediction.

Prevalence of cognitive impairment in home health physical therapy

BACKGROUND

The prevalence of cognitive impairment in home health physical therapy (HHPT) is unknown. We sought to identify the prevalence of cognitive impairment, including cognitive impairment no dementia (CIND) and dementia, among older adults who used HHPT, and if cognitive impairment prevalence was higher among those with HHPT-relevant characteristics.

METHODS

For our cross-sectional analysis, we identified 963 fee-for-service Medicare beneficiaries with HHPT claims (>85 years old: 28.8%, women: 63.7%, non-Hispanic White: 82.1%) in the 2014 and 2016 waves of the Health and Retirement Study (HRS) and used a validated algorithm to categorize cognitive status as normal, CIND, or dementia. We estimated the population prevalence and calculated age, gender, race/ethnicity adjusted odds ratio (aOR) of CIND and dementia for characteristics relevant to HHPT service delivery including depression, walking difficulty, fall history, incontinence, moderate-vigorous physical activity (MVPA) ≤1x/week, and community-initiated HHPT using multinomial logistic regression.

RESULTS

The population prevalence of cognitive impairment was 46.4% (CIND: 27.3%, dementia: 19.1%). The prevalence of cognitive impairment was greater among those with depression (46.7% vs. 39.5%), difficulty walking across the room (58.9% vs. 41.8%), fall history (49.1% vs. 42.9%), MVPA ≤1x/week (50.0% vs. 38.0%), and community-initiated HHPT (55.2% vs. 40.2%). Compared to normal cognitive status, the odds of cognitive impairment were greater for those with MVPA≤1x/week (CIND: aOR = 1.57 [95% CI: 1.05-2.33], dementia: aOR = 2.55 [95% CI: 1.54-4.22]), depression (dementia: aOR = 1.99 [95% CI: 1.19-3.30]), difficulty walking across the room (dementia: aOR = 2.54 [95% CI: 1.40-4.60]), fall history (dementia: aOR = 1.85 [95% CI: 1.20-2.83]), and community-initiated HHPT (dementia: aOR = 1.72 (95% CI: 1.13-2.61]).

CONCLUSION

There is a high prevalence of CIND and dementia in HHPT, and no characteristics had a low prevalence of cognitive impairment. Physical therapists should be ready to identify cognitive impairment and adapt home health service delivery for this vulnerable population of older adults.

Identification of risk factors for the onset of delirium associated with COVID-19 by mining nursing records

COVID-19 has a range of complications, from no symptoms to severe pneumonia. It can also affect multiple organs including the nervous system. COVID-19 affects the brain, leading to neurological symptoms such as delirium. Delirium, a sudden change in consciousness, can increase the risk of death and prolong the hospital stay. However, research on delirium prediction in patients with COVID-19 is insufficient. This study aimed to identify new risk factors that could predict the onset of delirium in patients with COVID-19 using machine learning (ML) applied to nursing records. This retrospective cohort study used natural language processing and ML to develop a model for classifying the nursing records of patients with delirium. We extracted the features of each word from the model and grouped similar words. To evaluate the usefulness of word groups in predicting the occurrence of delirium in patients with COVID-19, we analyzed the temporal changes in the frequency of occurrence of these word groups before and after the onset of delirium. Moreover, the sensitivity, specificity, and odds ratios were calculated. We identified (1) elimination-related behaviors and conditions and (2) abnormal patient behavior and conditions as risk factors for delirium. Group 1 had the highest sensitivity (0.603), whereas group 2 had the highest specificity and odds ratio (0.938 and 6.903, respectively). These results suggest that these parameters may be useful in predicting delirium in these patients. The risk factors for COVID-19-associated delirium identified in this study were more specific but less sensitive than the ICDSC (Intensive Care Delirium Screening Checklist) and CAM-ICU (Confusion Assessment Method for the Intensive Care Unit). However, they are superior to the ICDSC and CAM-ICU because they can predict delirium without medical staff and at no cost.

Natural Language Processing Applied to Clinical Documentation in Post-acute Care Settings: A Scoping Review

OBJECTIVES

To determine the scope of the application of natural language processing to free-text clinical notes in post-acute care and provide a foundation for future natural language processing-based research in these settings.

DESIGN

Scoping review; reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews guidelines.

SETTING AND PARTICIPANTS

Post-acute care (ie, home health care, long-term care, skilled nursing facilities, and inpatient rehabilitation facilities).

METHODS

PubMed, Cumulative Index of Nursing and Allied Health Literature, and Embase were searched in February 2023. Eligible studies had quantitative designs that used natural language processing applied to clinical documentation in post-acute care settings. The quality of each study was appraised.

RESULTS

Twenty-one studies were included. Almost all studies were conducted in home health care settings. Most studies extracted data from electronic health records to examine the risk for negative outcomes, including acute care utilization, medication errors, and suicide mortality. About half of the studies did not report age, sex, race, or ethnicity data or use standardized terminologies. Only 8 studies included variables from socio-behavioral domains. Most studies fulfilled all quality appraisal indicators.

CONCLUSIONS AND IMPLICATIONS

The application of natural language processing is nascent in post-acute care settings. Future research should apply natural language processing using standardized terminologies to leverage free-text clinical notes in post-acute care to promote timely, comprehensive, and equitable care. Natural language processing could be integrated with predictive models to help identify patients who are at risk of negative outcomes. Future research should incorporate socio-behavioral determinants and diverse samples to improve health equity in informatics tools.

Social Risk Factors are Associated with Risk for Hospitalization in Home Health Care: A Natural Language Processing Study

OBJECTIVE

This study aimed to develop a natural language processing (NLP) system that identified social risk factors in home health care (HHC) clinical notes and to examine the association between social risk factors and hospitalization or an emergency department (ED) visit.

DESIGN

Retrospective cohort study.

SETTING AND PARTICIPANTS

We used standardized assessments and clinical notes from one HHC agency located in the northeastern United States. This included 86,866 episodes of care for 65,593 unique patients. Patients received HHC services between 2015 and 2017.

METHODS

Guided by HHC experts, we created a vocabulary of social risk factors that influence hospitalization or ED visit risk in the HHC setting. We then developed an NLP system to automatically identify social risk factors documented in clinical notes. We used an adjusted logistic regression model to examine the association between the NLP-based social risk factors and hospitalization or an ED visit.

RESULTS

On the basis of expert consensus, the following social risk factors emerged: Social Environment, Physical Environment, Education and Literacy, Food Insecurity, Access to Care, and Housing and Economic Circumstances. Our NLP system performed "very good" with an F score of 0.91. Approximately 4% of clinical notes (33% episodes of care) documented a social risk factor. The most frequently documented social risk factors were Physical Environment and Social Environment. Except for Housing and Economic Circumstances, all NLP-based social risk factors were associated with higher odds of hospitalization and ED visits.

CONCLUSIONS AND IMPLICATIONS

HHC clinicians assess and document social risk factors associated with hospitalizations and ED visits in their clinical notes. Future studies can explore the social risk factors documented in HHC to improve communication across the health care system and to predict patients at risk for being hospitalized or visiting the ED.

Natural language processing with machine learning methods to analyze unstructured patient-reported outcomes derived from electronic health records: A systematic review

OBJECTIVE

Natural language processing (NLP) combined with machine learning (ML) techniques are increasingly used to process unstructured/free-text patient-reported outcome (PRO) data available in electronic health records (EHRs). This systematic review summarizes the literature reporting NLP/ML systems/toolkits for analyzing PROs in clinical narratives of EHRs and discusses the future directions for the application of this modality in clinical care.

METHODS

We searched PubMed, Scopus, and Web of Science for studies written in English between 1/1/2000 and 12/31/2020. Seventy-nine studies meeting the eligibility criteria were included. We abstracted and summarized information related to the study purpose, patient population, type/source/amount of unstructured PRO data, linguistic features, and NLP systems/toolkits for processing unstructured PROs in EHRs.

RESULTS

Most of the studies used NLP/ML techniques to extract PROs from clinical narratives (n = 74) and mapped the extracted PROs into specific PRO domains for phenotyping or clustering purposes (n = 26). Some studies used NLP/ML to process PROs for predicting disease progression or onset of adverse events (n = 22) or developing/validating NLP/ML pipelines for analyzing unstructured PROs (n = 19). Studies used different linguistic features, including lexical, syntactic, semantic, and contextual features, to process unstructured PROs. Among the 25 NLP systems/toolkits we identified, 15 used rule-based NLP, 6 used hybrid NLP, and 4 used non-neural ML algorithms embedded in NLP.

CONCLUSIONS

This study supports the potential utility of different NLP/ML techniques in processing unstructured PROs available in EHRs for clinical care. Though using annotation rules for NLP/ML to analyze unstructured PROs is dominant, deploying novel neural ML-based methods is warranted.

HomeADScreen: Developing Alzheimer's disease and related dementia risk identification model in home healthcare

BACKGROUND

More than 50 % of patients with Alzheimer's disease and related dementia (ADRD) remain undiagnosed. This is specifically the case for home healthcare (HHC) patients.

OBJECTIVES

This study aimed at developing HomeADScreen, an ADRD risk screening model built on the combination of HHC patients' structured data and information extracted from HHC clinical notes.

METHODS

The study's sample included 15,973 HHC patients with no diagnosis of ADRD and 8,901 patients diagnosed with ADRD across four follow-up time windows. First, we applied two natural language processing methods, Word2Vec and topic modeling methods, to extract ADRD risk factors from clinical notes. Next, we built the risk identification model on the combination of the Outcome and Assessment Information Set (OASIS-structured data collected in the HHC setting) and clinical notes-risk factors across the four-time windows.

RESULTS

The top-performing machine learning algorithm attained an Area under the Curve = 0.76 for a four-year risk prediction time window. After optimizing the cut-off value for screening patients with ADRD (cut-off-value = 0.31), we achieved sensitivity = 0.75 and an F1-score = 0.63. For the first-year time window, adding clinical note-derived risk factors to OASIS data improved the overall performance of the risk identification model by 60 %. We observed a similar trend of increasing the model's overall performance across other time windows. Variables associated with increased risk of ADRD were "hearing impairment" and "impaired patient ability in the use of telephone." On the other hand, being "non-Hispanic White" and the "absence of impairment with prior daily functioning" were associated with a lower risk of ADRD.

CONCLUSION

HomeADScreen has a strong potential to be translated into clinical practice and assist HHC clinicians in assessing patients' cognitive function and referring them for further neurological assessment.

Prevalence and predictors of incident ADRD diagnosis following a Medicare home health episode

INTRODUCTION

Home health (HH) may be an important source of care for those with early-stage/undiagnosed Alzheimer's Disease and Related Dementias (ADRD), but little is known regarding prevalence or predictors of incident ADRD diagnosis following HH.

METHODS

Using 2010-2012 linked Master Beneficiary Summary File (MBSF) and HH assessment data for 40,596 Medicare HH patients, we model incident ADRD diagnosis within 1 year of HH via multivariable logistic regression.

RESULTS

Among HH patients without diagnosed ADRD, 10% received an incident diagnosis within 1 year. In adjusted models, patients were three times more likely to receive an incident ADRD diagnosis if they had HH clinician-reported impaired overall cognition (compared to patients without reported impairment) and twice as likely if they were community-referred (compared to hospital-referred patients).

DISCUSSION

There is a pressing need to develop tailored HH clinical pathways and protect access to community-referred HH to support community-living older adults with early-stage/undiagnosed ADRD.

Optimizing healthcare system by amalgamation of text processing and deep learning: a systematic review

The explosion of clinical textual data has drawn the attention of researchers. Owing to the abundance of clinical data, it is becoming difficult for healthcare professionals to take real-time measures. The tools and methods are lacking when compared to the amount of clinical data generated every day. This review aims to survey the text processing pipeline with deep learning methods such as CNN, RNN, LSTM, and GRU in the healthcare domain and discuss various applications such as clinical concept detection and extraction, medically aware dialogue systems, sentiment analysis of drug reviews shared online, clinical trial matching, and pharmacovigilance. In addition, we highlighted the major challenges in deploying text processing with deep learning to clinical textual data and identified the scope of research in this domain. Furthermore, we have discussed various resources that can be used in the future to optimize the healthcare domain by amalgamating text processing and deep learning.

The reporting of neuropsychiatric symptoms in electronic health records of individuals with Alzheimer's disease: a natural language processing study

BACKGROUND

Neuropsychiatric symptoms (NPS) are prevalent in the early clinical stages of Alzheimer's disease (AD) according to proxy-based instruments. Little is known about which NPS clinicians report and whether their judgment aligns with proxy-based instruments. We used natural language processing (NLP) to classify NPS in electronic health records (EHRs) to estimate the reporting of NPS in symptomatic AD at the memory clinic according to clinicians. Next, we compared NPS as reported in EHRs and NPS reported by caregivers on the Neuropsychiatric Inventory (NPI).

METHODS

Two academic memory clinic cohorts were used: the Amsterdam UMC (n = 3001) and the Erasmus MC (n = 646). Patients included in these cohorts had MCI, AD dementia, or mixed AD/VaD dementia. Ten trained clinicians annotated 13 types of NPS in a randomly selected training set of n = 500 EHRs from the Amsterdam UMC cohort and in a test set of n = 250 EHRs from the Erasmus MC cohort. For each NPS, a generalized linear classifier was trained and internally and externally validated. Prevalence estimates of NPS were adjusted for the imperfect sensitivity and specificity of each classifier. Intra-individual comparison of the NPS classified in EHRs and NPS reported on the NPI were conducted in a subsample (59%).

RESULTS

Internal validation performance of the classifiers was excellent (AUC range: 0.81-0.91), but external validation performance decreased (AUC range: 0.51-0.93). NPS were prevalent in EHRs from the Amsterdam UMC, especially apathy (adjusted prevalence = 69.4%), anxiety (adjusted prevalence = 53.7%), aberrant motor behavior (adjusted prevalence = 47.5%), irritability (adjusted prevalence = 42.6%), and depression (adjusted prevalence = 38.5%). The ranking of NPS was similar for EHRs from the Erasmus MC, although not all classifiers obtained valid prevalence estimates due to low specificity. In both cohorts, there was minimal agreement between NPS classified in the EHRs and NPS reported on the NPI (all kappa coefficients < 0.28), with substantially more reports of NPS in EHRs than on NPI assessments.

CONCLUSIONS

NLP classifiers performed well in detecting a wide range of NPS in EHRs of patients with symptomatic AD visiting the memory clinic and showed that clinicians frequently reported NPS in these EHRs. Clinicians generally reported more NPS in EHRs than caregivers reported on the NPI.

Can Patients with Dementia Be Identified in Primary Care Electronic Medical Records Using Natural Language Processing?

Dementia and mild cognitive impairment can be underrecognized in primary care practice and research. Free-text fields in electronic medical records (EMRs) are a rich source of information which might support increased detection and enable a better understanding of populations at risk of dementia. We used natural language processing (NLP) to identify dementia-related features in EMRs and compared the performance of supervised machine learning models to classify patients with dementia. We assembled a cohort of primary care patients aged 66 + years in Ontario, Canada, from EMR notes collected until December 2016: 526 with dementia and 44,148 without dementia. We identified dementia-related features by applying published lists, clinician input, and NLP with word embeddings to free-text progress and consult notes and organized features into thematic groups. Using machine learning models, we compared the performance of features to detect dementia, overall and during time periods relative to dementia case ascertainment in health administrative databases. Over 900 dementia-related features were identified and grouped into eight themes (including symptoms, social, function, cognition). Using notes from all time periods, LASSO had the best performance (F1 score: 77.2%, sensitivity: 71.5%, specificity: 99.8%). Model performance was poor when notes written before case ascertainment were included (F1 score: 14.4%, sensitivity: 8.3%, specificity 99.9%) but improved as later notes were added. While similar models may eventually improve recognition of cognitive issues and dementia in primary care EMRs, our findings suggest that further research is needed to identify which additional EMR components might be useful to promote early detection of dementia.

Supplementary Information

The online version contains supplementary material available at 10.1007/s41666-023-00125-6.

Developing a Classification Algorithm for Prediabetes Risk Detection From Home Care Nursing Notes: Using Natural Language Processing

This study developed and validated a rule-based classification algorithm for prediabetes risk detection using natural language processing from home care nursing notes. First, we developed prediabetes-related symptomatic terms in English and Korean. Second, we used natural language processing to preprocess the notes. Third, we created a rule-based classification algorithm with 31 484 notes, excluding 315 instances of missing data. The final algorithm was validated by measuring accuracy, precision, recall, and the F1 score against a gold standard testing set (400 notes). The developed terms comprised 11 categories and 1639 words in Korean and 1181 words in English. Using the rule-based classification algorithm, 42.2% of the notes comprised one or more prediabetic symptoms. The algorithm achieved high performance when applied to the gold standard testing set. We proposed a rule-based natural language processing algorithm to optimize the classification of the prediabetes risk group, depending on whether the home care nursing notes contain prediabetes-related symptomatic terms. Tokenization based on white space and the rule-based algorithm were brought into effect to detect the prediabetes symptomatic terms. Applying this algorithm to electronic health records systems will increase the possibility of preventing diabetes onset through early detection of risk groups and provision of tailored intervention.

Novel technical and privacy-preserving technology for artificial intelligence in ophthalmology

PURPOSE OF REVIEW

The application of artificial intelligence (AI) in medicine and ophthalmology has experienced exponential breakthroughs in recent years in diagnosis, prognosis, and aiding clinical decision-making. The use of digital data has also heralded the need for privacy-preserving technology to protect patient confidentiality and to guard against threats such as adversarial attacks. Hence, this review aims to outline novel AI-based systems for ophthalmology use, privacy-preserving measures, potential challenges, and future directions of each.

RECENT FINDINGS

Several key AI algorithms used to improve disease detection and outcomes include: Data-driven, imagedriven, natural language processing (NLP)-driven, genomics-driven, and multimodality algorithms. However, deep learning systems are susceptible to adversarial attacks, and use of data for training models is associated with privacy concerns. Several data protection methods address these concerns in the form of blockchain technology, federated learning, and generative adversarial networks.

SUMMARY

AI-applications have vast potential to meet many eyecare needs, consequently reducing burden on scarce healthcare resources. A pertinent challenge would be to maintain data privacy and confidentiality while supporting AI endeavors, where data protection methods would need to rapidly evolve with AI technology needs. Ultimately, for AI to succeed in medicine and ophthalmology, a balance would need to be found between innovation and privacy.

Clinical notes: An untapped opportunity for improving risk prediction for hospitalization and emergency department visit during home health care

BACKGROUND/OBJECTIVE

Between 10 and 25% patients are hospitalized or visit emergency department (ED) during home healthcare (HHC). Given that up to 40% of these negative clinical outcomes are preventable, early and accurate prediction of hospitalization risk can be one strategy to prevent them. In recent years, machine learning-based predictive modeling has become widely used for building risk models. This study aimed to compare the predictive performance of four risk models built with various data sources for hospitalization and ED visits in HHC.

METHODS

Four risk models were built using different variables from two data sources: structured data (i.e., Outcome and Assessment Information Set (OASIS) and other assessment items from the electronic health record (EHR)) and unstructured narrative-free text clinical notes for patients who received HHC services from the largest non-profit HHC organization in New York between 2015 and 2017. Then, five machine learning algorithms (logistic regression, Random Forest, Bayesian network, support vector machine (SVM), and Naïve Bayes) were used on each risk model. Risk model performance was evaluated using the F-score and Precision-Recall Curve (PRC) area metrics.

RESULTS

During the study period, 8373/86,823 (9.6%) HHC episodes resulted in hospitalization or ED visits. Among five machine learning algorithms on each model, the SVM showed the highest F-score (0.82), while the Random Forest showed the highest PRC area (0.864). Adding information extracted from clinical notes significantly improved the risk prediction ability by up to 16.6% in F-score and 17.8% in PRC.

CONCLUSION

All models showed relatively good hospitalization or ED visit risk predictive performance in HHC. Information from clinical notes integrated with the structured data improved the ability to identify patients at risk for these emergent care events.

Sensory Processing Abnormalities in Community-Dwelling Older Adults with Cognitive Impairment: A Mixed Methods Study

Mild cognitive impairment (MCI) or dementia often leads to behavioral and psychiatric symptoms of dementia (BPSD). Sensory processing abnormalities may be associated with BPSD. The purpose of this study was to explore relationships among sensory processing, behavior, and environmental features within the homes of people with MCI or dementia. This project used mixed methods to assess participants' sensory processing, care partner perspectives on behaviors, and in situ observations of the home environment. Nine participants with cognitive impairment (MCI n = 8, early dementia = 1) and their care partners were included. Seven participants with cognitive impairment were reported to have abnormal sensory processing. Findings suggest that unique environmental adaptations, tailored to personal and sensory preferences for each participant, were associated with a decreased level of behavioral disruption during the observation periods. Implementing sensory-based approaches to maximize environment adaptation may be beneficial in reducing disruptive behaviors for adults with cognitive impairment.

Exploring Reasons for Delayed Start-of-Care Nursing Visits in Home Health Care: Algorithm Development and Data Science Study

BACKGROUND

Delayed start-of-care nursing visits in home health care (HHC) can result in negative outcomes, such as hospitalization. No previous studies have investigated why start-of-care HHC nursing visits are delayed, in part because most reasons for delayed visits are documented in free-text HHC nursing notes.

OBJECTIVE

The aims of this study were to (1) develop and test a natural language processing (NLP) algorithm that automatically identifies reasons for delayed visits in HHC free-text clinical notes and (2) describe reasons for delayed visits in a large patient sample.

METHODS

This study was conducted at the Visiting Nurse Service of New York (VNSNY). We examined data available at the VNSNY on all new episodes of care started in 2019 (N=48,497). An NLP algorithm was developed and tested to automatically identify and classify reasons for delayed visits.

RESULTS

The performance of the NLP algorithm was 0.8, 0.75, and 0.77 for precision, recall, and F-score, respectively. A total of one-third of HHC episodes (n=16,244) had delayed start-of-care HHC nursing visits. The most prevalent identified category of reasons for delayed start-of-care nursing visits was no answer at the door or phone (3728/8051, 46.3%), followed by patient/family request to postpone or refuse some HHC services (n=2858, 35.5%), and administrative or scheduling issues (n=1465, 18.2%). In 40% (n=16,244) of HHC episodes, 2 or more reasons were documented.

CONCLUSIONS

To avoid critical delays in start-of-care nursing visits, HHC organizations might examine and improve ways to effectively address the reasons for delayed visits, using effective interventions, such as educating patients or caregivers on the importance of a timely nursing visit and improving patients' intake procedures.

Use of Machine Learning and Artificial Intelligence Methods in Geriatric Mental Health Research Involving Electronic Health Record or Administrative Claims Data: A Systematic Review

Introduction: Electronic health records (EHR) and administrative healthcare data (AHD) are frequently used in geriatric mental health research to answer various health research questions. However, there is an increasing amount and complexity of data available that may lend itself to alternative analytic approaches using machine learning (ML) or artificial intelligence (AI) methods. We performed a systematic review of the current application of ML or AI approaches to the analysis of EHR and AHD in geriatric mental health. Methods: We searched MEDLINE, Embase, and PsycINFO to identify potential studies. We included all articles that used ML or AI methods on topics related to geriatric mental health utilizing EHR or AHD data. We assessed study quality either by Prediction model Risk OF Bias ASsessment Tool (PROBAST) or Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) checklist. Results: We initially identified 391 articles through an electronic database and reference search, and 21 articles met inclusion criteria. Among the selected studies, EHR was the most used data type, and the datasets were mainly structured. A variety of ML and AI methods were used, with prediction or classification being the main application of ML or AI with the random forest as the most common ML technique. Dementia was the most common mental health condition observed. The relative advantages of ML or AI techniques compared to biostatistical methods were generally not assessed. Only in three studies, low risk of bias (ROB) was observed according to all the PROBAST domains but in none according to QUADAS-2 domains. The quality of study reporting could be further improved. Conclusion: There are currently relatively few studies using ML and AI in geriatric mental health research using EHR and AHD methods, although this field is expanding. Aside from dementia, there are few studies of other geriatric mental health conditions. The lack of consistent information in the selected studies precludes precise comparisons between them. Improving the quality of reporting of ML and AI work in the future would help improve research in the field. Other courses of improvement include using common data models to collect/organize data, and common datasets for ML model validation.

Estimation of Psychological Distress in Japanese Youth Through Narrative Writing: Text-Based Stylometric and Sentiment Analyses

Background

Internalizing mental illnesses associated with psychological distress are often underdetected. Text-based detection using natural language processing (NLP) methods is increasingly being used to complement conventional detection efforts. However, these approaches often rely on self-disclosure through autobiographical narratives that may not always be possible, especially in the context of the collectivistic Japanese culture.

Objective

We propose the use of narrative writing as an alternative resource for mental illness detection in youth. Accordingly, in this study, we investigated the textual characteristics of narratives written by youth with psychological distress; our research focuses on the detection of psychopathological tendencies in written imaginative narratives.

Methods

Using NLP tools such as stylometric measures and lexicon-based sentiment analysis, we examined short narratives from 52 Japanese youth (mean age 19.8 years, SD 3.1) obtained through crowdsourcing. Participants wrote a short narrative introduction to an imagined story before completing a questionnaire to quantify their tendencies toward psychological distress. Based on this score, participants were categorized into higher distress and lower distress groups. The written narratives were then analyzed using NLP tools and examined for between-group differences. Although outside the scope of this study, we also carried out a supplementary analysis of narratives written by adults using the same procedure.

Results

Youth demonstrating higher tendencies toward psychological distress used significantly more positive (happiness-related) words, revealing differences in valence of the narrative content. No other significant differences were observed between the high and low distress groups.

Conclusions

Youth with tendencies toward mental illness were found to write more positive stories that contained more happiness-related terms. These results may potentially have widespread implications on psychological distress screening on online platforms, particularly in cultures such as Japan that are not accustomed to self-disclosure. Although the mechanisms that we propose in explaining our results are speculative, we believe that this interpretation paves the way for future research in online surveillance and detection efforts.