A Novel Model for Predicting Rehospitalization Risk Incorporating Physical Function, Cognitive Status, and Psychosocial Support Using Natural Language Processing

Classifying Individuals With Rheumatic Conditions as Financially Insecure Using Electronic Health Record Data and Natural Language Processing: Algorithm Derivation and Validation

OBJECTIVE

We aimed to examine the feasibility of applying natural language processing (NLP) to unstructured electronic health record (EHR) documents to detect the presence of financial insecurity among patients with rheumatologic disease enrolled in an integrated care management program (iCMP).

METHODS

We incorporated supervised, rule-based NLP and statistical methods to identify financial insecurity among patients with rheumatic conditions enrolled in an iCMP (n = 20,395) in a multihospital EHR system. We constructed a lexicon for financial insecurity using data from available knowledge sources and then reviewed EHR notes from 538 randomly selected individuals (training cohort n = 366, validation cohort n = 172). We manually categorized records as having "definite," "possible," or "no" mention of financial insecurity. All available notes were processed using Narrative Information Linear Extraction, a rule-based version of NLP. Models were trained using the NLP features for financial insecurity using logistic, least absolute shrinkage operator (LASSO), and random forest performance characteristic and were compared with the reference standard.

RESULTS

A total of 245,142 notes were processed from 538 individual patient records. Financial insecurity was present among 100 (27%) individuals in the training cohort and 63 (37%) in the validation cohort. The LASSO and random forest models performed identically and slightly better than logistic regression, with positive predictive values of 0.90, sensitivities of 0.29, and specificities of 0.98.

CONCLUSION

The development of a context-driven lexicon used with rule-based NLP to extract data that identify financial insecurity is feasible for use and improved the capture for presence of financial insecurity with high accuracy. In the absence of a standard lexicon and construct definition for financial insecurity status, additional studies are needed to optimize the sensitivity of algorithms to categorize financial insecurity with construct validity.

Large language models to identify social determinants of health in electronic health records

Social determinants of health (SDoH) play a critical role in patient outcomes, yet their documentation is often missing or incomplete in the structured data of electronic health records (EHRs). Large language models (LLMs) could enable high-throughput extraction of SDoH from the EHR to support research and clinical care. However, class imbalance and data limitations present challenges for this sparsely documented yet critical information. Here, we investigated the optimal methods for using LLMs to extract six SDoH categories from narrative text in the EHR: employment, housing, transportation, parental status, relationship, and social support. The best-performing models were fine-tuned Flan-T5 XL for any SDoH mentions (macro-F1 0.71), and Flan-T5 XXL for adverse SDoH mentions (macro-F1 0.70). Adding LLM-generated synthetic data to training varied across models and architecture, but improved the performance of smaller Flan-T5 models (delta F1 + 0.12 to +0.23). Our best-fine-tuned models outperformed zero- and few-shot performance of ChatGPT-family models in the zero- and few-shot setting, except GPT4 with 10-shot prompting for adverse SDoH. Fine-tuned models were less likely than ChatGPT to change their prediction when race/ethnicity and gender descriptors were added to the text, suggesting less algorithmic bias (p < 0.05). Our models identified 93.8% of patients with adverse SDoH, while ICD-10 codes captured 2.0%. These results demonstrate the potential of LLMs in improving real-world evidence on SDoH and assisting in identifying patients who could benefit from resource support.

Natural language processing to identify case factors in child protection court proceedings

Social work case files hold rich detail about the lives and needs of vulnerable groups. Traditional case-reading studies to gain generalisable knowledge are resource-intensive, however, and sample sizes thereby limited. The advent of ‘big data’ technology, and vast repositories of centrally stored electronic records offer social work researchers novel alternatives, including data linkage and predictive risk modelling using administrative data. Free-text documents, however – including assessments, reports, and case chronologies – remain a largely untapped resource. This paper describes how 5000 social work court statements held by the Child and Family Court Advisory Support Service in England (Cafcass) were analysed using natural language processing (NLP) based on simple rules and mathematical principles. Thirteen factors relating to harm and risk to children involved in care proceedings in England were identified by automated computer techniques, and almost 90% agreement with professional readers achieved when the factors were clear-cut. The study represents an innovative approach for social work research on complex social problems. In conclusion, the paper discusses learning points; practical implications; future research avenues; and the technical and ethical challenges of NLP.

Prediction of Future Health Care Utilization Through Note-extracted Psychosocial Factors

BACKGROUND

Persons with multimorbidity (≥2 chronic conditions) face an increased risk of poor health outcomes, especially as they age. Psychosocial factors such as social isolation, chronic stress, housing insecurity, and financial insecurity have been shown to exacerbate these outcomes, but are not routinely assessed during the clinical encounter. Our objective was to extract these concepts from chart notes using natural language processing and predict their impact on health care utilization for patients with multimorbidity.

METHODS

A cohort study to predict the 1-year likelihood of hospitalizations and emergency department visits for patients 65+ with multimorbidity with and without psychosocial factors. Psychosocial factors were extracted from narrative notes; all other covariates were extracted from electronic health record data from a large academic medical center using validated algorithms and concept sets. Logistic regression was performed to predict the likelihood of hospitalization and emergency department visit in the next year.

RESULTS

In all, 76,479 patients were eligible; the majority were White (89%), 54% were female, with mean age 73. Those with psychosocial factors were older, had higher baseline utilization, and more chronic illnesses. The 4 psychosocial factors all independently predicted future utilization (odds ratio=1.27-2.77, C -statistic=0.63). Accounting for demographics, specific conditions, and previous utilization, 3 of 4 of the extracted factors remained predictive (odds ratio=1.13-1.86) for future utilization. Compared with models with no psychosocial factors, they had improved discrimination. Individual predictions were mixed, with social isolation predicting depression and morbidity; stress predicting atherosclerotic cardiovascular disease onset; and housing insecurity predicting substance use disorder morbidity.

DISCUSSION

Psychosocial factors are known to have adverse health impacts, but are rarely measured; using natural language processing, we extracted factors that identified a higher risk segment of older adults with multimorbidity. Combining these extraction techniques with other measures of social determinants may help catalyze population health efforts to address psychosocial factors to mitigate their health impacts.

Classifying Social Determinants of Health from Unstructured Electronic Health Records Using Deep Learning-based Natural Language Processing

OBJECTIVE

Social determinants of health (SDOH) are non-medical factors that can profoundly impact patient health outcomes. However, SDOH are rarely available in structured electronic health record (EHR) data such as diagnosis codes, and more commonly found in unstructured narrative clinical notes. Hence, identifying social context from unstructured EHR data has become increasingly important. Yet, previous work on using natural language processing to automate extraction of SDOH from text (a) usually focuses on an ad hoc selection of SDOH, and (b) does not use the latest advances in deep learning. Our objective was to advance automatic extraction of SDOH from clinical text by (a) systematically creating a set of SDOH based on standard biomedical and psychiatric ontologies, and (b) training state-of-the-art deep neural networks to extract mentions of these SDOH from clinical notes.

DESIGN

A retrospective cohort study.

SETTING AND PARTICIPANTS

Data were extracted from the Medical Information Mart for Intensive Care (MIMIC-III) database. The corpus comprised 3,504 social related sentences from 2,670 clinical notes.

METHODS

We developed a framework for automated classification of multiple SDOH categories. Our dataset comprised narrative clinical notes under the "Social Work" category in the MIMIC-III Clinical Database. Using standard terminologies, SNOMED-CT and DSM-IV, we systematically curated a set of 13 SDOH categories and created annotation guidelines for these. After manually annotating the 3,504 sentences, we developed and tested three deep neural network (DNN) architectures - convolutional neural network (CNN), long short-term memory (LSTM) network, and the Bidirectional Encoder Representations from Transformers (BERT) - for automated detection of eight SDOH categories. We also compared these DNNs to three baselines models: (1) cTAKES, as well as (2) L2-regularized logistic regression and (3) random forests on bags-of-words. Model evaluation metrics included micro- and macro- F1, and area under the receiver operating characteristic curve (AUC).

RESULTS

All three DNN models accurately classified all SDOH categories (minimum micro-F1 =.632, minimum macro-AUC=.854). Compared to the CNN and LSTM, BERT performed best in most key metrics (micro-F1 = 0.690, macro-AUC=0.907). The BERT model most effectively identified the "occupational" category (F1=.774, AUC=.965) and least effectively identified the "non-SDOH" category (F=.491, AUC=.788). BERT outperformed cTAKES in distinguishing social vs non-social sentences (BERT F1 = .87 vs. cTAKES F1=.06), and outperformed logistic regression (micro-F1=0.649, macro-AUC=0.696) and random forest (micro-F1=0.502, macro-AUC=0.523) trained on bag-of-words.

CONCLUSIONS

Our study framework with DNN models demonstrated improved performance for efficiently identifying a systematic range of SDOH categories from clinical notes in the EHR. Improved identification of patient SDOH may further improve healthcare outcomes.

Extracting social determinants of health from electronic health records using natural language processing: a systematic review

OBJECTIVE

Social determinants of health (SDoH) are nonclinical dispositions that impact patient health risks and clinical outcomes. Leveraging SDoH in clinical decision-making can potentially improve diagnosis, treatment planning, and patient outcomes. Despite increased interest in capturing SDoH in electronic health records (EHRs), such information is typically locked in unstructured clinical notes. Natural language processing (NLP) is the key technology to extract SDoH information from clinical text and expand its utility in patient care and research. This article presents a systematic review of the state-of-the-art NLP approaches and tools that focus on identifying and extracting SDoH data from unstructured clinical text in EHRs.

MATERIALS AND METHODS

A broad literature search was conducted in February 2021 using 3 scholarly databases (ACL Anthology, PubMed, and Scopus) following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. A total of 6402 publications were initially identified, and after applying the study inclusion criteria, 82 publications were selected for the final review.

RESULTS

Smoking status (n = 27), substance use (n = 21), homelessness (n = 20), and alcohol use (n = 15) are the most frequently studied SDoH categories. Homelessness (n = 7) and other less-studied SDoH (eg, education, financial problems, social isolation and support, family problems) are mostly identified using rule-based approaches. In contrast, machine learning approaches are popular for identifying smoking status (n = 13), substance use (n = 9), and alcohol use (n = 9).

CONCLUSION

NLP offers significant potential to extract SDoH data from narrative clinical notes, which in turn can aid in the development of screening tools, risk prediction models, and clinical decision support systems.

Association Rules Mining for Hospital Readmission: A Case Study

As an indicator of healthcare quality and performance, hospital readmission incurs major costs for healthcare systems worldwide. Understanding the relationships between readmission factors, such as input features and readmission length, is challenging following intricate hospital readmission procedures. This study discovered the significant correlation between potential readmission factors (threshold of various settings for readmission length) and basic demographic variables. Association rule mining (ARM), particularly the Apriori algorithm, was utilised to extract the hidden input variable patterns and relationships among admitted patients by generating supervised learning rules. The mined rules were categorised into two outcomes to comprehend readmission data; (i) the rules associated with various readmission length and (ii) several expert-validated variables related to basic demographics (gender, race, and age group). The extracted rules proved useful to facilitate decision-making and resource preparation to minimise patient readmission.

Predictive modelling of hospital readmission: Evaluation of different preprocessing techniques on machine learning classifiers

Hospital readmission is a major cost for healthcare systems worldwide. If patients with a higher potential of readmission could be identified at the start, existing resources could be used more efficiently, and appropriate plans could be implemented to reduce the risk of readmission. Therefore, it is important to predict the right target patients. Medical data is usually noisy, incomplete, and inconsistent. Hence, before developing a prediction model, it is crucial to efficiently set up the predictive model so that improved predictive performance is achieved. The current study aims to analyse the impact of different preprocessing methods on the performance of different machine learning classifiers. The preprocessing applied by previous hospital readmission studies were compared, and the most common approaches highlighted such as missing value imputation, feature selection, data balancing, and feature scaling. The hyperparameters were selected using Bayesian optimisation. The different preprocessing pipelines were assessed using various performance metrics and computational costs. The results indicated that the preprocessing approaches helped improve the model’s prediction of hospital readmission.

Social and Behavioral Determinants of Health in the Era of Artificial Intelligence with Electronic Health Records: A Scoping Review

Background. There is growing evidence that social and behavioral determinants of health (SBDH) play a substantial effect in a wide range of health outcomes. Electronic health records (EHRs) have been widely employed to conduct observational studies in the age of artificial intelligence (AI). However, there has been limited review into how to make the most of SBDH information from EHRs using AI approaches.Methods. A systematic search was conducted in six databases to find relevant peer-reviewed publications that had recently been published. Relevance was determined by screening and evaluating the articles. Based on selected relevant studies, a methodological analysis of AI algorithms leveraging SBDH information in EHR data was provided.Results. Our synthesis was driven by an analysis of SBDH categories, the relationship between SBDH and healthcare-related statuses, natural language processing (NLP) approaches for extracting SBDH from clinical notes, and predictive models using SBDH for health outcomes.Discussion. The associations between SBDH and health outcomes are complicated and diverse; several pathways may be involved. Using NLP technology to support the extraction of SBDH and other clinical ideas simplifies the identification and extraction of essential concepts from clinical data, efficiently unlocks unstructured data, and aids in the resolution of unstructured data-related issues.Conclusion. Despite known associations between SBDH and diseases, SBDH factors are rarely investigated as interventions to improve patient outcomes. Gaining knowledge about SBDH and how SBDH data can be collected from EHRs using NLP approaches and predictive models improves the chances of influencing health policy change for patient wellness, ultimately promoting health and health equity.

Adaptation of an NLP system to a new healthcare environment to identify social determinants of health

Social determinants of health (SDoH) are increasingly important factors for population health, healthcare outcomes, and care delivery. However, many of these factors are not reliably captured within structured electronic health record (EHR) data. In this work, we evaluated and adapted a previously published NLP tool to include additional social risk factors for deployment at Vanderbilt University Medical Center in an Acute Myocardial Infarction cohort. We developed a transformation of the SDoH outputs of the tool into the OMOP common data model (CDM) for re-use across many potential use cases, yielding performance measures across 8 SDoH classes of precision 0.83 recall 0.74 and F-measure of 0.78.

Differences in Biologic Utilization and Surgery Rates in Pediatric and Adult Crohn's Disease: Results From a Large Electronic Medical Record-derived Cohort

BACKGROUND AND AIMS

Crohn's disease (CD) is a chronic illness that affects both the pediatric and adult populations with an increasing worldwide prevalence. We aim to identify a large, single-center cohort of patients with CD using natural language processing (NLP) in combination with codified data and extract surgical rates and medication usage from the electronic medical record (EMR).

METHODS

Patients with CD were identified from the entire Cleveland Clinic EMR using ICD codes and CD-specific terms identified by NLP to fit a logistic regression model. Cohorts were developed for pediatric-onset (younger than 18 years) and adult-onset (18 years and older) CD. Surgeries were identified using current procedural terminology (CPT) codes and NLP. Crohn's disease-related medications were extracted using physician orders in the EMR.

RESULTS

Patients with pediatric-onset (n = 2060) and adult-onset (n = 4973) CD were identified from 2000 to 2017 with a positive predictive value of 98.5%. Rate of CD-related abdominal surgery over time was significantly higher in adult-onset compared with pediatric-onset CD (10-year surgery rate 49.9% vs 37.7%, respectively; P < 0.001). Treatment with biologics was significantly higher in pediatric vs adult-onset CD cohorts (63.6% vs 49.2%; P < 0.001). The overall rate of CD-related abdominal surgery was significantly higher in those who received <6 months of a biologic compared with ≥6 months of a biologic for both cohorts (pediatric 64.1% vs 39.1%, P ≤ 0.001; adult 69.3% vs 56.5%, P ≤ 0.001). Additionally, 60.9% in pediatric-onset CD and 43.5% in adult-onset CD treated with ≥6 months of biologic therapy have not required abdominal surgery. On multivariable analysis, perianal surgery was a significant risk factor for abdominal surgery in both cohorts.

CONCLUSION

We used a combination of codified and NLP data to establish the largest, North American, single-center EMR cohort of pediatric- and adult-onset CD patients and determined that biologics are associated with lower rates of surgery over time, potentially altering the natural history of the disease.

Development of a predictive model for retention in HIV care using natural language processing of clinical notes

OBJECTIVE

Adherence to a treatment plan from HIV-positive patients is necessary to decrease their mortality and improve their quality of life, however some patients display poor appointment adherence and become lost to follow-up (LTFU). We applied natural language processing (NLP) to analyze indications towards or against LTFU in HIV-positive patients' notes.

MATERIALS AND METHODS

Unstructured lemmatized notes were labeled with an LTFU or Retained status using a 183-day threshold. An NLP and supervised machine learning system with a linear model and elastic net regularization was trained to predict this status. Prevalence of characteristics domains in the learned model weights were evaluated.

RESULTS

We analyzed 838 LTFU vs 2964 Retained notes and obtained a weighted F1 mean of 0.912 via nested cross-validation; another experiment with notes from the same patients in both classes showed substantially lower metrics. "Comorbidities" were associated with LTFU through, for instance, "HCV" (hepatitis C virus) and likewise "Good adherence" with Retained, represented with "Well on ART" (antiretroviral therapy).

DISCUSSION

Mentions of mental health disorders and substance use were associated with disparate retention outcomes, however history vs active use was not investigated. There remains further need to model transitions between LTFU and being retained in care over time.

CONCLUSION

We provided an important step for the future development of a model that could eventually help to identify patients who are at risk for falling out of care and to analyze which characteristics could be factors for this. Further research is needed to enhance this method with structured electronic medical record fields.

A comprehensive study of mobility functioning information in clinical notes: Entity hierarchy, corpus annotation, and sequence labeling

BACKGROUND

Secondary use of Electronic Health Records (EHRs) has mostly focused on health conditions (diseases and drugs). Function is an important health indicator in addition to morbidity and mortality. Nevertheless, function has been overlooked in accessing patients' health status. The World Health Organization (WHO)'s International Classification of Functioning, Disability and Health (ICF) is considered the international standard for describing and coding function and health states. We pioneer the first comprehensive analysis and identification of functioning concepts in the Mobility domain of the ICF.

RESULTS

Using physical therapy notes at the National Institutes of Health's Clinical Center, we induced a hierarchical order of mobility-related entities including 5 entities types, 3 relations, 8 attributes, and 33 attribute values. Two domain experts manually curated a gold standard corpus of 14,281 nested entity mentions from 400 clinical notes. Inter-annotator agreement (IAA) of exact matching averaged 92.3 % F1-score on mention text spans, and 96.6 % Cohen's kappa on attributes assignments. A high-performance Ensemble machine learning model for named entity recognition (NER) was trained and evaluated using the gold standard corpus. Average F1-score on exact entity matching of our Ensemble method (84.90 %) outperformed popular NER methods: Conditional Random Field (80.4 %), Recurrent Neural Network (81.82 %), and Bidirectional Encoder Representations from Transformers (82.33 %).

CONCLUSIONS

The results of this study show that mobility functioning information can be reliably captured from clinical notes once adequate resources are provided for sequence labeling methods. We expect that functioning concepts in other domains of the ICF can be identified in similar fashion.

Automated Coding of Under-Studied Medical Concept Domains: Linking Physical Activity Reports to the International Classification of Functioning, Disability, and Health

Linking clinical narratives to standardized vocabularies and coding systems is a key component of unlocking the information in medical text for analysis. However, many domains of medical concepts, such as functional outcomes and social determinants of health, lack well-developed terminologies that can support effective coding of medical text. We present a framework for developing natural language processing (NLP) technologies for automated coding of medical information in under-studied domains, and demonstrate its applicability through a case study on physical mobility function. Mobility function is a component of many health measures, from post-acute care and surgical outcomes to chronic frailty and disability, and is represented as one domain of human activity in the International Classification of Functioning, Disability, and Health (ICF). However, mobility and other types of functional activity remain under-studied in the medical informatics literature, and neither the ICF nor commonly-used medical terminologies capture functional status terminology in practice. We investigated two data-driven paradigms, classification and candidate selection, to link narrative observations of mobility status to standardized ICF codes, using a dataset of clinical narratives from physical therapy encounters. Recent advances in language modeling and word embedding were used as features for established machine learning models and a novel deep learning approach, achieving a macro-averaged F-1 score of 84% on linking mobility activity reports to ICF codes. Both classification and candidate selection approaches present distinct strengths for automated coding in under-studied domains, and we highlight that the combination of (i) a small annotated data set; (ii) expert definitions of codes of interest; and (iii) a representative text corpus is sufficient to produce high-performing automated coding systems. This research has implications for continued development of language technologies to analyze functional status information, and the ongoing growth of NLP tools for a variety of specialized applications in clinical care and research.

Hierarchical analysis of factors associated with hospital readmissions for coronary heart disease: A case-control study

AIMS AND OBJECTIVES

To analyse, hierarchically, factors associated with hospital readmissions for acute coronary syndrome.

BACKGROUND

Hospital readmissions have risen, especially in patients with multiple comorbidities, which are most often chronic. The leading causes of hospital readmission include acute coronary syndrome, which is costly and often preventable. Determining clinical and nonclinical variables that increase the chances of readmission is important to assess and evaluate patients hospitalised for coronary heart diseases.

DESIGN

A case-control study whose dependent variable was hospital readmission for acute coronary syndrome.

METHODS

The study included 277 inpatients, of whom 132 were in their first hospitalisation and 145 had already been hospitalised for acute coronary syndrome. The independent variables for this hierarchical model were sociodemographic conditions, life habits, access to health services and physical health measures. Data were obtained by interviews, anthropometric measurements and patient records. Logistic regression analysis was performed using the stepwise technique, with Microsoft Excel and R version 3.2.3. The research was reported via strengthening the reporting of observational studies in epidemiology (STROBE).

RESULTS

In the final hierarchical logistic model, the following risk factors were associated with readmission for acute coronary syndrome: inadequate drug therapy adherence, stress, history of smoking for 30 years or more, and the lack of use of primary healthcare services.

CONCLUSIONS

Clinical and nonclinical variables are related to hospital readmission for acute coronary syndrome and can increase the chance of readmission by up to six times.

RELEVANCE TO CLINICAL PRACTICE

The predictive model can be used to avoid readmission for acute coronary syndrome, and it represents an advance in the prediction of the occurrence of the outcome. This implies the need for a reorientation of the network for postdischarge care in the first hospitalisation for acute coronary syndrome.

Natural language processing of symptoms documented in free-text narratives of electronic health records: a systematic review

OBJECTIVE

Natural language processing (NLP) of symptoms from electronic health records (EHRs) could contribute to the advancement of symptom science. We aim to synthesize the literature on the use of NLP to process or analyze symptom information documented in EHR free-text narratives.

MATERIALS AND METHODS

Our search of 1964 records from PubMed and EMBASE was narrowed to 27 eligible articles. Data related to the purpose, free-text corpus, patients, symptoms, NLP methodology, evaluation metrics, and quality indicators were extracted for each study.

RESULTS

Symptom-related information was presented as a primary outcome in 14 studies. EHR narratives represented various inpatient and outpatient clinical specialties, with general, cardiology, and mental health occurring most frequently. Studies encompassed a wide variety of symptoms, including shortness of breath, pain, nausea, dizziness, disturbed sleep, constipation, and depressed mood. NLP approaches included previously developed NLP tools, classification methods, and manually curated rule-based processing. Only one-third (n = 9) of studies reported patient demographic characteristics.

DISCUSSION

NLP is used to extract information from EHR free-text narratives written by a variety of healthcare providers on an expansive range of symptoms across diverse clinical specialties. The current focus of this field is on the development of methods to extract symptom information and the use of symptom information for disease classification tasks rather than the examination of symptoms themselves.

CONCLUSION

Future NLP studies should concentrate on the investigation of symptoms and symptom documentation in EHR free-text narratives. Efforts should be undertaken to examine patient characteristics and make symptom-related NLP algorithms or pipelines and vocabularies openly available.

Clinical and Sociobehavioral Prediction Model of 30-Day Hospital Readmissions Among People With HIV and Substance Use Disorder: Beyond Electronic Health Record Data

BACKGROUND

Under the Affordable Care Act, hospitals receive reduced reimbursements for excessive 30-day readmissions. However, the Centers for Medicare and Medicaid Services does not consider social and behavioral variables in expected readmission rate calculations, which may unfairly penalize systems caring for socially disadvantaged patients, including patients with HIV.

SETTING

Randomized controlled trial of patient navigation with or without financial incentives in HIV-positive substance users recruited from the inpatient setting at 11 US hospitals.

METHODS

External validation of an existing 30-day readmission prediction model, using variables available in the electronic health record (EHR-only model), in a new multicenter cohort of HIV-positive substance users was assessed by C-statistic and Hosmer-Lemeshow testing. A second model evaluated sociobehavioral factors in improving the prediction model (EHR-plus model) using multivariable regression and C-statistic with cross-validation.

RESULTS

The mean age of the cohort was 44.1 years, and participants were predominantly males (67.4%), non-white (88.0%), and poor (62.8%, <$20,000/year). Overall, 17.5% individuals had a hospital readmission within 30 days of initial hospital discharge. The EHR-only model resulted in a C-statistic of 0.65 (95% confidence interval: 0.60 to 0.70). Inclusion of additional sociobehavioral variables, food insecurity and readiness for substance use treatment, in the EHR-plus model resulted in a C-statistic of 0.74 (0.71 after cross-validation, 95% confidence interval: 0.64 to 0.77).

CONCLUSIONS

Incorporation of detailed social and behavioral variables substantially improved the performance of a 30-day readmission prediction model for hospitalized HIV-positive substance users. Our findings highlight the importance of social determinants in readmission risk and the need to ask about, adjust for, and address them.

Validity of early-onset dementia diagnoses in VA electronic medical record administrative data

OBJECTIVE

To determine the validity of diagnoses indicative of early-onset dementia (EOD) obtained from an algorithm using administrative data, we examined Veterans Health Administration (VHA) electronic medical records (EMRs).

METHOD

A previously used method of identifying cases of dementia using administrative data was applied to a random sample of 176 cases of Post-9/11 deployed veterans under 65 years of age. Retrospective, cross-sectional examination of EMRs was conducted, using a combination of administrative data, chart abstraction, and review/consensus by board-certified neuropsychologists.

RESULTS

Approximately 73% of EOD diagnoses identified using existing algorithms were identified as false positives in the overall sample. This increased to approximately 76% among those with mental health conditions and approximately 85% among those with mild traumatic brain injury (TBI; i.e. concussion). Factors related to improved diagnostic accuracy included more severe TBI, diagnosing clinician type, presence of neuroimaging data, absence of a comorbid mental health condition diagnosis, and older age at time of diagnosis.

CONCLUSIONS

A previously used algorithm for detecting dementia using VHA administrative data was not supported for use in the younger adult samples and resulted in an unacceptably high number of false positives. Based on these findings, there is concern for possible misclassification in population studies using similar algorithms to identify rates of EOD among veterans. Further, we provide suggestions to develop an enhanced algorithm for more accurate dementia surveillance among younger populations.

Broadening horizons: the case for capturing function and the role of health informatics in its use

Background

Human activity and the interaction between health conditions and activity is a critical part of understanding the overall function of individuals. The World Health Organization’s International Classification of Functioning, Disability and Health (ICF) models function as all aspects of an individual’s interaction with the world, including organismal concepts such as individual body structures, functions, and pathologies, as well as the outcomes of the individual’s interaction with their environment, referred to as activity and participation. Function, particularly activity and participation outcomes, is an important indicator of health at both the level of an individual and the population level, as it is highly correlated with quality of life and a critical component of identifying resource needs. Since it reflects the cumulative impact of health conditions on individuals and is not disease specific, its use as a health indicator helps to address major barriers to holistic, patient-centered care that result from multiple, and often competing, disease specific interventions. While the need for better information on function has been widely endorsed, this has not translated into its routine incorporation into modern health systems.

Purpose

We present the importance of capturing information on activity as a core component of modern health systems and identify specific steps and analytic methods that can be used to make it more available to utilize in improving patient care. We identify challenges in the use of activity and participation information, such as a lack of consistent documentation and diversity of data specificity and representation across providers, health systems, and national surveys. We describe how activity and participation information can be more effectively captured, and how health informatics methodologies, including natural language processing (NLP), can enable automatically locating, extracting, and organizing this information on a large scale, supporting standardization and utilization with minimal additional provider burden. We examine the analytic requirements and potential challenges of capturing this information with informatics, and describe how data-driven techniques can combine with common standards and documentation practices to make activity and participation information standardized and accessible for improving patient care.

Recommendations

We recommend four specific actions to improve the capture and analysis of activity and participation information throughout the continuum of care: (1) make activity and participation annotation standards and datasets available to the broader research community; (2) define common research problems in automatically processing activity and participation information; (3) develop robust, machine-readable ontologies for function that describe the components of activity and participation information and their relationships; and (4) establish standards for how and when to document activity and participation status during clinical encounters. We further provide specific short-term goals to make significant progress in each of these areas within a reasonable time frame.

Moonstone: a novel natural language processing system for inferring social risk from clinical narratives

Background

Social risk factors are important dimensions of health and are linked to access to care, quality of life, health outcomes and life expectancy. However, in the Electronic Health Record, data related to many social risk factors are primarily recorded in free-text clinical notes, rather than as more readily computable structured data, and hence cannot currently be easily incorporated into automated assessments of health. In this paper, we present Moonstone, a new, highly configurable rule-based clinical natural language processing system designed to automatically extract information that requires inferencing from clinical notes. Our initial use case for the tool is focused on the automatic extraction of social risk factor information — in this case, housing situation, living alone, and social support — from clinical notes. Nursing notes, social work notes, emergency room physician notes, primary care notes, hospital admission notes, and discharge summaries, all derived from the Veterans Health Administration, were used for algorithm development and evaluation.

Results

An evaluation of Moonstone demonstrated that the system is highly accurate in extracting and classifying the three variables of interest (housing situation, living alone, and social support). The system achieved positive predictive value (i.e. precision) scores ranging from 0.66 (homeless/marginally housed) to 0.98 (lives at home/not homeless), accuracy scores ranging from 0.63 (lives in facility) to 0.95 (lives alone), and sensitivity (i.e. recall) scores ranging from 0.75 (lives in facility) to 0.97 (lives alone).

Conclusions

The Moonstone system is — to the best of our knowledge — the first freely available, open source natural language processing system designed to extract social risk factors from clinical text with good (lives in facility) to excellent (lives alone) performance. Although developed with the social risk factor identification task in mind, Moonstone provides a powerful tool to address a range of clinical natural language processing tasks, especially those tasks that require nuanced linguistic processing in conjunction with inference capabilities.

Automatically identifying social isolation from clinical narratives for patients with prostate Cancer

BACKGROUND

Social isolation is an important social determinant that impacts health outcomes and mortality among patients. The National Academy of Medicine recently recommended that social isolation be documented in electronic health records (EHR). However, social isolation usually is not recorded or obtained as coded data but rather collected from patient self-report or documented in clinical narratives. This study explores the feasibility and effectiveness of natural language processing (NLP) strategy for identifying patients who are socially isolated from clinical narratives.

METHOD

We used data from the Medical University of South Carolina (MUSC) Research Data Warehouse. Patients 18 years-of-age or older who were diagnosed with prostate cancer between January 1, 2014 and May 31, 2017 were eligible for this study. NLP pipelines identifying social isolation were developed via extraction of notes on progress, history and physical, consult, emergency department provider, telephone encounter, discharge summary, plan of care, and radiation oncology. Of 4195 eligible prostate cancer patients, we randomly sampled 3138 patients (75%) as a training dataset. The remaining 1057 patients (25%) were used as a test dataset to evaluate NLP algorithm performance. Standard performance measures for the NLP algorithm, including precision, recall, and F-measure, were assessed by expert manual review using the test dataset.

RESULTS

A total of 55,516 clinical notes from 3138 patients were included to develop the lexicon and NLP pipelines for social isolation. Of those, 35 unique patients (1.2%) had social isolation mention(s) in 217 notes. Among 24 terms relevant to social isolation, the most prevalent were "lack of social support," "lonely," "social isolation," "no friends," and "loneliness". Among 1057 patients in the test dataset, 17 patients (1.6%) were identified as having social isolation mention(s) in 40 clinical notes. Manual review identified four false positive mentions of social isolation and one false negatives in 154 notes from randomly selected 52 controls. The NLP pipeline demonstrated 90% precision, 97% recall, and 93% F-measure. The major reasons for a false positive included the ambiguities of the experiencer of social isolation, negation, and alternate meaning of words.

CONCLUSIONS

Our NLP algorithms demonstrate a highly accurate approach to identify social isolation.

Predictive models for hospital readmission risk: A systematic review of methods

OBJECTIVES

Hospital readmission risk prediction facilitates the identification of patients potentially at high risk so that resources can be used more efficiently in terms of cost-benefit. In this context, several models for readmission risk prediction have been proposed in recent years. The goal of this review is to give an overview of prediction models for hospital readmission, describe the data analysis methods and algorithms used for building the models, and synthesize their results.

METHODS

Studies that reported the predictive performance of a model for hospital readmission risk were included. We defined the scope of the review and accordingly built a search query to select the candidate papers. This query string was used as input for the chosen search engines, namely PubMed and Google Scholar. For each study, we recorded the population, feature selection method, classification algorithm, sample size, readmission threshold, readmission rate and predictive performance of the model.

RESULTS

We identified 77 studies that met the inclusion criteria, out of 265 citations. In 68% of the studies (n = 52) logistic regression or other regression techniques were utilized as the main method. Ten (13%) studies used survival analysis for model construction, while 14 (18%) used machine learning techniques for classification, of which decision tree-based methods and SVM were the most utilized algorithms. Among these, only four studies reported the use of any class imbalance addressing technique, of which resampling is the most frequent (75%). The performance of the models varied significantly among studies, with Area Under the ROC Curve (AUC) values in the ranges between 0.54 and 0.92.

CONCLUSION

Logistic regression and survival analysis have been traditionally the most widely used techniques for model building. Nevertheless, machine learning techniques are becoming increasingly popular in recent years. Recent comparative studies suggest that machine learning techniques can improve prediction ability over traditional statistical approaches. Regardless, the lack of an appropriate benchmark dataset of hospital readmissions makes a comparison of models' performance across different studies difficult.

Comparing clinician descriptions of frailty and geriatric syndromes using electronic health records: a retrospective cohort study

Background

Geriatric syndromes, including frailty, are common in older adults and associated with adverse outcomes. We compared patients described in clinical notes as “frail” to other older adults with respect to geriatric syndrome burden and healthcare utilization.

Methods

We conducted a retrospective cohort study on 18,341 Medicare Advantage enrollees aged 65+ (members of a large nonprofit medical group in Massachusetts), analyzing up to three years of administrative claims and structured and unstructured electronic health record (EHR) data. We determined the presence of ten geriatric syndromes (falls, malnutrition, dementia, severe urinary control issues, absence of fecal control, visual impairment, walking difficulty, pressure ulcers, lack of social support, and weight loss) from claims and EHR data, and the presence of frailty descriptions in clinical notes with a pattern-matching natural language processing (NLP) algorithm.

Results

Of the 18,341 patients, we found that 2202 (12%) were described as “frail” in clinical notes. “Frail” patients were older (82.3 ± 6.8 vs 75.9 ± 5.9, p < .001) and had higher rates of healthcare utilization, including number of inpatient hospitalizations and emergency department visits, than the rest of the population (p < .001). “Frail” patients had on average 4.85 ± 1.72 of the ten geriatric syndromes studied, while non-frail patients had 2.35 ± 1.71 (p = .013). Falls, walking difficulty, malnutrition, weight loss, lack of social support and dementia were more highly correlated with frailty descriptions. The most common geriatric syndrome pattern among “frail” patients was a combination of walking difficulty, lack of social support, falls, and weight loss.

Conclusions

Patients identified as “frail” by providers in clinical notes have higher rates of healthcare utilization and more geriatric syndromes than other patients. Certain geriatric syndromes were more highly correlated with descriptions of frailty than others.

Electronic supplementary material

The online version of this article (10.1186/s12877-017-0645-7) contains supplementary material, which is available to authorized users.

Identifying Nonclinical Factors Associated With 30-Day Readmission in Patients with Cardiovascular Disease: Protocol for an Observational Study

Background

Cardiovascular disease (CVD) is the leading cause of hospitalization in older adults and high readmission rates have attracted considerable attention as actionable targets to promote efficiency in care and to reduce costs. Despite a plethora of research over the past decade, current strategies to predict readmissions have been largely ineffective and efforts to identify novel clinical predictors have been largely unsuccessful.

Objective

The objective of this study is to examine a wide array of socioeconomic, psychosocial, behavioral, and clinical factors to predict risks of 30-day hospital readmission in cardiovascular patients.

Methods

The study includes patients (aged 18 years and older) admitted for the treatment of cardiovascular-related illnesses at the Duke Heart Center, which is among the nation’s largest and top-ranked cardiovascular care hospitals. The study uses a novel standardized survey to ascertain data on a comprehensive array of patient characteristics that will be linked to their electronic medical records. A series of univariate and multivariate models will be used to estimate the associations between the patient-level factors and 30-day readmissions. The performance of the risk models will be examined based on 2 components of accuracy—model calibration and discrimination—to determine how closely the predicted outcome agrees with the observed (actual) outcome and how well the model distinguishes patients who were readmitted and those who were not. The purpose of this paper is to present the protocol for the implementation of this study.

Results

The study was launched in February 2014 and is actively recruiting patients from the Heart Center. Approximately 550 patients have been enrolled to date and the study is expected to continue recruitment until February 2018. Preliminary results show that participants in the study were aged 63.6 years on average (SD 14.0), predominately male (61.2%), and primarily non-Hispanic white (64.6%) or non-Hispanic black (31.7%). The demographic characteristics of study participants were not significantly different from all patients admitted to the Heart Center during this period with an average age of 65.0 years (SD 15.3) and predominately male (58.6%), non-Hispanic white (62.9%) or non-Hispanic black (31.8%) The integration of the interview data with clinical data from the patient electronic medical records is currently underway. The study has received funding and ethical approval.

Conclusions

Many US hospitals continue to struggle with high readmission rates in patients with cardiovascular disease. The primary objective of this study is to collect and integrate a comprehensive array of patient attributes to develop a powerful yet parsimonious model to stratify risks of rehospitalization in cardiovascular patients. The results of this research also have the potential to identify actionable targets for tailored interventions to improve patient outcomes.