Screening pregnant women for suicidal behavior in electronic medical records: diagnostic codes vs. clinical notes processed by natural language processing

Defining Suicidal Thought and Behavior Phenotypes for Genetic Studies

Background

Standardized definitions of suicidality phenotypes, including suicidal ideation (SI), attempt (SA), and death (SD) are a critical step towards improving understanding and comparison of results in suicide research. The complexity of suicidality contributes to heterogeneity in phenotype definitions, impeding evaluation of clinical and genetic risk factors across studies and efforts to combine samples within consortia. Here, we present expert and data-supported recommendations for defining suicidality and control phenotypes to facilitate merging current/legacy samples with definition variability and aid future sample creation.

Methods

A subgroup of clinician researchers and experts from the Suicide Workgroup of the Psychiatric Genomics Consortium (PGC) reviewed existing PGC definitions for SI, SA, SD, and control groups and generated preliminary consensus guidelines for instrument-derived and international classification of disease (ICD) data. ICD lists were validated in two independent datasets (N = 9,151 and 12,394).

Results

Recommendations are provided for evaluated instruments for SA and SI, emphasizing selection of lifetime measures phenotype-specific wording. Recommendations are also provided for defining SI and SD from ICD data. As the SA ICD definition is complex, SA code list recommendations were validated against instrument results with sensitivity (range = 15.4% to 80.6%), specificity (range = 67.6% to 97.4%), and positive predictive values (range = 0.59-0.93) reported.

Conclusions

Best-practice guidelines are presented for the use of existing information to define SI/SA/SD in consortia research. These proposed definitions are expected to facilitate more homogeneous data aggregation for genetic and multisite studies. Future research should involve refinement, improved generalizability, and validation in diverse populations.

Predicting recurrent chat contact in a psychological intervention for the youth using natural language processing

Chat-based counseling hotlines emerged as a promising low-threshold intervention for youth mental health. However, despite the resulting availability of large text corpora, little work has investigated Natural Language Processing (NLP) applications within this setting. Therefore, this preregistered approach (OSF: XA4PN) utilizes a sample of approximately 19,000 children and young adults that received a chat consultation from a 24/7 crisis service in Germany. Around 800,000 messages were used to predict whether chatters would contact the service again, as this would allow the provision of or redirection to additional treatment. We trained an XGBoost Classifier on the words of the anonymized conversations, using repeated cross-validation and bayesian optimization for hyperparameter search. The best model was able to achieve an AUROC score of 0.68 (p < 0.01) on the previously unseen 3942 newest consultations. A shapely-based explainability approach revealed that words indicating younger age or female gender and terms related to self-harm and suicidal thoughts were associated with a higher chance of recontacting. We conclude that NLP-based predictions of recurrent contact are a promising path toward personalized care at chat hotlines.

Artificial Intelligence and Healthcare: A Journey through History, Present Innovations, and Future Possibilities

Artificial intelligence (AI) has emerged as a powerful tool in healthcare significantly impacting practices from diagnostics to treatment delivery and patient management. This article examines the progress of AI in healthcare, starting from the field's inception in the 1960s to present-day innovative applications in areas such as precision medicine, robotic surgery, and drug development. In addition, the impact of the COVID-19 pandemic on the acceleration of the use of AI in technologies such as telemedicine and chatbots to enhance accessibility and improve medical education is also explored. Looking forward, the paper speculates on the promising future of AI in healthcare while critically addressing the ethical and societal considerations that accompany the integration of AI technologies. Furthermore, the potential to mitigate health disparities and the ethical implications surrounding data usage and patient privacy are discussed, emphasizing the need for evolving guidelines to govern AI's application in healthcare.

Bridging information gaps in menopause status classification through natural language processing

Objective

To use natural language processing (NLP) of clinical notes to augment existing structured electronic health record (EHR) data for classification of a patient's menopausal status.

Materials and methods

A rule-based NLP system was designed to capture evidence of a patient's menopause status including dates of a patient's last menstrual period, reproductive surgeries, and postmenopause diagnosis as well as their use of birth control and menstrual interruptions. NLP-derived output was used in combination with structured EHR data to classify a patient's menopausal status. NLP processing and patient classification were performed on a cohort of 307 512 female Veterans receiving healthcare at the US Department of Veterans Affairs (VA).

Results

NLP was validated at 99.6% precision. Including the NLP-derived data into a menopause phenotype increased the number of patients with data relevant to their menopausal status by 118%. Using structured codes alone, 81 173 (27.0%) are able to be classified as postmenopausal or premenopausal. However, with the inclusion of NLP, this number increased 167 804 (54.6%) patients. The premenopausal category grew by 532.7% with the inclusion of NLP data.

Discussion

By employing NLP, it became possible to identify documented data elements that predate VA care, originate outside VA networks, or have no corresponding structured field in the VA EHR that would be otherwise inaccessible for further analysis.

Conclusion

NLP can be used to identify concepts relevant to a patient's menopausal status in clinical notes. Adding NLP-derived data to an algorithm classifying a patient's menopausal status significantly increases the number of patients classified using EHR data, ultimately enabling more detailed assessments of the impact of menopause on health outcomes.

Diabetes-related complications, glycemic levels, and healthcare utilization outcomes after therapeutic inertia in type 2 diabetes mellitus

AIMS

To assess diabetes-related complications, glycemic levels, and healthcare utilization 12 months after exposure to therapeutic inertia among patients with type 2 diabetes mellitus (T2D).

METHODS

This retrospective cohort study analyzed data from the OneFlorida Clinical Research Consortium (electronic health records from Florida practices/clinics). The cohort included adult patients (≥18 years old) with T2D who had an HbA1c≥7.0% (53 mmol/mol) recorded from January 1, 2014-September 30, 2019. Therapeutic inertia (exposed vs. not exposed) was evaluated during the six months following HbA1c≥7.0% (53 mmol/mol). The outcomes assessed during the 12-month follow-up period included diabetes-related complications (continuous Diabetes Complications and Severity Index (DCSI)), glycemic levels (continuous follow-up HbA1c lab), and healthcare utilization counts. We analyzed data using multivariable regression models, adjusting for covariates.

RESULTS

The cohort included 26,881 patients with T2D (58.94% White race, 49.72% female, and mean age of 58.82 (SD=13.09)). After adjusting for covariates, therapeutic inertia exposure was associated with lower DCSI (estimate=-0.14 (SE=0.03), p < 0.001), higher follow-up HbA1c (estimate=0.14 (SE=0.04), p < 0.001), and lower rates of ambulatory visits (rate ratio=0.79, 95% CI=0.75-0.82).

CONCLUSIONS

Findings communicate the clinical practice implications and public health implications for combating therapeutic inertia in diabetes care.

Applying Natural Language Processing to Textual Data From Clinical Data Warehouses: Systematic Review

BACKGROUND

In recent years, health data collected during the clinical care process have been often repurposed for secondary use through clinical data warehouses (CDWs), which interconnect disparate data from different sources. A large amount of information of high clinical value is stored in unstructured text format. Natural language processing (NLP), which implements algorithms that can operate on massive unstructured textual data, has the potential to structure the data and make clinical information more accessible.

OBJECTIVE

The aim of this review was to provide an overview of studies applying NLP to textual data from CDWs. It focuses on identifying the (1) NLP tasks applied to data from CDWs and (2) NLP methods used to tackle these tasks.

METHODS

This review was performed according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. We searched for relevant articles in 3 bibliographic databases: PubMed, Google Scholar, and ACL Anthology. We reviewed the titles and abstracts and included articles according to the following inclusion criteria: (1) focus on NLP applied to textual data from CDWs, (2) articles published between 1995 and 2021, and (3) written in English.

RESULTS

We identified 1353 articles, of which 194 (14.34%) met the inclusion criteria. Among all identified NLP tasks in the included papers, information extraction from clinical text (112/194, 57.7%) and the identification of patients (51/194, 26.3%) were the most frequent tasks. To address the various tasks, symbolic methods were the most common NLP methods (124/232, 53.4%), showing that some tasks can be partially achieved with classical NLP techniques, such as regular expressions or pattern matching that exploit specialized lexica, such as drug lists and terminologies. Machine learning (70/232, 30.2%) and deep learning (38/232, 16.4%) have been increasingly used in recent years, including the most recent approaches based on transformers. NLP methods were mostly applied to English language data (153/194, 78.9%).

CONCLUSIONS

CDWs are central to the secondary use of clinical texts for research purposes. Although the use of NLP on data from CDWs is growing, there remain challenges in this field, especially with regard to languages other than English. Clinical NLP is an effective strategy for accessing, extracting, and transforming data from CDWs. Information retrieved with NLP can assist in clinical research and have an impact on clinical practice.

Identifying suicide documentation in clinical notes through zero-shot learning

Background and Aims

In deep learning, a major difficulty in identifying suicidality and its risk factors in clinical notes is the lack of training samples given the small number of true positive instances among the number of patients screened. This paper describes a novel methodology that identifies suicidality in clinical notes by addressing this data sparsity issue through zero-shot learning. Our general aim was to develop a tool that leveraged zero-shot learning to effectively identify suicidality documentation in all types of clinical notes.

Methods

US Veterans Affairs clinical notes served as data. The training data set label was determined using diagnostic codes of suicide attempt and self-harm. We used a base string associated with the target label of suicidality to provide auxiliary information by narrowing the positive training cases to those containing the base string. We trained a deep neural network by mapping the training documents' contents to a semantic space. For comparison, we trained another deep neural network using the identical training data set labels, and bag-of-words features.

Results

The zero-shot learning model outperformed the baseline model in terms of area under the curve, sensitivity, specificity, and positive predictive value at multiple probability thresholds. In applying a 0.90 probability threshold, the methodology identified notes documenting suicidality but not associated with a relevant ICD-10-CM code, with 94% accuracy.

Conclusion

This method can effectively identify suicidality without manual annotation.

Suicidal behavior across a broad range of psychiatric disorders

Suicide is a leading cause of death worldwide. In 2020, some 12.2 million Americans seriously contemplated suicide, 3.2 million planned suicide attempts, and 1.2 million attempted suicide. Traditionally, the approach to treating suicidal behavior (SB) has been to treat the "underlying" psychiatric disorder. However, the number of diagnoses associated with SB is considerable. We could find no studies describing the range of disorders reported to be comorbid with SB. This narrative review summarizes literature documenting the occurrence of SB across the lifespan and the full range of psychiatric diagnoses, not only BPD and those that comprise MDE, It also describes the relevance of these observations to clinical practice, research, and nosology. The literature searches contained the terms "suicid*" and each individual psychiatric diagnosis and identified 587 studies. We did not include case reports, case series, studies only addressing suicidal ideation or non-suicidal self-injury (NSSI), studies on self-harm, not distinguishing between SB and NSSI and studies that did not include any individuals that met criteria for a specific DSM-5 diagnosis (n = 366). We found that SB (suicide and/or suicide attempt) was reported to be associated with 72 out of 145 diagnoses, although data quality varied. Thus, SB is not exclusively germane to Major Depressive Episode (MDE) and Borderline Personality Disorder (BPD), the only conditions for which it is a diagnostic criterion. That SB co-occurs with so many diagnoses reinforces the need to assess current and past SB regardless of diagnosis, and supports the addition of charting codes to the DSM-5 to indicate current or past SB. It also comports with new data that specific genes are associated with SB independent of psychiatric diagnoses, and suggests that SB should be managed with specific suicide prevention interventions in addition to treatments indicated for co-occurring diagnoses. SB diagnostic codes would help researchers and clinicians document and measure SB's trajectory and response to treatment over time, and, ultimately, help develop secondary and tertiary prevention strategies. As a separate diagnosis, SB would preclude situations in which a potentially life-threatening behavior is not accounted for by a diagnosis, a problem that is particularly salient when no mental disorder is present, as is sometimes the case.

Opportunities and challenges in application of artificial intelligence in pharmacology

Artificial intelligence (AI) is a machine science that can mimic human behaviour like intelligent analysis of data. AI functions with specialized algorithms and integrates with deep and machine learning. Living in the digital world can generate a huge amount of medical data every day. Therefore, we need an automated and reliable evaluation tool that can make decisions more accurately and faster. Machine learning has the potential to learn, understand and analyse the data used in healthcare systems. In the last few years, AI is known to be employed in various fields in pharmaceutical science especially in pharmacological research. It helps in the analysis of preclinical (laboratory animals) and clinical (in human) trial data. AI also plays important role in various processes such as drug discovery/manufacturing, diagnosis of big data for disease identification, personalized treatment, clinical trial research, radiotherapy, surgical robotics, smart electronic health records, and epidemic outbreak prediction. Moreover, AI has been used in the evaluation of biomarkers and diseases. In this review, we explain various models and general processes of machine learning and their role in pharmacological science. Therefore, AI with deep learning and machine learning could be relevant in pharmacological research.

Application of Natural Language Processing (NLP) in Detecting and Preventing Suicide Ideation: A Systematic Review

(1) Introduction: Around a million people are reported to die by suicide every year, and due to the stigma associated with the nature of the death, this figure is usually assumed to be an underestimate. Machine learning and artificial intelligence such as natural language processing has the potential to become a major technique for the detection, diagnosis, and treatment of people. (2) Methods: PubMed, EMBASE, MEDLINE, PsycInfo, and Global Health databases were searched for studies that reported use of NLP for suicide ideation or self-harm. (3) Result: The preliminary search of 5 databases generated 387 results. Removal of duplicates resulted in 158 potentially suitable studies. Twenty papers were finally included in this review. (4) Discussion: Studies show that combining structured and unstructured data in NLP data modelling yielded more accurate results than utilizing either alone. Additionally, to reduce suicides, people with mental problems must be continuously and passively monitored. (5) Conclusions: The use of AI&ML opens new avenues for considerably guiding risk prediction and advancing suicide prevention frameworks. The review's analysis of the included research revealed that the use of NLP may result in low-cost and effective alternatives to existing resource-intensive methods of suicide prevention.

Natural Language Processing for Computer-Assisted Chart Review to Assess Documentation of Substance use and Psychopathology in Heart Failure Patients Awaiting Cardiac Resynchronization Therapy

CONTEXT

Advanced heart failure (HF) patients often experience distressing psychological symptoms, frequently meeting diagnostic criteria for psychological disorders, including anxiety, depression, and substance use disorder. Patients with device-based HF therapies have added risk for psychological disorders, with consequences for their physiological functioning, including adverse cardiac outcomes.

OBJECTIVES

This study used natural language processing (NLP) for computer-assisted chart review to assess documentation of mental health and substance use in HF patients awaiting cardiac resynchronization therapy (CRT), a device-based HF therapy.

METHODS

We applied NLP to clinical notes from electronic health records (EHR) of 965 consecutive patients, with 9821 total clinical notes, at two academic medical centers between 2004 and 2015. We developed and validated a keyword library capturing terms related to mental health and substance use, while balancing specificity and sensitivity.

RESULTS

Mean age was 71.6 years (SD = 11.8), 78% male, and 87% non-Hispanic White. Of the 544 patients (56.4%) with documentation of mental health history, 9.7% had their mental health assessed and 6.6% had a plan documented. Of the 773 patients (80.1%) with documentation of substance use history, 10 (1.0%) had an assessment, and 3 (0.3%) had a plan.

CONCLUSION

Despite clinical recommendations and standards of care, clinicians are under documenting assessments and plans prior to CRT. Future research should develop an algorithm to prompt clinicians to document this content. Such quality improvement efforts may ensure adherence to standards of care and clinical guidelines.

Improving ascertainment of suicidal ideation and suicide attempt with natural language processing

Methods relying on diagnostic codes to identify suicidal ideation and suicide attempt in Electronic Health Records (EHRs) at scale are suboptimal because suicide-related outcomes are heavily under-coded. We propose to improve the ascertainment of suicidal outcomes using natural language processing (NLP). We developed information retrieval methodologies to search over 200 million notes from the Vanderbilt EHR. Suicide query terms were extracted using word2vec. A weakly supervised approach was designed to label cases of suicidal outcomes. The NLP validation of the top 200 retrieved patients showed high performance for suicidal ideation (area under the receiver operator curve [AUROC]: 98.6, 95% confidence interval [CI] 97.1-99.5) and suicide attempt (AUROC: 97.3, 95% CI 95.2-98.7). Case extraction produced the best performance when combining NLP and diagnostic codes and when accounting for negated suicide expressions in notes. Overall, we demonstrated that scalable and accurate NLP methods can be developed to identify suicidal behavior in EHRs to enhance prevention efforts, predictive models, and precision medicine.

Scalable relevance ranking algorithm via semantic similarity assessment improves efficiency of medical chart review

OBJECTIVE

Accurately assigning phenotype information to individual patients via computational phenotyping using Electronic Health Records (EHRs) has been seen as the first step towards enabling EHRs for precision medicine research. Chart review labels annotated by clinical experts, also known as "gold standard" labels, are essential for the development and validation of computational phenotyping algorithms. However, given the complexity of EHR systems, the process of chart review is both labor intensive and time consuming. We propose a fully automated algorithm, referred to as pGUESS, to rank EHR notes according to their relevance to a given phenotype. By identifying the most relevant notes, pGUESS can greatly improve the efficiency and accuracy of chart reviews.

METHOD

pGUESS uses prior guided semantic similarity to measure the informativeness of a clinical note to a given phenotype. We first select candidate clinical concepts from a pool of comprehensive medical concepts using public knowledge sources and then derive the semantic embedding vector (SEV) for a reference article (SEV_ref) and each note (SEV_note). The algorithm scores the relevance of a note as the cosine similarity between SEV_note and SEV_ref.

RESULTS

The algorithm was validated against four sets of 200 notes that were manually annotated by clinical experts to assess their informativeness to one of three disease phenotypes. pGUESS algorithm substantially outperforms existing unsupervised approaches for classifying the relevance status with respect to both accuracy and scalability across phenotypes. Averaging over the three phenotypes, the rank correlation between the algorithm ranking and gold standard label was 0.64 for pGUESS, but only 0.47 and 0.35 for the next two best performing algorithms. pGUESS is also much more computationally scalable compared to existing algorithms.

CONCLUSION

pGUESS algorithm can substantially reduce the burden of chart review and holds potential in improving the efficiency and accuracy of human annotation.

Frequency of Clinicians' Assessments for Access to Lethal Means in Persons at Risk for Suicide

OBJECTIVE

We measured the frequency of clinicians' assessments for access to lethal means, including firearms and medications in patients at risk of suicide from electronic medical and mental health records in outpatient and emergency settings.

METHODS

We included adult patients who reported suicide ideation on the PHQ-9 depression screener in behavioral health and primary care outpatient settings of a large integrated health system in the U.S. and those with suicidal behavior treated in the emergency department. Two separate natural language processing queries were developed on medical record text documentation: (1) assessment for access to firearms (8,994 patients), (2) assessment for access to medications (4,939 patients).

RESULTS

Only 35% of patients had documentation of firearm or medication assessment in the month following treatment for suicidal behavior in the emergency setting. Among those reporting suicidal ideation in outpatient setting, 31% had documentation of firearm assessment and 23% for medication assessment. The accuracy of the estimates was very good for firearm assessment (F1 = 89%) and medication assessment in the outpatient setting (F1 = 91%) and fair for medication assessment in the emergency setting (F1 = 70%) due to more varied documentation styles.

CONCLUSIONS

Lethal means assessment following report of suicidal ideation or behavior is low in a nonacademic health care setting. Until health systems implement more structured documentation to measure lethal means assessment, such as discrete data field, NLP methods may be used to conduct research and surveillance of this important prevention practice in real-world settings.

Antidepressant use and risk of intubation or death in hospitalized patients with COVID-19: A retrospective cohort study of clinical effectiveness

Initial controlled trials of the serotonergic antidepressant fluvoxamine showed promise for treatment of mild to moderate COVID-19 in outpatients, although more recent outpatient data have been less encouraging. Turning to studies of hospitalized patients, a retrospective cohort study by Hoertel and associates in 2021 found a markedly reduced risk of intubation or death among patients hospitalized with COVID-19 who were receiving serotonergic antidepressants at the time of admission vs. those not receiving antidepressants. In an attempt to replicate these latter findings, we performed a similarly designed study of 500 individuals hospitalized with COVID-19 in a large academic hospital system who were taking a serotonergic antidepressant at the time of admission compared with two groups (N = 573 and N = 593) not receiving an antidepressant. In analyses controlling for demographic and clinical variables, we found no significant difference in effect between the antidepressant group and either of the two comparison groups [hazard ratios (95% CI) for intubation or death 1.1 (0.83-1.5) and 1.1 (0.86-1.5); and for death alone 1.3 (0.93-1.8) and 1.1 (0.85-1.7)]. Examining the results of our study, along with those of Hoertel et al. and three additional retrospective cohort studies in inpatients published in the interim, the data permit only very limited conclusions, with the findings on the effect of serotonergic antidepressants ranging from a strongly protective effect to no effect. Although there are numerous threats to validity that might account for this wide range of findings, we could not identify any principal factor or set of factors that could clearly explain the differences.

Diagnosing post-traumatic stress disorder using electronic medical record data

This study proposes a predictive model that uses structured data and unstructured narrative notes from Electronic Medical Records to accurately identify patients diagnosed with Post-Traumatic Stress Disorder (PTSD). We utilize data from primary care clinicians participating in the Manitoba Primary Care Research Network (MaPCReN) representing 154,118 patients. A reference sample of 195 patients that had their PTSD diagnosis confirmed using a manual chart review of structured data and narrative notes, and PTSD negative patients is used as the gold standard data for model training, validation and testing. We assess structured and unstructured data from eight tables in the MaPCReN namely, patient demographics, disease case, examinations, medication, billing records, health condition, risk factors, and encounter notes. Feature engineering is applied to convert data into proper representation for predictive modeling. We explore serial and parallel mixed data models that are trained on both structured and unstructured data to identify PTSD. Model performances were calculated based on a highly skewed hold-out test dataset. The serial model that uses both structured and text data as input, yielded the highest values in sensitivity (0.77), F-measure (0.76), and AUC (0.88) and the parallel model that uses both structured and text data as the input obtained the highest positive predicted value (PPV) (0.75). Diseases such as PTSD are difficult to diagnose. Information recorded in the chart note over multiple visits of the patients with the primary care physicians has higher predictive power than structured data and combining these two data types can increase the predictive capabilities of machine learning models in diagnosing PTSD. While the deep-learning model outperformed the traditional ensemble model in processing text data, the ensemble classifier obtained better results in ingesting a combination of features obtained from both data types in the serial mixed model. The study demonstrated that unstructured encounter notes enhance a model's ability to identify patients diagnosed with PTSD. These findings can enhance quality improvement, research, and disease surveillance related to PTSD in primary care populations.

Using Verb Fluency, Natural Language Processing, and Machine Learning to Detect Alzheimer's Disease

Alzheimer's disease (AD) causes significant impairments in memory and other cognitive domains. As there is no cure to the disease yet, early detection and delay of disease progression are critical for management of AD. Verbal fluency is one of the most common and sensitive neuropsychological methods used for detection and evaluation of the cognitive declines in AD, in which a subject is required to name as many items as possible in 30 or 60 seconds that belong to a certain category. In this study, we develop an approach to detect AD using a verb fluency (VF) task, a specific subset of verbal fluency analyzing the subjects' listing of verbs in a given time period. We use machine learning techniques including random forest (RF), neural network (NN), recurrent NN (RNN), and natural language processing (NLP) to detect the risk of AD. The results show that the developed models can stratify subjects into the corresponding AD and control groups with up to 76% accuracy using RF, but at a cost of having to preprocess the data. This accuracy is slightly lower, but not significantly, at 67% using RNN and NLP, which involves almost no manual preprocessing of the data. This study opens up a powerful approach of using simple VF tasks for early detection of AD.

Developing a Natural Language Processing tool to identify perinatal self-harm in electronic healthcare records

BACKGROUND

Self-harm occurring within pregnancy and the postnatal year ("perinatal self-harm") is a clinically important yet under-researched topic. Current research likely under-estimates prevalence due to methodological limitations. Electronic healthcare records (EHRs) provide a source of clinically rich data on perinatal self-harm.

AIMS

(1) To create a Natural Language Processing (NLP) tool that can, with acceptable precision and recall, identify mentions of acts of perinatal self-harm within EHRs. (2) To use this tool to identify service-users who have self-harmed perinatally, based on their EHRs.

METHODS

We used the Clinical Record Interactive Search system to extract de-identified EHRs of secondary mental healthcare service-users at South London and Maudsley NHS Foundation Trust. We developed a tool that applied several layers of linguistic processing based on the spaCy NLP library for Python. We evaluated mention-level performance in the following domains: span, status, temporality and polarity. Evaluation was done against a manually coded reference standard. Mention-level performance was reported as precision, recall, F-score and Cohen's kappa for each domain. Performance was also assessed at 'service-user' level and explored whether a heuristic rule improved this. We report per-class statistics for service-user performance, as well as likelihood ratios and post-test probabilities.

RESULTS

Mention-level performance: micro-averaged F-score, precision and recall for span, polarity and temporality >0.8. Kappa for status 0.68, temporality 0.62, polarity 0.91. Service-user level performance with heuristic: F-score, precision, recall of minority class 0.69, macro-averaged F-score 0.81, positive LR 9.4 (4.8-19), post-test probability 69.0% (53-82%). Considering the task difficulty, the tool performs well, although temporality was the attribute with the lowest level of annotator agreement.

CONCLUSIONS

It is feasible to develop an NLP tool that identifies, with acceptable validity, mentions of perinatal self-harm within EHRs, although with limitations regarding temporality. Using a heuristic rule, it can also function at a service-user-level.

ICD-10-Coding of Medically Unexplained Physical Symptoms and Somatoform Disorders-A Survey With German GPs

Background: General practitioners (GPs) are reluctant to use codes that correspond to somatization syndromes. Aim: To quantify GPs' views on coding of medically unexplained physical symptoms (MUPS), somatoform disorders, and associated factors. Design and Setting: Survey with German GPs. Methods: We developed six survey items [response options "does not apply at all (1)"-"does fully apply (6)"], invited a random sample of 12.004 GPs to participate in the self-administered cross-sectional survey and analysed data using descriptive statistics and logistic regression analyses. Results: Response rate was 15.2% with N = 1,731 valid responses (54.3% female). Participants considered themselves familiar with ICD-10 criteria for somatoform disorders (M = 4.52; SD =.036) and considered adequate coding as essential prerequisite for treatment (M = 5.02; SD = 1.21). All other item means were close to the scale mean: preference for symptom or functional codes (M = 3.40; SD = 1.21), consideration of the possibility of stigmatisation (M = 3.30; SD = 1.35) and other disadvantages (M = 3.28; SD = 1.30) and coding only if psychotherapy is intended (M = 3.39; SD = 1.46). Exposure, guideline knowledge, and experience were most strongly associated with GPs' self-reported coding behaviour. Conclusions: Subjective exposure, guideline knowledge, and experience as a GP, but no sociodemographic variable being associated with GPs' subjective coding behaviour could indicate that GPs offer a relatively homogeneous approach to coding and handling of MUPS and somatoform disorders. Strengthening guideline knowledge and implementation, and practise with simulated patients could increase the subjective competence to cope with the challenge that patients with MUPS and somatoform disorders present.

What Every Reader Should Know About Studies Using Electronic Health Record Data but May Be Afraid to Ask

Coincident with the tsunami of COVID-19–related publications, there has been a surge of studies using real-world data, including those obtained from the electronic health record (EHR). Unfortunately, several of these high-profile publications were retracted because of concerns regarding the soundness and quality of the studies and the EHR data they purported to analyze. These retractions highlight that although a small community of EHR informatics experts can readily identify strengths and flaws in EHR-derived studies, many medical editorial teams and otherwise sophisticated medical readers lack the framework to fully critically appraise these studies. In addition, conventional statistical analyses cannot overcome the need for an understanding of the opportunities and limitations of EHR-derived studies. We distill here from the broader informatics literature six key considerations that are crucial for appraising studies utilizing EHR data: data completeness, data collection and handling (eg, transformation), data type (ie, codified, textual), robustness of methods against EHR variability (within and across institutions, countries, and time), transparency of data and analytic code, and the multidisciplinary approach. These considerations will inform researchers, clinicians, and other stakeholders as to the recommended best practices in reviewing manuscripts, grants, and other outputs from EHR-data derived studies, and thereby promote and foster rigor, quality, and reliability of this rapidly growing field.

Selection of Clinical Text Features for Classifying Suicide Attempts

Research has demonstrated cohort misclassification when studies of suicidal thoughts and behaviors (STBs) rely on ICD-9/10-CM diagnosis codes. Electronic health record (EHR) data are being explored to better identify patients, a process called EHR phenotyping. Most STB phenotyping studies have used structured EHR data, but some are beginning to incorporate unstructured clinical text. In this study, we used a publicly-accessible natural language processing (NLP) program for biomedical text (MetaMap) and iterative elastic net regression to extract and select predictive text features from the discharge summaries of 810 inpatient admissions of interest. Initial sets of 5,866 and 2,709 text features were reduced to 18 and 11, respectively. The two models fit with these features obtained an area under the receiver operating characteristic curve of 0.866-0.895 and an area under the precision-recall curve of 0.800-0.838, demonstrating the approach's potential to identify textual features to incorporate in phenotyping models.

Artificial Intelligence and Suicide Prevention: A Systematic Review of Machine Learning Investigations

Suicide is a leading cause of death that defies prediction and challenges prevention efforts worldwide. Artificial intelligence (AI) and machine learning (ML) have emerged as a means of investigating large datasets to enhance risk detection. A systematic review of ML investigations evaluating suicidal behaviors was conducted using PubMed/MEDLINE, PsychInfo, Web-of-Science, and EMBASE, employing search strings and MeSH terms relevant to suicide and AI. Databases were supplemented by hand-search techniques and Google Scholar. Inclusion criteria: (1) journal article, available in English, (2) original investigation, (3) employment of AI/ML, (4) evaluation of a suicide risk outcome. N = 594 records were identified based on abstract search, and 25 hand-searched reports. N = 461 reports remained after duplicates were removed, n = 316 were excluded after abstract screening. Of n = 149 full-text articles assessed for eligibility, n = 87 were included for quantitative synthesis, grouped according to suicide behavior outcome. Reports varied widely in methodology and outcomes. Results suggest high levels of risk classification accuracy (>90%) and Area Under the Curve (AUC) in the prediction of suicidal behaviors. We report key findings and central limitations in the use of AI/ML frameworks to guide additional research, which hold the potential to impact suicide on broad scale.

Identifying and Predicting Intentional Self-Harm in Electronic Health Record Clinical Notes: Deep Learning Approach

BACKGROUND

Suicide is an important public health concern in the United States and around the world. There has been significant work examining machine learning approaches to identify and predict intentional self-harm and suicide using existing data sets. With recent advances in computing, deep learning applications in health care are gaining momentum.

OBJECTIVE

This study aimed to leverage the information in clinical notes using deep neural networks (DNNs) to (1) improve the identification of patients treated for intentional self-harm and (2) predict future self-harm events.

METHODS

We extracted clinical text notes from electronic health records (EHRs) of 835 patients with International Classification of Diseases (ICD) codes for intentional self-harm and 1670 matched controls who never had any intentional self-harm ICD codes. The data were divided into training and holdout test sets. We tested a number of algorithms on clinical notes associated with the intentional self-harm codes using the training set, including several traditional bag-of-words-based models and 2 DNN models: a convolutional neural network (CNN) and a long short-term memory model. We also evaluated the predictive performance of the DNNs on a subset of patients who had clinical notes 1 to 6 months before the first intentional self-harm event. Finally, we evaluated the impact of a pretrained model using Word2vec (W2V) on performance.

RESULTS

The area under the receiver operating characteristic curve (AUC) for the CNN on the phenotyping task, that is, the detection of intentional self-harm in clinical notes concurrent with the events was 0.999, with an F1 score of 0.985. In the predictive task, the CNN achieved the highest performance with an AUC of 0.882 and an F1 score of 0.769. Although pretraining with W2V shortened the DNN training time, it did not improve performance.

CONCLUSIONS

The strong performance on the first task, namely, phenotyping based on clinical notes, suggests that such models could be used effectively for surveillance of intentional self-harm in clinical text in an EHR. The modest performance on the predictive task notwithstanding, the results using DNN models on clinical text alone are competitive with other reports in the literature using risk factors from structured EHR data.

Validation of a Semiautomated Natural Language Processing-Based Procedure for Meta-Analysis of Cancer Susceptibility Gene Penetrance

PURPOSE

Quantifying the risk of cancer associated with pathogenic mutations in germline cancer susceptibility genes-that is, penetrance-enables the personalization of preventive management strategies. Conducting a meta-analysis is the best way to obtain robust risk estimates. We have previously developed a natural language processing (NLP) -based abstract classifier which classifies abstracts as relevant to penetrance, prevalence of mutations, both, or neither. In this work, we evaluate the performance of this NLP-based procedure.

MATERIALS AND METHODS

We compared the semiautomated NLP-based procedure, which involves automated abstract classification and text mining, followed by human review of identified studies, with the traditional procedure that requires human review of all studies. Ten high-quality gene-cancer penetrance meta-analyses spanning 16 gene-cancer associations were used as the gold standard by which to evaluate the performance of our procedure. For each meta-analysis, we evaluated the number of abstracts that required human review (workload) and the ability to identify the studies that were included by the authors in their quantitative analysis (coverage).

RESULTS

Compared with the traditional procedure, the semiautomated NLP-based procedure led to a lower workload across all 10 meta-analyses, with an overall 84% reduction (2,774 abstracts v 16,941 abstracts) in the amount of human review required. Overall coverage was 93%-we are able to identify 132 of 142 studies-before reviewing references of identified studies. Reasons for the 10 missed studies included blank and poorly written abstracts. After reviewing references, nine of the previously missed studies were identified and coverage improved to 99% (141 of 142 studies).

CONCLUSION

We demonstrated that an NLP-based procedure can significantly reduce the review workload without compromising the ability to identify relevant studies. NLP algorithms have promising potential for reducing human efforts in the literature review process.

Construction of a semi-automatic ICD-10 coding system

Background

The International Classification of Diseases, 10th Revision (ICD-10) has been widely used to describe the diagnosis information of patients. Automatic ICD-10 coding is important because manually assigning codes is expensive, time consuming and error prone. Although numerous approaches have been developed to explore automatic coding, few of them have been applied in practice. Our aim is to construct a practical, automatic ICD-10 coding machine to improve coding efficiency and quality in daily work.

Methods

In this study, we propose the use of regular expressions (regexps) to establish a correspondence between diagnosis codes and diagnosis descriptions in outpatient settings and at admission and discharge. The description models of the regexps were embedded in our upgraded coding system, which queries a diagnosis description and assigns a unique diagnosis code. Like most studies, the precision (P), recall (R), F-measure (F) and overall accuracy (A) were used to evaluate the system performance. Our study had two stages. The datasets were obtained from the diagnosis information on the homepage of the discharge medical record. The testing sets were from October 1, 2017 to April 30, 2018 and from July 1, 2018 to January 31, 2019.

Results

The values of P were 89.27 and 88.38% in the first testing phase and the second testing phase, respectively, which demonstrate high precision. The automatic ICD-10 coding system completed more than 160,000 codes in 16 months, which reduced the workload of the coders. In addition, a comparison between the amount of time needed for manual coding and automatic coding indicated the effectiveness of the system-the time needed for automatic coding takes nearly 100 times less than manual coding.

Conclusions

Our automatic coding system is well suited for the coding task. Further studies are warranted to perfect the description models of the regexps and to develop synthetic approaches to improve system performance.

Identifying Predictors of Suicide in Severe Mental Illness: A Feasibility Study of a Clinical Prediction Rule (Oxford Mental Illness and Suicide Tool or OxMIS)

BACKGROUND

Oxford Mental Illness and Suicide tool (OxMIS) is a brief, scalable, freely available, structured risk assessment tool to assess suicide risk in patients with severe mental illness (schizophrenia-spectrum disorders or bipolar disorder). OxMIS requires further external validation, but a lack of large-scale cohorts with relevant variables makes this challenging. Electronic health records provide possible data sources for external validation of risk prediction tools. However, they contain large amounts of information within free-text that is not readily extractable. In this study, we examined the feasibility of identifying suicide predictors needed to validate OxMIS in routinely collected electronic health records.

METHODS

In study 1, we manually reviewed electronic health records of 57 patients with severe mental illness to calculate OxMIS risk scores. In study 2, we examined the feasibility of using natural language processing to scale up this process. We used anonymized free-text documents from the Clinical Record Interactive Search database to train a named entity recognition model, a machine learning technique which recognizes concepts in free-text. The model identified eight concepts relevant for suicide risk assessment: medication (antidepressant/antipsychotic treatment), violence, education, self-harm, benefits receipt, drug/alcohol use disorder, suicide, and psychiatric admission. We assessed model performance in terms of precision (similar to positive predictive value), recall (similar to sensitivity) and F1 statistic (an overall performance measure).

RESULTS

In study 1, we estimated suicide risk for all patients using the OxMIS calculator, giving a range of 12 month risk estimates from 0.1-3.4%. For 13 out of 17 predictors, there was no missing information in electronic health records. For the remaining 4 predictors missingness ranged from 7-26%; to account for these missing variables, it was possible for OxMIS to estimate suicide risk using a range of scores. In study 2, the named entity recognition model had an overall precision of 0.77, recall of 0.90 and F1 score of 0.83. The concept with the best precision and recall was medication (precision 0.84, recall 0.96) and the weakest were suicide (precision 0.37), and drug/alcohol use disorder (recall 0.61).

CONCLUSIONS

It is feasible to estimate suicide risk with the OxMIS tool using predictors identified in routine clinical records. Predictors could be extracted using natural language processing. However, electronic health records differ from other data sources, particularly for family history variables, which creates methodological challenges.

Validation of an Electronic Health Record-Based Suicide Risk Prediction Modeling Approach Across Multiple Health Care Systems

Importance

Suicide is a leading cause of mortality, with suicide-related deaths increasing in recent years. Automated methods for individualized risk prediction have great potential to address this growing public health threat. To facilitate their adoption, they must first be validated across diverse health care settings.

Objective

To evaluate the generalizability and cross-site performance of a risk prediction method using readily available structured data from electronic health records in predicting incident suicide attempts across multiple, independent, US health care systems.

Design, Setting, and Participants

For this prognostic study, data were extracted from longitudinal electronic health record data comprising International Classification of Diseases, Ninth Revision diagnoses, laboratory test results, procedures codes, and medications for more than 3.7 million patients from 5 independent health care systems participating in the Accessible Research Commons for Health network. Across sites, 6 to 17 years' worth of data were available, up to 2018. Outcomes were defined by International Classification of Diseases, Ninth Revision codes reflecting incident suicide attempts (with positive predictive value >0.70 according to expert clinician medical record review). Models were trained using naive Bayes classifiers in each of the 5 systems. Models were cross-validated in independent data sets at each site, and performance metrics were calculated. Data analysis was performed from November 2017 to August 2019.

Main Outcomes and Measures

The primary outcome was suicide attempt as defined by a previously validated case definition using International Classification of Diseases, Ninth Revision codes. The accuracy and timeliness of the prediction were measured at each site.

Results

Across the 5 health care systems, of the 3 714 105 patients (2 130 454 female [57.2%]) included in the analysis, 39 162 cases (1.1%) were identified. Predictive features varied by site but, as expected, the most common predictors reflected mental health conditions (eg, borderline personality disorder, with odds ratios of 8.1-12.9, and bipolar disorder, with odds ratios of 0.9-9.1) and substance use disorders (eg, drug withdrawal syndrome, with odds ratios of 7.0-12.9). Despite variation in geographical location, demographic characteristics, and population health characteristics, model performance was similar across sites, with areas under the curve ranging from 0.71 (95% CI, 0.70-0.72) to 0.76 (95% CI, 0.75-0.77). Across sites, at a specificity of 90%, the models detected a mean of 38% of cases a mean of 2.1 years in advance.

Conclusions and Relevance

Across 5 diverse health care systems, a computationally efficient approach leveraging the full spectrum of structured electronic health record data was able to detect the risk of suicidal behavior in unselected patients. This approach could facilitate the development of clinical decision support tools that inform risk reduction interventions.

Automatic extraction and assessment of lifestyle exposures for Alzheimer's disease using natural language processing

INTRODUCTION

Previous biomedical studies identified many lifestyle exposures that could possibly represent risk factors for dementia in general or dementia due to Alzheimer's disease (AD). These lifestyle exposures are mainly mentioned in free-text electronic health records (EHRs). However, automatic extraction and assessment of these exposures using EHRs remains understudied.

METHODS

A natural language processing (NLP) approach was adopted to extract lifestyle exposures and intervention strategies from the clinical notes of 260 patients with clinical diagnoses of AD dementia and 260 age-matched cognitively unimpaired persons. Statistics of lifestyle exposures were compared between these two groups. The mapping results of the NLP extraction were evaluated by comparing the results with data captured independently by clinicians.

RESULTS

Thirty out of fifty-five potentially relevant lifestyle exposures were mentioned in our clinical note dataset. Twenty-two dietary factors and three substance abuses that were potentially relevant were not found in clinical notes. Patients with AD dementia were significantly exposed to more of the potential risk factors compared to the cognitively unimpaired subjects (χ2 = 120.31, p-value < 0.001). The average accuracy of the automated extraction was 74.0% in comparison with the manual review of randomly selected 50 sample documents.

DISCUSSION AND CONCLUSION

We illustrated the feasibility of NLP techniques for the automated evaluation of a large number lifestyle habits using free-text EHR data. We found that AD dementia patients were exposed to more of the potential risk factors than the comparison group. Our results also demonstrated the feasibility and accuracy of investigating putative risk factors using NLP techniques.

Machine learning discovery of longitudinal patterns of depression and suicidal ideation

Background and aim

Depression is often accompanied by thoughts of self-harm, which are a strong predictor of subsequent suicide attempt and suicide death. Few empirical data are available regarding the temporal correlation between depression symptoms and suicidal ideation. We investigated the anecdotal concern that suicidal ideation may increase during a period of depression improvement.

Data

Longitudinal Patient Health Questionnaire (PHQ)-9 is a questionnaire of 9 multiple-choice questions to assess the frequency of depressive symptoms within the previous two weeks. We analyzed a chronic depression treatment population’s electronic health record (EHR) data, containing 610 patients’ longitudinal PHQ-9 scores (62% age 45 and older; 68% female) within 40 weeks.

Methods

The irregular and sparse EHR data were transformed into continuous trajectories using Gaussian process regression. We first estimated the correlations between the symptoms (total score of the first 8 questions; PHQ-8) and suicide ideation (9th question score; Item 9) using the cross-correlation function. We then used an artificial neural network (ANN) to discover subtypes of depression patterns from the fitted depression trajectories. In addition, we conducted a separate analysis using the unfitted raw PHQ scores to examine PHQ-8’s and Item 9’s pattern changes.

Results

Results showed that the majority of patients’ PHQ-8 and Item 9 scores displayed strong temporal correlations. We found five patterns in the PHQ-8 and the Item 9 trajectories. We also found 8% - 13% of the patients have experienced an increase in suicidal ideation during the improvement of their PHQ-8. Using a trajectory-based method for subtype pattern detection in depression progression, we provided a better understanding of temporal correlations between depression symptoms over time.

Associations of antepartum suicidal behaviour with adverse infant and obstetric outcomes

BACKGROUND

Relatively little is known about antepartum suicidal behaviour and pregnancy outcomes. We examined associations of antepartum suicidal behaviour, alone and in combination with psychiatric disorders, with adverse infant and obstetric outcomes.

METHODS

We included 188 925 singleton livebirths from a retrospective cohort (1996-2016). Suicidal behaviour, psychiatric disorders, and outcomes were derived from electronic medical records. We performed multivariable logistic regressions with generalised estimating equations to estimate adjusted odds ratios (aOR) with 95% confidence intervals (95%CI).

RESULTS

The prevalence of antepartum suicidal behaviour was 152.44 per 100 000 singleton livebirths. Nearly two-thirds (64.24%) of women with suicidal behaviour also had psychiatric disorders. Compared to women without psychiatric disorders and suicidal behaviour, women with psychiatric disorders alone had 1.3-fold to 1.4-fold increased odds of delivering low birthweight or preterm infants and 1.2-fold increased odds of experiencing obstetric complications. Women with suicidal behaviour alone had increased odds of preterm labour (aOR 2.05, 95% CI 1.16, 3.62). Women with both suicidal behaviour and psychiatric disorders had > twofold increased odds of delivering low birthweight (aOR 2.52, 95% CI 1.40, 4.54), preterm birth (aOR 2.44, 95% CI 1.63, 3.66), and low birthweight/preterm birth (aOR 2.30, 95% CI 1.54, 3.44) infants; the odds of preterm labour (aOR 1.62, 95% CI 1.06, 2.47), placental abruption (aOR 2.33, 95% CI 1.20, 4.51), preterm rupture of membranes (aOR 1.63, 95% CI 1.08, 2.46), and postpartum haemorrhage (aOR 1.93, 95%CI 1.09, 3.40) were elevated.

CONCLUSIONS

Antepartum suicidal behaviour, when co-occurring with psychiatric disorders, is associated with increased odds of adverse infant and obstetric outcomes. Future studies are warranted to understand the causal roles of suicidal behaviour and psychiatric disorders in pregnancy.

Use of natural language processing in electronic medical records to identify pregnant women with suicidal behavior: towards a solution to the complex classification problem

We developed algorithms to identify pregnant women with suicidal behavior using information extracted from clinical notes by natural language processing (NLP) in electronic medical records. Using both codified data and NLP applied to unstructured clinical notes, we first screened pregnant women in Partners HealthCare for suicidal behavior. Psychiatrists manually reviewed clinical charts to identify relevant features for suicidal behavior and to obtain gold-standard labels. Using the adaptive elastic net, we developed algorithms to classify suicidal behavior. We then validated algorithms in an independent validation dataset. From 275,843 women with codes related to pregnancy or delivery, 9331 women screened positive for suicidal behavior by either codified data (N = 196) or NLP (N = 9,145). Using expert-curated features, our algorithm achieved an area under the curve of 0.83. By setting a positive predictive value comparable to that of diagnostic codes related to suicidal behavior (0.71), we obtained a sensitivity of 0.34, specificity of 0.96, and negative predictive value of 0.83. The algorithm identified 1423 pregnant women with suicidal behavior among 9331 women screened positive. Mining unstructured clinical notes using NLP resulted in a 11-fold increase in the number of pregnant women identified with suicidal behavior, as compared to solely reliance on diagnostic codes.