Evaluating Automatic Methods to Extract Patients' Supplement Use from Clinical Reports

Patient Dietary Supplements Use: Do Results from Natural Language Processing of Clinical Notes Agree with Survey Data?

There is widespread use of dietary supplements, some prescribed but many taken without a physician's guidance. There are many potential interactions between supplements and both over-the-counter and prescription medications in ways that are unknown to patients. Structured medical records do not adequately document supplement use; however, unstructured clinical notes often contain extra information on supplements. We studied a group of 377 patients from three healthcare facilities and developed a natural language processing (NLP) tool to detect supplement use. Using surveys of these patients, we investigated the correlation between self-reported supplement use and NLP extractions from the clinical notes. Our model achieved an F1 score of 0.914 for detecting all supplements. Individual supplement detection had a variable correlation with survey responses, ranging from an F1 of 0.83 for calcium to an F1 of 0.39 for folic acid. Our study demonstrated good NLP performance while also finding that self-reported supplement use is not always consistent with the documented use in clinical records.

The Use of Artificial Intelligence in Pharmacovigilance: A Systematic Review of the Literature

INTRODUCTION

Artificial intelligence through machine learning uses algorithms and prior learnings to make predictions. Recently, there has been interest to include more artificial intelligence in pharmacovigilance of products already in the market and pharmaceuticals in development.

OBJECTIVE

The aim of this study was to identify and describe the uses of artificial intelligence in pharmacovigilance through a systematic literature review.

METHODS

Embase and MEDLINE database searches were conducted for articles published from January 1, 2015 to July 9, 2021 using search terms such as 'pharmacovigilance,' 'patient safety,' 'artificial intelligence,' and 'machine learning' in the title or abstract. Scientific articles that contained information on the use of artificial intelligence in all modalities of patient safety or pharmacovigilance were reviewed and synthesized using a pre-specified data extraction template. Articles with incomplete information and letters to editor, notes, and commentaries were excluded.

RESULTS

Sixty-six articles were identified for evaluation. Most relevant articles on artificial intelligence focused on machine learning, and it was used in patient safety in the identification of adverse drug events (ADEs) and adverse drug reactions (ADRs) (57.6%), processing safety reports (21.2%), extraction of drug-drug interactions (7.6%), identification of populations at high risk for drug toxicity or guidance for personalized care (7.6%), prediction of side effects (3.0%), simulation of clinical trials (1.5%), and integration of prediction uncertainties into diagnostic classifiers to increase patient safety (1.5%). Artificial intelligence has been used to identify safety signals through automated processes and training with machine learning models; however, the findings may not be generalizable given that there were different types of data included in each source.

CONCLUSION

Artificial intelligence allows for the processing and analysis of large amounts of data and can be applied to various disease states. The automation and machine learning models can optimize pharmacovigilance processes and provide a more efficient way to analyze information relevant to safety, although more research is needed to identify if this optimization has an impact on the quality of safety analyses. It is expected that its use will increase in the near future, particularly with its role in the prediction of side effects and ADRs.

Using natural language processing methods to classify use status of dietary supplements in clinical notes

Background

Despite widespread use, the safety of dietary supplements is open to doubt due to the fact that they can interact with prescribed medications, leading to dangerous clinical outcomes. Electronic health records (EHRs) provide a potential way for active pharmacovigilance on dietary supplements since a fair amount of dietary supplement information, especially those on use status, can be found in clinical notes. Extracting such information is extremely significant for subsequent supplement safety research.

Methods

In this study, we collected 2500 sentences for 25 commonly used dietary supplements and annotated into four classes: Continuing (C), Discontinued (D), Started (S) and Unclassified (U). Both rule-based and machine learning-based classifiers were developed on the same training set and evaluated using the hold-out test set. The performances of the two classifiers were also compared.

Results

The rule-based classifier achieved F-measure of 0.90, 0.85, 0.90, and 0.86 in C, D, S, and U status, respectively. The optimal machine learning-based classifier (Maximum Entropy) achieved F-measure of 0.90, 0.92, 0.91 and 0.88 in C, D, S, and U status, respectively. The comparison result shows that the machine learning-based classifier has a better performance, which is more efficient and scalable especially when the sample size doubles.

Conclusions

Machine learning-based classifier outperforms rule-based classifier in categorization of the use status of dietary supplements in clinical notes. Future work includes applying deep learning methods and developing a hybrid system to approach use status classification task.

Detecting Signals of Associations between Dietary Supplement Use and Mental Disorders from Twitter

In this study, we attempted to detect signals of association between dietary supplement use and mental disorders from Twitter. We collected tweets ranging from 2016 to 2017 which mention five dietary supplements. A case cohort of 257 users were identified by adapting a natural language processing method with further manually verified to have taken one supplement. We then randomly selected 257 users who had not taken any dietary supplement as the control cohort and compared the sentiment and mental health signals of their tweets to the case cohort. We have observed significant differences in the sad, anxious, and negative sentiment between the two cohorts. These results have shown that Twitter is a potential source for detecting signals of association between dietary supplement use and anxiety disorders, depression, and mood disorders.