Filtering big data from social media--Building an early warning system for adverse drug reactions

Enhancing the Efficiency of the Individual Case Safety Report (ICSR) Quality and Compliance through Automation

BACKGROUND

Over the past few years, major inspection findings have been identified in the "management of adverse reactions" that may be due to increasing workload in pharmaceutical organizations impacting the correctness of information in individual case safety reports (ICSRs). Although retrospective quality check (Retro-QC) and late submission analyses are important steps in ensuring ICSR quality, their manual application poses several challenges that can be overcome through automation.

OBJECTIVES

To improve the efficiency of the Retro-QC analysis and late submission analysis using a computer-operated tool called Compliance and Metrics Management (CMM) tool, and to measure the tool's effectiveness in terms of productivity, time, and cost savings by comparing against the manual process.

METHODS

Time savings were calculated by measuring the difference in time taken during the manual process versus the automated process. Cost savings were measured in terms of hourly remuneration for the time saved. Productivity was calculated as the difference between the number of cases handled in the manual versus automated process. Thus, the overall efficiency was measured in terms of time and cost savings along with increased productivity.

RESULTS

Automation resulted in time savings of 49% and cost savings of 43% for Retro-QC analysis, and the productivity level increased by 67%. For late submission analysis, the CMM tool resulted in time savings of 88% and cost savings of 87%.

CONCLUSION

CMM tool enhanced the efficiency of both Retro-QC and late submission analyses by increasing productivity along with time and cost savings. It also reduced the number of errors, thereby enhancing the accuracy of the process and overall compliance.

Integrating unsupervised and supervised learning techniques to predict traumatic brain injury: A population-based study

This work aimed to identify pre-existing health conditions of patients with traumatic brain injury (TBI) and develop predictive models for the first TBI event and its external causes by employing a combination of unsupervised and supervised learning algorithms. We acquired up to five years of pre-injury diagnoses for 488,107 patients with TBI and 488,107 matched control patients who entered the emergency department or acute care hospitals between April 1st, 2002, and March 31st, 2020. Diagnoses were obtained from the Ontario Health Insurance Plan (OHIP) database which contains province-wide claims data by physicians in Ontario, Canada for inpatient and outpatient services. A screening process was conducted on the OHIP diagnostic codes to limit the subsequent analysis to codes that were predictive of TBI, which concluded that 314 codes were significantly associated with TBI. The Latent Dirichlet Allocation (LDA) model was applied to the diagnostic codes and generated an optimal number of 19 topics that concur with published literature but also suggest other unexplored areas. Estimated word-topic probabilities from the LDA model helped us detect pre-morbid conditions among patients with TBI by uncovering the underlying patterns of diagnoses, meanwhile estimated document-topic probabilities were utilized in variable creation as form of a dimension reduction. We created 19 topic scores for each patient in the cohort which were utilized along with socio-demographic factors for Random Forest binary classifier models. Test set performances evaluated using area under the receiver operating characteristic curve (AUC) were: TBI event (AUC = 0.85), external cause of injury: falls (AUC = 0.85), struck by/against (AUC = 0.83), cyclist collision (AUC = 0.76), motor vehicle collision (AUC = 0.83). Our analysis successfully demonstrated the feasibility of using machine learning to predict TBI due to various external causes and identified the most important factors that contribute to this prediction.

COVID-19 Vaccine Adverse Event Detection Based on Multi-Label Classification With Various Label Selection Strategies

Analyzing massive VAERS reports without medical context may lead to incorrect conclusions about vaccine adverse events (VAE). Facilitating VAE detection promotes continual safety improvement for new vaccines. This study proposes a multi-label classification method with various term-and topic-based label selection strategies to improve the accuracy and efficiency of VAE detection. Topic modeling methods are first used to generate rule-based label dependencies from Medical Dictionary for Regulatory Activities terms in VAE reports with two hyper-parameters. Multiple label selection strategies, namely one-vs-rest (OvsR), problem transformation (PT), algorithm adaption (AA), and deep learning (DL) methods, are used in multi-label classification to examine the model performance, respectively. Experimental results indicated that the topic-based PT methods improve the accuracy by up to 33.69% using a COVID-19 VAE reporting data set, which improves the robustness and interpretability of our models. In addition, the topic-based OvsR methods achieve an optimal accuracy of up to 98.88%. The accuracy of the AA methods with topic-based labels increased by up to 87.36%. By contrast, the state-of-art LSTM- and BERT-based DL methods have relatively poor performance with accuracy rates of 71.89% and 64.63%, respectively. Our findings reveal that the proposed method effectively improves the model accuracy and strengthens VAE interpretability by using different label selection strategies and domain knowledge in multi-label classification for VAE detection.

A Disease-Prediction Protocol Integrating Triage Priority and BERT-Based Transfer Learning for Intelligent Triage

Large hospitals can be complex, with numerous discipline and subspecialty settings. Patients may have limited medical knowledge, making it difficult for them to determine which department to visit. As a result, visits to the wrong departments and unnecessary appointments are common. To address this issue, modern hospitals require a remote system capable of performing intelligent triage, enabling patients to perform self-service triage. To address the challenges outlined above, this study presents an intelligent triage system based on transfer learning, capable of processing multilabel neurological medical texts. The system predicts a diagnosis and corresponding department based on the patient’s input. It utilizes the triage priority (TP) method to label diagnostic combinations found in medical records, converting a multilabel problem into a single-label one. The system considers disease severity and reduces the “class overlapping” of the dataset. The BERT model classifies the chief complaint text, predicting a primary diagnosis corresponding to the complaint. To address data imbalance, a composite loss function based on cost-sensitive learning is added to the BERT architecture. The study results indicate that the TP method achieves a classification accuracy of 87.47% on medical record text, outperforming other problem transformation methods. By incorporating the composite loss function, the system’s accuracy rate improves to 88.38% surpassing other loss functions. Compared to traditional methods, this system does not introduce significant complexity, yet substantially improves triage accuracy, reduces patient input confusion, and enhances hospital triage capabilities, ultimately improving the patient’s medical experience. The findings could provide a reference for intelligent triage development.

Vaccine Vigilance System: Considerations on the Effectiveness of Vigilance Data Use in COVID-19 Vaccination

(1) Background: The safety of medicines has been receiving increased attention to ensure that the risks of taking medicines do not outweigh the benefits. This is the reason why, over several decades, the pharmacovigilance system has been developed. The post-authorization pharmacovigilance system is based on reports from healthcare professionals and patients on observed adverse reactions. The reports are collected in databases and progressively evaluated. However, there are emerging concerns about the effectiveness of the established passive pharmacovigilance system in accelerating circumstances, such as the COVID-19 pandemic, when billions of doses of new vaccines were administered without a long history of use. Currently, health professionals receive fragmented new information on the safety of medicines from competent authorities after a lengthy evaluation process. Simultaneously, in the context of accelerated mass vaccination, health professionals need to have access to operational information-at least on organ systems at higher risk. Therefore, the aim of this study was to perform a primary data analysis of publicly available data on suspected COVID-19 vaccine-related adverse reactions in Europe, in order to identify the predominant groups of reported medical conditions after vaccination and their association with vaccine groups, as well as to evaluate the data accessibility on specific syndromes. (2) Methods: To achieve the objectives, the data publicly available in the EudraVigilance European Database for Suspected Adverse Drug Reaction Reports were analyzed. The following tasks were defined to: (1) Identify the predominant groups of medical conditions mentioned in adverse reaction reports; (2) determine the relative frequency of reports within vaccine groups; (3) assess the feasibility of obtaining information on a possibly associated syndrome-myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS). (3) Results: The data obtained demonstrate that the predominant medical conditions induced after vaccination are relevant to the following categories: (1) "General disorders and administration site conditions", (2) "nervous system disorders", and (3) "musculoskeletal and connective tissue disorders". There are more reports for mRNA vaccines, but the relative frequency of reports per dose administered, is lower for this group of vaccines. Information on ME/CFS was not available, but reports of "chronic fatigue syndrome" are included in the database and accessible for primary analysis. (4) Conclusions: The information obtained on the predominantly reported medical conditions and the relevant vaccine groups may be useful for health professionals, patients, researchers, and medicine manufacturers. Policymakers could benefit from reflecting on the design of an active pharmacovigilance model, making full use of modern information technologies, including big data analysis of social media and networks for the detection of primary signals and building an early warning system.

Artificial Intelligent Context-Aware Machine-Learning Tool to Detect Adverse Drug Events from Social Media Platforms

INTRODUCTION

Pharmacovigilance (PV) has proven to detect post-marketing adverse drug events (ADE). Previous research used the natural language processing (NLP) tool to extract unstructured texts relevant to ADEs. However, texts without context reduce the efficiency of such algorithms. Our objective was to develop and validate an innovative NLP tool, aTarantula, using a context-aware machine-learning algorithm to detect existing ADEs from social media using an aggregated lexicon.

METHOD

aTarantula utilized FastText embeddings and an aggregated lexicon to extract contextual data from three patient forums (i.e., MedHelp, MedsChat, and PatientInfo) taking warfarin. The lexicon used warfarin package inserts and synonyms of warfarin ADEs from UMLS and FAERS databases. Data was stored on SQLite and then refined and manually checked by three clinical pharmacists for validation.

RESULTS

Multiple organ systems where the most frequent ADE were reported at 1.50%, followed by CNS side effects at 1.19%. Lymphatic system ADEs were the least common side effect reported at 0.09%. The overall Spearman rank correlation coefficient between patient-reported data from the forums and FAERS was 0.19. As determined by pharmacist validation, aTarantula had a sensitivity of 84.2% and a specificity of 98%. Three clinical pharmacists manually validated our results. Finally, we created an aggregated lexicon for mining ADEs from social media.

CONCLUSION

We successfully developed aTarantula, a machine-learning algorithmn based on artificial intelligence to extract warfarin-related ADEs from online social discussion forums automatically. Our study shows that it is feasible to use aTarantula to detect ADEs. Future researchers can validate aTarantula on the diverse dataset.

Language-agnostic deep learning framework for automatic monitoring of population-level mental health from social networks

In many countries, mental health issues are among the most serious public health concerns. National mental health statistics are frequently collected from reported patient cases or government-sponsored surveys, which have restricted coverage, frequency, and timeliness. Many domains of study, including public healthcare and biomedical informatics, have recently adopted social media data as a feasible real-time alternative to traditional methods of gathering representative information at the population level in a variety of contexts. However, because of the limits of fundamental natural language processing tools and labeled corpora in countries with limited natural language resources, such as Thailand, implementing social media systems to monitor mental health signals could be challenging. This paper presents LAPoMM, a novel framework for monitoring real-time mental health indicators from social media data without using labeled datasets in low-resource languages. Specifically, we use cross-lingual methods to train language-agnostic models and validate our framework by examining cross-correlations between the aggregate predicted mental signals and real-world administrative data from Thailand's Department of Mental Health, which includes monthly depression patients and reported cases of suicidal attempts. A combination of a language-agnostic representation and a deep learning classification model outperforms all other cross-lingual techniques for recognizing various mental signals in Tweets, such as emotions, sentiments, and suicidal tendencies. The correlation analyses discover a strong positive relationship between actual depression cases and the predicted negative sentiment signals as well as suicide attempts and negative signals (e.g., fear, sadness, and disgust) and suicidal tendency. These findings establish the effectiveness of our proposed framework and its potential applications in monitoring population-level mental health using large-scale social media data. Furthermore, because the language-agnostic model utilized in the methodology is capable of supporting a wide range of languages, the proposed LAPoMM framework can be easily generalized for analogous applications in other countries with limited language resources.

Identifying Adverse Drug Reaction-Related Text from Social Media: A Multi-View Active Learning Approach with Various Document Representations

Adverse drug reactions (ADRs) are a huge public health issue. Identifying text that mentions ADRs from a large volume of social media data is important. However, we need to address two challenges for high-performing ADR-related text detection: the data imbalance problem and the requirement of simultaneously using data-driven information and handcrafted information. Therefore, we propose an approach named multi-view active learning using domain-specific and data-driven document representations (MVAL4D), endeavoring to enhance the predictive capability and alleviate the requirement of labeled data. Specifically, a new view-generation mechanism is proposed to generate multiple views by simultaneously exploiting various document representations obtained using handcrafted feature engineering and by performing deep learning methods. Moreover, different from previous active learning studies in which all instances are chosen using the same selection criterion, MVAL4D adopts different criteria (i.e., confidence and informativeness) to select potentially positive instances and potentially negative instances for manual annotation. The experimental results verify the effectiveness of MVAL4D. The proposed approach can be generalized to many other text classification tasks. Moreover, it can offer a solid foundation for the ADR mention extraction task, and improve the feasibility of monitoring drug safety using social media data.

An Attention-Based Multi-Representational Fusion Method for Social-Media-Based Text Classification

There exist various text-classification tasks using user-generated contents (UGC) on social media in the big data era. In view of advantages and disadvantages of feature-engineering-based machine-learning models and deep-learning models, we argue that fusing handcrafted-text representation via feature engineering and data-driven deep-text representations extracted by performing deep-learning methods is conducive to enhancing text-classification capability. Given the characteristics of different deep neural networks, their complementary effect needs to be investigated. Moreover, contributions of these representations need to be adaptively learned when it comes to addressing different tasks or predicting different samples. Therefore, in this paper, we propose a novel fused deep-neural-network architecture with a hierarchical attention mechanism for text classification with social media data. Specifically, in the context that handcraft features are available, we employ the attention mechanism to adaptively fuse totally data-driven-text representation and handcrafted representation. For the generation of the data-driven-text representation, we propose a data-driven encoder that fuses text representations derived from three deep-learning methods with the attention mechanism, to adaptively select discriminative representation and explore their complementary effect. To verify the effectiveness of our approach, we performed two text-classification tasks, i.e., identifying adverse drug reaction (ADR)-relevant tweets from social media and identifying comparative-relevant reviews from an E-commerce platform. Experimental results demonstrate that our approach outperforms other baselines.

Big data analytics meets social media: A systematic review of techniques, open issues, and future directions

•

A comprehensive systematic review on social big data analytic approaches is provided.

•

The main methods, pros, cons, evaluation methods, and parameters are discussed.

•

A scientific taxonomy of social big data analytic approaches is presented.

•

A detailed list of challenges and future research directions is outlined.

Social Networking Services (SNSs) connect people worldwide, where they communicate through sharing contents, photos, videos, posting their first-hand opinions, comments, and following their friends. Social networks are characterized by velocity, volume, value, variety, and veracity, the 5 V’s of big data. Hence, big data analytic techniques and frameworks are commonly exploited in Social Network Analysis (SNA). By the ever-increasing growth of social networks, the analysis of social data, to describe and find communication patterns among users and understand their behaviors, has attracted much attention. In this paper, we demonstrate how big data analytics meets social media, and a comprehensive review is provided on big data analytic approaches in social networks to search published studies between 2013 and August 2020, with 74 identified papers. The findings of this paper are presented in terms of main journals/conferences, yearly distributions, and the distribution of studies among publishers. Furthermore, the big data analytic approaches are classified into two main categories: Content-oriented approaches and network-oriented approaches. The main ideas, evaluation parameters, tools, evaluation methods, advantages, and disadvantages are also discussed in detail. Finally, the open challenges and future directions that are worth further investigating are discussed.

Spontaneously Generated Online Patient Experience of Modafinil: A Qualitative and NLP Analysis

Objective: To compare the findings from a qualitative and a natural language processing (NLP) based analysis of online patient experience posts on patient experience of the effectiveness and impact of the drug Modafinil.

Methods: Posts (n = 260) from 5 online social media platforms where posts were publicly available formed the dataset/corpus. Three platforms asked posters to give a numerical rating of Modafinil. Thematic analysis: data was coded and themes generated. Data were categorized into PreModafinil, Acquisition, Dosage, and PostModafinil and compared to identify each poster's own view of whether taking Modafinil was linked to an identifiable outcome. We classified this as positive, mixed, negative, or neutral and compared this with numerical ratings. NLP: Corpus text was speech tagged and keywords and key terms extracted. We identified the following entities: drug names, condition names, symptoms, actions, and side-effects. We searched for simple relationships, collocations, and co-occurrences of entities. To identify causal text, we split the corpus into PreModafinil and PostModafinil and used n-gram analysis. To evaluate sentiment, we calculated the polarity of each post between −1 (negative) and +1 (positive). NLP results were mapped to qualitative results.

Results: Posters had used Modafinil for 33 different primary conditions. Eight themes were identified: the reason for taking (condition or symptom), impact of symptoms, acquisition, dosage, side effects, other interventions tried or compared to, effectiveness of Modafinil, and quality of life outcomes. Posters reported perceived effectiveness as follows: 68% positive, 12% mixed, 18% negative. Our classification was consistent with poster ratings. Of the most frequent 100 keywords/keyterms identified by term extraction 88/100 keywords and 84/100 keyterms mapped directly to the eight themes. Seven keyterms indicated negation and temporal states. Sentiment was as follows 72% positive sentiment 4% neutral 24% negative. Matching of sentiment between the qualitative and NLP methods was accurate in 64.2% of posts. If we allow for one category difference matching was accurate in 85% of posts.

Conclusions: User generated patient experience is a rich resource for evaluating real world effectiveness, understanding patient perspectives, and identifying research gaps. Both methods successfully identified the entities and topics contained in the posts. In contrast to current evidence, posters with a wide range of other conditions found Modafinil effective. Perceived causality and effectiveness were identified by both methods demonstrating the potential to augment existing knowledge.

Assessing Patient-Perceived Hospital Service Quality and Sentiment in Malaysian Public Hospitals Using Machine Learning and Facebook Reviews

Social media is emerging as a new avenue for hospitals and patients to solicit input on the quality of care. However, social media data is unstructured and enormous in volume. Moreover, no empirical research on the use of social media data and perceived hospital quality of care based on patient online reviews has been performed in Malaysia. The purpose of this study was to investigate the determinants of positive sentiment expressed in hospital Facebook reviews in Malaysia, as well as the association between hospital accreditation and sentiments expressed in Facebook reviews. From 2017 to 2019, we retrieved comments from 48 official public hospitals' Facebook pages. We used machine learning to build a sentiment analyzer and service quality (SERVQUAL) classifier that automatically classifies the sentiment and SERVQUAL dimensions. We utilized logistic regression analysis to determine our goals. We evaluated a total of 1852 reviews and our machine learning sentiment analyzer detected 72.1% of positive reviews and 27.9% of negative reviews. We classified 240 reviews as tangible, 1257 reviews as trustworthy, 125 reviews as responsive, 356 reviews as assurance, and 1174 reviews as empathy using our machine learning SERVQUAL classifier. After adjusting for hospital characteristics, all SERVQUAL dimensions except Tangible were associated with positive sentiment. However, no significant relationship between hospital accreditation and online sentiment was discovered. Facebook reviews powered by machine learning algorithms provide valuable, real-time data that may be missed by traditional hospital quality assessments. Additionally, online patient reviews offer a hitherto untapped indication of quality that may benefit all healthcare stakeholders. Our results confirm prior studies and support the use of Facebook reviews as an adjunct method for assessing the quality of hospital services in Malaysia.

Assessing Patient Perceptions and Experiences of Paracetamol in France: Infodemiology Study Using Social Media Data Mining

BACKGROUND

Individuals frequently turning to social media to discuss medical conditions and medication, sharing their experiences and information and asking questions among themselves. These online discussions can provide valuable insights into individual perceptions of medical treatment, and increasingly, studies are focusing on the potential use of this information to improve health care management.

OBJECTIVE

The objective of this infodemiology study was to identify social media posts mentioning paracetamol-containing products to develop a better understanding of patients' opinions and perceptions of the drug.

METHODS

Posts between January 2003 and March 2019 containing at least one mention of paracetamol were extracted from 18 French forums in May 2019 with the use of the Detec't (Kap Code) web crawler. Posts were then analyzed using the automated Detec't tool, which uses machine learning and text mining methods to inspect social media posts and extract relevant content. Posts were classified into groups: Paracetamol Only, Paracetamol and Opioids, Paracetamol and Others, and the Aggregate group.

RESULTS

Overall, 44,283 posts were analyzed from 20,883 different users. Post volume over the study period showed a peak in activity between 2009 and 2012, as well as a spike in 2017 in the Aggregate group. The number of posts tended to be higher during winter each year. Posts were made predominantly by women (14,897/20,883, 71.34%), with 12.00% (2507/20,883) made by men and 16.67% (3479/20,883) by individuals of unknown gender. The mean age of web users was 39 (SD 19) years. In the Aggregate group, pain was the most common medical concept discussed (22,257/37,863, 58.78%), and paracetamol risk was the most common discussion topic, addressed in 20.36% (8902/43,725) of posts. Doliprane was the most common medication mentioned (14,058/44,283, 31.74%) within the Aggregate group, and tramadol was the most commonly mentioned drug in combination with paracetamol in the Aggregate group (1038/19,587, 5.30%). The most common unapproved indication mentioned within the Paracetamol Only group was fatigue (190/616, with 16.32% positive for an unapproved indication), with reference to dependence made by 1.61% (136/8470) of the web users, accounting for 1.33% (171/12,843) of the posts in the Paracetamol Only group. Dependence mentions in the Paracetamol and Opioids group were provided by 6.94% (248/3576) of web users, accounting for 5.44% (342/6281) of total posts. Reference to overdose was made by 245 web users across 291 posts within the Paracetamol Only group. The most common potential adverse event detected was nausea (306/12843, 2.38%) within the Paracetamol Only group.

CONCLUSIONS

The use of social media mining with the Detec't tool provided valuable information on the perceptions and understanding of the web users, highlighting areas where providing more information for the general public on paracetamol, as well as other medications, may be of benefit.

The Role of Advanced Technologies Supplemented with Traditional Methods in Pharmacovigilance Sciences

BACKGROUND

The immediate automatic systemic monitoring and reporting of adverse drug reactions, improving the efficacy is the utmost need of the medical informatics community. The venturing of advanced digital technologies into the health sector has opened new avenues for rapid monitoring. In recent years, data shared through social media, mobile apps, and other social websites has increased manifolds requiring data mining techniques.

OBJECTIVE

The objective of this report is to highlight the role of advanced technologies together with the traditional methods to proactively aid in the early detection of adverse drug reactions concerned with drug safety and pharmacovigilance.

METHODS

A thorough search was conducted on papers and patents regarding pharmacovigilance. All articles with respect to the relevant subject were explored and mined from public repositories such as Pubmed, Google Scholar, Springer, ScienceDirect (Elsevier), Web of Science, etc. Results: The European Union's Innovative Medicines Initiative WEB-RADR project has emphasized the development of mobile applications and social media data for reporting adverse effects. Only relevant data has to be captured through the data mining algorithms (DMAs) as it plays an important role in timely prediction of risk with high accuracy using two popular approaches; the frequentist and Bayesian approach. Pharmacovigilance at the pre-marketing stage is useful for the prediction of adverse drug reactions in the early developmental stage of a drug. Later, post-marketing safety reports and clinical data reports are important to be monitored through electronic health records, prescription-event monitoring, spontaneous reporting databases, etc. Conclusion: The advanced technologies supplemented with traditional technologies are the need of the hour for evaluating a product's risk profile and reducing risk in population especially with comorbid conditions and on concomitant medications.

Categorising patient concerns using natural language processing techniques

OBJECTIVES

Patient feedback is critical to identify and resolve patient safety and experience issues in healthcare systems. However, large volumes of unstructured text data can pose problems for manual (human) analysis. This study reports the results of using a semiautomated, computational topic-modelling approach to analyse a corpus of patient feedback.

METHODS

Patient concerns were received by Alberta Health Services between 2011 and 2018 (n=76 163), regarding 806 care facilities in 163 municipalities, including hospitals, clinics, community care centres and retirement homes, in a province of 4.4 million. Their existing framework requires manual labelling of pre-defined categories. We applied an automated latent Dirichlet allocation (LDA)-based topic modelling algorithm to identify the topics present in these concerns, and thereby produce a framework-free categorisation.

RESULTS

The LDA model produced 40 topics which, following manual interpretation by researchers, were reduced to 28 coherent topics. The most frequent topics identified were communication issues causing delays (frequency: 10.58%), community care for elderly patients (8.82%), interactions with nurses (8.80%) and emergency department care (7.52%). Many patient concerns were categorised into multiple topics. Some were more specific versions of categories from the existing framework (eg, communication issues causing delays), while others were novel (eg, smoking in inappropriate settings).

DISCUSSION

LDA-generated topics were more nuanced than the manually labelled categories. For example, LDA found that concerns with community care were related to concerns about nursing for seniors, providing opportunities for insight and action.

CONCLUSION

Our findings outline the range of concerns patients share in a large health system and demonstrate the usefulness of using LDA to identify categories of patient concerns.

Determination of Patient Sentiment and Emotion in Ophthalmology: Infoveillance Tutorial on Web-Based Health Forum Discussions

Background

Clinical data in social media are an underused source of information with great potential to allow for a deeper understanding of patient values, attitudes, and preferences.

Objective

This tutorial aims to describe a novel, robust, and modular method for the sentiment analysis and emotion detection of free text from web-based forums and the factors to consider during its application.

Methods

We mined the discussion and user information of all posts containing search terms related to a medical subspecialty (oculoplastics) from MedHelp, the largest web-based platform for patient health forums. We used data cleaning and processing tools to define the relevant subset of results and prepare them for sentiment analysis. We executed sentiment and emotion analyses by using IBM Watson Natural Language Understanding to generate sentiment and emotion scores for the posts and their associated keywords. The keywords were aggregated using natural language processing tools.

Results

Overall, 39 oculoplastic-related search terms resulted in 46,381 eligible posts within 14,329 threads. Posts were written by 18,319 users (117 doctors; 18,202 patients) and included 201,611 associated keywords. Keywords that occurred ≥500 times in the corpus were used to identify the most prominent topics, including specific symptoms, medication, and complications. The sentiment and emotion scores of these keywords and eligible posts were analyzed to provide concrete examples of the potential of this methodology to allow for a better understanding of patients’ attitudes. The overall sentiment score reflects a positive, neutral, or negative sentiment, whereas the emotion scores (anger, disgust, fear, joy, and sadness) represent the likelihood of the presence of the emotion. In keyword grouping analyses, medical signs, symptoms, and diseases had the lowest overall sentiment scores (−0.598). Complications were highly associated with sadness (0.485). Forum posts mentioning body parts were related to sadness (0.416) and fear (0.321). Administration was the category with the highest anger score (0.146). The top 6 forum subgroups had an overall negative sentiment score; the most negative one was the Neurology forum, with a score of −0.438. The Undiagnosed Symptoms forum had the highest sadness score (0.448). The least likely fearful posts were those from the Eye Care forum, with a score of 0.260. The overall sentiment score was much more negative before the doctor replied. The anger, disgust, fear, and sadness emotion scores decreased in likelihood, whereas joy was slightly more likely to be expressed after doctors replied.

Conclusions

This report allows physicians and researchers to efficiently mine and perform sentiment analysis on social media to better understand patients’ perspectives and promote patient-centric care. Important factors to be considered during its application include evaluating the scope of the search; selecting search terms and understanding their linguistic usages; and establishing selection, filtering, and processing criteria for posts and keywords tailored to the desired results.

HTwitt: a hadoop-based platform for analysis and visualization of streaming Twitter data

Twitter produces a massive amount of data due to its popularity that is one of the reasons underlying big data problems. One of those problems is the classification of tweets due to use of sophisticated and complex language, which makes the current tools insufficient. We present our framework HTwitt, built on top of the Hadoop ecosystem, which consists of a MapReduce algorithm and a set of machine learning techniques embedded within a big data analytics platform to efficiently address the following problems: (1) traditional data processing techniques are inadequate to handle big data; (2) data preprocessing needs substantial manual effort; (3) domain knowledge is required before the classification; (4) semantic explanation is ignored. In this work, these challenges are overcome by using different algorithms combined with a Naïve Bayes classifier to ensure reliability and highly precise recommendations in virtualization and cloud environments. These features make HTwitt different from others in terms of having an effective and practical design for text classification in big data analytics. The main contribution of the paper is to propose a framework for building landslide early warning systems by pinpointing useful tweets and visualizing them along with the processed information. We demonstrate the results of the experiments which quantify the levels of overfitting in the training stage of the model using different sizes of real-world datasets in machine learning phases. Our results demonstrate that the proposed system provides high-quality results with a score of nearly 95% and meets the requirement of a Hadoop-based classification system.

Text classification models for the automatic detection of nonmedical prescription medication use from social media

BACKGROUND

Prescription medication (PM) misuse/abuse has emerged as a national crisis in the United States, and social media has been suggested as a potential resource for performing active monitoring. However, automating a social media-based monitoring system is challenging-requiring advanced natural language processing (NLP) and machine learning methods. In this paper, we describe the development and evaluation of automatic text classification models for detecting self-reports of PM abuse from Twitter.

METHODS

We experimented with state-of-the-art bi-directional transformer-based language models, which utilize tweet-level representations that enable transfer learning (e.g., BERT, RoBERTa, XLNet, AlBERT, and DistilBERT), proposed fusion-based approaches, and compared the developed models with several traditional machine learning, including deep learning, approaches. Using a public dataset, we evaluated the performances of the classifiers on their abilities to classify the non-majority "abuse/misuse" class.

RESULTS

Our proposed fusion-based model performs significantly better than the best traditional model (F1-score [95% CI]: 0.67 [0.64-0.69] vs. 0.45 [0.42-0.48]). We illustrate, via experimentation using varying training set sizes, that the transformer-based models are more stable and require less annotated data compared to the other models. The significant improvements achieved by our best-performing classification model over past approaches makes it suitable for automated continuous monitoring of nonmedical PM use from Twitter.

CONCLUSIONS

BERT, BERT-like and fusion-based models outperform traditional machine learning and deep learning models, achieving substantial improvements over many years of past research on the topic of prescription medication misuse/abuse classification from social media, which had been shown to be a complex task due to the unique ways in which information about nonmedical use is presented. Several challenges associated with the lack of context and the nature of social media language need to be overcome to further improve BERT and BERT-like models. These experimental driven challenges are represented as potential future research directions.

Approaches to text mining for analyzing treatment plan of quit smoking with free-text medical records: A PRISMA-compliant meta-analysis

BACKGROUND

Smoking is a complex behavior associated with multiple factors such as personality, environment, genetics, and emotions. Text data are a rich source of information. However, pure text data requires substantial human resources and time to extract and apply the knowledge, resulting in many details not being discovered and used. This study proposes a novel approach that explores a text mining flow to capture the behavior of smokers quitting tobacco from their free-text medical records. More importantly, the paper examines the impact of these changes on smokers. The goal is to help smokers quit smoking. The study population included adult patients that were >20 years old of age who consulted the medical center's smoking cessation outpatient clinic from January to December 2016. A total of 246 patients visited the clinic in the study period. After excluding incomplete medical records or lost follow up, there were 141 patients included in the final analysis. There are 141 valid data points for patients who only treated once and patients with empty medical records. Two independent review authors will make the study selection based on the study eligibility criteria. Our participants are from all the patients that were involved in this study and the staff of Division of Family Medicine, National Taiwan University Hospital. Interventions and study appraisal are not required.

METHODS

The paper develops an algorithm for analyzing smoking cessation treatment plans documented in free-text medical records. The approach involves the development of an information extraction flow that uses a combination of data mining techniques, including text mining. It can use not only to help others quit smoking but also for other medical records with similar data elements. The Apriori associations of our algorithm from the text mining revealed several important clinical implications for physicians during smoking cessation. For example, an apparent association between nicotine replacement therapy (NRT) and other medications such as Inderal, Rivotril, Dogmatyl, and Solaxin. Inderal and Rivotril use in patients with anxiety disorders as anxiolytics frequently.

RESULTS

Finally, we find that the rules associating with NRT combination with blood tests may imply that the use of NRT combination therapy in smokers with chronic illness may result in lower abstinence. Further large-scale surveys comparing varenicline or bupropion with NRT combination in smokers with a chronic disease are warranted. The Apriori algorithm suffers from some weaknesses despite being transparent and straightforward. The main limitation is the costly wasting of time to hold a vast number of candidates sets with frequent itemsets, low minimum support, or large itemsets.

CONCLUSION

In the paper, the most visible areas for the therapeutic application of text mining are the integration and transfer of advances made in basic sciences, as well as a better understanding of the processes involved in smoking cessation. Text mining may also be useful for supporting decision-making processes associated with smoking cessation. Systematic review registration number is not registered.

Mining Social Media Data for Biomedical Signals and Health-Related Behavior

Social media data have been increasingly used to study biomedical and health-related phenomena. From cohort-level discussions of a condition to population-level analyses of sentiment, social media have provided scientists with unprecedented amounts of data to study human behavior associated with a variety of health conditions and medical treatments. Here we review recent work in mining social media for biomedical, epidemiological, and social phenomena information relevant to the multilevel complexity of human health. We pay particular attention to topics where social media data analysis has shown the most progress, including pharmacovigilance and sentiment analysis, especially for mental health. We also discuss a variety of innovative uses of social media data for health-related applications as well as important limitations of social media data access and use.

Adverse drug reaction detection on social media with deep linguistic features

Adverse reactions caused by drugs are one of the most important public health problems. Social media has encouraged more patients to share their drug use experiences and has become a major source for the detection of professionally unreported adverse drug reactions (ADRs). Since a large number of user posts do not mention any ADR, accurate detection of the presence of ADRs in each user post is necessary before further research can be conducted. Previous feature-based methods focus on extracting more shallow linguistic features that are unable to capture deep and subtle information in the context, ultimately failing to provide satisfactory accuracy. To overcome the limitations of previous studies, this paper proposes a novel method that can extract deep linguistic features and then combine them with shallow linguistic features for ADR detection. We first extract predicate-ADR pairs under the guidance of extended syntactic dependencies and ADR lexicon. Then, we extract semantic and part-of-speech (POS) features for each pair and pool the features of different pairs to generate a holistic representation of deep linguistic features. Finally, we use the collection of deep features and several shallow features to train the predictive models. A series of experiments are performed on data sets collected from DailyStrength and Twitter. Our approach can achieve AUCs of 94.44% and 88.97% on the two data sets, respectively, outperforming other state-of-the-art methods. The results demonstrate the potential benefits of deep linguistic features for ADR detection on social data. This method can be applied to multiple other healthcare and text analysis tasks and can be used to support pharmacovigilance research.

Predicting potential adverse events using safety data from marketed drugs

BACKGROUND

While clinical trials are considered the gold standard for detecting adverse events, often these trials are not sufficiently powered to detect difficult to observe adverse events. We developed a preliminary approach to predict 135 adverse events using post-market safety data from marketed drugs. Adverse event information available from FDA product labels and scientific literature for drugs that have the same activity at one or more of the same targets, structural and target similarities, and the duration of post market experience were used as features for a classifier algorithm. The proposed method was studied using 54 drugs and a probabilistic approach of performance evaluation using bootstrapping with 10,000 iterations.

RESULTS

Out of 135 adverse events, 53 had high probability of having high positive predictive value. Cross validation showed that 32% of the model-predicted safety label changes occurred within four to nine years of approval (median: six years).

CONCLUSIONS

This approach predicts 53 serious adverse events with high positive predictive values where well-characterized target-event relationships exist. Adverse events with well-defined target-event associations were better predicted compared to adverse events that may be idiosyncratic or related to secondary target effects that were poorly captured. Further enhancement of this model with additional features, such as target prediction and drug binding data, may increase accuracy.

Utilizing Advanced Technologies to Augment Pharmacovigilance Systems: Challenges and Opportunities

There are significant challenges and opportunities in deploying and utilizing advanced information technology (IT) within pharmacovigilance (PV) systems and across the pharmaceutical industry. Various aspects of PV will benefit from automation (e.g., by improving standardization or increasing data quality). Several themes are developed, highlighting the challenges faced, exploring solutions, and assessing the potential for further research. Automation of the workflow for processing of individual case safety reports (ICSRs) is adopted as a use case. This involves a logical progression through a series of steps that when linked together comprise the complete work process required for the effective management of ICSRs. We recognize that the rapid development of new technologies will invariably outpace the regulations applicable to PV systems. Nevertheless, we believe that such systems may be improved by intelligent automation. It is incumbent on the owners of these systems to explore opportunities presented by new technologies with regulators in order to evaluate the applicability, design, deployment, performance, validation and maintenance of advanced technologies to ensure that the PV system continues to be fit for purpose. Proposed approaches to the validation of automated PV systems are presented. A series of definitions and a critical appraisal of important considerations are provided in the form of use cases. We summarize progress made and opportunities for the development of automation of future systems. The overall goal of automation is to provide high quality safety data in the correct format, in context, more quickly, and with less manual effort. This will improve the evidence available for scientific assessment and helps to inform and expedite decisions about the minimization of risks associated with medicines.

Predicting the occurrence of surgical site infections using text mining and machine learning

In this study we propose the use of text mining and machine learning methods to predict and detect Surgical Site Infections (SSIs) using textual descriptions of surgeries and post-operative patients’ records, mined from the database of a high complexity University hospital. SSIs are among the most common adverse events experienced by hospitalized patients; preventing such events is fundamental to ensure patients’ safety. Knowledge on SSI occurrence rates may also be useful in preventing future episodes. We analyzed 15,479 surgery descriptions and post-operative records testing different preprocessing strategies and the following machine learning algorithms: Linear SVC, Logistic Regression, Multinomial Naive Bayes, Nearest Centroid, Random Forest, Stochastic Gradient Descent, and Support Vector Classification (SVC). For prediction purposes, the best result was obtained using the Stochastic Gradient Descent method (79.7% ROC-AUC); for detection, Logistic Regression yielded the best performance (80.6% ROC-AUC).

Classifying adverse drug reactions from imbalanced twitter data

BACKGROUND

Nowadays, social media are often being used by general public to create and share public messages related to their health. With the global increase in social media usage, there is a trend of posting information related to adverse drug reactions (ADR). Mining the social media data for this type of information will be helpful for pharmacological post-marketing surveillance and monitoring. Although the concept of using social media to facilitate pharmacovigilance is convincing, construction of automatic ADR detection systems remains a challenge because the corpora compiled from social media tend to be highly imbalanced, posing a major obstacle to the development of classifiers with reliable performance.

METHODS

Several methods have been proposed to address the challenge of imbalanced corpora. However, we are not aware of any studies that investigated the effectiveness of the strategies of dealing with the problem of imbalanced data in the context of ADR detection from social media. In light of this, we evaluated a variety of imbalanced techniques and proposed a novel word embedding-based synthetic minority over-sampling technique (WESMOTE), which synthesizes new training examples from the sentence representation based on word embeddings. We compared the performance of all methods on two large imbalanced datasets released for the purpose of detecting ADR posts.

RESULTS

In comparison with the state-of-the-art approaches, the classifiers that incorporated imbalanced classification techniques achieved comparable or better F-scores. All of our best performing configurations combined random under-sampling with techniques including the proposed WESMOTE, boosting and ensemble, implying that an integration of these approaches with under-sampling provides a reliable solution for large imbalanced social media datasets. Furthermore, ensemble-based methods like vote-based under-sampling (VUE) and random under-sampling boosting can be alternatives for the hybrid synthetic methods because both methods increase the diversity of the created weak classifiers, leading to better recall and overall F-scores for the minority classes.

CONCLUSIONS

Data collected from the social media are usually very large and highly imbalanced. In order to maximize the performance of a classifier trained on such data, applications of imbalanced strategies are required. We considered several practical methods for handling imbalanced Twitter data along with their performance on the binary classification task with respect to ADRs. In conclusion, the following practical insights are gained: 1) When dealing with text classification, the proposed word embedding-based synthetic minority over-sampling technique is more effective than traditional synthetic-based over-sampling methods. 2) In cases where large amounts of training data are available, the imbalanced strategies combined with under-sampling techniques are preferred. 3) Finally, employment of advanced methods does not guarantee better performance than simpler ones such as VUE, which achieved high performance with advantages like faster building time and ease of development.

Detecting Potential Adverse Drug Reactions Using a Deep Neural Network Model

BACKGROUND

Adverse drug reactions (ADRs) are common and are the underlying cause of over a million serious injuries and deaths each year. The most familiar method to detect ADRs is relying on spontaneous reports. Unfortunately, the low reporting rate of spontaneous reports is a serious limitation of pharmacovigilance.

OBJECTIVE

The objective of this study was to identify a method to detect potential ADRs of drugs automatically using a deep neural network (DNN).

METHODS

We designed a DNN model that utilizes the chemical, biological, and biomedical information of drugs to detect ADRs. This model aimed to fulfill two main purposes: identifying the potential ADRs of drugs and predicting the possible ADRs of a new drug. For improving the detection performance, we distributed representations of the target drugs in a vector space to capture the drug relationships using the word-embedding approach to process substantial biomedical literature. Moreover, we built a mapping function to address new drugs that do not appear in the dataset.

RESULTS

Using the drug information and the ADRs reported up to 2009, we predicted the ADRs of drugs recorded up to 2012. There were 746 drugs and 232 new drugs, which were only recorded in 2012 with 1325 ADRs. The experimental results showed that the overall performance of our model with mean average precision at top-10 achieved is 0.523 and the rea under the receiver operating characteristic curve (AUC) score achieved is 0.844 for ADR prediction on the dataset.

CONCLUSIONS

Our model is effective in identifying the potential ADRs of a drug and the possible ADRs of a new drug. Most importantly, it can detect potential ADRs irrespective of whether they have been reported in the past.

Innovation in Pharmacovigilance: Use of Artificial Intelligence in Adverse Event Case Processing

Automation of pharmaceutical safety case processing represents a significant opportunity to affect the strongest cost driver for a company's overall pharmacovigilance budget. A pilot was undertaken to test the feasibility of using artificial intelligence and robotic process automation to automate processing of adverse event reports. The pilot paradigm was used to simultaneously test proposed solutions of three commercial vendors. The result confirmed the feasibility of using artificial intelligence–based technology to support extraction from adverse event source documents and evaluation of case validity. In addition, the pilot demonstrated viability of the use of safety database data fields as a surrogate for otherwise time‐consuming and costly direct annotation of source documents. Finally, the evaluation and scoring method used in the pilot was able to differentiate vendor capabilities and identify the best candidate to move into the discovery phase.

An automated pipeline for analyzing medication event reports in clinical settings

BACKGROUND

Medication events in clinical settings are significant threats to patient safety. Analyzing and learning from the medication event reports is an important way to prevent the recurrence of these events. Currently, the analysis of medication event reports is ineffective and requires heavy workloads for clinicians. An automated pipeline is proposed to help clinicians deal with the accumulated reports, extract valuable information and generate feedback from the reports. Thus, the strategy of medication event prevention can be further developed based on the lessons learned.

METHODS

In order to build the automated pipeline, four classic machine learning classifiers (i.e., support vector machine, Naïve Bayes, random forest, and multi-layer perceptron) were compared to identify the event originating stages, event types, and event causes from the medication event reports. The precision, recall and F-1 measure were calculated to assess the performance of the classifiers. Further, a strategy to measure the similarity of medication event reports in our pipeline was established and evaluated by human subjects through a questionnaire.

RESULTS

We developed three classifiers to identify the medication event originating stages, event types and causes, respectively. For the event originating stages, a support vector machine classifier obtains the best performance with an F-1 measure of 0.792. For the event types, a support vector machine classifier exhibits the best performance with an F-1 measure of 0.758. And for the event causes, a random forest classifier reaches an F-1 measure of 0.925. The questionnaire results show that the similarity measurement is consistent with the domain experts in the task of identifying similar reports.

CONCLUSION

We developed and evaluated an automated pipeline that could identify three attributes from the medication event reports and calculate the similarity scores between the reports based on the attributes. The pipeline is expected to improve the efficiency of analyzing the medication event reports and to learn from the reports in a timely manner.

Authenticity and credibility aware detection of adverse drug events from social media

OBJECTIVES

Adverse drug events (ADEs) are among the top causes of hospitalization and death. Social media is a promising open data source for the timely detection of potential ADEs. In this paper, we study the problem of detecting signals of ADEs from social media.

METHODS

Detecting ADEs whose drug and AE may be reported in different posts of a user leads to major concerns regarding the content authenticity and user credibility, which have not been addressed in previous studies. Content authenticity concerns whether a post mentions drugs or adverse events that are actually consumed or experienced by the writer. User credibility indicates the degree to which chronological evidence from a user's sequence of posts should be trusted in the ADE detection. We propose AC-SPASM, a Bayesian model for the authenticity and credibility aware detection of ADEs from social media. The model exploits the interaction between content authenticity, user credibility and ADE signal quality. In particular, we argue that the credibility of a user correlates with the user's consistency in reporting authentic content.

RESULTS

We conduct experiments on a real-world Twitter dataset containing 1.2 million posts from 13,178 users. Our benchmark set contains 22 drugs and 8089 AEs. AC-SPASM recognizes authentic posts with F₁ - the harmonic mean of precision and recall of 80%, and estimates user credibility with precision@10 = 90% and NDCG@10 - a measure for top-10 ranking quality of 96%. Upon validation against known ADEs, AC-SPASM achieves F₁ = 91%, outperforming state-of-the-art baseline models by 32% (p < 0.05). Also, AC-SPASM obtains precision@456 = 73% and NDCG@456 = 94% in detecting and prioritizing unknown potential ADE signals for further investigation. Furthermore, the results show that AC-SPASM is scalable to large datasets.

CONCLUSIONS

Our study demonstrates that taking into account the content authenticity and user credibility improves the detection of ADEs from social media. Our work generates hypotheses to reduce experts' guesswork in identifying unknown potential ADEs.

Authenticity and credibility aware detection of adverse drug events from social media

OBJECTIVES

Adverse drug events (ADEs) are among the top causes of hospitalization and death. Social media is a promising open data source for the timely detection of potential ADEs. In this paper, we study the problem of detecting signals of ADEs from social media.

METHODS

Detecting ADEs whose drug and AE may be reported in different posts of a user leads to major concerns regarding the content authenticity and user credibility, which have not been addressed in previous studies. Content authenticity concerns whether a post mentions drugs or adverse events that are actually consumed or experienced by the writer. User credibility indicates the degree to which chronological evidence from a user's sequence of posts should be trusted in the ADE detection. We propose AC-SPASM, a Bayesian model for the authenticity and credibility aware detection of ADEs from social media. The model exploits the interaction between content authenticity, user credibility and ADE signal quality. In particular, we argue that the credibility of a user correlates with the user's consistency in reporting authentic content.

RESULTS

We conduct experiments on a real-world Twitter dataset containing 1.2 million posts from 13,178 users. Our benchmark set contains 22 drugs and 8089 AEs. AC-SPASM recognizes authentic posts with F₁ - the harmonic mean of precision and recall of 80%, and estimates user credibility with precision@10 = 90% and NDCG@10 - a measure for top-10 ranking quality of 96%. Upon validation against known ADEs, AC-SPASM achieves F₁ = 91%, outperforming state-of-the-art baseline models by 32% (p < 0.05). Also, AC-SPASM obtains precision@456 = 73% and NDCG@456 = 94% in detecting and prioritizing unknown potential ADE signals for further investigation. Furthermore, the results show that AC-SPASM is scalable to large datasets.

CONCLUSIONS

Our study demonstrates that taking into account the content authenticity and user credibility improves the detection of ADEs from social media. Our work generates hypotheses to reduce experts' guesswork in identifying unknown potential ADEs.

The Missing Variable in Big Data for Social Sciences: The Decision-Maker

The value of big data for social sciences and social impact is professed to be high. This potential value is related, however, to the capacity of using extracted information in decision-making. In all of this, one important point has been overlooked: when “humans” retain a role in the decision-making process, the value of information is no longer an objective feature but depends on the knowledge and mindset of end users. A new big data cycle has been proposed in this paper, where the decision-maker is placed at the centre of the process. The proposed cycle is tested through two cases and, as a result of the suggested approach, two operations—filtering and framing—which are routinely carried out independently by scientists and end users in an unconscious manner, become clear and transparent. The result is a new cycle where four dimensions guide the interactions for creating value.

Annotation and detection of drug effects in text for pharmacovigilance

Pharmacovigilance (PV) databases record the benefits and risks of different drugs, as a means to ensure their safe and effective use. Creating and maintaining such resources can be complex, since a particular medication may have divergent effects in different individuals, due to specific patient characteristics and/or interactions with other drugs being administered. Textual information from various sources can provide important evidence to curators of PV databases about the usage and effects of drug targets in different medical subjects. However, the efficient identification of relevant evidence can be challenging, due to the increasing volume of textual data. Text mining (TM) techniques can support curators by automatically detecting complex information, such as interactions between drugs, diseases and adverse effects. This semantic information supports the quick identification of documents containing information of interest (e.g., the different types of patients in which a given adverse drug reaction has been observed to occur). TM tools are typically adapted to different domains by applying machine learning methods to corpora that are manually labelled by domain experts using annotation guidelines to ensure consistency. We present a semantically annotated corpus of 597 MEDLINE abstracts, PHAEDRA, encoding rich information on drug effects and their interactions, whose quality is assured through the use of detailed annotation guidelines and the demonstration of high levels of inter-annotator agreement (e.g., 92.6% F-Score for identifying named entities and 78.4% F-Score for identifying complex events, when relaxed matching criteria are applied). To our knowledge, the corpus is unique in the domain of PV, according to the level of detail of its annotations. To illustrate the utility of the corpus, we have trained TM tools based on its rich labels to recognise drug effects in text automatically. The corpus and annotation guidelines are available at: http://www.nactem.ac.uk/PHAEDRA/ .

Pharmacovigilance from social media: An improved random subspace method for identifying adverse drug events

OBJECTIVE

Recent advances in Web 2.0 technologies have seen significant strides towards utilizing patient-generated content for pharmacovigilance. Social media-based pharmacovigilance has great potential to augment current efforts and provide regulatory authorities with valuable decision aids. Among various pharmacovigilance activities, identifying adverse drug events (ADEs) is very important for patient safety. However, in health-related discussion forums, ADEs may confound with drug indications and beneficial effects, etc. Therefore, the focus of this study is to develop a strategy to identify ADEs from other semantic types, and meanwhile to determine the drug that an ADE is associated with.

MATERIALS AND METHODS

In this study, two groups of features, i.e., shallow linguistic features and semantic features, are explored. Moreover, motivated and inspired by the characteristics of explored two feature categories for social media-based ADE identification, an improved random subspace method, called Stratified Sampling-based Random Subspace (SSRS), is proposed. Unlike conventional random subspace method that applies random sampling for subspace selection, SSRS adopts stratified sampling-based subspace selection strategy.

RESULTS

A case study on heart disease discussion forums is performed to evaluate the effectiveness of the SSRS method. Experimental results reveal that the proposed SSRS method significantly outperforms other compared ensemble methods and existing approaches for ADE identification.

DISCUSSION AND CONCLUSION

Our proposed method is easy to implement since it is based on two feature sets that can be naturally derived, and therefore, can omit artificial stratum generation efforts. Moreover, SSRS has great potential of being applied to deal with other high-dimensional problems that can represent original data from two different aspects.

Utility of social media and crowd-intelligence data for pharmacovigilance: a scoping review

BACKGROUND

A scoping review to characterize the literature on the use of conversations in social media as a potential source of data for detecting adverse events (AEs) related to health products.

METHODS

Our specific research questions were (1) What social media listening platforms exist to detect adverse events related to health products, and what are their capabilities and characteristics? (2) What is the validity and reliability of data from social media for detecting these adverse events? MEDLINE, EMBASE, Cochrane Library, and relevant websites were searched from inception to May 2016. Any type of document (e.g., manuscripts, reports) that described the use of social media data for detecting health product AEs was included. Two reviewers independently screened citations and full-texts, and one reviewer and one verifier performed data abstraction. Descriptive synthesis was conducted.

RESULTS

After screening 3631 citations and 321 full-texts, 70 unique documents with 7 companion reports available from 2001 to 2016 were included. Forty-six documents (66%) described an automated or semi-automated information extraction system to detect health product AEs from social media conversations (in the developmental phase). Seven pre-existing information extraction systems to mine social media data were identified in eight documents. Nineteen documents compared AEs reported in social media data with validated data and found consistent AE discovery in all except two documents. None of the documents reported the validity and reliability of the overall system, but some reported on the performance of individual steps in processing the data. The validity and reliability results were found for the following steps in the data processing pipeline: data de-identification (n = 1), concept identification (n = 3), concept normalization (n = 2), and relation extraction (n = 8). The methods varied widely, and some approaches yielded better results than others.

CONCLUSIONS

Our results suggest that the use of social media conversations for pharmacovigilance is in its infancy. Although social media data has the potential to supplement data from regulatory agency databases; is able to capture less frequently reported AEs; and can identify AEs earlier than official alerts or regulatory changes, the utility and validity of the data source remains under-studied.

TRIAL REGISTRATION

Open Science Framework ( https://osf.io/kv9hu/ ).

Mining Patients' Narratives in Social Media for Pharmacovigilance: Adverse Effects and Misuse of Methylphenidate

Background: The Food and Drug Administration (FDA) in the United States and the European Medicines Agency (EMA) have recognized social media as a new data source to strengthen their activities regarding drug safety.

Objective: Our objective in the ADR-PRISM project was to provide text mining and visualization tools to explore a corpus of posts extracted from social media. We evaluated this approach on a corpus of 21 million posts from five patient forums, and conducted a qualitative analysis of the data available on methylphenidate in this corpus.

Methods: We applied text mining methods based on named entity recognition and relation extraction in the corpus, followed by signal detection using proportional reporting ratio (PRR). We also used topic modeling based on the Correlated Topic Model to obtain the list of the matics in the corpus and classify the messages based on their topics.

Results: We automatically identified 3443 posts about methylphenidate published between 2007 and 2016, among which 61 adverse drug reactions (ADR) were automatically detected. Two pharmacovigilance experts evaluated manually the quality of automatic identification, and a f-measure of 0.57 was reached. Patient's reports were mainly neuro-psychiatric effects. Applying PRR, 67% of the ADRs were signals, including most of the neuro-psychiatric symptoms but also palpitations. Topic modeling showed that the most represented topics were related to Childhood and Treatment initiation, but also Side effects. Cases of misuse were also identified in this corpus, including recreational use and abuse.

Conclusion: Named entity recognition combined with signal detection and topic modeling have demonstrated their complementarity in mining social media data. An in-depth analysis focused on methylphenidate showed that this approach was able to detect potential signals and to provide better understanding of patients' behaviors regarding drugs, including misuse.

Descriptions of Adverse Drug Reactions Are Less Informative in Forums Than in the French Pharmacovigilance Database but Provide More Unexpected Reactions

Background: Social media have drawn attention for their potential use in Pharmacovigilance. Recent work showed that it is possible to extract information concerning adverse drug reactions (ADRs) from posts in social media. The main objective of the Vigi4MED project was to evaluate the relevance and quality of the information shared by patients on web forums about drug safety and its potential utility for pharmacovigilance.

Methods: After selecting websites of interest, we manually evaluated the relevance of the content of posts for pharmacovigilance related to six drugs (agomelatine, baclofen, duloxetine, exenatide, strontium ranelate, and tetrazepam). We compared forums to the French Pharmacovigilance Database (FPVD) to (1) evaluate whether they contained relevant information to characterize a pharmacovigilance case report (patient’s age and sex; treatment indication, dose and duration; time-to-onset (TTO) and outcome of the ADR, and drug dechallenge and rechallenge) and (2) perform impact analysis (nature, seriousness, unexpectedness, and outcome of the ADR).

Results: The cases in the FPVD were significantly more informative than posts in forums for patient description (age, sex), treatment description (dose, duration, TTO), and outcome of the ADR, but the indication for the treatment was more often found in forums. Cases were more often serious in the FPVD than in forums (46% vs. 4%), but forums more often contained an unexpected ADR than the FPVD (24% vs. 17%). Moreover, 197 unexpected ADRs identified in forums were absent from the FPVD and the distribution of the MedDRA System Organ Classes (SOCs) was different between the two data sources.

Discussion: This study is the first to evaluate if patients’ posts may qualify as potential and informative case reports that should be stored in a pharmacovigilance database in the same way as case reports submitted by health professionals. The posts were less informative (except for the indication) and focused on less serious ADRs than the FPVD cases, but more unexpected ADRs were presented in forums than in the FPVD and their SOCs were different. Thus, web forums should be considered as a secondary, but complementary source for pharmacovigilance.

Big Data in Public Health: Terminology, Machine Learning, and Privacy

The digital world is generating data at a staggering and still increasing rate. While these "big data" have unlocked novel opportunities to understand public health, they hold still greater potential for research and practice. This review explores several key issues that have arisen around big data. First, we propose a taxonomy of sources of big data to clarify terminology and identify threads common across some subtypes of big data. Next, we consider common public health research and practice uses for big data, including surveillance, hypothesis-generating research, and causal inference, while exploring the role that machine learning may play in each use. We then consider the ethical implications of the big data revolution with particular emphasis on maintaining appropriate care for privacy in a world in which technology is rapidly changing social norms regarding the need for (and even the meaning of) privacy. Finally, we make suggestions regarding structuring teams and training to succeed in working with big data in research and practice.

Detection of Cases of Noncompliance to Drug Treatment in Patient Forum Posts: Topic Model Approach

Background

Medication nonadherence is a major impediment to the management of many health conditions. A better understanding of the factors underlying noncompliance to treatment may help health professionals to address it. Patients use peer-to-peer virtual communities and social media to share their experiences regarding their treatments and diseases. Using topic models makes it possible to model themes present in a collection of posts, thus to identify cases of noncompliance.

Objective

The aim of this study was to detect messages describing patients’ noncompliant behaviors associated with a drug of interest. Thus, the objective was the clustering of posts featuring a homogeneous vocabulary related to nonadherent attitudes.

Methods

We focused on escitalopram and aripiprazole used to treat depression and psychotic conditions, respectively. We implemented a probabilistic topic model to identify the topics that occurred in a corpus of messages mentioning these drugs, posted from 2004 to 2013 on three of the most popular French forums. Data were collected using a Web crawler designed by Kappa Santé as part of the Detec’t project to analyze social media for drug safety. Several topics were related to noncompliance to treatment.

Results

Starting from a corpus of 3650 posts related to an antidepressant drug (escitalopram) and 2164 posts related to an antipsychotic drug (aripiprazole), the use of latent Dirichlet allocation allowed us to model several themes, including interruptions of treatment and changes in dosage.

The topic model approach detected cases of noncompliance behaviors with a recall of 98.5% (272/276) and a precision of 32.6% (272/844).

Conclusions

Topic models enabled us to explore patients’ discussions on community websites and to identify posts related with noncompliant behaviors. After a manual review of the messages in the noncompliance topics, we found that noncompliance to treatment was present in 6.17% (276/4469) of the posts.

HARNESSING SOCIAL MEDIA FOR HEALTH INFORMATION MANAGEMENT

The remarkable upsurge of social media has dramatic impacts on health care research and practice in the past decade. Social media are reshaping health information management in a variety of ways, ranging from providing cost-effective ways to improve clinician-patient communication and exchange health-related information and experience, to enabling the discovery of new medical knowledge and information. Despite some demonstrated initial success, social media use and analytics for improving health as a research field is still at its infancy. Information systems researchers can potentially play a key role in advancing the field. This study proposes a conceptual framework for social media-based health information management by drawing on multi-disciplinary research. With the guidance of the framework, this research presents related research challenges, identifies important yet under-explored research issues, and discusses promising directions for future research.

The Patient's Voice in Pharmacovigilance: Pragmatic Approaches to Building a Patient-Centric Drug Safety Organization

Patient-centeredness has become an acknowledged hallmark of not only high-quality health care but also high-quality drug development. Biopharmaceutical companies are actively seeking to be more patient-centric in drug research and development by involving patients in identifying target disease conditions, participating in the design of, and recruitment for, clinical trials, and disseminating study results. Drug safety departments within the biopharmaceutical industry are at a similar inflection point. Rising rates of per capita prescription drug use underscore the importance of having robust pharmacovigilance systems in place to detect and assess adverse drug reactions (ADRs). At the same time, the practice of pharmacovigilance is being transformed by a host of recent regulatory guidances and related initiatives which emphasize the importance of the patient’s perspective in drug safety. Collectively, these initiatives impact the full range of activities that fall within the remit of pharmacovigilance, including ADR reporting, signal detection and evaluation, risk management, medication error assessment, benefit–risk assessment and risk communication. Examples include the fact that manufacturing authorization holders are now expected to monitor all digital sources under their control for potential reports of ADRs, and the emergence of new methods for collecting, analysing and reporting patient-generated ADR reports for signal detection and evaluation purposes. A drug safety department’s ability to transition successfully into a more patient-centric organization will depend on three defining attributes: (1) a patient-centered culture; (2) deployment of a framework to guide patient engagement activities; and (3) demonstrated proficiency in patient-centered competencies, including patient engagement, risk communication and patient preference assessment. Whether, and to what extent, drug safety departments embrace the new patient-centric imperative, and the methods and processes they implement to achieve this end effectively and efficiently, promise to become distinguishing factors in the highly competitive biopharmaceutical industry landscape.