Postmarket drug surveillance without trial costs: discovery of adverse drug reactions through large-scale analysis of web search queries

A large-scale observational comparison of antidepressants and their effects

BACKGROUND

Selective Serotonin Reuptake Inhibitors (SSRIs) represent a diverse class of medications widely prescribed for depression and anxiety. Despite their common use, there is an absence of large-scale, real-world evidence capturing the heterogeneity in their effects on individuals. This study addresses this gap by utilizing naturalistic search data to explore the varied impact of six different SSRIs on user behavior.

METHODS

The study sample included ∼508 thousand Bing users with searches for one of six SSRIs (citalopram, escitalopram, fluoxetine, fluvoxamine, paroxetine, sertraline) from April-December 2022, comprising 510 million queries. Cox proportional hazard models were employed to examine 30 topics (e.g., shopping, tourism, health) and 195 health symptoms (e.g., anxiety, weight gain, impotence), using each SSRI as a reference. We assessed the relative hazard ratios between drugs and, where feasible, ranked the SSRIs based on their observed effects. We used Cox proportional hazard models in order to account for both the likelihood of users searching for a particular topic or symptom and the associated time to that search. The temporal aspect aided in distinguishing between potential symptoms of the disorder, short-term medication side effects, and later appearing side effects.

RESULTS

Differences were found in search behaviors associated with each SSRI. E.g., fluvoxamine was associated with a significantly higher likelihood of searching weight gain compared to all other SSRIs (HRs 1.85-2.93). Searches following citalopram were associated with significantly higher rates of later impotence queries compared to all other SSRIs (HRs 5.11-7.76), except fluvoxamine. Fluvoxamine was associated with a significantly higher rate of health related searches than all other SSRIs (HRs 2.11-2.36).

CONCLUSIONS

Our study reveals new insights into the varying SSRI impacts, suggesting distinct symptom profiles. This novel use of large-scale, naturalistic search data contributes to pharmacovigilance efforts, enhancing our understanding of intra-class variation among SSRIs, potentially uncovering previously unidentified drug effects.

Estimating the household secondary attack rate and serial interval of COVID-19 using social media

We propose a method to estimate the household secondary attack rate (hSAR) of COVID-19 in the United Kingdom based on activity on the social media platform X, formerly known as Twitter. Conventional methods of hSAR estimation are resource intensive, requiring regular contact tracing of COVID-19 cases. Our proposed framework provides a complementary method that does not rely on conventional contact tracing or laboratory involvement, including the collection, processing, and analysis of biological samples. We use a text classifier to identify reports of people tweeting about themselves and/or members of their household having COVID-19 infections. A probabilistic analysis is then performed to estimate the hSAR based on the number of self or household, and self and household tweets of COVID-19 infection. The analysis includes adjustments for a reluctance of Twitter users to tweet about household members, and the possibility that the secondary infection was not acquired within the household. Experimental results for the UK, both monthly and weekly, are reported for the period from January 2020 to February 2022. Our results agree with previously reported hSAR estimates, varying with the primary variants of concern, e.g. delta and omicron. The serial interval (SI) is based on the time between the two tweets that indicate a primary and secondary infection. Experimental results, though larger than the consensus, are qualitatively similar. The estimation of hSAR and SI using social media data constitutes a new tool that may help in characterizing, forecasting and managing outbreaks and pandemics in a faster, affordable, and more efficient manner.

Social media and internet search data to inform drug utilization: A systematic scoping review

Introduction

Drug utilization is currently assessed through traditional data sources such as big electronic medical records (EMRs) databases, surveys, and medication sales. Social media and internet data have been reported to provide more accessible and more timely access to medications' utilization.

Objective

This review aims at providing evidence comparing web data on drug utilization to other sources before the COVID-19 pandemic.

Methods

We searched Medline, EMBASE, Web of Science, and Scopus until November 25th, 2019, using a predefined search strategy. Two independent reviewers conducted screening and data extraction.

Results

Of 6,563 (64%) deduplicated publications retrieved, 14 (0.2%) were included. All studies showed positive associations between drug utilization information from web and comparison data using very different methods. A total of nine (64%) studies found positive linear correlations in drug utilization between web and comparison data. Five studies reported association using other methods: One study reported similar drug popularity rankings using both data sources. Two studies developed prediction models for future drug consumption, including both web and comparison data, and two studies conducted ecological analyses but did not quantitatively compare data sources. According to the STROBE, RECORD, and RECORD-PE checklists, overall reporting quality was mediocre. Many items were left blank as they were out of scope for the type of study investigated.

Conclusion

Our results demonstrate the potential of web data for assessing drug utilization, although the field is still in a nascent period of investigation. Ultimately, social media and internet search data could be used to get a quick preliminary quantification of drug use in real time. Additional studies on the topic should use more standardized methodologies on different sets of drugs in order to confirm these findings. In addition, currently available checklists for study quality of reporting would need to be adapted to these new sources of scientific information.

Identifying Profiles and Symptoms of Patients With Long COVID in France: Data Mining Infodemiology Study Based on Social Media

BACKGROUND

Long COVID-a condition with persistent symptoms post COVID-19 infection-is the first illness arising from social media. In France, the French hashtag #ApresJ20 described symptoms persisting longer than 20 days after contracting COVID-19. Faced with a lack of recognition from medical and official entities, patients formed communities on social media and described their symptoms as long-lasting, fluctuating, and multisystemic. While many studies on long COVID relied on traditional research methods with lengthy processes, social media offers a foundation for large-scale studies with a fast-flowing outburst of data.

OBJECTIVE

We aimed to identify and analyze Long Haulers' main reported symptoms, symptom co-occurrences, topics of discussion, difficulties encountered, and patient profiles.

METHODS

Data were extracted based on a list of pertinent keywords from public sites (eg, Twitter) and health-related forums (eg, Doctissimo). Reported symptoms were identified via the MedDRA dictionary, displayed per the volume of posts mentioning them, and aggregated at the user level. Associations were assessed by computing co-occurrences in users' messages, as pairs of preferred terms. Discussion topics were analyzed using the Biterm Topic Modeling; difficulties and unmet needs were explored manually. To identify patient profiles in relation to their symptoms, each preferred term's total was used to create user-level hierarchal clusters.

RESULTS

Between January 1, 2020, and August 10, 2021, overall, 15,364 messages were identified as originating from 6494 patients of long COVID or their caregivers. Our analyses revealed 3 major symptom co-occurrences: asthenia-dyspnea (102/289, 35.3%), asthenia-anxiety (65/289, 22.5%), and asthenia-headaches (50/289, 17.3%). The main reported difficulties were symptom management (150/424, 35.4% of messages), psychological impact (64/424,15.1%), significant pain (51/424, 12.0%), deterioration in general well-being (52/424, 12.3%), and impact on daily and professional life (40/424, 9.4% and 34/424, 8.0% of messages, respectively). We identified 3 profiles of patients in relation to their symptoms: profile A (n=406 patients) reported exclusively an asthenia symptom; profile B (n=129) expressed anxiety (n=129, 100%), asthenia (n=28, 21.7%), dyspnea (n=15, 11.6%), and ageusia (n=3, 2.3%); and profile C (n=141) described dyspnea (n=141, 100%), and asthenia (n=45, 31.9%). Approximately 49.1% of users (79/161) continued expressing symptoms after more than 3 months post infection, and 20.5% (33/161) after 1 year.

CONCLUSIONS

Long COVID is a lingering condition that affects people worldwide, physically and psychologically. It impacts Long Haulers' quality of life, everyday tasks, and professional activities. Social media played an undeniable role in raising and delivering Long Haulers' voices and can potentially rapidly provide large volumes of valuable patient-reported information. Since long COVID was a self-titled condition by patients themselves via social media, it is imperative to continuously include their perspectives in related research. Our results can help design patient-centric instruments to be further used in clinical practice to better capture meaningful dimensions of long COVID.

The Role of Information Boxes in Search Engine Results for Symptom Searches: Analysis of Archival Data

Background

Search engines provide health information boxes as part of search results to address information gaps and misinformation for commonly searched symptoms. Few prior studies have sought to understand how individuals who are seeking information about health symptoms navigate different types of page elements on search engine results pages, including health information boxes.

Objective

Using real-world search engine data, this study sought to investigate how users searching for common health-related symptoms with Bing interacted with health information boxes (info boxes) and other page elements.

Methods

A sample of searches (N=28,552 unique searches) was compiled for the 17 most common medical symptoms queried on Microsoft Bing by users in the United States between September and November 2019. The association between the page elements that users saw, their characteristics, and the time spent on elements or clicks was investigated using linear and logistic regression.

Results

The number of searches ranged by symptom type from 55 searches for cramps to 7459 searches for anxiety. Users searching for common health-related symptoms saw pages with standard web results (n=24,034, 84%), itemized web results (n=23,354, 82%), ads (n=13,171, 46%), and info boxes (n=18,215, 64%). Users spent on average 22 (SD 26) seconds on the search engine results page. Users who saw all page elements spent 25% (7.1 s) of their time on the info box, 23% (6.1 s) on standard web results, 20% (5.7 s) on ads, and 10% (10 s) on itemized web results, with significantly more time on the info box compared to other elements and the least amount of time on itemized web results. Info box characteristics such as reading ease and appearance of related conditions were associated with longer time on the info box. Although none of the info box characteristics were associated with clicks on standard web results, info box characteristics such as reading ease and related searches were negatively correlated with clicks on ads.

Conclusions

Info boxes were attended most by users compared with other page elements, and their characteristics may influence future web searching. Future studies are needed that further explore the utility of info boxes and their influence on real-world health-seeking behaviors.

Classifying Drug Ratings Using User Reviews with Transformer-Based Language Models

Drug review websites such as Drugs.com provide users' textual reviews and numeric ratings of drugs. These reviews along with the ratings are used for the consumers for choosing a drug. However, the numeric ratings may not always be consistent with text reviews and purely relying on the rating score for finding positive/negative reviews may not be reliable. Automatic classification of user ratings based on textual review can create a more reliable rating for drugs. In this project, we built classification models to classify drug review ratings using textual reviews with traditional machine learning and deep learning models. Traditional machine learning models including Random Forest and Naive Bayesian classifiers were built using TF-IDF features as input. Also, transformer-based neural network models including BERT, Bio_ClinicalBERT, RoBERTa, XLNet, ELECTRA, and ALBERT were built using the raw text as input. Overall, Bio_ClinicalBERT model outperformed the other models with an overall accuracy of 87%. We further identified concepts of the Unified Medical Language System (UMLS) from the postings and analyzed their semantic types stratified by class types. This research demonstrated that transformer-based models can be used to classify drug reviews based solely on textual reviews.

Providing early indication of regional anomalies in COVID-19 case counts in England using search engine queries

Prior work has shown the utility of using Internet searches to track the incidence of different respiratory illnesses. Similarly, people who suffer from COVID-19 may query for their symptoms prior to accessing the medical system (or in lieu of it). To assist in the UK government's response to the COVID-19 pandemic we analyzed searches for relevant symptoms on the Bing web search engine from users in England to identify areas of the country where unexpected rises in relevant symptom searches occurred. These were reported weekly to the UK Health Security Agency to assist in their monitoring of the pandemic. Our analysis shows that searches for "fever" and "cough" were the most correlated with future case counts during the initial stages of the pandemic, with searches preceding case counts by up to 21 days. Unexpected rises in search patterns were predictive of anomalous rises in future case counts within a week, reaching an Area Under Curve of 0.82 during the initial phase of the pandemic, and later reducing due to changes in symptom presentation. Thus, analysis of regional searches for symptoms can provide an early indicator (of more than one week) of increases in COVID-19 case counts.

Impact of online mental health screening tools on help-seeking, care receipt, and suicidal ideation and suicidal intent: Evidence from internet search behavior in a large U.S. cohort

INTRODUCTION

Most people with psychiatric illnesses do not receive treatment for almost a decade after disorder onset. Online mental health screens reflect one mechanism designed to shorten this lag in help-seeking, yet there has been limited research on the effectiveness of screening tools in naturalistic settings.

MATERIAL AND METHODS

We examined a cohort of persons directed to a mental health screening tool via the Bing search engine (n = 126,060). We evaluated the impact of tool content on later searches for mental health self-references, self-diagnosis, care seeking, psychoactive medications, suicidal ideation, and suicidal intent. Website characteristics were evaluated by pairs of independent raters to ascertain screen type and content. These included the presence/absence of a suggestive diagnosis, a message on interpretability, as well as referrals to digital treatments, in-person treatments, and crisis services.

RESULTS

Using machine learning models, the results suggested that screen content predicted later searches with mental health self-references (AUC = 0·73), mental health self-diagnosis (AUC = 0·69), mental health care seeking (AUC = 0·61), psychoactive medications (AUC = 0·55), suicidal ideation (AUC = 0·58), and suicidal intent (AUC = 0·60). Cox-proportional hazards models suggested individuals utilizing tools with in-person care referral were significantly more likely to subsequently search for methods to actively end their life (HR = 1·727, p = 0·007).

DISCUSSION

Online screens may influence help-seeking behavior, suicidal ideation, and suicidal intent. Websites with referrals to in-person treatments could put persons at greater risk of active suicidal intent. Further evaluation using large-scale randomized controlled trials is needed.

Medical Needs Extraction for Breast Cancer Patients from Question and Answer Services: Natural Language Processing-Based Approach

BACKGROUND

A large number of patient narratives are available on various web services. As for web question and answer services, patient questions often relate to medical needs, and we expect these questions to provide clues for a better understanding of patients' medical needs.

OBJECTIVE

This study aimed to extract patients' needs and classify them into thematic categories. Clarifying patient needs is the first step in solving social issues that patients with cancer encounter.

METHODS

For this study, we used patient question texts containing the key phrase "breast cancer," available at the Yahoo! Japan question and answer service, Yahoo! Chiebukuro, which contains over 60,000 questions on cancer. First, we converted the question text into a vector representation. Next, the relevance between patient needs and existing cancer needs categories was calculated based on cosine similarity.

RESULTS

The proportion of correct classifications in our proposed method was approximately 70%. Considering the results of classifying questions, we found the variation and the number of needs.

CONCLUSIONS

We created 3 corpora to classify the problems of patients with cancer. The proposed method was able to classify the problems considering the question text. Moreover, as an application example, the question text that included the side effect signaling of drugs and the unmet needs of cancer patients could be extracted. Revealing these needs is important to fulfill the medical needs of patients with cancer.

Using Infodemiology Metrics to Assess Public Interest in Liver Transplantation: Google Trends Analysis

Background

Liver transplantation (LT) is the only curative treatment for end-stage liver disease. Less than 10% of global transplantation needs are met worldwide, and the need for LT is still increasing. The death rates on the waiting list remain too high.

Objective

It is, therefore, critical to raise awareness among the public and health care providers and in turn increasingly acquire donors.

Methods

We performed a Google Trends search using the search terms liver transplantation and liver transplant on October 15, 2020. On the basis of the resulting monthly data, the annual average Google Trends indices were calculated for the years 2004 to 2018. We not only investigated the trend worldwide but also used data from the United Network for Organ Sharing (UNOS), Spain, and Eurotransplant. Using pairwise Spearman correlations, Google Trends indices were examined over time and compared with the total number of liver transplants retrieved from the respective official websites of UNOS, the Organización Nacional de Trasplantes, and Eurotransplant.

Results

From 2004 to 2018, there was a significant decrease in the worldwide Google Trends index from 78.2 in 2004 to 20.5 in 2018 (–71.2%). This trend was more evident in UNOS than in the Eurotransplant group. In the same period, the number of transplanted livers increased worldwide. The waiting list mortality rate was 31% for Eurotransplant and 29% for UNOS. However, in Spain, where there are excellent awareness programs, the Google Trends index remained stable over the years with comparable, increasing LT numbers but a significantly lower waiting list mortality (15%).

Conclusions

Public awareness in LT has decreased significantly over the past two decades. Therefore, novel awareness programs should be initialized.

Analysis of a Vaping-Associated Lung Injury Outbreak through Participatory Surveillance and Archival Internet Data

The US Centers for Disease Control and Prevention alerted of a suspected outbreak of lung illness associated with using E-cigarette products in September 2019. At the time that the CDC published its alert little was known about the causes of the outbreak or who was at risk for it. Here we provide insights into the outbreak through analysis of passive reporting and participatory surveillance. We collected data about vaping habits and associated adverse reactions from four data sources pertaining to people in the USA: A participatory surveillance platform (YouVape), Reddit, Google Trends, and Bing. Data were analyzed to identify vaping behaviors and reported adverse events. These were correlated among sources and with prior reports. Data was obtained from 720 YouVape users, 4331 Reddit users, and over 1 million Bing users. Large geographic variation was observed across vaping products. Significant correlation was found among the data sources in reported adverse reactions. Models of participatory surveillance data found specific product and adverse reaction associations. Specifically, cannabidiol was found to be associated with fever, while tetrahydrocannabinol was found to be correlated with diarrhea. Our results demonstrate that utilization of different, complementary, online data sources provide a holistic view of vaping associated lung injury while augmenting traditional data sources.

Data Sharing and Privacy in Pharmaceutical Studies

Adverse drug events have been a long-standing concern for the wide-ranging harms to public health, and the substantial disease burden. The key to diminish or eliminate the impacts is to build a comprehensive pharmacovigilance system. Application of the "big data" approach has been proved to assist the detection of adverse drug events by involving previously unavailable data sources and promoting health information exchange. Even though challenges and potential risks still remain. The lack of effective privacy-preserving measures in the flow of medical data is the most important Accepted: one, where urgent actions are required to prevent the threats and facilitate the construction of pharmacovigilance systems. Several privacy protection methods are reviewed in this article, which may be helpful to break the barrier.

Quantifying Public Interest in Police Reforms by Mining Internet Search Data Following George Floyd's Death

BACKGROUND

The death of George Floyd while in police custody has resurfaced serious questions about police conduct that result in the deaths of unarmed persons.

OBJECTIVE

Data-driven strategies that identify and prioritize the public's needs may engender a public health response to improve policing. We assessed how internet searches indicative of interest in police reform changed after Mr Floyd's death.

METHODS

We monitored daily Google searches (per 10 million total searches) that included the terms "police" and "reform(s)" (eg, "reform the police," "best police reforms," etc) originating from the United States between January 1, 2010, through July 5, 2020. We also monitored searches containing the term "police" with "training," "union(s)," "militarization," or "immunity" as markers of interest in the corresponding reform topics.

RESULTS

The 41 days following Mr Floyd's death corresponded with the greatest number of police "reform(s)" searches ever recorded, with 1,350,000 total searches nationally. Searches increased significantly in all 50 states and Washington DC. By reform topic, nationally there were 1,220,000 total searches for "police" and "union(s)"; 820,000 for "training"; 360,000 for "immunity"; and 72,000 for "militarization." In terms of searches for all policy topics by state, 33 states searched the most for "training," 16 for "union(s)," and 2 for "immunity." States typically in the southeast had fewer queries related to any police reform topic than other states. States that had a greater percentage of votes for President Donald Trump during the 2016 election searched more often for police "union(s)" while states favoring Secretary Hillary Clinton searched more for police "training."

CONCLUSIONS

The United States is at a historical juncture, with record interest in topics related to police reform with variability in search terms across states. Policy makers can respond to searches by considering the policies their constituencies are searching for online, notably police training and unions. Public health leaders can respond by engaging in the subject of policing and advocating for evidence-based policy reforms.

Combining Social Media and FDA Adverse Event Reporting System to Detect Adverse Drug Reactions

INTRODUCTION

Adverse drug reactions (ADRs) are unintended reactions caused by a drug or combination of drugs taken by a patient. The current safety surveillance system relies on spontaneous reporting systems (SRSs) and more recently on observational health data; however, ADR detection may be delayed and lack geographic diversity. The broad scope of social media conversations, such as those on Twitter, can include health-related topics. Consequently, these data could be used to detect potentially novel ADRs with less latency. Although research regarding ADR detection using social media has made progress, findings are based on single information sources, and no study has yet integrated drug safety evidence from both an SRS and Twitter.

OBJECTIVE

The aim of this study was to combine signals from an SRS and Twitter to facilitate the detection of safety signals and compare the performance of the combined system with signals generated by individual data sources.

METHODS

We extracted potential drug-ADR posts from Twitter, used Monte Carlo expectation maximization to generate drug safety signals from both the US FDA Adverse Event Reporting System and posts from Twitter, and then integrated these signals using a Bayesian hierarchical model. The results from the integrated system and two individual sources were evaluated using a reference standard derived from drug labels. Area under the receiver operating characteristics curve (AUC) was computed to measure performance.

RESULTS

We observed a significant improvement in the AUC of the combined system when comparing it with Twitter alone, and no improvement when comparing with the SRS alone. The AUCs ranged from 0.587 to 0.637 for the combined SRS and Twitter, from 0.525 to 0.534 for Twitter alone, and from 0.612 to 0.642 for the SRS alone. The results varied because different preprocessing procedures were applied to Twitter.

CONCLUSION

The accuracy of signal detection using social media can be improved by combining signals with those from SRSs. However, the combined system cannot achieve better AUC performance than data from FAERS alone, which may indicate that Twitter data are not ready to be integrated into a purely data-driven combination system.

Infodemiology and Infoveillance: Scoping Review

BACKGROUND

Web-based sources are increasingly employed in the analysis, detection, and forecasting of diseases and epidemics, and in predicting human behavior toward several health topics. This use of the internet has come to be known as infodemiology, a concept introduced by Gunther Eysenbach. Infodemiology and infoveillance studies use web-based data and have become an integral part of health informatics research over the past decade.

OBJECTIVE

The aim of this paper is to provide a scoping review of the state-of-the-art in infodemiology along with the background and history of the concept, to identify sources and health categories and topics, to elaborate on the validity of the employed methods, and to discuss the gaps identified in current research.

METHODS

The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines were followed to extract the publications that fall under the umbrella of infodemiology and infoveillance from the JMIR, PubMed, and Scopus databases. A total of 338 documents were extracted for assessment.

RESULTS

Of the 338 studies, the vast majority (n=282, 83.4%) were published with JMIR Publications. The Journal of Medical Internet Research features almost half of the publications (n=168, 49.7%), and JMIR Public Health and Surveillance has more than one-fifth of the examined studies (n=74, 21.9%). The interest in the subject has been increasing every year, with 2018 featuring more than one-fourth of the total publications (n=89, 26.3%), and the publications in 2017 and 2018 combined accounted for more than half (n=171, 50.6%) of the total number of publications in the last decade. The most popular source was Twitter with 45.0% (n=152), followed by Google with 24.6% (n=83), websites and platforms with 13.9% (n=47), blogs and forums with 10.1% (n=34), Facebook with 8.9% (n=30), and other search engines with 5.6% (n=19). As for the subjects examined, conditions and diseases with 17.2% (n=58) and epidemics and outbreaks with 15.7% (n=53) were the most popular categories identified in this review, followed by health care (n=39, 11.5%), drugs (n=40, 10.4%), and smoking and alcohol (n=29, 8.6%).

CONCLUSIONS

The field of infodemiology is becoming increasingly popular, employing innovative methods and approaches for health assessment. The use of web-based sources, which provide us with information that would not be accessible otherwise and tackles the issues arising from the time-consuming traditional methods, shows that infodemiology plays an important role in health informatics research.

Assessment of the Frequency of Online Searches for Symptoms Before Diagnosis: Analysis of Archival Data

Background

Surveys suggest that a large proportion of people use the internet to search for information on medical symptoms they experience and that around one-third of the people in the United States self-diagnose using online information. However, surveys are known to be biased, and the true rates at which people search for information on their medical symptoms before receiving a formal medical diagnosis are unknown.

Objective

This study aimed to estimate the rate at which people search for information on their medical symptoms before receiving a formal medical diagnosis by a health professional.

Methods

We collected queries made on a general-purpose internet search engine by people in the United States who self-identified their diagnosis from 1 of 20 medical conditions. We focused on conditions that have evident symptoms and are neither screened systematically nor a part of usual medical care. Thus, they are generally diagnosed after the investigation of specific symptoms. We evaluated how many of these people queried for symptoms associated with their medical condition before their formal diagnosis. In addition, we used a survey questionnaire to assess the familiarity of laypeople with the symptoms associated with these conditions.

Results

On average, 15.49% (1792/12,367, SD 8.4%) of people queried about symptoms associated with their medical condition before receiving a medical diagnosis. A longer duration between the first query for a symptom and the corresponding diagnosis was correlated with an increased likelihood of people querying about those symptoms (rho=0.6; P=.005); similarly, unfamiliarity with the association between a condition and its symptom was correlated with an increased likelihood of people querying about those symptoms (rho=−0.47; P=.08). In addition, worrying symptoms were 14% more likely to be queried about.

Conclusions

Our results indicate that there is large variability in the percentage of people who query the internet for their symptoms before a formal medical diagnosis is made. This finding has important implications for systems that attempt to screen for medical conditions.

Lymelight: forecasting Lyme disease risk using web search data

Lyme disease is the most common tick-borne disease in the Northern Hemisphere. Existing estimates of Lyme disease spread are delayed a year or more. We introduce Lymelight-a new method for monitoring the incidence of Lyme disease in real-time. We use a machine-learned classifier of web search sessions to estimate the number of individuals who search for possible Lyme disease symptoms in a given geographical area for two years, 2014 and 2015. We evaluate Lymelight using the official case count data from CDC and find a 92% correlation (p < 0.001) at county level. Importantly, using web search data allows us not only to assess the incidence of the disease, but also to examine the appropriateness of treatments subsequently searched for by the users. Public health implications of our work include monitoring the spread of vector-borne diseases in a timely and scalable manner, complementing existing approaches through real-time detection, which can enable more timely interventions. Our analysis of treatment searches may also help reduce misdiagnosis of the disease.

Adverse events associated with colonoscopy; an examination of online concerns

Background

Colonoscopy as a screening and diagnostic tool is generally safe and well-tolerated, and significant complications are rare. The rate of more mild adverse effects is difficult to estimate, particularly when such effects do not result in hospital admission. We aimed to identify the rate and timing of adverse effects as reported by users querying symptoms on an internet search engine.

Methods

We identified queries made to Bing originating from users in the United States containing the word “colonoscopy” during a 12-month period and identified those queries in which the timing of colonoscopy could be estimated. We then identified queries from those same users for medical symptoms during the time span from 5 days before through 30 days after the colonoscopy date.

Results

Of 641,223 users mentioning colonoscopy, 7013 (1.1%) had a query that enabled identification of their colonoscopy date. The majority of queries about colonoscopy preceded the procedure, and concerned diet. 28% of colonoscopy-related queries were made afterwards, and included queries about diarrhea and cramps, with 2.6% of users querying respiratory symptoms after the procedure, including cough (1.2%) and pneumonia (0.6%). Respiratory symptoms rose significantly at days 7–10 after the colonoscopy.

Conclusions

Internet search queries for respiratory symptoms rose approximately one week after queries relating to colonoscopy, raising the possibility that such symptoms are an under-reported late adverse effect of the procedure. Given the widespread use of colonoscopy as a screening modality and the rise of anesthesia-assisted colonoscopy in the United States in recent years, this signal is of potential public health concern.

Adverse Events Due to Insomnia Drugs Reported in a Regulatory Database and Online Patient Reviews: Comparative Study

BACKGROUND

Patient online drug reviews are a resource for other patients seeking information about the practical benefits and drawbacks of drug therapies. Patient reviews may also serve as a source of postmarketing safety data that are more user-friendly than regulatory databases. However, the reliability of online reviews has been questioned, because they do not undergo professional review and lack means of verification.

OBJECTIVE

We evaluated online reviews of hypnotic medications, because they are commonly used and their therapeutic efficacy is particularly amenable to patient self-evaluation. Our primary objective was to compare the types and frequencies of adverse events reported to the Food and Drug Administration Adverse Event Reporting System (FAERS) with analogous information in patient reviews on the consumer health website Drugs.com. The secondary objectives were to describe patient reports of efficacy and adverse events and assess the influence of medication cost, effectiveness, and adverse events on user ratings of hypnotic medications.

METHODS

Patient ratings and narratives were retrieved from 1407 reviews on Drugs.com between February 2007 and March 2018 for eszopiclone, ramelteon, suvorexant, zaleplon, and zolpidem. Reviews were coded to preferred terms in the Medical Dictionary for Regulatory Activities. These reviews were compared to 5916 cases in the FAERS database from January 2015 to September 2017.

RESULTS

Similar adverse events were reported to both Drugs.com and FAERS. Both resources identified a lack of efficacy as a common complaint for all five drugs. Both resources revealed that amnesia commonly occurs with eszopiclone, zaleplon, and zolpidem, while nightmares commonly occur with suvorexant. Compared to FAERS, online reviews of zolpidem reported a much higher frequency of amnesia and partial sleep activities. User ratings were highest for zolpidem and lowest for suvorexant. Statistical analyses showed that patient ratings are influenced by considerations of efficacy and adverse events, while drug cost is unimportant.

CONCLUSIONS

For hypnotic medications, online patient reviews and FAERS emphasized similar adverse events. Online reviewers rated drugs based on perception of efficacy and adverse events. We conclude that online patient reviews of hypnotics are a valid source that can supplement traditional adverse event reporting systems.

Deconstructing the Pharmacovigilance Hype Cycle

Data science is making increasing contributions to pharmacovigilance. Although the technical innovation of these works are indisputable, efficient progress in real-world pharmacovigilance signal detection may be hampered by corresponding technology life cycle effects, with a resulting tendency to conclude that, with large enough datasets and intricate algorithms, "the numbers speak for themselves," discounting the importance of clinical and scientific judgment. A practical consequence is overzealous declarations regarding the safety or lack of safety of drugs. We describe these concerns through a critical discussion of key results and conclusions from case studies selected to illustrate these points.

Can internet search engine queries be used to diagnose diabetes? Analysis of archival search data

AIMS

Diabetes is often diagnosed late. This study aimed to assess the possibility for earlier detection of diabetes from search data, using predictive models trained on large-scale data.

METHODS

We extracted all English-language queries made by people in the USA to Bing during 1 year and identified queries containing symptoms of diabetes. We compared the ability of four different prediction models (linear regression, logistic regression, decision tree and random forest) to distinguish between users who stated that they were diagnosed with diabetes and users who did not refer to diabetes or diabetes drugs but queried about at least one of the symptoms.

RESULTS

We identified 11,050 "new diabetes users" who stated they had been diagnosed with diabetes and approximately 11.5 million "control users" who queried about symptoms without querying for terms related to diabetes. Both the logistic regression and the random forest models were able to distinguish between the populations with an area under curve of 0.92 which translates to a positive predictive value of 56% at a false-positive rate of 1%. The model could identify patients up to 240 days before they mentioned being diagnosed.

CONCLUSIONS

Some undiagnosed diabetes patients can be detected accurately according to their symptom queries to a search engine. Such earlier diagnosis, especially in cases of type 1 diabetes, could be clinically meaningful. The ability of search engines to serve as a population-wide screening tool could potentially be improved using additional data provided by users.

Classifying adverse drug reactions from imbalanced twitter data

BACKGROUND

Nowadays, social media are often being used by general public to create and share public messages related to their health. With the global increase in social media usage, there is a trend of posting information related to adverse drug reactions (ADR). Mining the social media data for this type of information will be helpful for pharmacological post-marketing surveillance and monitoring. Although the concept of using social media to facilitate pharmacovigilance is convincing, construction of automatic ADR detection systems remains a challenge because the corpora compiled from social media tend to be highly imbalanced, posing a major obstacle to the development of classifiers with reliable performance.

METHODS

Several methods have been proposed to address the challenge of imbalanced corpora. However, we are not aware of any studies that investigated the effectiveness of the strategies of dealing with the problem of imbalanced data in the context of ADR detection from social media. In light of this, we evaluated a variety of imbalanced techniques and proposed a novel word embedding-based synthetic minority over-sampling technique (WESMOTE), which synthesizes new training examples from the sentence representation based on word embeddings. We compared the performance of all methods on two large imbalanced datasets released for the purpose of detecting ADR posts.

RESULTS

In comparison with the state-of-the-art approaches, the classifiers that incorporated imbalanced classification techniques achieved comparable or better F-scores. All of our best performing configurations combined random under-sampling with techniques including the proposed WESMOTE, boosting and ensemble, implying that an integration of these approaches with under-sampling provides a reliable solution for large imbalanced social media datasets. Furthermore, ensemble-based methods like vote-based under-sampling (VUE) and random under-sampling boosting can be alternatives for the hybrid synthetic methods because both methods increase the diversity of the created weak classifiers, leading to better recall and overall F-scores for the minority classes.

CONCLUSIONS

Data collected from the social media are usually very large and highly imbalanced. In order to maximize the performance of a classifier trained on such data, applications of imbalanced strategies are required. We considered several practical methods for handling imbalanced Twitter data along with their performance on the binary classification task with respect to ADRs. In conclusion, the following practical insights are gained: 1) When dealing with text classification, the proposed word embedding-based synthetic minority over-sampling technique is more effective than traditional synthetic-based over-sampling methods. 2) In cases where large amounts of training data are available, the imbalanced strategies combined with under-sampling techniques are preferred. 3) Finally, employment of advanced methods does not guarantee better performance than simpler ones such as VUE, which achieved high performance with advantages like faster building time and ease of development.

Symptoms Prompting Interest in Celiac Disease and the Gluten-Free Diet: Analysis of Internet Search Term Data

BACKGROUND

Celiac disease, a common immune-based disease triggered by gluten, has diverse clinical manifestations, and the relative distribution of symptoms leading to diagnosis has not been well characterized in the population.

OBJECTIVE

This study aimed to use search engine data to identify a set of symptoms and conditions that would identify individuals at elevated likelihood of a subsequent celiac disease diagnosis. We also measured the relative prominence of these search terms before versus after a search related to celiac disease.

METHODS

We extracted English-language queries submitted to the Bing search engine in the United States and identified those who submitted a new celiac-related query during a 1-month period, without any celiac-related queries in the preceding 9 months. We compared the ratio between the number of times that each symptom or condition was asked in the 14 days preceding the first celiac-related query of each person and the number of searches for that same symptom or condition in the 14 days after the celiac-related query.

RESULTS

We identified 90,142 users who made a celiac-related query, of whom 6528 (7%) exhibited sustained interest, defined as making a query on more than 1 day. Though a variety of symptoms and associated conditions were also queried before a celiac-related query, the maximum area under the receiver operating characteristic curve was 0.53. The symptom most likely to be queried more before than after a celiac-related query was diarrhea (query ratio [QR] 1.28). Extraintestinal symptoms queried before a celiac disease query included headache (QR 1.26), anxiety (QR 1.10), depression (QR 1.03), and attention-deficit hyperactivity disorder (QR 1.64).

CONCLUSIONS

We found an increase in antecedent searches for symptoms known to be associated with celiac disease, a rise in searches for depression and anxiety, and an increase in symptoms that are associated with celiac disease but may not be reported to health care providers. The protean clinical manifestations of celiac disease are reflected in the diffuse nature of antecedent internet queries of those interested in celiac disease, underscoring the challenge of effective case-finding strategies.

Internet Searches for Medical Symptoms Before Seeking Information on 12-Step Addiction Treatment Programs: A Web-Search Log Analysis

Background

Brief intervention is a critical method for identifying patients with problematic substance use in primary care settings and for motivating them to consider treatment options. However, despite considerable evidence of delay discounting in patients with substance use disorders, most brief advice by physicians focuses on the long-term negative medical consequences, which may not be the best way to motivate patients to seek treatment information.

Objective

Identification of the specific symptoms that most motivate individuals to seek treatment information may offer insights for further improving brief interventions. To this end, we used anonymized internet search engine data to investigate which medical conditions and symptoms preceded searches for 12-step meeting locators and general 12-step information.

Methods

We extracted all queries made by people in the United States on the Bing search engine from November 2016 to July 2017. These queries were filtered for those who mentioned seeking Alcoholics Anonymous (AA) or Narcotics Anonymous (NA); in addition, queries that contained a medical symptom or condition or a synonym thereof were analyzed. We identified medical symptoms and conditions that predicted searches for seeking treatment at different time lags. Specifically, symptom queries were first determined to be significantly predictive of subsequent 12-step queries if the probability of querying a medical symptom by those who later sought information about the 12-step program exceeded the probability of that same query being made by a comparison group of all other Bing users in the United States. Second, we examined symptom queries preceding queries on the 12-step program at time lags of 0-7 days, 7-14 days, and 14-30 days, where the probability of asking about a medical symptom was greater in the 30-day time window preceding 12-step program information-seeking as compared to all previous times that the symptom was queried.

Results

In our sample of 11,784 persons, we found 10 medical symptoms that predicted AA information seeking and 9 symptoms that predicted NA information seeking. Of these symptoms, a substantial number could be categorized as nonsevere in nature. Moreover, when medical symptom persistence was examined across a 1-month time period, a substantial number of nonsevere, yet persistent, symptoms were identified.

Conclusions

Our results suggest that many common or nonsevere medical symptoms and conditions motivate subsequent interest in AA and NA programs. In addition to highlighting severe long-term consequences, brief interventions could be restructured to highlight how increasing substance misuse can worsen discomfort from common medical symptoms in the short term, as well as how these worsening symptoms could exacerbate social embarrassment or decrease physical attractiveness.

Online interest regarding violent attacks, gun control, and gun purchase: A causal analysis

BACKGROUND

Increased interest about gun ownership and gun control are oftentimes driven by informational shocks in a common factor, namely violent attacks, and the perceived need for higher levels of safety. A causal depiction of the societal interest around violent attacks, gun control and gun purchase, both synchronous and over time, should be a stepping stone for designing future strategies regarding the safety concerns of the U.S. population.

OBJECTIVE

Examine the causal relationships between unexpected increases in population interest about violent attacks, gun control, and gun purchase.

METHODS

Relationships among online searches for information about violent attacks, gun control, and gun purchase occurring between 2004 and 2017 in the U.S. are explained through a novel structural vector autoregressive time series model to account for simultaneous causal relationships.

RESULTS

More than 20% of the stationary variability in each of gun control and gun purchase interest can be explained by the remaining factors. Gun control interest appears to be caused, in part, by violent attacks informational shocks, yet violent attacks, although impactful, have a lesser effect than gun control debate on long-term gun ownership interests.

CONCLUSIONS

The form in which gun control has been introduced in public debate may have further increased gun ownership interest. Reactive gun purchase interest may be an unintended side effect of gun control debate. U.S. policymakers may need to rethink current approaches to promotion of gun control, and whether societal policy debate without policy outcomes could be having unintended effects.

Web-Based Signal Detection Using Medical Forums Data in France: Comparative Analysis

BACKGROUND

While traditional signal detection methods in pharmacovigilance are based on spontaneous reports, the use of social media is emerging. The potential strength of Web-based data relies on their volume and real-time availability, allowing early detection of signals of disproportionate reporting (SDRs).

OBJECTIVE

This study aimed (1) to assess the consistency of SDRs detected from patients' medical forums in France compared with those detected from the traditional reporting systems and (2) to assess the ability of SDRs in identifying earlier than the traditional reporting systems.

METHODS

Messages posted on patients' forums between 2005 and 2015 were used. We retained 8 disproportionality definitions. Comparison of SDRs from the forums with SDRs detected in VigiBase was done by describing the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), accuracy, receiver operating characteristics curve, and the area under the curve (AUC). The time difference in months between the detection dates of SDRs from the forums and VigiBase was provided.

RESULTS

The comparison analysis showed that the sensitivity ranged from 29% to 50.6%, the specificity from 86.1% to 95.5%, the PPV from 51.2% to 75.4%, the NPV from 68.5% to 91.6%, and the accuracy from 68% to 87.7%. The AUC reached 0.85 when using the metric empirical Bayes geometric mean. Up to 38% (12/32) of the SDRs were detected earlier in the forums than that in VigiBase.

CONCLUSIONS

The specificity, PPV, and NPV were high. The overall performance was good, showing that data from medical forums may be a valuable source for signal detection. In total, up to 38% (12/32) of the SDRs could have been detected earlier, thus, ensuring the increased safety of patients. Further enhancements are needed to investigate the reliability and validation of patients' medical forums worldwide, the extension of this analysis to all possible drugs or at least to a wider selection of drugs, as well as to further assess performance against established signals.

From Big Data to Smart Data for Pharmacovigilance: The Role of Healthcare Databases and Other Emerging Sources

In the last decade 'big data' has become a buzzword used in several industrial sectors, including but not limited to telephony, finance and healthcare. Despite its popularity, it is not always clear what big data refers to exactly. Big data has become a very popular topic in healthcare, where the term primarily refers to the vast and growing volumes of computerized medical information available in the form of electronic health records, administrative or health claims data, disease and drug monitoring registries and so on. This kind of data is generally collected routinely during administrative processes and clinical practice by different healthcare professionals: from doctors recording their patients' medical history, drug prescriptions or medical claims to pharmacists registering dispensed prescriptions. For a long time, this data accumulated without its value being fully recognized and leveraged. Today big data has an important place in healthcare, including in pharmacovigilance. The expanding role of big data in pharmacovigilance includes signal detection, substantiation and validation of drug or vaccine safety signals, and increasingly new sources of information such as social media are also being considered. The aim of the present paper is to discuss the uses of big data for drug safety post-marketing assessment.

The effectiveness of public health advertisements to promote health: a randomized-controlled trial on 794,000 participants

As public health advertisements move online, it becomes possible to run inexpensive randomized-controlled trials (RCTs) thereof. Here we report the results of an online RCT to improve food choices and integrate exercise into daily activities of internet users. People searching for pre-specified terms were randomized to receive one of several professionally developed campaign advertisements or the “status quo” (ads that would otherwise have been served). For 1-month pre-intervention and post-intervention, their searches for health-promoting goods or services were recorded. Our results show that 48% of people who were exposed to the ads made future searches for weight loss information, compared with 32% of those in the control group—a 50% increase. The advertisements varied in efficacy. However, the effectiveness of the advertisements may be greatly improved by targeting individuals based on their lifestyle preferences and/or sociodemographic characteristics, which together explain 49% of the variation in response to the ads. These results demonstrate that online advertisements hold promise as a mechanism for changing population health behaviors. They also provide researchers powerful ways to measure and improve the effectiveness of online public health interventions. Finally, we show that corporations that use these sophisticated tools to promote unhealthy products can potentially be outbid and outmaneuvered.

People who see specific health-promoting messages after searching online for weight-related terms are more likely to subsequently search for information on weight loss interventions. A team led by Elad Yom-Tov from Microsoft Research Israel in Herzeliya conducted a randomized trial involving 794,000 users of the Bing search engine who queried terms related to weight, diet, and exercise. Randomly chosen subjects were shown advertisements designed to promote healthy living, while all other users were shown standard ads. The researchers found that 48% of people exposed to the health-promoting advertisements made searches within the next month for weight loss information, compared with only 32% of those in the control group. The findings suggest that targeted online messaging can help change population health behaviors.

Utility of social media and crowd-intelligence data for pharmacovigilance: a scoping review

BACKGROUND

A scoping review to characterize the literature on the use of conversations in social media as a potential source of data for detecting adverse events (AEs) related to health products.

METHODS

Our specific research questions were (1) What social media listening platforms exist to detect adverse events related to health products, and what are their capabilities and characteristics? (2) What is the validity and reliability of data from social media for detecting these adverse events? MEDLINE, EMBASE, Cochrane Library, and relevant websites were searched from inception to May 2016. Any type of document (e.g., manuscripts, reports) that described the use of social media data for detecting health product AEs was included. Two reviewers independently screened citations and full-texts, and one reviewer and one verifier performed data abstraction. Descriptive synthesis was conducted.

RESULTS

After screening 3631 citations and 321 full-texts, 70 unique documents with 7 companion reports available from 2001 to 2016 were included. Forty-six documents (66%) described an automated or semi-automated information extraction system to detect health product AEs from social media conversations (in the developmental phase). Seven pre-existing information extraction systems to mine social media data were identified in eight documents. Nineteen documents compared AEs reported in social media data with validated data and found consistent AE discovery in all except two documents. None of the documents reported the validity and reliability of the overall system, but some reported on the performance of individual steps in processing the data. The validity and reliability results were found for the following steps in the data processing pipeline: data de-identification (n = 1), concept identification (n = 3), concept normalization (n = 2), and relation extraction (n = 8). The methods varied widely, and some approaches yielded better results than others.

CONCLUSIONS

Our results suggest that the use of social media conversations for pharmacovigilance is in its infancy. Although social media data has the potential to supplement data from regulatory agency databases; is able to capture less frequently reported AEs; and can identify AEs earlier than official alerts or regulatory changes, the utility and validity of the data source remains under-studied.

TRIAL REGISTRATION

Open Science Framework ( https://osf.io/kv9hu/ ).

Trends of infodemiology studies: a scoping review

INTRODUCTION

The health care industry is rich in data and information. Web technologies, such as search engines and social media, have provided an opportunity for the management of user generated data in real time in the form of infodemiology studies. The aim of this study was to investigate infodemiology studies conducted during 2002-2016, and compare them based on developed, developing and in transition countries.

METHODS

This scoping review was conducted in 2017 with the help of the PRISMA guidelines. PubMed, Scopus, Science Direct, Web of Knowledge, Google Scholar, Wiley and Springer databases were searched between the years 2002 and 2016. Finally, 56 articles were included in the review and analysed.

RESULTS

The initial infodemiology studies pertain to the quality assessment of the hospital's websites. Most of the studies were on developed countries, based on flu, and published in the Journal of Medical Internet Research.

CONCLUSION

The infodemiology approach provides unmatched opportunities for the management of health data and information generated by the users. Using this potential will provide unique opportunities for the health information need assessment in real time by health librarians and thereby provide evidence based health information to the people.

Detection of Adverse Drug Reactions using Medical Named Entities on Twitter

Adverse Drug Reactions (ADRs) are unintentional reactions caused by a drug or combination of drugs taken by a patient. The current ADR reporting systems inevitably have delays in reporting such events. The broad scope of social media conversations on sites such as Twitter means that inevitably health-related topics will be covered. This means that these sites could then be used to detect potentially novel ADRs with less latency for subsequent further investigation. In this work, we investigate ADR surveillance using a large corpus of Twitter data, containing around 50 billion tweets spanning 3 years (2012-2014), and evaluate against over 3000 drugs reported in the FAERS database. This is both a larger corpus and broader selection of drugs than previous work in the domain. We compare the ADRs identified using our method to the FDA Adverse Event Reporting System (FAERS) database of ADRs reported using more traditional techniques, and find that Twitter is a useful resource for ADR detection up to 72% micro-averaged precision. Micro-averaged recall of 6% is achievable using only 10% of Twitter, indicating that with a higher-volume or targeted feed it would be possible to detect a large percentage of ADRs.

Respiratory syncytial virus tracking using internet search engine data

Background

Respiratory Syncytial Virus (RSV) is the leading cause of hospitalization in children less than 1 year of age in the United States. Internet search engine queries may provide high resolution temporal and spatial data to estimate and predict disease activity.

Methods

After filtering an initial list of 613 symptoms using high-resolution Bing search logs, we used Google Trends data between 2004 and 2016 for a smaller list of 50 terms to build predictive models of RSV incidence for five states where long-term surveillance data was available. We then used domain adaptation to model RSV incidence for the 45 remaining US states.

Results

Surveillance data sources (hospitalization and laboratory reports) were highly correlated, as were laboratory reports with search engine data. The four terms which were most often statistically significantly correlated as time series with the surveillance data in the five state models were RSV, flu, pneumonia, and bronchiolitis. Using our models, we tracked the spread of RSV by observing the time of peak use of the search term in different states. In general, the RSV peak moved from south-east (Florida) to the north-west US.

Conclusions

Our study represents the first time that RSV has been tracked using Internet data results and highlights successful use of search filters and domain adaptation techniques, using data at multiple resolutions. Our approach may assist in identifying spread of both local and more widespread RSV transmission and may be applicable to other seasonal conditions where comprehensive epidemiological data is difficult to collect or obtain.

Evidence From Web-Based Dietary Search Patterns to the Role of B12 Deficiency in Non-Specific Chronic Pain: A Large-Scale Observational Study

Background

Profound vitamin B12 deficiency is a known cause of disease, but the role of low or intermediate levels of B12 in the development of neuropathy and other neuropsychiatric symptoms, as well as the relationship between eating meat and B12 levels, is unclear.

Objective

The objective of our study was to investigate the role of low or intermediate levels of B12 in the development of neuropathy and other neuropsychiatric symptoms.

Methods

We used food-related Internet search patterns from a sample of 8.5 million people based in the US as a proxy for B12 intake and correlated these searches with Internet searches related to possible effects of B12 deficiency.

Results

Food-related search patterns were highly correlated with known consumption and food-related searches (ρ=.69). Awareness of B12 deficiency was associated with a higher consumption of B12-rich foods and with queries for B12 supplements. Searches for terms related to neurological disorders were correlated with searches for B12-poor foods, in contrast with control terms. Popular medicines, those having fewer indications, and those which are predominantly used to treat pain, were more strongly correlated with the ability to predict neuropathic pain queries using the B12 contents of food.

Conclusions

Our findings show that Internet search patterns are a useful way of investigating health questions in large populations, and suggest that low B12 intake may be associated with a broader spectrum of neurological disorders than previously thought.

Toward multimodal signal detection of adverse drug reactions

OBJECTIVE

Improving mechanisms to detect adverse drug reactions (ADRs) is key to strengthening post-marketing drug safety surveillance. Signal detection is presently unimodal, relying on a single information source. Multimodal signal detection is based on jointly analyzing multiple information sources. Building on, and expanding the work done in prior studies, the aim of the article is to further research on multimodal signal detection, explore its potential benefits, and propose methods for its construction and evaluation.

MATERIAL AND METHODS

Four data sources are investigated; FDA's adverse event reporting system, insurance claims, the MEDLINE citation database, and the logs of major Web search engines. Published methods are used to generate and combine signals from each data source. Two distinct reference benchmarks corresponding to well-established and recently labeled ADRs respectively are used to evaluate the performance of multimodal signal detection in terms of area under the ROC curve (AUC) and lead-time-to-detection, with the latter relative to labeling revision dates.

RESULTS

Limited to our reference benchmarks, multimodal signal detection provides AUC improvements ranging from 0.04 to 0.09 based on a widely used evaluation benchmark, and a comparative added lead-time of 7-22 months relative to labeling revision dates from a time-indexed benchmark.

CONCLUSIONS

The results support the notion that utilizing and jointly analyzing multiple data sources may lead to improved signal detection. Given certain data and benchmark limitations, the early stage of development, and the complexity of ADRs, it is currently not possible to make definitive statements about the ultimate utility of the concept. Continued development of multimodal signal detection requires a deeper understanding the data sources used, additional benchmarks, and further research on methods to generate and synthesize signals.

Adverse Reactions Associated With Cannabis Consumption as Evident From Search Engine Queries

Background

Cannabis is one of the most widely used psychoactive substances worldwide, but adverse drug reactions (ADRs) associated with its use are difficult to study because of its prohibited status in many countries.

Objective

Internet search engine queries have been used to investigate ADRs in pharmaceutical drugs. In this proof-of-concept study, we tested whether these queries can be used to detect the adverse reactions of cannabis use.

Methods

We analyzed anonymized queries from US-based users of Bing, a widely used search engine, made over a period of 6 months and compared the results with the prevalence of cannabis use as reported in the US National Survey on Drug Use in the Household (NSDUH) and with ADRs reported in the Food and Drug Administration’s Adverse Drug Reporting System. Predicted prevalence of cannabis use was estimated from the fraction of people making queries about cannabis, marijuana, and 121 additional synonyms. Predicted ADRs were estimated from queries containing layperson descriptions to 195 ICD-10 symptoms list.

Results

Our results indicated that the predicted prevalence of cannabis use at the US census regional level reaches an R² of .71 NSDUH data. Queries for ADRs made by people who also searched for cannabis reveal many of the known adverse effects of cannabis (eg, cough and psychotic symptoms), as well as plausible unknown reactions (eg, pyrexia).

Conclusions

These results indicate that search engine queries can serve as an important tool for the study of adverse reactions of illicit drugs, which are difficult to study in other settings.

[Patterns of access to information on protection against UV during the Brazilian summer: is there such a thing as the "summer effect"?]

Internet search patterns associated with "windows" of collective interest have been increasingly investigated in the field of public health. This article sets out to identify search patterns relating to the quest for information on skin protection after the perception of excessive exposure to UV radiation - the so-called "summer effect" as it is commonly referred to in Brazil. To calculate the number of hits on the Brazilian National Cancer Institute website - a renowned source of information resources on prevention - log analyzer software was used to measure the volume of hits on specific content pages. The pages on skin protection and self-examination (pages of interest) were monitored over a 48-month period. It was seen that, although the monthly average of hits on pages of interest revealed statistically significant annual growth, the results for the analysis of variance showed no significant differences between the number of hits in the summer compared with other months (p = 0.7491). In short, the perception of intense exposure to the summer sun did not encourage further interest to search for information on prevention.

Combination of Deep Recurrent Neural Networks and Conditional Random Fields for Extracting Adverse Drug Reactions from User Reviews

Adverse drug reactions (ADRs) are an essential part of the analysis of drug use, measuring drug use benefits, and making policy decisions. Traditional channels for identifying ADRs are reliable but very slow and only produce a small amount of data. Text reviews, either on specialized web sites or in general-purpose social networks, may lead to a data source of unprecedented size, but identifying ADRs in free-form text is a challenging natural language processing problem. In this work, we propose a novel model for this problem, uniting recurrent neural architectures and conditional random fields. We evaluate our model with a comprehensive experimental study, showing improvements over state-of-the-art methods of ADR extraction.

Predicting Drug Recalls From Internet Search Engine Queries

Batches of pharmaceuticals are sometimes recalled from the market when a safety issue or a defect is detected in specific production runs of a drug. Such problems are usually detected when patients or healthcare providers report abnormalities to medical authorities. Here, we test the hypothesis that defective production lots can be detected earlier by monitoring queries to Internet search engines. We extracted queries from the USA to the Bing search engine, which mentioned one of the 5195 pharmaceutical drugs during 2015 and all recall notifications issued by the Food and Drug Administration (FDA) during that year. By using attributes that quantify the change in query volume at the state level, we attempted to predict if a recall of a specific drug will be ordered by FDA in a time horizon ranging from 1 to 40 days in future. Our results show that future drug recalls can indeed be identified with an AUC of 0.791 and a lift at 5% of approximately 6 when predicting a recall occurring one day ahead. This performance degrades as prediction is made for longer periods ahead. The most indicative attributes for prediction are sudden spikes in query volume about a specific medicine in each state. Recalls of prescription drugs and those estimated to be of medium-risk are more likely to be identified using search query data. These findings suggest that aggregated Internet search engine data can be used to facilitate in early warning of faulty batches of medicines.

Using Web-Based Search Data to Study the Public's Reactions to Societal Events: The Case of the Sandy Hook Shooting

Background

Internet search is the most common activity on the World Wide Web and generates a vast amount of user-reported data regarding their information-seeking preferences and behavior. Although this data has been successfully used to examine outbreaks, health care utilization, and outcomes related to quality of care, its value in informing public health policy remains unclear.

Objective

The aim of this study was to evaluate the role of Internet search query data in health policy development. To do so, we studied the public’s reaction to a major societal event in the context of the 2012 Sandy Hook School shooting incident.

Methods

Query data from the Yahoo! search engine regarding firearm-related searches was analyzed to examine changes in user-selected search terms and subsequent websites visited for a period of 14 days before and after the shooting incident.

Results

A total of 5,653,588 firearm-related search queries were analyzed. In the after period, queries increased for search terms related to “guns” (+50.06%), “shooting incident” (+333.71%), “ammunition” (+155.14%), and “gun-related laws” (+535.47%). The highest increase (+1054.37%) in Web traffic was seen by news websites following “shooting incident” queries whereas searches for “guns” (+61.02%) and “ammunition” (+173.15%) resulted in notable increases in visits to retail websites. Firearm-related queries generally returned to baseline levels after approximately 10 days.

Conclusions

Search engine queries present a viable infodemiology metric on public reactions and subsequent behaviors to major societal events and could be used by policymakers to inform policy development.

Impact of Predicting Health Care Utilization Via Web Search Behavior: A Data-Driven Analysis

BACKGROUND

By recent estimates, the steady rise in health care costs has deprived more than 45 million Americans of health care services and has encouraged health care providers to better understand the key drivers of health care utilization from a population health management perspective. Prior studies suggest the feasibility of mining population-level patterns of health care resource utilization from observational analysis of Internet search logs; however, the utility of the endeavor to the various stakeholders in a health ecosystem remains unclear.

OBJECTIVE

The aim was to carry out a closed-loop evaluation of the utility of health care use predictions using the conversion rates of advertisements that were displayed to the predicted future utilizers as a surrogate. The statistical models to predict the probability of user's future visit to a medical facility were built using effective predictors of health care resource utilization, extracted from a deidentified dataset of geotagged mobile Internet search logs representing searches made by users of the Baidu search engine between March 2015 and May 2015.

METHODS

We inferred presence within the geofence of a medical facility from location and duration information from users' search logs and putatively assigned medical facility visit labels to qualifying search logs. We constructed a matrix of general, semantic, and location-based features from search logs of users that had 42 or more search days preceding a medical facility visit as well as from search logs of users that had no medical visits and trained statistical learners for predicting future medical visits. We then carried out a closed-loop evaluation of the utility of health care use predictions using the show conversion rates of advertisements displayed to the predicted future utilizers. In the context of behaviorally targeted advertising, wherein health care providers are interested in minimizing their cost per conversion, the association between show conversion rate and predicted utilization score, served as a surrogate measure of the model's utility.

RESULTS

We obtained the highest area under the curve (0.796) in medical visit prediction with our random forests model and daywise features. Ablating feature categories one at a time showed that the model performance worsened the most when location features were dropped. An online evaluation in which advertisements were served to users who had a high predicted probability of a future medical visit showed a 3.96% increase in the show conversion rate.

CONCLUSIONS

Results from our experiments done in a research setting suggest that it is possible to accurately predict future patient visits from geotagged mobile search logs. Results from the offline and online experiments on the utility of health utilization predictions suggest that such prediction can have utility for health care providers.

Identifying Complementary and Alternative Medicine Usage Information from Internet Resources. A Systematic Review

OBJECTIVES

Identify and highlight research issues and methods used in studying Complementary and Alternative Medicine (CAM) information needs, access, and exchange over the Internet.

METHODS

A literature search was conducted using Preferred Reporting Items for Systematic Reviews and Meta-Analysis guidelines from PubMed to identify articles that have studied Internet use in the CAM context. Additional searches were conducted at Nature.com and Google Scholar.

RESULTS

The Internet provides a major medium for attaining CAM information and can also serve as an avenue for conducting CAM related surveys. Based on the literature analyzed in this review, there seems to be significant interest in developing methodologies for identifying CAM treatments, including the analysis of search query data and social media platform discussions. Several studies have also underscored the challenges in developing approaches for identifying the reliability of CAM-related information on the Internet, which may not be supported with reliable sources. The overall findings of this review suggest that there are opportunities for developing approaches for making available accurate information and developing ways to restrict the spread and sale of potentially harmful CAM products and information.

CONCLUSIONS

Advances in Internet research are yet to be used in context of understanding CAM prevalence and perspectives. Such approaches may provide valuable insights into the current trends and needs in context of CAM use and spread.

Differences in physical status, mental state and online behavior of people in pro-anorexia web communities

BACKGROUND

There is a debate about the effects of pro-anorexia (colloquially referred to as pro-ana) websites. Research suggests that the effect of these websites is not straightforward. Indeed, the actual function of these sites is disputed, with studies indicating both negative and positive effects.

AIM

This is the first study which systematically examined the differences between pro-anorexia web communities in four main aspects: web language used (posts); web interests/search behaviors (queries); users' self-reported weight status and weight goals; and associated self-reported mood/pathology.

METHODS

We collected three primary sources of data, including messages posed on three pro-ana websites, a survey completed by over 1000 participants of a pro-ana website, and the searches made on the Bing search engine of pro-anorexia users. These data were analyzed for content, reported demographics and pathology, and behavior over time.

RESULTS

Although members of the main pro-ana website investigated appear to be depressed, with high rates of self-harm and suicide attempts, users are significantly more interested in treatment, have wishes of procreation and reported the highest goal weights among the investigated sites. In contrast, users of other pro-ana websites investigated, are more interested in morbid themes including depression, self-harm and suicide. The percentage of severely malnourished website users, in general, appears to be small (20%).

CONCLUSIONS

Our results indicate that a new strategy is required to facilitate the communication between mental health specialists and pro-ana web users, recognizing the differences in harm associated with different websites.

A curated and standardized adverse drug event resource to accelerate drug safety research

Identification of adverse drug reactions (ADRs) during the post-marketing phase is one of the most important goals of drug safety surveillance. Spontaneous reporting systems (SRS) data, which are the mainstay of traditional drug safety surveillance, are used for hypothesis generation and to validate the newer approaches. The publicly available US Food and Drug Administration (FDA) Adverse Event Reporting System (FAERS) data requires substantial curation before they can be used appropriately, and applying different strategies for data cleaning and normalization can have material impact on analysis results. We provide a curated and standardized version of FAERS removing duplicate case records, applying standardized vocabularies with drug names mapped to RxNorm concepts and outcomes mapped to SNOMED-CT concepts, and pre-computed summary statistics about drug-outcome relationships for general consumption. This publicly available resource, along with the source code, will accelerate drug safety research by reducing the amount of time spent performing data management on the source FAERS reports, improving the quality of the underlying data, and enabling standardized analyses using common vocabularies.

In silico methods for drug repurposing and pharmacology

Data in the biological, chemical, and clinical domains are accumulating at ever-increasing rates and have the potential to accelerate and inform drug development in new ways. Challenges and opportunities now lie in developing analytic tools to transform these often complex and heterogeneous data into testable hypotheses and actionable insights. This is the aim of computational pharmacology, which uses in silico techniques to better understand and predict how drugs affect biological systems, which can in turn improve clinical use, avoid unwanted side effects, and guide selection and development of better treatments. One exciting application of computational pharmacology is drug repurposing-finding new uses for existing drugs. Already yielding many promising candidates, this strategy has the potential to improve the efficiency of the drug development process and reach patient populations with previously unmet needs such as those with rare diseases. While current techniques in computational pharmacology and drug repurposing often focus on just a single data modality such as gene expression or drug-target interactions, we argue that methods such as matrix factorization that can integrate data within and across diverse data types have the potential to improve predictive performance and provide a fuller picture of a drug's pharmacological action. WIREs Syst Biol Med 2016, 8:186-210. doi: 10.1002/wsbm.1337 For further resources related to this article, please visit the WIREs website.