Natural Language Processing and Its Implications for the Future of Medication Safety: A Narrative Review of Recent Advances and Challenges

Development and validation of HBV surveillance models using big data and machine learning

BACKGROUND

The construction of a robust healthcare information system is fundamental to enhancing countries' capabilities in the surveillance and control of hepatitis B virus (HBV). Making use of China's rapidly expanding primary healthcare system, this innovative approach using big data and machine learning (ML) could help towards the World Health Organization's (WHO) HBV infection elimination goals of reaching 90% diagnosis and treatment rates by 2030. We aimed to develop and validate HBV detection models using routine clinical data to improve the detection of HBV and support the development of effective interventions to mitigate the impact of this disease in China.

METHODS

Relevant data records extracted from the Family Medicine Clinic of the University of Hong Kong-Shenzhen Hospital's Hospital Information System were structuralized using state-of-the-art Natural Language Processing techniques. Several ML models have been used to develop HBV risk assessment models. The performance of the ML model was then interpreted using the Shapley value (SHAP) and validated using cohort data randomly divided at a ratio of 2:1 using a five-fold cross-validation framework.

RESULTS

The patterns of physical complaints of patients with and without HBV infection were identified by processing 158,988 clinic attendance records. After removing cases without any clinical parameters from the derivation sample (n = 105,992), 27,392 cases were analysed using six modelling methods. A simplified model for HBV using patients' physical complaints and parameters was developed with good discrimination (AUC = 0.78) and calibration (goodness of fit test p-value >0.05).

CONCLUSIONS

Suspected case detection models of HBV, showing potential for clinical deployment, have been developed to improve HBV surveillance in primary care setting in China. (Word count: 264).

Development of a text mining algorithm for identifying adverse drug reactions in electronic health records

Objective

Adverse drug reactions (ADRs) are a significant healthcare concern. They are often documented as free text in electronic health records (EHRs), making them challenging to use in clinical decision support systems (CDSS). The study aimed to develop a text mining algorithm to identify ADRs in free text of Dutch EHRs.

Materials and Methods

In Phase I, our previously developed CDSS algorithm was recoded and improved upon with the same relatively large dataset of 35 000 notes (Step A), using R to identify possible ADRs with Medical Dictionary for Regulatory Activities (MedDRA) terms and the related Systematized Nomenclature of Medicine Clinical Terms (SNOMED-CT) (Step B). In Phase II, 6 existing text-mining R-scripts were used to detect and present unique ADRs, and positive predictive value (PPV) and sensitivity were observed.

Results

In Phase IA, the recoded algorithm performed better than the previously developed CDSS algorithm, resulting in a PPV of 13% and a sensitivity of 93%. For The sensitivity for serious ADRs was 95%. The algorithm identified 58 additional possible ADRs. In Phase IB, the algorithm achieved a PPV of 10%, a sensitivity of 86%, and an F-measure of 0.18. In Phase II, four R-scripts enhanced the sensitivity and PPV of the algorithm, resulting in a PPV of 70%, a sensitivity of 73%, an F-measure of 0.71, and a 63% sensitivity for serious ADRs.

Discussion and Conclusion

The recoded Dutch algorithm effectively identifies ADRs from free-text Dutch EHRs using R-scripts and MedDRA/SNOMED-CT. The study details its limitations, highlighting the algorithm's potential and significant improvements.

Applications of Natural Language Processing for Automated Clinical Data Analysis in Orthopaedics

Natural language processing is an exciting and emerging field in health care that can transform the field of orthopaedics. It can aid in the process of automated clinical data analysis, changing the way we extract data for various purposes including research and registry formation, diagnosis, and medical billing. This scoping review will look at the various applications of NLP in orthopaedics. Specific examples of NLP applications include identification of essential data elements from surgical and imaging reports, patient feedback analysis, and use of AI conversational agents for patient engagement. We will demonstrate how NLP has proven itself to be a powerful and valuable tool. Despite these potential advantages, there are drawbacks we must consider. Concerns with data quality, bias, privacy, and accessibility may stand as barriers in the way of widespread implementation of NLP technology. As natural language processing technology continues to develop, it has the potential to revolutionize orthopaedic research and clinical practices and enhance patient outcomes.

Navigating the Complexities of Artificial Intelligence-Enabled Real-World Data Collection for Oncology Pharmacovigilance

This new editorial discusses the promise and challenges of successful integration of natural language processing methods into electronic health records for timely, robust, and fair oncology pharmacovigilance.

Paediatric medication incident reporting: a multicentre comparison study of medication errors identified at audit, detected by staff and reported to an incident system

OBJECTIVES

To compare medication errors identified at audit and via direct observation with medication errors reported to an incident reporting system at paediatric hospitals and to investigate differences in types and severity of errors detected and reported by staff.

METHODS

This is a comparison study at two tertiary referral paediatric hospitals between 2016 and 2020 in Australia. Prescribing errors were identified from a medication chart audit of 7785 patient records. Medication administration errors were identified from a prospective direct observational study of 5137 medication administration doses to 1530 patients. Medication errors reported to the hospitals' incident reporting system were identified and matched with errors identified at audit and observation.

RESULTS

Of 11 302 clinical prescribing errors identified at audit, 3.2 per 1000 errors (95% CI 2.3 to 4.4, n=36) had an incident report. Of 2224 potentially serious prescribing errors from audit, 26.1% (95% CI 24.3 to 27.9, n=580) were detected by staff and 11.2 per 1000 errors (95% CI 7.6 to 16.5, n=25) were reported to the incident system. Although the prescribing error detection rates varied between the two hospitals, there was no difference in incident reporting rates regardless of error severity. Of 40 errors associated with actual patient harm, only 7 (17.5%; 95% CI 8.7% to 31.9%) were detected by staff and 4 (10.0%; 95% CI 4.0% to 23.1%) had an incident report. None of the 2883 clinical medication administration errors observed, including 903 potentially serious errors and 144 errors associated with actual patient harm, had incident reports.

CONCLUSION

Incident reporting data do not provide an accurate reflection of medication errors and related harm to children in hospitals. Failure to detect medication errors is likely to be a significant contributor to low error reporting rates. In an era of electronic health records, new automated approaches to monitor medication safety should be pursued to provide real-time monitoring.

Artificial intelligence / machine-learning tool for post-market surveillance of in vitro diagnostic assays

The study compares an artificial intelligence technology with traditional manual search of literature databases to assess the accuracy and efficiency of retrieving relevant articles for post-market surveillance of in vitro diagnostic and medical devices under the Medical Device Regulation and In Vitro Diagnostic Medical Device Regulation. Over a 3-year period, literature searches and technical assessment searches were performed manually or using the Huma.AI platform to retrieve relevant articles related to the safety and performance of selected in vitro diagnostic and medical devices. The manual search involved refined keyword searches, screening of titles/abstracts / full text, and extraction of relevant information. The Huma.AI search utilized advanced caching techniques and a natural language processing system to identify relevant reports. Searches were conducted on PubMed and PubMed Central. The number of identified relevant reports, precision rates, and time requirements for each approach were analyzed. The Huma.AI system outperformed the manual search in terms of the number of identified relevant articles in almost all cases. The average precision rates per year were significantly higher and more consistent with the Huma.AI search compared with the manual search. The Huma.AI system also took significantly less time to perform the searches and analyze the outputs than the manual search. The study demonstrated that the Huma.AI platform was more effective and efficient in identifying relevant articles compared with the manual approach.

A large dataset of annotated incident reports on medication errors

Incident reports of medication errors are valuable learning resources for improving patient safety. However, pertinent information is often contained within unstructured free text, which prevents automated analysis and limits the usefulness of these data. Natural language processing can structure this free text automatically and retrieve relevant past incidents and learning materials, but to be able to do so requires a large, fully annotated and validated corpus of incident reports. We present a corpus of 58,658 machine-annotated incident reports of medication errors that can be used to advance the development of information extraction models and subsequent incident learning. We report the best F1-scores for the annotated dataset: 0.97 and 0.76 for named entity recognition and intention/factuality analysis, respectively, for the cross-validation exercise. Our dataset contains 478,175 named entities and differentiates between incident types by recognising discrepancies between what was intended and what actually occurred. We explain our annotation workflow and technical validation and provide access to the validation datasets and machine annotator for labelling future incident reports of medication errors.

Exploring patient medication adherence and data mining methods in clinical big data: A contemporary review

BACKGROUND

Increasingly, patient medication adherence data are being consolidated from claims databases and electronic health records (EHRs). Such databases offer an indirect avenue to gauge medication adherence in our data-rich healthcare milieu. The surge in data accessibility, coupled with the pressing need for its conversion to actionable insights, has spotlighted data mining, with machine learning (ML) emerging as a pivotal technique. Nonadherence poses heightened health risks and escalates medical costs. This paper elucidates the synergistic interaction between medical database mining for medication adherence and the role of ML in fostering knowledge discovery.

METHODS

We conducted a comprehensive review of EHR applications in the realm of medication adherence, leveraging ML techniques. We expounded on the evolution and structure of medical databases pertinent to medication adherence and harnessed both supervised and unsupervised ML paradigms to delve into adherence and its ramifications.

RESULTS

Our study underscores the applications of medical databases and ML, encompassing both supervised and unsupervised learning, for medication adherence in clinical big data. Databases like SEER and NHANES, often underutilized due to their intricacies, have gained prominence. Employing ML to excavate patient medication logs from these databases facilitates adherence analysis. Such findings are pivotal for clinical decision-making, risk stratification, and scholarly pursuits, aiming to elevate healthcare quality.

CONCLUSION

Advanced data mining in the era of big data has revolutionized medication adherence research, thereby enhancing patient care. Emphasizing bespoke interventions and research could herald transformative shifts in therapeutic modalities.

Use of Electronic Health Record Data for Drug Safety Signal Identification: A Scoping Review

INTRODUCTION

Pharmacovigilance programs protect patient health and safety by identifying adverse event signals through postmarketing surveillance of claims data and spontaneous reports. Electronic health records (EHRs) provide new opportunities to address limitations of traditional approaches and promote discovery-oriented pharmacovigilance.

METHODS

To evaluate the current state of EHR-based medication safety signal identification, we conducted a scoping literature review of studies aimed at identifying safety signals from routinely collected patient-level EHR data. We extracted information on study design, EHR data elements utilized, analytic methods employed, drugs and outcomes evaluated, and key statistical and data analysis choices.

RESULTS

We identified 81 eligible studies. Disproportionality methods were the predominant analytic approach, followed by data mining and regression. Variability in study design makes direct comparisons difficult. Studies varied widely in terms of data, confounding adjustment, and statistical considerations.

CONCLUSION

Despite broad interest in utilizing EHRs for safety signal identification, current efforts fail to leverage the full breadth and depth of available data or to rigorously control for confounding. The development of best practices and application of common data models would promote the expansion of EHR-based pharmacovigilance.

Natural language processing-driven state machines to extract social factors from unstructured clinical documentation

Objective

This study sought to create natural language processing algorithms to extract the presence of social factors from clinical text in 3 areas: (1) housing, (2) financial, and (3) unemployment. For generalizability, finalized models were validated on data from a separate health system for generalizability.

Materials and Methods

Notes from 2 healthcare systems, representing a variety of note types, were utilized. To train models, the study utilized n-grams to identify keywords and implemented natural language processing (NLP) state machines across all note types. Manual review was conducted to determine performance. Sampling was based on a set percentage of notes, based on the prevalence of social need. Models were optimized over multiple training and evaluation cycles. Performance metrics were calculated using positive predictive value (PPV), negative predictive value, sensitivity, and specificity.

Results

PPV for housing rose from 0.71 to 0.95 over 3 training runs. PPV for financial rose from 0.83 to 0.89 over 2 training iterations, while PPV for unemployment rose from 0.78 to 0.88 over 3 iterations. The test data resulted in PPVs of 0.94, 0.97, and 0.95 for housing, financial, and unemployment, respectively. Final specificity scores were 0.95, 0.97, and 0.95 for housing, financial, and unemployment, respectively.

Discussion

We developed 3 rule-based NLP algorithms, trained across health systems. While this is a less sophisticated approach, the algorithms demonstrated a high degree of generalizability, maintaining >0.85 across all predictive performance metrics.

Conclusion

The rule-based NLP algorithms demonstrated consistent performance in identifying 3 social factors within clinical text. These methods may be a part of a strategy to measure social factors within an institution.

Improving drug safety with adverse event detection using natural language processing

INTRODUCTION

Pharmacovigilance (PV) involves monitoring and aggregating adverse event information from a variety of data sources, including health records, biomedical literature, spontaneous adverse event reports, product labels, and patient-generated content like social media posts, but the most pertinent details in these sources are typically available in narrative free-text formats. Natural language processing (NLP) techniques can be used to extract clinically relevant information from PV texts to inform decision-making.

AREAS COVERED

We conducted a non-systematic literature review by querying the PubMed database to examine the uses of NLP in drug safety and distilled the findings to present our expert opinion on the topic.

EXPERT OPINION

New NLP techniques and approaches continue to be applied for drug safety use cases; however, systems that are fully deployed and in use in a clinical environment remain vanishingly rare. To see high-performing NLP techniques implemented in the real setting will require long-term engagement with end users and other stakeholders and revised workflows in fully formulated business plans for the targeted use cases. Additionally, we found little to no evidence of extracted information placed into standardized data models, which should be a way to make implementations more portable and adaptable.

Towards artificial intelligence-enabled extracellular vesicle precision drug delivery

Extracellular Vesicles (EVs), particularly exosomes, recently exploded into nanomedicine as an emerging drug delivery approach due to their superior biocompatibility, circulating stability, and bioavailability in vivo. However, EV heterogeneity makes molecular targeting precision a critical challenge. Deciphering key molecular drivers for controlling EV tissue targeting specificity is in great need. Artificial intelligence (AI) brings powerful prediction ability for guiding the rational design of engineered EVs in precision control for drug delivery. This review focuses on cutting-edge nano-delivery via integrating large-scale EV data with AI to develop AI-directed EV therapies and illuminate the clinical translation potential. We briefly review the current status of EVs in drug delivery, including the current frontier, limitations, and considerations to advance the field. Subsequently, we detail the future of AI in drug delivery and its impact on precision EV delivery. Our review discusses the current universal challenge of standardization and critical considerations when using AI combined with EVs for precision drug delivery. Finally, we will conclude this review with a perspective on future clinical translation led by a combined effort of AI and EV research.

Developing Validated Tools to Identify Pulmonary Embolism in Electronic Databases: Rationale and Design of the PE-EHR+ Study

BACKGROUND

 Contemporary pulmonary embolism (PE) research, in many cases, relies on data from electronic health records (EHRs) and administrative databases that use International Classification of Diseases (ICD) codes. Natural language processing (NLP) tools can be used for automated chart review and patient identification. However, there remains uncertainty with the validity of ICD-10 codes or NLP algorithms for patient identification.

METHODS

 The PE-EHR+ study has been designed to validate ICD-10 codes as Principal Discharge Diagnosis, or Secondary Discharge Diagnoses, as well as NLP tools set out in prior studies to identify patients with PE within EHRs. Manual chart review by two independent abstractors by predefined criteria will be the reference standard. Sensitivity, specificity, and positive and negative predictive values will be determined. We will assess the discriminatory function of code subgroups for intermediate- and high-risk PE. In addition, accuracy of NLP algorithms to identify PE from radiology reports will be assessed.

RESULTS

 A total of 1,734 patients from the Mass General Brigham health system have been identified. These include 578 with ICD-10 Principal Discharge Diagnosis codes for PE, 578 with codes in the secondary position, and 578 without PE codes during the index hospitalization. Patients within each group were selected randomly from the entire pool of patients at the Mass General Brigham health system. A smaller subset of patients will also be identified from the Yale-New Haven Health System. Data validation and analyses will be forthcoming.

CONCLUSIONS

 The PE-EHR+ study will help validate efficient tools for identification of patients with PE in EHRs, improving the reliability of efficient observational studies or randomized trials of patients with PE using electronic databases.

Thematic mapping of perioperative incident reporting data to relational coordination domains

Communication failure is a common root cause of adverse clinical events. Problematic communication domains are difficult to decipher, and communication improvement strategies are scarce. This study compared perioperative incident reports (IR) identifying potential communication failures with the results of a contemporaneous peri-operative Relational Coordination (RC) survey. We hypothesised that IR-prevalent themes would map to areas-of-weakness identified in the RC survey. Perioperative IRs filed between 2018 and 2020 (n = 6,236) were manually reviewed to identify communication failures (n = 1049). The IRs were disaggregated into seven RC theory domains and compared with the RC survey. Report disaggregation ratings demonstrated a three-way inter-rater agreement of 91.2%. Of the 1,049 communication failure-related IRs, shared knowledge deficits (n = 479, 46%) or accurate communication (n = 465, 44%) were most frequently identified. Communication frequency failures (n = 3, 0.3%) were rarely coded. Comparatively, shared knowledge was the weakest domain in the RC survey, while communication frequency was the strongest, correlating well with our IR data. Linking IR with RC domains offers a novel approach to assessing the specific elements of communication failures with an acute care facility. This approach provides a deployable mechanism to trend intra- and inter-domain progress in communication success, and develop targeted interventions to mitigate against communication failure-related adverse events.

Biological gene extraction path based on knowledge graph and natural language processing

The continuous progress of society and the vigorous development of science and technology have brought people the dawn of maintaining health and preventing and controlling diseases. At the same time, with the update and iteration of bioinformatics technology, the current biological gene research has also undergone revolutionary changes. However, a long-standing problem in genetic research has always plagued researchers, that is, how to find the most needed sample genes from a large number of sample genes, so as to reduce unnecessary research and reduce research costs. By studying the extraction path of biological genes, it can help researchers to extract the most valuable research genes and avoid wasting time and energy. In order to solve the above problems, this paper used the Bhattacharyya distance index and the Gini index to screen the sample genes when extracting the characteristic genes of breast cancer. In the selected 49 public genes, 6 principal components were extracted by principal component analysis (PCA), and finally the experimental results were tested. It was found that when the optimal number of characteristic genes was selected as 5, the recognition rate of genes reached the highest 90.31%, which met the experimental requirements. In addition, the experiment also proved that the characteristic gene extraction method designed in this paper had a removal rate of 99.75% of redundant genes, which can greatly reduce the time and money cost of research.

Adverse drug event detection using natural language processing: A scoping review of supervised learning methods

To reduce adverse drug events (ADEs), hospitals need a system to support them in monitoring ADE occurrence routinely, rapidly, and at scale. Natural language processing (NLP), a computerized approach to analyze text data, has shown promising results for the purpose of ADE detection in the context of pharmacovigilance. However, a detailed qualitative assessment and critical appraisal of NLP methods for ADE detection in the context of ADE monitoring in hospitals is lacking. Therefore, we have conducted a scoping review to close this knowledge gap, and to provide directions for future research and practice. We included articles where NLP was applied to detect ADEs in clinical narratives within electronic health records of inpatients. Quantitative and qualitative data items relating to NLP methods were extracted and critically appraised. Out of 1,065 articles screened for eligibility, 29 articles met the inclusion criteria. Most frequent tasks included named entity recognition (n = 17; 58.6%) and relation extraction/classification (n = 15; 51.7%). Clinical involvement was reported in nine studies (31%). Multiple NLP modelling approaches seem suitable, with Long Short Term Memory and Conditional Random Field methods most commonly used. Although reported overall performance of the systems was high, it provides an inflated impression given a steep drop in performance when predicting the ADE entity or ADE relation class. When annotating corpora, treating an ADE as a relation between a drug and non-drug entity seems the best practice. Future research should focus on semi-automated methods to reduce the manual annotation effort, and examine implementation of the NLP methods in practice.

Natural language processing: state of the art, current trends and challenges

Natural language processing (NLP) has recently gained much attention for representing and analyzing human language computationally. It has spread its applications in various fields such as machine translation, email spam detection, information extraction, summarization, medical, and question answering etc. In this paper, we first distinguish four phases by discussing different levels of NLP and components of Natural Language Generation followed by presenting the history and evolution of NLP. We then discuss in detail the state of the art presenting the various applications of NLP, current trends, and challenges. Finally, we present a discussion on some available datasets, models, and evaluation metrics in NLP.

Development and evaluation of an interoperable natural language processing system for identifying pneumonia across clinical settings of care and institutions

Objective

To evaluate the feasibility, accuracy, and interoperability of a natural language processing (NLP) system that extracts diagnostic assertions of pneumonia in different clinical notes and institutions.

Materials and Methods

A rule-based NLP system was designed to identify assertions of pneumonia in 3 types of clinical notes from electronic health records (EHRs): emergency department notes, radiology reports, and discharge summaries. The lexicon and classification logic were tailored for each note type. The system was first developed and evaluated using annotated notes from the Department of Veterans Affairs (VA). Interoperability was assessed using data from the University of Utah (UU).

Results

The NLP system was comprised of 782 rules and achieved moderate-to-high performance in all 3 note types in VA (precision/recall/f1: emergency = 88.1/86.0/87.1; radiology = 71.4/96.2/82.0; discharge = 88.3/93.0/90.1). When applied to UU data, performance was maintained in emergency and radiology but decreased in discharge summaries (emergency = 84.7/94.3/89.3; radiology = 79.7/100.0/87.9; discharge = 65.5/92.7/76.8). Customization with 34 additional rules increased performance for all note types (emergency = 89.3/94.3/91.7; radiology = 87.0/100.0/93.1; discharge = 75.0/95.1/83.4).

Conclusion

NLP can be used to accurately identify the diagnosis of pneumonia across different clinical settings and institutions. A limited amount of customization to account for differences in lexicon, clinical definition of pneumonia, and EHR structure can achieve high accuracy without substantial modification.

Development of a Novel Scoring System to Quantify the Severity of Incident Reports: An Exploratory Research Study

Incident reporting systems have been widely adopted to collect information about patient safety incidents. Much of the value of incident reports lies in the free-text section. Computer processing of semantic information may be helpful to analyze this. We developed a novel scoring system for decision making to assess the severity of incidents using the semantic characteristics of the text in incident reports, and compared its results with experts' opinions. We retrospectively analyzed free-text data from incident reports from January 2012 to September 2021 at Nagoya University Hospital, Aichi, Japan. The sample was allocated to training and validation datasets using the hold-out method. Morphological analysis was used to segment terms in the training dataset. We calculated a severity term score, a severity report score and severity group score, by report volume size, and compared these with conventional severity classifications by patient safety experts and reporters. We allocated 96,082 incident reports into two groups. We calculated 1,802 severity term scores from the 48,041 reports in the training dataset. There was a significant difference in severity report score between reports categorized as severe and not severe by experts (95% confidence interval [CI] -0.83 to -0.80, p < 0.001, d = 0.81). Severity group scores were positively associated with severity ratings from experts and reporters (correlation coefficients 0.73 [95% CI 0.63-0.80, p < 0.001] and 0.79 [95% CI 0.71-0.85, p < 0.001]) for all departments. Our severity scoring system could therefore contribute to better organizational patient safety.

Current patient safety issues: the role of pharmacovigilance

In the current conditions of the pandemic, the burden on the healthcare system, including the pharmacovigilance system monitoring the safety of pharmacotherapy, has significantly increased in all countries. An integral component in ensuring the safety of pharmacotherapy is the identification and prevention of the development of adverse drug reactions (ADR), which are a serious health problem worldwide. One of the modern problems of healthcare, including pharmacovigilance, was the lack of vaccines and drugs for the treatment and prevention of COVID-19 in the first waves of the pandemic, which led to the use of off-label a large number of drugs (hydroxychloroquine, azithromycin, ivermectin) for the treatment of patients with COVID-19 despite the fact that scientific data their benefits were of poor quality and based on in vitro studies. The accelerated approval of drugs and vaccines to combat the COVID-19 pandemic also highlighted the need for rapid data on the safety of drugs in the post-marketing period. However, despite the fact that pharmacovigilance is developing, it still lags behind the impressive scientific and technological achievements achieved in other areas of medicine. Unfortunately, spontaneous reporting does not assess the true prevalence of ADR well, since reporting indicators can vary significantly depending on the motivation, availability of time, qualifications, fear of punishment and similar factors of the sender. Given these known limitations of the spontaneous messaging method, additional strategies for detecting ADR are often used, including trigger tools, manual viewing of medical records and automated monitoring.

Spontaneously generated online patient experience data - how and why is it being used in health research: an umbrella scoping review

PURPOSE

Social media has led to fundamental changes in the way that people look for and share health related information. There is increasing interest in using this spontaneously generated patient experience data as a data source for health research. The aim was to summarise the state of the art regarding how and why SGOPE data has been used in health research. We determined the sites and platforms used as data sources, the purposes of the studies, the tools and methods being used, and any identified research gaps.

METHODS

A scoping umbrella review was conducted looking at review papers from 2015 to Jan 2021 that studied the use of SGOPE data for health research. Using keyword searches we identified 1759 papers from which we included 58 relevant studies in our review.

RESULTS

Data was used from many individual general or health specific platforms, although Twitter was the most widely used data source. The most frequent purposes were surveillance based, tracking infectious disease, adverse event identification and mental health triaging. Despite the developments in machine learning the reviews included lots of small qualitative studies. Most NLP used supervised methods for sentiment analysis and classification. Very early days, methods need development. Methods not being explained. Disciplinary differences - accuracy tweaks vs application. There is little evidence of any work that either compares the results in both methods on the same data set or brings the ideas together.

CONCLUSION

Tools, methods, and techniques are still at an early stage of development, but strong consensus exists that this data source will become very important to patient centred health research.

Using the Electronic Health Record to Characterize the Hepatitis C Virus Care Cascade

OBJECTIVES

Chronic hepatitis C virus (HCV) infection is one of the main causes of hepatocellular carcinoma. Before initiating a multilevel HCV screening intervention, we sought to (1) describe concordance between the electronic health record (EHR) data warehouse and manual medical record review in recording aspects of HCV testing and treatment and (2) estimate the percentage of patients with chronic HCV infection who initiated and completed HCV treatment using manual medical record review.

METHODS

We examined the medical records for 177 patients (100 randomly selected patients born during 1945-1965 without evidence of HCV testing and 77 adult patients of any birth cohort who had completed HCV testing) with a primary care or relevant specialist visit at an academic health care system in Tampa, Florida, from 2015 through 2018. We used the Cohen κ coefficient to examine the degree of concordance between the searchable data warehouse and the medical record review abstractions. Descriptive statistics characterized referral to and receipt of treatment among patients with chronic HCV infection from medical record review.

RESULTS

We found generally good concordance between the data warehouse abstraction and medical record review for HCV testing data (κ ranged from 0.66 to 0.87). However, the data warehouse failed to capture data on HCV treatment variables. According to medical record review, 28 patients had chronic HCV infection; 16 patients were prescribed treatment, 14 initiated treatment, and 9 achieved and had a reported posttreatment undetected HCV viral load.

CONCLUSIONS

Using data warehouse data provides generally reliable HCV testing information. However, without the use of natural language processing and purposeful EHR design, manual medical record reviews will likely be required to characterize treatment initiation and completion.

Predictive Modelling of Susceptibility to Substance Abuse, Mortality and Drug-Drug Interactions in Opioid Patients

Objective: Opioids are a class of drugs that are known for their use as pain relievers. They bind to opioid receptors on nerve cells in the brain and the nervous system to mitigate pain. Addiction is one of the chronic and primary adverse events of prolonged usage of opioids. They may also cause psychological disorders, muscle pain, depression, anxiety attacks etc. In this study, we present a collection of predictive models to identify patients at risk of opioid abuse and mortality by using their prescription histories. Also, we discover particularly threatening drug-drug interactions in the context of opioid usage.

Methods and Materials: Using a publicly available dataset from MIMIC-III, two models were trained, Logistic Regression with L2 regularization (baseline) and Extreme Gradient Boosting (enhanced model), to classify the patients of interest into two categories based on their susceptibility to opioid abuse. We’ve also used K-Means clustering, an unsupervised algorithm, to explore drug-drug interactions that might be of concern.

Results: The baseline model for classifying patients susceptible to opioid abuse has an F1 score of 76.64% (accuracy 77.16%) while the enhanced model has an F1 score of 94.45% (accuracy 94.35%). These models can be used as a preliminary step towards inferring the causal effect of opioid usage and can help monitor the prescription practices to minimize the opioid abuse.

Discussion and Conclusion: Results suggest that the enhanced model provides a promising approach in preemptive identification of patients at risk for opioid abuse. By discovering and correlating the patterns contributing to opioid overdose or abuse among a variety of patients, machine learning models can be used as an efficient tool to help uncover the existing gaps and/or fraudulent practices in prescription writing. To quote an example of one such incidental finding, our study discovered that insulin might possibly be interacting with opioids in an unfavourable way leading to complications in diabetic patients. This indicates that diabetic patients under long term opioid usage might need to take increased amounts of insulin to make it more effective. This observation backs up prior research studies done on a similar aspect. To increase the translational value of our work, the predictive models and the associated software code are made available under the MIT License.

Text Mining of Adverse Events in Clinical Trials: Deep Learning Approach

BACKGROUND

Pharmacovigilance and safety reporting, which involve processes for monitoring the use of medicines in clinical trials, play a critical role in the identification of previously unrecognized adverse events or changes in the patterns of adverse events.

OBJECTIVE

This study aims to demonstrate the feasibility of automating the coding of adverse events described in the narrative section of the serious adverse event report forms to enable statistical analysis of the aforementioned patterns.

METHODS

We used the Unified Medical Language System (UMLS) as the coding scheme, which integrates 217 source vocabularies, thus enabling coding against other relevant terminologies such as the International Classification of Diseases-10th Revision, Medical Dictionary for Regulatory Activities, and Systematized Nomenclature of Medicine). We used MetaMap, a highly configurable dictionary lookup software, to identify the mentions of the UMLS concepts. We trained a binary classifier using Bidirectional Encoder Representations from Transformers (BERT), a transformer-based language model that captures contextual relationships, to differentiate between mentions of the UMLS concepts that represented adverse events and those that did not.

RESULTS

The model achieved a high F1 score of 0.8080, despite the class imbalance. This is 10.15 percent points lower than human-like performance but also 17.45 percent points higher than that of the baseline approach.

CONCLUSIONS

These results confirmed that automated coding of adverse events described in the narrative section of serious adverse event reports is feasible. Once coded, adverse events can be statistically analyzed so that any correlations with the trialed medicines can be estimated in a timely fashion.

Leveraging Case Narratives to Enhance Patient Age Ascertainment from Adverse Event Reports

INTRODUCTION

Missing age presents a significant challenge when evaluating individual case safety reports (ICSRs) in the FDA Adverse Event Reporting System (FAERS). When age is missing in an ICSR's structured field, it may be in the report's free-text narrative.

OBJECTIVES

This study aimed to evaluate the performance and assess the potential impact of a rule-based natural language processing (NLP) tool that utilizes a text string search to identify patients' numerical age from unstructured narratives.

METHODS

Using FAERS ICSRs from 2002 to 2018, we evaluated the annual proportion of ICSRs with age missing in the structured field before and after NLP application. Reviewers manually identified patients' age from ICSR narratives (gold standard) from a random sample of 1500 ICSRs. The gold standard was compared to the NLP-identified age.

RESULTS

During the study period, the percentage of ICSRs missing age in the structured field increased from 21.9 to 43.8%. The NLP tool performed well among the random sample: sensitivity 98.5%, specificity 92.9%, positive predictive value (PPV) 94.9%, and F-measure 96.7%. It also performed well for the subset of ICSRs missing age in the structured field; when applied to these cases, NLP identified age for an additional one million ICSRs (10% of the total number of ICSRs from 2002 to 2018) and decreased the percentage of ICSRs missing age to 27% overall.

CONCLUSIONS

NLP has potential utility to extract patients' age from ICSR narratives. Use of this tool would enhance pharmacovigilance and research using FAERS data.

Deep learning-based natural language processing in ophthalmology: applications, challenges and future directions

PURPOSE OF REVIEW

Artificial intelligence (AI) is the fourth industrial revolution in mankind's history. Natural language processing (NLP) is a type of AI that transforms human language, to one that computers can interpret and process. NLP is still in the formative stages of development in healthcare, with promising applications and potential challenges in its applications. This review provides an overview of AI-based NLP, its applications in healthcare and ophthalmology, next-generation use case, as well as potential challenges in deployment.

RECENT FINDINGS

The integration of AI-based NLP systems into existing clinical care shows considerable promise in disease screening, risk stratification, and treatment monitoring, amongst others. Stakeholder collaboration, greater public acceptance, and advancing technologies will continue to shape the NLP landscape in healthcare and ophthalmology.

SUMMARY

Healthcare has always endeavored to be patient centric and personalized. For AI-based NLP systems to become an eventual reality in larger-scale applications, it is pertinent for key stakeholders to collaborate and address potential challenges in application. Ultimately, these would enable more equitable and generalizable use of NLP systems for the betterment of healthcare and society.

The use of narrative electronic prescribing instructions in pharmacoepidemiology: A scoping review for the International Society for Pharmacoepidemiology

Narrative electronic prescribing instructions (NEPIs) are text that convey information on the administration or co‐administration of a drug as directed by a prescriber. For researchers, NEPIs have the potential to advance our understanding of the risks and benefits of medications in populations; however, due to their unstructured nature, they are not often utilized. The goal of this scoping review was to evaluate how NEPIs are currently employed in research, identify opportunities and challenges for their broader application, and provide recommendations on their future use. The scoping review comprised a comprehensive literature review and a survey of key stakeholders. From the literature review, we identified 33 primary articles that described the use of NEPIs. The majority of articles (n = 19) identified issues with the quality of information in NEPIs compared with structured prescribing information; nine articles described the development of novel algorithms that performed well in extracting information from NEPIs, and five described the used of manual or simpler algorithms to extract prescribing information from NEPIs. A survey of 19 stakeholders indicated concerns for the quality of information in NEPIs and called for standardization of NEPIs to reduce data variability/errors. Nevertheless, stakeholders believed NEPIs present an opportunity to identify prescriber's intent for the prescription and to study temporal treatment patterns. In summary, NEPIs hold much promise for advancing the field of pharmacoepidemiology. Researchers should take advantage of addressing important questions that can be uniquely answered with NEPIs, but exercise caution when using this information and carefully consider the quality of the data.

[Utilizing social media data in post-market safety surveillance]

Post-marketing surveillance is the principal means to ensure drug use safety. The spontaneous report is the essential method of post-marketing surveillance for drug safety. Often, most spontaneous reports come from medical staff and sometimes come from patients who use the drug. The posts published by individuals on social media platforms that contain drugs and related adverse reaction content have gradually been seen as a new data source similar to spontaneous reports from drug users in recent years. Those user-generated posts potentially provide researchers and regulators with new opportunities to conduct post-marketing surveillance for drug safety from patients' perspectives mostly rather than medical professionals and can afford the possibility theoretically to discover drug-related safety issues earlier than traditional methods. Social media data as a new data source for safety signal detection and signal reinforcement have the unique advantages, such as population coverage, type of drugs, type of adverse reactions, data timeliness and quantity. Most of the social media data used in post-marketing surveillance research for drug safety are still text data in English, and even multiple languages are used by different people worldwide on several social media platforms. Unfortunately, there is still a controversy in the academic circles whether social media data can be used as reliable data sources for routine post-marketing surveillance for drug safety. A couple of obstacles of data, methods and ethics must be overcome before leveraging social media data for post-marketing surveillance. The number of Chinese social media users is large, and the social media data in the Chinese language is rapidly snowballing, which can be employed as the potential data source for post-marketing surveillance for drug safety. However, due to the Chinese language's specific characteristics, the text's diversity is different from the English text, and there is not enough accepted corpus in medical scenarios. Besides, the lack of domestic laws and regulations on privacy and security protection of social media data poses more challenges for applying Chinese social media data for post-market surveillance. The significance of social media data to post-marketing surveillance for drug safety is undoubtedly significant. It will be an essential development direction for future research to overcome the challenges of using social media data by developing new technologies and establishing new mechanisms.

Differences in medication usage of dental patients by age, gender, race/ethnicity and insurance status

BACKGROUND

Limited studies have investigated the medication profile of young adult dental patients despite the high prevalence of prescription opioid abuse in this population.

OBJECTIVE

This study investigated the extent and differences in medication usage of dental patients older than 18 years by age, race/ethnicity, gender, insurance status and mechanism of action in an academic dental clinic setting.

METHODS

Using an automated approach, medication names in the Electronic Dental Record were retrieved and classified according to the National Drug Code directory. Descriptive statistics, multivariable ANOVA and Post hoc tests were performed to detect differences in the number of medications by patient demographics.

RESULTS

Of the 11,220 adult patients, 53 percent reported taking at least one medication with significant differences in medication usage by demographics. Hydroxymethylglutaryl-coenzyme A reductase inhibitors (21-36%), and angiotensin-converting enzyme inhibitors (19-23%) ranked the top two medication classes among patients 55 years and older. Opioid agonists (7-14%), and Selective Serotonin Reuptake Inhibitors (SSRIs) (5-12%) ranked the top two medication classes among patients aged 18-54 years.

CONCLUSIONS

The results underscore the importance of dental providers to review medical and medication histories of patients regardless of their age to avoid adverse events and to determine patient's risk for opioid abuse.

Validation of natural language processing to determine the presence and size of abdominal aortic aneurysms in a large integrated health system

OBJECTIVE

Previous studies of the natural history of abdominal aortic aneurysms (AAAs) have been limited by small cohort sizes or heterogeneous analyses of pooled data. By quickly and efficiently extracting imaging data from the health records, natural language processing (NLP) has the potential to substantially improve how we study and care for patients with AAAs. The aim of the present study was to test the ability of an NLP tool to accurately identify the presence or absence of AAAs and detect the maximal abdominal aortic diameter in a large dataset of imaging study reports.

METHODS

Relevant imaging study reports (n = 230,660) from 2003 to 2017 were obtained for 32,778 patients followed up in a prospective aneurysm surveillance registry within a large, diverse, integrated healthcare system. A commercially available NLP algorithm was used to assess the presence of AAAs, confirm the absence of AAAs, and extract the maximal diameter of the abdominal aorta, if stated. A blinded expert manual review of 18,000 randomly selected imaging reports was used as the reference standard. The positive predictive value (PPV or precision), sensitivity (recall), and the kappa statistics were calculated.

RESULTS

Of the randomly selected 18,000 studies that underwent expert manual review, 48.7% were positive for AAAs. In confirming the presence of an AAA, the interrater reliability of the NLP compared with the expert review showed a kappa value of 0.84 (95% confidence interval [CI], 0.83-0.85), with a PPV of 95% and sensitivity of 88.5%. The NLP algorithm showed similar results for confirming the absence of an AAA, with a kappa of 0.79 (95% CI, 0.799-0.80), PPV of 77.7%, and sensitivity of 91.9%. The kappa, PPV, and sensitivity of the NLP for correctly identifying the maximal aortic diameter was 0.88 (95% CI, 0.87-0.89), 88.8%, and 88.2% respectively.

CONCLUSIONS

The use of NLP software can accurately analyze large volumes of radiology report data to detect AAA disease and assemble a contemporary aortic diameter-based cohort of patients for longitudinal analysis to guide surveillance, medical management, and operative decision making. It can also potentially be used to identify from the electronic medical records pre- and postoperative AAA patients "lost to follow-up," leverage human resources engaged in the ongoing surveillance of patients with AAAs, and facilitate the construction and implementation of AAA screening programs.

Prediction of general medical admission length of stay with natural language processing and deep learning: a pilot study

Length of stay (LOS) and discharge destination predictions are key parts of the discharge planning process for general medical hospital inpatients. It is possible that machine learning, using natural language processing, may be able to assist with accurate LOS and discharge destination prediction for this patient group. Emergency department triage and doctor notes were retrospectively collected on consecutive general medical and acute medical unit admissions to a single tertiary hospital from a 2-month period in 2019. These data were used to assess the feasibility of predicting LOS and discharge destination using natural language processing and a variety of machine learning models. 313 patients were included in the study. The artificial neural network achieved the highest accuracy on the primary outcome of predicting whether a patient would remain in hospital for > 2 days (accuracy 0.82, area under the received operator curve 0.75, sensitivity 0.47 and specificity 0.97). When predicting LOS as an exact number of days, the artificial neural network achieved a mean absolute error of 2.9 and a mean squared error of 16.8 on the test set. For the prediction of home as a discharge destination (vs any non-home alternative), all models performed similarly with an accuracy of approximately 0.74. This study supports the feasibility of using natural language processing to predict general medical inpatient LOS and discharge destination. Further research is indicated with larger, more detailed, datasets from multiple centres to optimise and examine the accuracy that may be achieved with such predictions.

Evaluation of Use of Technologies to Facilitate Medical Chart Review

INTRODUCTION

While medical chart review remains the gold standard to validate health conditions or events identified in administrative claims and electronic health record databases, it is time consuming, expensive and can involve subjective decisions.

AIM

The aim of this study was to describe the landscape of technology-enhanced approaches that could be used to facilitate medical chart review within and across distributed data networks.

METHOD

We conducted a semi-structured survey regarding processes for medical chart review with organizations that either routinely do medical chart review or use technologies that could facilitate chart review.

RESULTS

Fifteen out of 17 interviewed organizations used optical character recognition (OCR) or natural language processing (NLP) in their chart review process. None used handwriting recognition software. While these organizations found OCR and NLP to be useful for expediting extraction of useful information from medical charts, they also mentioned several challenges. Quality of medical scans can be variable, interfering with the accuracy of OCR. Additionally, linguistic complexity in medical notes and heterogeneity in reporting templates used by different healthcare systems can reduce the transportability of NLP-based algorithms to diverse healthcare settings.

CONCLUSION

New technologies including OCR and NLP are currently in use by various organizations involved in medical chart review. While technology-enhanced approaches could scale up capacity to validate key variables and make information about important clinical variables from medical records more generally available for research purposes, they often require considerable customization when employed in a distributed data environment with multiple, diverse healthcare settings.

Utilizing Advanced Technologies to Augment Pharmacovigilance Systems: Challenges and Opportunities

There are significant challenges and opportunities in deploying and utilizing advanced information technology (IT) within pharmacovigilance (PV) systems and across the pharmaceutical industry. Various aspects of PV will benefit from automation (e.g., by improving standardization or increasing data quality). Several themes are developed, highlighting the challenges faced, exploring solutions, and assessing the potential for further research. Automation of the workflow for processing of individual case safety reports (ICSRs) is adopted as a use case. This involves a logical progression through a series of steps that when linked together comprise the complete work process required for the effective management of ICSRs. We recognize that the rapid development of new technologies will invariably outpace the regulations applicable to PV systems. Nevertheless, we believe that such systems may be improved by intelligent automation. It is incumbent on the owners of these systems to explore opportunities presented by new technologies with regulators in order to evaluate the applicability, design, deployment, performance, validation and maintenance of advanced technologies to ensure that the PV system continues to be fit for purpose. Proposed approaches to the validation of automated PV systems are presented. A series of definitions and a critical appraisal of important considerations are provided in the form of use cases. We summarize progress made and opportunities for the development of automation of future systems. The overall goal of automation is to provide high quality safety data in the correct format, in context, more quickly, and with less manual effort. This will improve the evidence available for scientific assessment and helps to inform and expedite decisions about the minimization of risks associated with medicines.

Natural Language Processing Combined with ICD-9-CM Codes as a Novel Method to Study the Epidemiology of Allergic Drug Reactions

BACKGROUND

Allergic drug reaction epidemiologic data are sparse because it remains difficult to identify true cases in large data sets using manual chart review.

OBJECTIVE

To develop and validate a novel informatics method based on natural language processing (NLP) in combination with International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes that identifies allergic drug reactions in the electronic health record.

METHODS

Previously studied and high-yield ICD-9-CM codes were used to screen for possible allergic drug reactions among all inpatients admitted in 2007 and 2008. A random sample was selected for manual chart review to identify true cases of allergic drug reactions. A rule-based NLP algorithm was then developed to identify allergic drug reactions using free-text clinical notes and discharge summaries from the filtered cases. The performance of using manual chart review of ICD-9-CM codes alone was compared with ICD-9-CM codes in combination with NLP.

RESULTS

Of 3907 cases identified by ICD-9-CM codes, 725 (19%) were randomly selected for manual chart review; 335 were confirmed as allergic drug reactions, resulting in a positive predictive value (PPV) of 46% (range: 18%-79%) when using ICD-9-CM codes alone. Our NLP algorithm in combination with ICD-9-CM codes achieved a PPV of 86% (range: 69%-100%). Among the 335 confirmed positive cases, NLP identified 259 true cases, resulting in a recall/sensitivity of 77% (range: 26%-100%). Among the 390 negative cases, NLP achieved a specificity of 89% (range: 69%-100%).

CONCLUSION

Using NLP with ICD-9-CM codes improved identification of allergic drug reactions. The resulting decrease in manual chart review effort will facilitate large epidemiology studies of this understudied area.

Digital Health and the State of Interoperable Electronic Health Records

Digital health systems and innovative care delivery within these systems have great potential to improve national health care and positively impact the health outcomes of patients. However, currently, very few countries have systems that can implement digital interventions at scale. This is partly because of the lack of interoperable electronic health records (EHRs). It is difficult to make decisions for an individual or population when the data on that person or population are dispersed over multiple incompatible systems. This viewpoint paper has highlighted some key obstacles of current EHRs and some promising successes, with the goal of promoting EHR evolution and advocating for frameworks that develop digital health systems that serve populations—a critical goal as we move further into this data-rich century with an ever-increasing number of patients who live longer and depend on health care services where resources may already be strained. This paper aimed to analyze the evolution, obstacles, and current landscape of EHRs and identify fundamental areas of hindrance for interoperability. It also aimed to highlight countries where advances have been made and extract best practices from these examples. The obstacles to EHR interoperability are not easily solved, but improving the current situation in countries where a national policy is not in place will require a focused inquiry into solutions from various sources in the public and private sector. Effort must be made on a national scale to seek solutions for optimally interoperable EHRs beyond status quo solutions. A list of considerations for best practices is suggested.

Mining social media data to assess the risk of skin and soft tissue infections from allergen immunotherapy

BACKGROUND

Allergen immunotherapy (AIT) treatment for allergic rhinitis and asthma is used by 2.6 million Americans annually. Clinical and sterility testing studies identify no risk of contamination or infection from extracts prepared using recommended aseptic techniques, but regulatory concerns persist. Social media can be used to investigate rare adverse effects not captured by traditional studies.

OBJECTIVE

We sought to investigate large social media databases for suggestion of AIT skin and soft tissue infection (SSTI) risk and compare this risk to a comparator procedure with a sterile pharmaceutical.

METHODS

We analyzed US-restricted data from more than 10 common text-based social media platforms including Facebook, Twitter, and Reddit between 2012 and 2016. We used natural language processing (NLP) to identify posts related to AIT and, separately, influenza vaccination (comparator procedure). NLP was followed by manual review to identify posts suggesting a possible SSTI associated with either AIT or influenza vaccination. SSTI frequencies with 95% CIs were compared.

RESULTS

We identified 25,126 AIT posts, which were matched by social media platform to 25,126 influenza vaccination-related posts. NLP identified 4088 (16.3%) AIT posts that required manual review, with 6 posts (0.02%; 95% CI, 0.005%-0.043%) indicative of possible AIT-related SSTI. NLP identified 2689 (10.7%) influenza posts that required manual review, with 7 posts (0.03%; 95% CI, 0.007%-0.048%) indicative of possible influenza vaccination-related SSTI.

CONCLUSIONS

Social media data suggest that SSTI from AIT and influenza vaccination are equally rare events. Given that AIT's SSTI risk appears comparable to the risk using a sterile pharmaceutical based on social media data, current aseptic technique procedures seem safe.

The Adverse Drug Reactions From Patient Reports in Social Media Project: Protocol for an Evaluation Against a Gold Standard

Background

Social media is a potential source of information on postmarketing drug safety surveillance that still remains unexploited nowadays. Information technology solutions aiming at extracting adverse reactions (ADRs) from posts on health forums require a rigorous evaluation methodology if their results are to be used to make decisions. First, a gold standard, consisting of manual annotations of the ADR by human experts from the corpus extracted from social media, must be implemented and its quality must be assessed. Second, as for clinical research protocols, the sample size must rely on statistical arguments. Finally, the extraction methods must target the relation between the drug and the disease (which might be either treated or caused by the drug) rather than simple co-occurrences in the posts.

Objective

We propose a standardized protocol for the evaluation of a software extracting ADRs from the messages on health forums. The study is conducted as part of the Adverse Drug Reactions from Patient Reports in Social Media project.

Methods

Messages from French health forums were extracted. Entity recognition was based on Racine Pharma lexicon for drugs and Medical Dictionary for Regulatory Activities terminology for potential adverse events (AEs). Natural language processing–based techniques automated the ADR information extraction (relation between the drug and AE entities). The corpus of evaluation was a random sample of the messages containing drugs and/or AE concepts corresponding to recent pharmacovigilance alerts. A total of 2 persons experienced in medical terminology manually annotated the corpus, thus creating the gold standard, according to an annotator guideline. We will evaluate our tool against the gold standard with recall, precision, and f-measure. Interannotator agreement, reflecting gold standard quality, will be evaluated with hierarchical kappa. Granularities in the terminologies will be further explored.

Results

Necessary and sufficient sample size was calculated to ensure statistical confidence in the assessed results. As we expected a global recall of 0.5, we needed at least 384 identified ADR concepts to obtain a 95% CI with a total width of 0.10 around 0.5. The automated ADR information extraction in the corpus for evaluation is already finished. The 2 annotators already completed the annotation process. The analysis of the performance of the ADR information extraction module as compared with gold standard is ongoing.

Conclusions

This protocol is based on the standardized statistical methods from clinical research to create the corpus, thus ensuring the necessary statistical power of the assessed results. Such evaluation methodology is required to make the ADR information extraction software useful for postmarketing drug safety surveillance.

International Registered Report Identifier (IRRID)

RR1-10.2196/11448

Using Electronic Health Records To Generate Phenotypes For Research

Electronic health records contain patient-level data collected during and for clinical care. Data within the electronic health record include diagnostic billing codes, procedure codes, vital signs, laboratory test results, clinical imaging, and physician notes. With repeated clinic visits, these data are longitudinal, providing important information on disease development, progression, and response to treatment or intervention strategies. The near universal adoption of electronic health records nationally has the potential to provide population-scale real-world clinical data accessible for biomedical research, including genetic association studies. For this research potential to be realized, high-quality research-grade variables must be extracted from these clinical data warehouses. We describe here common and emerging electronic phenotyping approaches applied to electronic health records, as well as current limitations of both the approaches and the biases associated with these clinically collected data that impact their use in research. © 2018 by John Wiley & Sons, Inc.

Extraction and Standardization of Patient Complaints from Electronic Medication Histories for Pharmacovigilance: Natural Language Processing Analysis in Japanese

BACKGROUND

Despite the growing number of studies using natural language processing for pharmacovigilance, there are few reports on manipulating free text patient information in Japanese.

OBJECTIVE

This study aimed to establish a method of extracting and standardizing patient complaints from electronic medication histories accumulated in a Japanese community pharmacy for the detection of possible adverse drug event (ADE) signals.

METHODS

Subjective information included in electronic medication history data provided by a Japanese pharmacy operating in Hiroshima, Japan from September 1, 2015 to August 31, 2016, was used as patients' complaints. We formulated search rules based on morphological analysis and daily (nonmedical) speech and developed a system that automatically executes the search rules and annotates free text data with International Classification of Diseases, Tenth Revision (ICD-10) codes. The performance of the system was evaluated through comparisons with data manually annotated by health care workers for a data set of 5000 complaints.

RESULTS

Of 5000 complaints, the system annotated 2236 complaints with ICD-10 codes, whereas health care workers annotated 2348 statements. There was a match in the annotation of 1480 complaints between the system and manual work. System performance was .66 regarding precision, .63 in recall, and .65 for the F-measure.

CONCLUSIONS

Our results suggest that the system may be helpful in extracting and standardizing patients' speech related to symptoms from massive amounts of free text data, replacing manual work. After improving the extraction accuracy, we expect to utilize this system to detect signals of possible ADEs from patients' complaints in the future.