DDC-Outlier: Preventing Medication Errors Using Unsupervised Learning

Deep learning classification of drug-related problems from pharmaceutical interventions issued by hospital clinical pharmacists during medication prescription review: a large-scale descriptive retrospective study in a French university hospital

OBJECTIVES

Pharmaceutical interventions are proposals made by hospital clinical pharmacists to address sub-optimal uses of medications during prescription review. Pharmaceutical interventions include the identification of drug-related problems, their prevention and resolution. The objective of this study was to exploit a newly developed deep neural network classifier to identify drug-related problems from pharmaceutical interventions and perform a large retrospective descriptive analysis of them in a French university hospital over a 3-year period.

METHODS

Data were collected from prescription support software from 2018 to 2020. A classifier running in Python 3.8 and using Keras library was then used to automatically categorise drug-related problems from pharmaceutical interventions according to the coding of the French Society of Clinical Pharmacy.

RESULTS

2 930 656 prescription lines were analysed for a total of 119 689 patients. Among these prescription lines, 153 335 (5.2%) resulted in pharmaceutical interventions (n=48 202 patients; 40.2%). Pharmaceutical interventions were predominantly observed in patients aged 65 years or older (n=26 141 patients out of 53 186; 49.1%) and in patients taking five or more medications (44 702 patients out of 93 419; 47.8%). The most frequently identified types of drug-related problems associated with pharmaceutical interventions were 'Non-conformity to guidelines or contra-indication' (n=88 523; 57.7%), 'Overdosage' (16 975; 11.1%) and 'Improper administration' (13 898; 9.1%). The most frequently encountered drugs were: paracetamol (n=10 585; 6.9%), esomeprazole (6031; 3.9%), hydrochlorothiazide (2951; 1.9%), enoxaparin (2191; 1.4%), tramadol (1879; 1.2%), calcium (2073; 1.3%), perindopril (1950; 1.2%), amlodipine (1716; 1.1%), simvastatin (1560; 1.0%) and insulin (1019; 0.7%).

CONCLUSIONS

The deep neural network classifier used met the challenge of automatically classifying drug-related problems from pharmaceutical interventions from a large database without mobilising significant human resources. The use of such a classifier can lead to alerting caregivers about certain risky practices in prescription and administration, and triggering actions to improve patients' therapeutic outcomes.

Using machine learning or deep learning models in a hospital setting to detect inappropriate prescriptions: a systematic review

OBJECTIVES

The emergence of artificial intelligence (AI) is catching the interest of hospital pharmacists. A massive collection of health data is now available to train AI models and hold the promise of disrupting codes and practices. The objective of this systematic review was to examine the state of the art of machine learning or deep learning models that detect inappropriate hospital medication orders.

METHODS

A systematic review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement. MEDLINE and Embase databases were searched from inception to May 2023. Studies were included if they reported and described an AI model intended for use by clinical pharmacists in hospitals. Risk of bias was assessed using the Prediction model Risk Of Bias ASsessment Tool (PROBAST).

RESULTS

13 articles were selected after review: 12 studies were judged to have high risk of bias; 11 studies were published between 2020 and 2023; 8 were conducted in North America and Asia; 6 analysed orders and detected inappropriate prescriptions according to patient profiles and medication orders; and 7 detected specific inappropriate prescriptions, such as detecting antibiotic resistance, dosage abnormality in prescriptions, high alert drugs errors from prescriptions or predicting the risk of adverse drug events. Various AI models were used, mainly supervised learning techniques. The training datasets used were very heterogeneous; the length of study varied from 2 weeks to 7 years and the number of prescription orders analysed went from 31 to 5 804 192.

CONCLUSIONS

This systematic review points out that, to date, few original research studies report AI tools based on machine or deep learning in the field of hospital clinical pharmacy. However, these original articles, while preliminary, highlighted the potential value of integrating AI into clinical hospital pharmacy practice.

Natural language processing assisted detection of inappropriate proton pump inhibitor use in adult hospitalised patients

OBJECTIVES

To establish a clinical application monitoring system for proton pump inhibitors (PPI-MS) and to enhance the detection and intervention of inappropriate PPI use in adult hospitalised patients.

METHODS

Natural language processing technology was applied to indication recognition of therapeutic PPI applications and the assessment of admission record recognition for preventive PPI applications. Symptom judgement was based on the tense-negation model and regular expressions. Evidence-based rules for clinical PPI application were embedded for the construction of PPI-MS. A total of 9421 patient records using PPI from July 2022 to July 2023 were analysed to validate the performance of the system and to identify common issues related to inappropriate clinical PPI use.

RESULTS

Out of 9421 hospitalised patients detected using PPI, 4736 (50.27%) were used for prophylaxis and the rest for therapeutic use. Among the prophylactic medications, 2274 patients (48.02%) were identified as receiving inappropriate prophylactic PPI. The main reasons were inappropriate prophylaxis without indication. Additionally, 258 cases of inappropriate therapeutic PPI use were identified, mainly involving the use of esomeprazole for peptic ulcers and Zollinger-Ellison syndrome. The efficiency of the PPI rational medication monitoring system, when coupled with human involvement, was 32 times that of manual monitoring. Among cases of inappropriate prophylactic PPI use, 45.29% were due to lack of indications, 28.34% involved inappropriate administration routes, 15.74% were related to inappropriate dosing frequencies and 10.62% were attributed to inappropriate drug selection. There were 933 cases related to the use of antiplatelet and anticoagulant drugs and 708 cases related to the use of non-steroidal anti-inflammatory drugs. The overall accuracy of the PPI-MS system was 88.69%, with a recall rate of 99.33%, and the F1 score was 93.71%.

CONCLUSIONS

Establishing a PPI medication monitoring system through natural language processing technology, while ensuring accuracy and recall rates, improves evaluation efficiency and homogeneity. This provides a new solution for timely detection of issues relating to clinical PPI usage.

Artificial intelligence in the field of pharmacy practice: A literature review

Artificial intelligence (AI) is a transformative technology used in various industrial sectors including healthcare. In pharmacy practice, AI has the potential to significantly improve medication management and patient care. This review explores various AI applications in the field of pharmacy practice. The incorporation of AI technologies provides pharmacists with tools and systems that help them make accurate and evidence-based clinical decisions. By using AI algorithms and Machine Learning, pharmacists can analyze a large volume of patient data, including medical records, laboratory results, and medication profiles, aiding them in identifying potential drug-drug interactions, assessing the safety and efficacy of medicines, and making informed recommendations tailored to individual patient requirements. Various AI models have been developed to predict and detect adverse drug events, assist clinical decision support systems with medication-related decisions, automate dispensing processes in community pharmacies, optimize medication dosages, detect drug-drug interactions, improve adherence through smart technologies, detect and prevent medication errors, provide medication therapy management services, and support telemedicine initiatives. By incorporating AI into clinical practice, health care professionals can augment their decision-making processes and provide patients with personalized care. AI allows for greater collaboration between different healthcare services provided to a single patient. For patients, AI may be a useful tool for providing guidance on how and when to take a medication, aiding in patient education, and promoting medication adherence and AI may be used to know how and where to obtain the most cost-effective healthcare and how best to communicate with healthcare professionals, optimize the health monitoring using wearables devices, provide everyday lifestyle and health guidance, and integrate diet and exercise.

Using automated methods to detect safety problems with health information technology: a scoping review

OBJECTIVE

To summarize the research literature evaluating automated methods for early detection of safety problems with health information technology (HIT).

MATERIALS AND METHODS

We searched bibliographic databases including MEDLINE, ACM Digital, Embase, CINAHL Complete, PsycINFO, and Web of Science from January 2010 to June 2021 for studies evaluating the performance of automated methods to detect HIT problems. HIT problems were reviewed using an existing classification for safety concerns. Automated methods were categorized into rule-based, statistical, and machine learning methods, and their performance in detecting HIT problems was assessed. The review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta Analyses extension for Scoping Reviews statement.

RESULTS

Of the 45 studies identified, the majority (n = 27, 60%) focused on detecting use errors involving electronic health records and order entry systems. Machine learning (n = 22) and statistical modeling (n = 17) were the most common methods. Unsupervised learning was used to detect use errors in laboratory test results, prescriptions, and patient records while supervised learning was used to detect technical errors arising from hardware or software issues. Statistical modeling was used to detect use errors, unauthorized access, and clinical decision support system malfunctions while rule-based methods primarily focused on use errors.

CONCLUSIONS

A wide variety of rule-based, statistical, and machine learning methods have been applied to automate the detection of safety problems with HIT. Many opportunities remain to systematically study their application and effectiveness in real-world settings.

Detecting Unusual Intravenous Infusion Alerting Patterns with Machine Learning Algorithms

OBJECTIVE

To detect unusual infusion alerting patterns using machine learning (ML) algorithms as a first step to advance safer inpatient intravenous administration of high-alert medications.

MATERIALS AND METHODS

We used one year of detailed propofol infusion data from a hospital. Interpretable and clinically relevant variables were feature engineered, and data points were aggregated per calendar day. A univariate (maximum times-limit) moving range (mr) control chart was used to simulate clinicians' common approach to identifying unusual infusion alerting patterns. Three different unsupervised multivariate ML-based anomaly detection algorithms (Local Outlier Factor, Isolation Forest, and k-Nearest Neighbors) were used for the same purpose. Results from the control chart and ML algorithms were compared.

RESULTS

The propofol data had 3,300 infusion alerts, 92% of which were generated during the day shift and seven of which had a times-limit greater than 10. The mr-chart identified 15 alert pattern anomalies. Different thresholds were set to include the top 15 anomalies from each ML algorithm. A total of 31 unique ML anomalies were grouped and ranked by agreeability. All algorithms agreed on 10% of the anomalies, and at least two algorithms agreed on 36%. Each algorithm detected one specific anomaly that the mr-chart did not detect. The anomaly represented a day with 71 propofol alerts (half of which were overridden) generated at an average rate of 1.06 per infusion, whereas the moving alert rate for the week was 0.35 per infusion.

DISCUSSION

These findings show that ML-based algorithms are more robust than control charts in detecting unusual alerting patterns. However, we recommend using a combination of algorithms, as multiple algorithms serve a benchmarking function and allow researchers to focus on data points with the highest algorithm agreeability.

CONCLUSION

Unsupervised ML algorithms can assist clinicians in identifying unusual alert patterns as a first step toward achieving safer infusion practices.

Detecting Unusual Intravenous Infusion Alerting Patterns with Machine Learning Algorithms

OBJECTIVE

To detect unusual infusion alerting patterns using machine learning (ML) algorithms as a first step to advance safer inpatient intravenous administration of high-alert medications.

MATERIALS AND METHODS

We used one year of detailed propofol infusion data from a hospital. Interpretable and clinically relevant variables were feature engineered, and data points were aggregated per calendar day. A univariate (maximum times-limit) moving range (mr) control chart was used to simulate clinicians' common approach to identifying unusual infusion alerting patterns. Three different unsupervised multivariate ML-based anomaly detection algorithms (Local Outlier Factor, Isolation Forest, and k-Nearest Neighbors) were used for the same purpose. Results from the control chart and ML algorithms were compared.

RESULTS

The propofol data had 3,300 infusion alerts, 92% of which were generated during the day shift and seven of which had a times-limit greater than 10. The mr-chart identified 15 alert pattern anomalies. Different thresholds were set to include the top 15 anomalies from each ML algorithm. A total of 31 unique ML anomalies were grouped and ranked by agreeability. All algorithms agreed on 10% of the anomalies, and at least two algorithms agreed on 36%. Each algorithm detected one specific anomaly that the mr-chart did not detect. The anomaly represented a day with 71 propofol alerts (half of which were overridden) generated at an average rate of 1.06 per infusion, whereas the moving alert rate for the week was 0.35 per infusion.

DISCUSSION

These findings show that ML-based algorithms are more robust than control charts in detecting unusual alerting patterns. However, we recommend using a combination of algorithms, as multiple algorithms serve a benchmarking function and allow researchers to focus on data points with the highest algorithm agreeability.

CONCLUSION

Unsupervised ML algorithms can assist clinicians in identifying unusual alert patterns as a first step toward achieving safer infusion practices.

Detection of overdose and underdose prescriptions-An unsupervised machine learning approach

Overdose prescription errors sometimes cause serious life-threatening adverse drug events, while underdose errors lead to diminished therapeutic effects. Therefore, it is important to detect and prevent these errors. In the present study, we used the one-class support vector machine (OCSVM), one of the most common unsupervised machine learning algorithms for anomaly detection, to identify overdose and underdose prescriptions. We extracted prescription data from electronic health records in Kyushu University Hospital between January 1, 2014 and December 31, 2019. We constructed an OCSVM model for each of the 21 candidate drugs using three features: age, weight, and dose. Clinical overdose and underdose prescriptions, which were identified and rectified by pharmacists before administration, were collected. Synthetic overdose and underdose prescriptions were created using the maximum and minimum doses, defined by drug labels or the UpToDate database. We applied these prescription data to the OCSVM model and evaluated its detection performance. We also performed comparative analysis with other unsupervised outlier detection algorithms (local outlier factor, isolation forest, and robust covariance). Twenty-seven out of 31 clinical overdose and underdose prescriptions (87.1%) were detected as abnormal by the model. The constructed OCSVM models showed high performance for detecting synthetic overdose prescriptions (precision 0.986, recall 0.964, and F-measure 0.973) and synthetic underdose prescriptions (precision 0.980, recall 0.794, and F-measure 0.839). In comparative analysis, OCSVM showed the best performance. Our models detected the majority of clinical overdose and underdose prescriptions and demonstrated high performance in synthetic data analysis. OCSVM models, constructed using features such as age, weight, and dose, are useful for detecting overdose and underdose prescriptions.

The use of narrative electronic prescribing instructions in pharmacoepidemiology: A scoping review for the International Society for Pharmacoepidemiology

Narrative electronic prescribing instructions (NEPIs) are text that convey information on the administration or co‐administration of a drug as directed by a prescriber. For researchers, NEPIs have the potential to advance our understanding of the risks and benefits of medications in populations; however, due to their unstructured nature, they are not often utilized. The goal of this scoping review was to evaluate how NEPIs are currently employed in research, identify opportunities and challenges for their broader application, and provide recommendations on their future use. The scoping review comprised a comprehensive literature review and a survey of key stakeholders. From the literature review, we identified 33 primary articles that described the use of NEPIs. The majority of articles (n = 19) identified issues with the quality of information in NEPIs compared with structured prescribing information; nine articles described the development of novel algorithms that performed well in extracting information from NEPIs, and five described the used of manual or simpler algorithms to extract prescribing information from NEPIs. A survey of 19 stakeholders indicated concerns for the quality of information in NEPIs and called for standardization of NEPIs to reduce data variability/errors. Nevertheless, stakeholders believed NEPIs present an opportunity to identify prescriber's intent for the prescription and to study temporal treatment patterns. In summary, NEPIs hold much promise for advancing the field of pharmacoepidemiology. Researchers should take advantage of addressing important questions that can be uniquely answered with NEPIs, but exercise caution when using this information and carefully consider the quality of the data.

Pharmacists' perceptions of a machine learning model for the identification of atypical medication orders

OBJECTIVES

The study sought to assess the clinical performance of a machine learning model aiming to identify unusual medication orders.

MATERIALS AND METHODS

This prospective study was conducted at CHU Sainte-Justine, Canada, from April to August 2020. An unsupervised machine learning model based on GANomaly and 2 baselines were trained to learn medication order patterns from 10 years of data. Clinical pharmacists dichotomously (typical or atypical) labeled orders and pharmacological profiles (patients' medication lists). Confusion matrices, areas under the precision-recall curve (AUPRs), and F1 scores were calculated.

RESULTS

A total of 12 471 medication orders and 1356 profiles were labeled by 25 pharmacists. Medication order predictions showed a precision of 35%, recall (sensitivity) of 26%, and specificity of 97% as compared with pharmacist labels, with an AUPR of 0.25 and an F1 score of 0.30. Profile predictions showed a precision of 49%, recall of 75%, and specificity of 82%, with an AUPR of 0.60, and an F1 score of 0.59. The model performed better than the baselines. According to the pharmacists, the model was a useful screening tool, and 9 of 15 participants preferred predictions by medication, rather than by profile.

DISCUSSION

Predictions for profiles had higher F1 scores and recall compared with medication order predictions. Although the performance was much better for profile predictions, pharmacists generally preferred medication order predictions.

CONCLUSIONS

Based on the AUPR, this model showed better performance for the identification of atypical pharmacological profiles than for medication orders. Pharmacists considered the model a useful screening tool. Improving these predictions should be prioritized in future research to maximize clinical impact.

Demystifying machine learning: a primer for physicians

Machine learning is a tool for analysing digitised data sets and formulating predictions that can optimise clinical decision-making. It aims to identify complex patterns in large data sets and encode them into models that can then classify new unseen cases or make predictions on new data. Machine learning methods take several forms and individual models can be of many different types. More than 50 models have been approved for use in routine healthcare, and the numbers continue to grow exponentially. The reliability and robustness of any model depends on multiple factors, including the quality and quantity of the data used to develop the models, and the selection of features in the data considered most important to maximising accuracy. In ensuring models are safe, effective and reproducible in routine care, physicians need to have some understanding of how these models are developed and evaluated, and to collaborate with data and computer scientists in their design and validation. This narrative review introduces principles, methods and examples of machine learning in a way that does not require mastery of highly complex statistical and computational concepts.

Data-Driven Cervical Cancer Prediction Model with Outlier Detection and Over-Sampling Methods

Globally, cervical cancer remains as the foremost prevailing cancer in females. Hence, it is necessary to distinguish the importance of risk factors of cervical cancer to classify potential patients. The present work proposes a cervical cancer prediction model (CCPM) that offers early prediction of cervical cancer using risk factors as inputs. The CCPM first removes outliers by using outlier detection methods such as density-based spatial clustering of applications with noise (DBSCAN) and isolation forest (iForest) and by increasing the number of cases in the dataset in a balanced way, for example, through synthetic minority over-sampling technique (SMOTE) and SMOTE with Tomek link (SMOTETomek). Finally, it employs random forest (RF) as a classifier. Thus, CCPM lies on four scenarios: (1) DBSCAN + SMOTETomek + RF, (2) DBSCAN + SMOTE+ RF, (3) iForest + SMOTETomek + RF, and (4) iForest + SMOTE + RF. A dataset of 858 potential patients was used to validate the performance of the proposed method. We found that combinations of iForest with SMOTE and iForest with SMOTETomek provided better performances than those of DBSCAN with SMOTE and DBSCAN with SMOTETomek. We also observed that RF performed the best among several popular machine learning classifiers. Furthermore, the proposed CCPM showed better accuracy than previously proposed methods for forecasting cervical cancer. In addition, a mobile application that can collect cervical cancer risk factors data and provides results from CCPM is developed for instant and proper action at the initial stage of cervical cancer.