Clinical prediction rule to identify high-risk inpatients for adverse drug events: the JADE Study

Machine learning as a tool to identify inpatients who are not at risk of adverse drug events in a large dataset of a tertiary care hospital in the USA

AIMS

Adverse drug events (ADEs) are a major threat to inpatients in the United States of America (USA). It is unknown how well machine learning (ML) is able to predict whether or not a patient will suffer from an ADE during hospital stay based on data available at hospital admission for emergency department patients of all ages (binary classification task). It is further unknown whether ML is able to outperform logistic regression (LR) in doing so, and which variables are the most important predictors.

METHODS

In this study, 5 ML models- namely a random forest, gradient boosting machine (GBM), ridge regression, least absolute shrinkage and selection operator (LASSO) regression, and elastic net regression-as well as a LR were trained and tested for the prediction of inpatient ADEs identified using ICD-10-CM codes based on comprehensive previous work in a diverse population. In total, 210 181 observations from patients who were admitted to a large tertiary care hospital after emergency department stay between 2011 and 2019 were included. The area under the receiver operating characteristics curve (AUC) and AUC-precision-recall (AUC-PR) were used as primary performance indicators.

RESULTS

Tree-based models performed best with respect to AUC and AUC-PR. The gradient boosting machine (GBM) reached an AUC of 0.747 (95% confidence interval (CI): 0.735 to 0.759) and an AUC-PR of 0.134 (95% CI: 0.131 to 0.137) on unforeseen test data, while the random forest reached an AUC of 0.743 (95% CI: 0.731 to 0.755) and an AUC-PR of 0.139 (95% CI: 0.135 to 0.142), respectively. ML statistically significantly outperformed LR both on AUC and AUC-PR. Nonetheless, overall, models did not differ much with respect to their performance. Most important predictors were admission type, temperature and chief complaint for the best performing model (GBM).

CONCLUSIONS

The study demonstrated a first application of ML to predict inpatient ADEs based on ICD-10-CM codes, and a comparison with LR. Future research should address concerns arising from low precision and related problems.

Comparison of prioritisation algorithms for the selection of patients for medication reviews in the emergency department: a cross-sectional study

BACKGROUND

Risk prioritisation algorithms provide patients with a risk category that guides pharmacists to choose those needing medication reviews (MRs) the most. For this study the Medicine Risk Score (MERIS) and a modified Assessment of Risk Tool (ART) were used.

AIM

To examine how the selection of patients by the clinical pharmacists in an emergency department for MRs compared with the categorisation provided by MERIS and a modified version of ART (mART). Furthermore, examine the agreement between MERIS and mART.

METHOD

A cross-sectional study was conducted using data on all admitted patients during a two-month period. Data were entered into the prioritisation algorithms and independently ranked by the six pharmacists who were observed as they selected patients for MR. Risk scores and categorisations were compared between the algorithms and the pharmacists' ranking using t-test, Z-test, Chi square, Kruskal Wallis H-test, or Kappa statistics.

RESULTS

The study included 1133 patients. Significant differences were found between the pharmacists and the algorithms. The sensitivity and specificity of MERIS were 37.8% and 73.6%, for mART, 33.0% and 75.9%. Kappa was 0.58, showing moderate agreement. No significant differences were observed between the individual pharmacists' selection, but differences were significant between how pharmacists ranked the importance of the provided MRs.

CONCLUSION

Pharmacists disagreed with the risk categorisation provided by MERIS and mART. However, MERIS and mART had similar sensitivity, specificity, and moderate agreement. Further research should focus on how clinical algorithms affect the selection of patients and on the importance of the MRs carried out by pharmacists.

Systematic Review of Risk Factors Assessed in Predictive Scoring Tools for Drug-Related Problems in Inpatients

Drug-related problems (DRP, defined as adverse drug events/reactions and medication errors) are a common threat for patient safety. With the aim to aid improved allocation of specialist resources and to improve detection and prevention of DRP, numerous predictive scoring tools have been proposed. The external validation and evidence for the transferability of these tools still faces limitations. However, the proposed scoring tools include partly overlapping sets of similar factors, which may allow a new approach to estimate the external usability and validity of individual risk factors. Therefore, we conducted this systematic review and analysis. We identified 14 key studies that assessed 844 candidate risk factors for inclusion into predictive scoring tools. After consolidation to account for overlapping terminology and variable definitions, we assessed each risk factor in the number of studies it was assessed, and, if it was found to be a significant predictor of DRP, whether it was included in a final scoring tool. The latter included intake of ≥ 8 drugs, drugs of the Anatomical Therapeutic Chemical (ATC) class N, ≥1 comorbidity, an estimated glomerular filtration rate (eGFR) <30 mL/min and age ≥60 years. The methodological approach and the individual risk factors presented in this review may provide a new starting point for improved risk assessment.

Is a High Medication Risk Score Associated With Increased Risk of 30-Day Readmission? A Population-Based Cohort Study From CROSS-TRACKS

OBJECTIVES

The primary aim of this study was to evaluate whether a high Medication Risk Score (MERIS) upon admission to an emergency department is associated with increased risk of 30-day readmission in patients discharged directly home. Mortality, visit to general practitioner, and drug changes within 30 days were included as secondary outcomes.

METHODS

This is a historical cohort study with data from the Danish population-based open-cohort CROSS-TRACKS. Cox regression analyses were used to determine whether a high MERIS score was associated with increased risk of 30-day readmission and mortality. Visit to general practitioner and drug changes were tested with χ2 test and Wilcoxon rank sum test.

RESULTS

A total of 2106 patients were eligible: 2017 had a MERIS score lower than 14 (low-risk group), and 89 had a score of 14 or higher (high-risk group). The proportion of patients in the high-risk group who were readmitted was 21.3% compared with 16.3% in the low-risk group, resulting in a hazard ratio for readmission of 1.43 (95% confidence interval, 0.9-1.3). The hazard ratio for mortality was 8.3 (95% confidence interval, 3.0-22.8). No statistical significant difference was found in general practitioner visits; however, significantly more drug changes were observed in the high-risk group.

CONCLUSIONS

A high MERIS score was associated with increased risk of readmissions and can potentially assist healthcare professionals in the prioritizing of patients who may benefit from further exam, for example, additional medication review in acute care setting. Further investigation of MERIS and exploration of causal inferences between medication-related harm and medication-related readmissions are warranted.

Epidemiology of adverse drug events and medication errors in four nursing homes in Japan: the Japan Adverse Drug Events (JADE) Study

BACKGROUND

Worldwide, the emergence of super-ageing societies has increased the number of older people requiring support for daily activities. Many elderly residents of nursing homes (NHs) take drugs to treat chronic conditions; however, there are few reports of medication safety in NHs, especially from non-western countries.

OBJECTIVE

We examined the incidence and nature of adverse drug events (ADEs) and medication errors (MEs) in NHs for the elderly in Japan.

DESIGN, SETTING, AND PARTICIPANTS

The Japan Adverse Drug Events Study for NHs is a prospective cohort study that was conducted among all residents, except for short-term admissions, at four NHs for older people in Japan for 1 year.

MEASUREMENTS

Trained physicians and psychologists, five and six in number, respectively, reviewed all charts of the residents to identify suspected ADEs and MEs, which were then classified by the physicians into ADEs, potential ADEs and other MEs after the exclusion of ineligible events, for the assessment of their severity and preventability. The kappa score for presence of an ADE and preventability were 0.89 and 0.79, respectively.

RESULTS

We enrolled 459 residents, and this yielded 3315 resident-months of observation time. We identified 1207 ADEs and 600 MEs (incidence: 36.4 and 18.1 per 100 resident-months, respectively) during the study period. About one-third of ADEs were preventable, and MEs were most frequently observed in the monitoring stage (72%, 433/600), with 71% of the MEs occurring due to inadequate observation following the physician's prescription.

CONCLUSION

In Japan, ADEs and MEs are common among elderly residents of NHs. The assessment and appropriate adjustment of medication preadmission and postadmission to NHs are needed to improve medication safety, especially when a single physician is responsible for prescribing most medications for the residents, as is usually the case in Japan.

Clinical usefulness of prediction tools to identify adult hospitalized patients at risk of drug-related problems: A systematic review of clinical prediction models and risk assessment tools

AIMS

This study aimed to review systematically all available prediction tools identifying adult hospitalized patients at risk of drug-related problems, and to synthesize the evidence on performance and clinical usefulness.

METHODS

PubMed, Scopus, Web of Science, Embase, and CINAHL databases were searched for relevant studies. Titles, abstracts and full-text studies were sequentially screened for inclusion by two independent reviewers. The Prediction Model Risk of Bias Assessment Tool (PROBAST) and the Revised Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) checklists were used to assess risk of bias and applicability of prediction tools. A narrative synthesis was performed.

RESULTS

A total of 21 studies were included, 14 of which described the development of new prediction tools (four risk assessment tools and ten clinical prediction models) and six studies were validation based and one an impact study. There were variations in tool development processes, outcome measures and included predictors. Overall, tool performance had limitations in reporting and consistency, with the discriminatory ability based on area under the curve receiver operating characteristics (AUROC) ranging from poor to good (0.62-0.81), sensitivity and specificity ranging from 57.0% to 89.9% and 30.2% to 88.0%, respectively. The Medicines Optimisation Assessment tool and Assessment of Risk tool were prediction tools with the lowest risk of bias and low concern for applicability. Studies reporting external validation and impact on patient outcomes were scarce.

CONCLUSION

Most prediction tools have limitations in development and validation processes, as well as scarce evidence of clinical usefulness. Future studies should attempt to either refine currently available tools or apply a rigorous process capturing evidence of acceptance, usefulness, performance and outcomes.

Development and validation of the Adverse Inpatient Medication Event model (AIME)

AIMS

Medication harm has negative clinical and economic consequences, contributing to hospitalisation, morbidity and mortality. The incidence ranges from 4 to 14%, of which up to 50% of events may be preventable. A predictive model for identifying high-risk inpatients can guide a timely and systematic approach to prioritisation. The aim of this study is to develop and internally validate a risk prediction model for prioritisation of hospitalised patients at risk of medication harm.

METHODS

A retrospective cohort study was conducted in general medical and geriatric specialties at an Australian hospital over six months. Medication harm was identified using International Classification of Disease (ICD-10) codes and the hospital's incident database. Sixty-eight variables, including medications and laboratory results, were extracted from the hospital's databases. Multivariable logistic regression was used to develop the final risk model. Performance was evaluated using area under the receiver operative characteristic curve (AuROC) and clinical utility was determined using decision curve analysis.

RESULTS

The study cohort included 1982 patients with median age 74 years, of which 136 (7%) experienced at least one adverse medication event(s). The model included: length of stay, hospital re-admission within 12 months, venous or arterial thrombosis and/or embolism, ≥ 8 medications, serum sodium < 126 mmol/L, INR > 3, anti-psychotic, antiarrhythmic and immunosuppressant medications, and history of medication allergy. Validation gave an AuROC of 0.70 (95% CI: 0.65-0.74). Decision curve analysis identified that the AIME may be clinically useful to help guide decision making in practice.

CONCLUSION

We have developed a predictive model with reasonable performance. Future steps include external validation and impact evaluation.

The Double Burden of the COVID-19 Pandemic and Polypharmacy on Geriatric Population - Public Health Implications

COVID-19 pandemic is inducing acute respiratory distress syndrome, multi-organ failure, and eventual death. Respiratory failure is the leading cause of mortality in the elderly population with pre-existing medical conditions. This group is particularly vulnerable to infections due to a declined immune system, comorbidities, geriatric syndrome, and potentially inappropriate polypharmacy. These conditions make the elderly population more susceptible to the harmful effects of medications and the deleterious consequences of infections, including MERS-CoV, SARS-CoV, and SARS-CoV-2. Chronic diseases among elderlies, including respiratory diseases, hypertension, diabetes, and coronary heart diseases, present a significant challenge for healthcare professionals. To comply with the clinical guidelines, the practitioner may prescribe a complex medication regimen that adds up to the burden of pre-existing treatment, potentially inducing adverse drug reactions and leading to harmful side-effects. Consequently, the geriatric population is at increased risk of falls, frailty, and dependence that enhances their susceptibility to morbidity and mortality due to SARS-CoV-2 respiratory syndrome, particularly interstitial pneumonia. The major challenge resides in the detection of infection that may present as atypical manifestations in this age group. Healthy aging can be possible with adequate preventive measures and appropriate medication regimen and follow-up. Adherence to the guidelines and recommendations of WHO, CDC, and other national/regional/international agencies can reduce the risks of SARS-CoV-2 infection. Better training programs are needed to enhance the skill of health care professionals and patient’s caregivers. This review explains the public health implications associated with polypharmacy on the geriatric population with pre-existing co-morbidities during the COVID-19 pandemic.

Effects of stratified medication review in high-risk patients at admission to hospital: a randomised controlled trial

Background:

Patients at high risk of medication errors will potentially benefit most from medication reviews. An algorithm, MERIS, can identify the patients who are at highest risk of medication errors. The aim of this study was to examine the effects of performing stratified medication reviews on patients who according to MERIS were at highest risk of medication errors.

Methods:

A randomised controlled trial was performed at the Acute Admissions Unit, Aarhus University Hospital, Denmark. Patients were included at admission to the hospital and were randomised to control or intervention. The intervention consisted of stratified medication review at admission on patients with a high MERIS score. Clinical pharmacists and clinical pharmacologists performed the medication reviews; the clinical pharmacologists performed the reviews on patients with the highest MERIS score. The primary outcome measure was the number of prescribing errors during the hospitalisation. Secondary outcomes included self-experienced quality of life, health-care utilisation and mortality measured at follow-up 90 days after discharge.

Results:

A total of 375 patients were included, of which medication reviews were performed in 64 patients. The medication reviews addressed 63 prescribing errors in 37 patients and 60 other drug-related problems. No difference in the number of prescribing errors during hospitalisation between the intervention group (n = 165) and control group (n = 153) was found, corresponding to 0.11 prescribing errors per drug (95% confidence interval (CI): 0.08–0.14) versus 0.13 per drug (95% CI: 0.09–0.16), respectively. No differences in secondary outcomes were observed.

Conclusion:

A stratified medication review approach based on the individual patient’s risk of medication errors did not show impact on the chosen outcomes.

Plain language summary

How does a medication review at admission affect patients who are in high risk of medication errors?

Patients are at risk of medication errors at admission to hospital. Medication reviews aim to detect and solve these. Yet, due to limited resources in healthcare, it would be beneficial to detect the patients who are most at risk of medication errors and perform medication reviews on those patients.

In this study we investigated whether an algorithm, MERIS, could detect patients who are at highest risk of medication errors; we also studied whether performing medication reviews on patients at highest risk of medication errors would have an effect on, for example, the number of medication errors during hospitalisation, qualify of life and number of readmissions. We included 375 patients in a Danish acute admission unit and they were divided into control group and intervention group. Patients in the intervention group received a medication review at admission if they were considered at high risk of medication errors, assessed with the aid of MERIS. In summary, 64 patients in the intervention group were most at risk of medication errors and therefore received a medication review.

We conclude in the study that MERIS was useful in identifying relevant patients for medication reviews. Yet, the medication reviews performed at admission did not impact on the chosen outcomes.

Italian Emergency Department Visits and Hospitalizations for Outpatients' Adverse Drug Events: 12-Year Active Pharmacovigilance Surveillance (The MEREAFaPS Study)

Background

Adverse drug event (ADEs) are a significant cause of emergency department (ED) visits and consequent hospitalization. Preventing ADEs and their related ED visits in outpatients remains a public health safety challenge. In this context, the aims of the present study were to describe the frequency, seriousness and preventability of outpatients' ADE-related ED visits and hospitalizations in the Italian general population, and to identify the presence of potential predictors of ADE-related hospitalization.

Methods

We performed a nationwide, multicentre, observational, retrospective study based on reports of suspected ADEs collected between January 1, 2007 and December 31, 2018 in 94 EDs involved in the MEREAFaPS project. Patients' demographic characteristics, their clinical status, suspected and concomitant drugs, ADE description, and its degree of seriousness, were collected. Causality and preventability were assessed using validated algorithms, and logistic regression analyses were used to estimate the reporting odds ratios (RORs) with 95% confidence intervals (CIs) of ADE-related hospitalization, considering the following covariates: age, sex, ethnicity, number of implicated medications, parenteral administration, presence of interaction, therapeutic error, and/or complementary and alternative medicines (CAM).

Results

Within 12 years, 61,855 reports of suspected ADE were collected, of which 18,918 (30.6%) resulted in hospitalization (ADE defined as serious). Patients were mostly female (56.6%) and Caucasians (87.7%), with a mean age of 57.5 ± 25.0 years. 58% of patients were treated with more than two drugs, and 47% of ADEs leading to hospitalization were preventable. Anticoagulants, antibiotics, and nonsteroidal anti-inflammatory drugs (NSAIDs) were the most frequently implicated agents for ED visits and/or hospitalization, which included clinically significant ADEs, such as haemorrhage for anticoagulants, moderate to severe allergic reactions for antibiotics, and dermatologic reactions and gastrointestinal disturbances for NSAIDs. Older age (1.54 [1.48-1.60]), higher number of concomitantly taken drugs (2.22 [2.14-2.31]), the presence of drug-drug interactions (1.52 [1.28-1.81]), and therapeutic error (1.54 [1.34-1.78]), were significantly associated with an increased risk of hospitalization.

Conclusion

Our long-term active pharmacovigilance study in ED provided a valid estimation of ADE-related hospitalization in a representative sample of the Italian general population and can suggest further focus on medication safety in outpatients, in order to early recognise and prevent ADEs.

Improving the assessment of adverse drug reactions using the Naranjo Algorithm in daily practice: The Japan Adverse Drug Events Study

It is difficult to determine adverse drug reactions (ADRs) in daily complicated clinical practice in which many kinds of drugs are prescribed. We evaluated how well the Naranjo Algorithm (NA) categorized ADRs among suspected ADRs. The Japan Adverse Drug Events (JADE) study was a prospective cohort study of 3459 inpatients. After all suspected ADRs were reported from research assistants, a single physician reviewer independently assigned an NA score to each. After all NA score of suspected ADRs were scored, two physician reviewers discussed and determined ADRs based on the literature. We investigated the sensitivity and specificity of NA and each component to categorize ADRs among suspected ADRs. A total of 1579 suspected ADRs were reported in 962 patients. Physician reviewers determined 997 ADRs. The percentage of ADRs was 94% if the total NA score reached 5. The modified NA consisted of 5 components that showed high classification abilities; its area under the curve (AUC) was 0.92 for categorizing ADRs, the same as the original. When we set the total NA score cut-off value to 5, specificity was 0.95 and sensitivity was 0.59. When we reclassified NA components as binary variables, the specificity increased to 0.98 with a cut-off value of 4 and yielded an AUC of 0.93. In conclusion, we showed that both NA and modified NA could categorize ADRs among suspected ADRs with a high likelihood in daily clinical practice.

Machine Learning to Predict, Detect, and Intervene Older Adults Vulnerable for Adverse Drug Events in the Emergency Department

Adverse drug events (ADEs) are common and have serious consequences in older adults. ED visits are opportunities to identify and alter the course of such vulnerable patients. Current practice, however, is limited by inaccurate reporting of medication list, time-consuming medication reconciliation, and poor ADE assessment. This manuscript describes a novel approach to predict, detect, and intervene vulnerable older adults at risk of ADE using machine learning. Toxicologists' expertise in ADE is essential to creating the machine learning algorithm. Leveraging the existing electronic health records to better capture older adults at risk of ADE in the ED may improve their care.

Systematic review of predictive risk models for adverse drug events in hospitalized patients

AIM

An emerging approach to reducing hospital adverse drug events is the use of predictive risk scores. The aim of this systematic review was to critically appraise models developed for predicting adverse drug event risk in inpatients.

METHODS

Embase, PubMed, CINAHL and Scopus databases were used to identify studies of predictive risk models for hospitalized adult inpatients. Studies had to have used multivariable logistic regression for model development, resulting in a score or rule with two or more variables, to predict the likelihood of inpatient adverse drug events. The Checklist for the critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) was used to critically appraise eligible studies.

RESULTS

Eleven studies met the inclusion criteria and were included in the review. Ten described the development of a new model, whilst one study revalidated and updated an existing score. Studies used different definitions for outcome but were synonymous with or closely related to adverse drug events. Four studies undertook external validation, five internally validated and two studies did not validate their model. No studies evaluated impact of risk scores on patient outcomes.

CONCLUSION

Adverse drug event risk prediction is a complex endeavour but could help to improve patient safety and hospital resource management. Studies in this review had some limitations in their methods for model development, reporting and validation. Two studies, the BADRI and Trivalle's risk scores, used better model development and validation methods and reported reasonable performance, and so could be considered for further research.

Review of structured guides for deprescribing

Avoiding inappropriate polypharmacy has become increasingly recognised as a safety imperative for older patient care. Deprescribing is an active process of reviewing all medications being used by individual patients that prompts clinicians to consider which medications have unfavourable risk-benefit trade-offs in the context of illness severity, advanced age, multi-morbidity, physical and emotional capacity, life expectancy, care goals and personal preferences. Structured guides to deprescribing include algorithms, flow charts or tables which are patient-directed and aim to guide the clinician through sequential steps in deciding which medications should be targeted for discontinuation. In this narrative review, we describe seven structured deprescribing guides whose stated purpose included the reduction of polypharmacy, their use was not restricted to a single drug or drug class and they had undergone some form of efficacy testing. There was considerable heterogeneity in guide design and content, with some guides constituting little more than a set of principles while others entail detailed processes and sub-steps which addressed multiple determinants of drug appropriateness. Evidence of effectiveness for each guide was limited in that none have been evaluated in randomised controlled trials, that pilot or feasibility studies have involved relatively small patient samples, that intra-rater and inter-rater reliabilities have not been determined and that most have been studied in hospital settings. Which is most useful to clinicians is unknown in the absence of head-to-head comparisons. While most guides have face validity, more research is needed for determining effectiveness and ease of use in routine clinical practice, especially in primary care settings.

Risk of prescribing errors in acutely admitted patients: a pilot study

Background Prescribing errors in emergency settings occur frequently. Knowing which patients have the highest risk of errors could improve patient outcomes. Objective The aim of this study was to test an algorithm designed to assess prescribing error risk in individual patients, and to test the feasibility of medication reviews in high-risk patients. Setting The study was performed at the Acute Admissions Unit at Aarhus University Hospital, Denmark. Methods The study was an interventional pilot study. Patients included were assessed according to risk of prescribing errors with the aid of an algorithm called 'Medication Risk Score' (MERIS). Based on the score, high-risk patients were offered a medication review. The clinical relevance of the medication reviews was assessed retrospectively. Main outcome measure The number and nature of prescribing errors during the patients' hospitalisation. Results The study included 103 patients, all of whom could be risk assessed with the algorithm MERIS. MERIS stratified 38 patients as high-risk patients and 65 as low-risk patients. The 103 patients were prescribed a total of 848 drugs in which 88 prescribing errors were found (10.4 %). Sixty-two of these were found in patients in the high-risk group. In general, the medication reviews were found to be clinically relevant and approximately 50 % of recommendations were implemented. Conclusion MERIS was found to be applicable in a clinical setting and stratified most patients with prescribing errors into the high-risk group. The medication reviews were feasible and found to be clinically relevant by most raters.

Detection of Patients at High Risk of Medication Errors: Development and Validation of an Algorithm

Medication errors (MEs) are preventable and can result in patient harm and increased expenses in the healthcare system in terms of hospitalization, prolonged hospitalizations and even death. We aimed to develop a screening tool to detect acutely admitted patients at low or high risk of MEs comprised by items found by literature search and the use of theoretical weighting. Predictive variables used for the development of the risk score were found by the literature search. Three retrospective patient populations and one prospective pilot population were used for modelling. The final risk score was evaluated for precision by the use of sensitivity, specificity and area under the ROC (receiver operating characteristic) curves. The variables used in the final risk score were reduced renal function, the total number of drugs and the risk of individual drugs to cause harm and drug-drug interactions. We found a risk score in the prospective population with an area under the ROC curve of 0.76. The final risk score was found to be quite robust as it showed an area under the ROC curve of 0.87 in a recent patient population, 0.74 in a population of internal medicine and 0.66 in an orthopaedic population. We developed a simple and robust score, MERIS, with the ability to detect patients and divide them according to low and high risk of MEs in a general population admitted at acute admissions unit. The accuracy of the risk score was at least as good as other models reported using multiple regression analysis.