Iterative Development and Evaluation of a Pharmacogenomic-Guided Clinical Decision Support System for Warfarin Dosing

The PrescIT platform: An interoperable Clinical Decision Support System for ePrescription to Prevent Adverse Drug Reactions and Drug-Drug Interactions

INTRODUCTION

Preventable medication errors have been proven to cause significant public health burden, and ePrescription is a key part of the process where medication errors and adverse effects could be prevented. Information systems and "intelligent" computational approaches could provide a valuable tool to prevent such errors with profound impact in clinical practice.

OBJECTIVES

The PrescIT platform is a Clinical Decision Support System (CDSS) that aims to facilitate the prevention of adverse drug reactions (ADRs) and drug-drug interactions (DDIs) in the phase of ePrescription in Greece. The proposed platform could be relatively easily localized for use in other contexts too.

METHODS

The PrescIT platform is based on the use of Knowledge Engineering (ΚΕ) approaches, i.e., the use of Ontologies and Knowledge Graphs (KGs) developed upon openly available data sources. Open standards (i.e., RDF, OWL, SPARQL) are used for the development of the platform enabling the integration with already existing IT systems or for standalone use. The main KG is based on the use of DrugBank, MedDRA, SemMedDB and OpenPVSignal. In addition, the Business Process Management Notation (BPMN) has been used to model long-term therapeutic protocols used during the ePrescription process. Finally, the produced software has been pilot tested in three hospitals by 18 clinical professionals via in-person think-aloud sessions.

RESULTS

The PrescIT platform has been successfully integrated in a transparent fashion in a proprietary Hospital Information System (HIS), and it has also been used as a standalone application. Furthermore, it has been successfully integrated with the Greek National ePrescription system. During the pilot phase, one psychiatric therapeutic protocol was used as a testbed to collect end-users' feedback. Summarizing the feedback from the end-users, they have generally acknowledged the usefulness of such a system while also identifying some challenges in terms of usability and the overall user experience.

CONCLUSIONS

The PrescIT platform has been successfully deployed and piloted in real-world environments to evaluate its ability to support safer medication prescriptions.

PillHarmonics: An Orchestrated Pharmacogenetics Medication Clinical Decision Support Service

OBJECTIVES

 Pharmacogenetics (PGx) is increasingly important in individualizing therapeutic management plans, but is often implemented apart from other types of medication clinical decision support (CDS). The lack of integration of PGx into existing CDS may result in incomplete interaction information, which may pose patient safety concerns. We sought to develop a cloud-based orchestrated medication CDS service that integrates PGx with a broad set of drug screening alerts and evaluate it through a clinician utility study.

METHODS

 We developed the PillHarmonics service for implementation per the CDS Hooks protocol, algorithmically integrating a wide range of drug interaction knowledge using cloud-based screening services from First Databank (drug-drug/allergy/condition), PharmGKB (drug-gene), and locally curated content (drug-renal/hepatic/race). We performed a user study, presenting 13 clinicians and pharmacists with a prototype of the system's usage in synthetic patient scenarios. We collected feedback via a standard questionnaire and structured interview.

RESULTS

 Clinician assessment of PillHarmonics via the Technology Acceptance Model questionnaire shows significant evidence of perceived utility. Thematic analysis of structured interviews revealed that aggregated knowledge, concise actionable summaries, and information accessibility were highly valued, and that clinicians would use the service in their practice.

CONCLUSION

 Medication safety and optimizing efficacy of therapy regimens remain significant issues. A comprehensive medication CDS system that leverages patient clinical and genomic data to perform a wide range of interaction checking and presents a concise and holistic view of medication knowledge back to the clinician is feasible and perceived as highly valuable for more informed decision-making. Such a system can potentially address many of the challenges identified with current medication-related CDS.

Trust and acceptability of data-driven clinical recommendations in everyday practice: A scoping review

BACKGROUND

Increasing attention is being given to the analysis of large health datasets to derive new clinical decision support systems (CDSS). However, few data-driven CDSS are being adopted into clinical practice. Trust in these tools is believed to be fundamental for acceptance and uptake but to date little attention has been given to defining or evaluating trust in clinical settings.

OBJECTIVES

A scoping review was conducted to explore how and where acceptability and trustworthiness of data-driven CDSS have been assessed from the health professional's perspective.

METHODS

Medline, Embase, PsycInfo, Web of Science, Scopus, ACM Digital, IEEE Xplore and Google Scholar were searched in March 2022 using terms expanded from: "data-driven" AND "clinical decision support" AND "acceptability". Included studies focused on healthcare practitioner-facing data-driven CDSS, relating directly to clinical care. They included trust or a proxy as an outcome, or in the discussion. The preferred reporting items for systematic reviews and meta-analyses extension for scoping reviews (PRISMA-ScR) is followed in the reporting of this review.

RESULTS

3291 papers were screened, with 85 primary research studies eligible for inclusion. Studies covered a diverse range of clinical specialisms and intended contexts, but hypothetical systems (24) outnumbered those in clinical use (18). Twenty-five studies measured trust, via a wide variety of quantitative, qualitative and mixed methods. A further 24 discussed themes of trust without it being explicitly evaluated, and from these, themes of transparency, explainability, and supporting evidence were identified as factors influencing healthcare practitioner trust in data-driven CDSS.

CONCLUSION

There is a growing body of research on data-driven CDSS, but few studies have explored stakeholder perceptions in depth, with limited focused research on trustworthiness. Further research on healthcare practitioner acceptance, including requirements for transparency and explainability, should inform clinical implementation.

Pharmacogenomics implementation across multiple clinic settings: a qualitative evaluation

Aim: To advance clinical adoption and implementation of pharmacogenomics (PGx) testing, barriers and facilitators to these efforts must be understood. This study identified and examined barriers and facilitators to active implementation of a PGx program across multiple clinic settings in an academic healthcare system. Materials & methods: 28 contributors to the PGx implementation (e.g., clinical providers, informatics specialists) completed an interview to elicit their perceptions of the implementation. Results: Qualitative analysis identified several barriers and facilitators that spanned different stages of the implementation process. Specifically, unclear test payment mechanisms, decision support tool development, rigid workflows and provider education were noted as barriers to the PGx implementation. A multidisciplinary implementation team and leadership support emerged as key facilitators. Furthermore, participants also suggested strategies to overcome or maintain these factors. Conclusion: Assessing real-world implementation perceptions and suggested strategies from a range of implementation contributors facilitates a more comprehensive framework and best-practice guidelines for PGx implementation.

Human factors and safety analysis methods used in the design and redesign of electronic medication management systems: A systematic review

INTRODUCTION

Poorly designed electronic medication management systems (EMMS) or computerized physician order entry (CPOE) systems in hospital settings can result in usability issues and in turn, patient safety risks. As a safety science, human factors and safety analysis methods have potential to support the safe and usable design of EMMS.

OBJECTIVE

To identify and describe human factors and safety analysis methods that have been used in the design or redesign of EMMS used in hospital settings.

MATERIALS AND METHODS

A systematic review, following PRISMA guidelines, was conducted by searching online databases and relevant journals from January 2011 to May 2022. Studies were included if they described the practical application of human factors and safety analysis methods to support the design or redesign of a clinician-facing EMMS, or its components. Methods used were extracted and mapped to human centered design (HCD) activities: understanding context of use; specifying user requirements; producing design solutions; and evaluating the design.

RESULTS

Twenty-one papers met the inclusion criteria. Overall, 21 human factors and safety analysis methods were used in the design or redesign of EMMS with prototyping, usability testing, participant surveys/questionnaires and interviews the most frequent. Human factors and safety analysis methods were most frequently used to evaluate the design of a system (n = 67; 56.3%). Nineteen of 21 (90%) methods used aimed to identify usability issues and/or support iterative design; only one paper utilized a safety-oriented method and one, a mental workload assessment method.

DISCUSSION AND CONCLUSION

While the review identified 21 methods, EMMS design primarily utilized a subset of available methods, and rarely a method focused on safety. Given the high-risk nature of medication management in complex hospital environments, and the potential for harm due to poorly designed EMMS, there is significant potential to apply more safety-oriented human factors and safety analysis methods to support EMMS design.

In search of lost time: A timing evaluation of antimicrobial prescribing with and without a computerized decision support system using clinical vignettes

BACKGROUND

We implemented a computerized decision support system (CDSS) integrated in the in-house computerized physician order entry (CPOE) system to assist physicians with antimicrobial prescribing decisions in the context of the multicenter cluster-randomized COMPASS trial (NCT03120975). Some physicians in the intervention wards complained about the perceived extra-time associated with the use of the CDSS compared with routine prescribing through CPOE. The aim of this study was to compare the time needed to prescribe antimicrobials with and without the CDSS.

METHODS

Physicians with and without previous experience with the COMPASS CDSS working at our hospital in Geneva, Switzerland, were recruited to prescribe antimicrobials using clinical vignettes. Physicians without experience received a brief explanation of the CDSS. Each physician received 2 groups of 5-7 clinical vignettes randomly selected from a pool of 28. Each group of vignettes included prescriptions with different levels of complexity (empiric versus targeted or pre-defined treatment, dose adjustment for renal function, oral switch, treatments for which COMPASS does not provide recommendations or where a deviation was necessary). Prescriptions were completed using the standard CPOE (first set), then using COMPASS (second set). A print version of the local antimicrobial guidelines was available for consultation. Time to complete each prescription was recorded (including time needed to consult paper guidelines). The Mann-Whitney test was used for comparisons. Consultation of guidelines booklet and concordance with local guidelines were assessed.

RESULTS

Twenty-five physicians were recruited. Thirteen (52%) had previously used COMPASS. Among them, 11 (85%) estimated the extra-time being above 1 min. We evaluated a total of 296 vignettes. Overall, the median time to complete a prescription was 55.5 s (IQR 38-86) using COMPASS and 50 s (IQR 31-88) using the standard CPOE (p = 0.24). Concordance of prescriptions with local guidelines was similar with the 2 systems (127/148, 85.8% for both), but consultation of paper guidelines was more frequent when prescribing without the CDSS (49.3% (73/148) vs 22.3% (33/148)).

CONCLUSIONS

The increased time required for prescribing using COMPASS is overestimated by end-users. Information collected in the study will be used to streamline the prescribing process via COMPASS and increase acceptance.

The Endodontic Complexity Assessment Tool (E-CAT): A digital form for assessing root canal treatment case difficulty

AIMS

To develop an evidence-based, valid and reliable assessment tool that educational establishments and dental practitioners may use to assess the complexity of root canal treatment (RCT) utilizing digital advancements. The study also aimed to provide a more objective definition of the term 'uncomplicated' root canal treatment as described by the Association for Dental Education in Europe (ADEE) and the European Society of Endodontology (ESE) undergraduate curriculum guidelines for Endodontology.

METHODOLOGY

The development process involved a narrative review of the literature to identify the complexity factors associated with root canal treatment on permanent teeth; an iterative development and analysis process to assess the weighting of these factors; and the programming of digital software to enhance the efficiency and user interface of the assessment form. Validation of the tool was sought with a panel of 35 specialist endodontists to assess clinical scenarios and assess the consensus inter-examiner agreement with the outcomes provided by the E-CAT. The inter-user and intra-user reliability studies were conducted with 15 dentists to evaluate the same clinical cases and by repeating the experiment 9 months later. The ease of use of the form was also assessed.

RESULTS

The E-CAT was successfully developed with a total of 19 complexity criteria and hosted on a secure server under the domain of www.e-cat.uk. The tool provides a smart interactive filtering mechanism and automatic background calculation of the risk scores. Three levels of complexity were defined: class I (uncomplicated), class II (moderately complicated) and class III (highly complicated). The consensus of the panel of endodontists had excellent agreement with the outcome of the E-CAT. The inter-user and intra-user reliability was found to be 0.80 and 0.90, respectively. The average time to assess a case was 1:36 min.

CONCLUSION

The E-CAT gave promising results providing an efficient and reliable platform to assess the complexity of cases undergoing root canal treatments. The study design allowed the formulation of a more objective definition to describe 'uncomplicated' root canal treatment as referred to by the ESE and ADEE guidelines. This study is advantageous for educational, public health and referral pathways.

Biomedical Data Science and Informatics Challenges to Implementing Pharmacogenomics with Electronic Health Records

Pharmacogenomic information must be incorporated into electronic health records (EHRs) with clinical decision support in order to fully realize its potential to improve drug therapy. Supported by various clinical knowledge resources, pharmacogenomic workflows have been implemented in several healthcare systems. Little standardization exists across these efforts, however, which limits scalability both within and across clinical sites. Limitations in information standards, knowledge management, and the capabilities of modern EHRs remain challenges for the widespread use of pharmacogenomics in the clinic, but ongoing efforts are addressing these challenges. Although much work remains to use pharmacogenomic information more effectively within clinical systems, the experiences of pioneering sites and lessons learned from those programs may be instructive for other clinical areas beyond genomics. We present a vision of what can be achieved as informatics and data science converge to enable further adoption of pharmacogenomics in the clinic.

Noninterruptive Clinical Decision Support Decreases Ordering of Respiratory Viral Panels during Influenza Season

OBJECTIVE

A growing body of evidence suggests that testing for influenza virus alone is more appropriate than multiplex respiratory viral panel (RVP) testing for general populations of patients with respiratory tract infections. We aimed to decrease the proportion of RVPs out of total respiratory viral testing ordered during influenza season.

METHODS

We implemented two consecutive interventions: reflex testing for RVPs only after a negative influenza test, and noninterruptive clinical decision support (CDS) including modifications of the computerized physician order entry search behavior and cost display. We conducted an interrupted time series of RVPs and influenza polymerase chain reaction tests pre- and postintervention, and performed a mixed-effects logistic regression analysis with a primary outcome of proportion of RVPs out of total respiratory viral tests. The primary predictor was the intervention period, and covariates included the provider, clinical setting, associated diagnoses, and influenza incidence.

RESULTS

From March 2013 to April 2019, there were 24,294 RVPs and 26,012 influenza tests (n = 50,306). Odds of ordering an RVP decreased during the reflex testing period (odds ratio: 0.432, 95% confidence interval: 0.397-0.469), and decreased more dramatically during the noninterruptive CDS period (odds ratio: 0.291, 95% confidence interval: 0.259-0.327).

DISCUSSION

The odds of ordering an RVP were 71% less with the noninterruptive CDS intervention, which projected 4,773 fewer RVPs compared with baseline. Assuming a cost equal to Medicare reimbursement rates for RVPs and influenza tests, this would generate an estimated averted cost of $1,259,474 per year.

CONCLUSION

Noninterruptive CDS interventions are effective in reducing unnecessary and expensive testing, and avoid typical pitfalls such as alert fatigue.

Interventions designed to improve the safety and quality of therapeutic anticoagulation in an inpatient electronic medical record

IMPORTANCE

Anticoagulants are high-risk medications with the potential to cause significant patient harm or death. Digital transformation is occurring in hospital practice and it is essential to implement effective, evidence-based strategies for these medications in an electronic medical record (EMR).

OBJECTIVE

To systematically appraise the literature to determine which EMR interventions have improved the safety and quality of therapeutic anticoagulation in an inpatient hospital setting.

METHODS

PubMed, Embase, CINAHL, and the International Pharmaceutical Database were searched for suitable publications. Articles that met eligibility criteria up to September 2018 were included. The review was registered with PROSPERO (CRD42018104899). The web-based software platform Covidence® was used for screening and data extraction. Studies were grouped according to the type of intervention and the outcomes measured. Where relevant, a bias assessment was performed.

RESULTS

We found 2624 candidate articles and 27 met inclusion criteria. They included 3 randomised controlled trials, 4 cohort studies and 20 pre/post observational studies. There were four major interventions; computerised physician order entry (CPOE) (n = 4 studies), clinical decision support system (CDSS) methods (n = 21), dashboard utilisation (n = 1) and EMR implementation in general (n = 1). Seven outcomes were used to summarise the study results. Most research focused on prescribing or documentation compliance (n = 18). The remaining study outcome measures were: medication errors (n = 9), adverse drug events (n = 5), patient outcomes (morbidity/mortality/length of hospital stay/re-hospitalisation) (n = 5), quality use of anticoagulant (n = 4), end-user acceptance (n = 4), cost effectiveness (n = 1).

CONCLUSION

Despite the research cited, limited benefits have been demonstrated to date. It appears healthcare organisations are yet to determine optimal, evidence-based-methods to improve EMR utilisation. Further evaluation, collaboration and work are necessary to measure and leverage the potential benefits of digital health systems. Most research evaluating therapeutic anticoagulation management within an EMR focused on prescribing or documentation compliance, with less focus on clinical impact to the patient or cost effectiveness. Evidence suggests that CPOE in conjunction with CDSS is needed to effectively manage therapeutic anticoagulation. Targets for robust research include the integration of 'stealth' alerts, nomograms into digital systems and the use of dashboards within clinical practice.

Utilizing a user-centered approach to develop and assess pharmacogenomic clinical decision support for thiopurine methyltransferase

Background

A pharmacogenomic clinical decision support tool (PGx-CDS) for thiopurine medications can help physicians incorporate pharmacogenomic results into prescribing decisions by providing up-to-date, real-time decision support. However, the PGx-CDS user interface may introduce errors and promote alert fatigue. The objective of this study was to develop and evaluate a prototype of a PGx-CDS user interface for thiopurine medications with user-centered design methods.

Methods

This study had two phases: In phase I, we conducted qualitative interviews to assess providers’ information needs. Interview transcripts were analyzed through a combination of inductive and deductive qualitative analysis to develop design requirements for a PGx-CDS user interface. Using these requirements, we developed a user interface prototype and evaluated its usability (phase II).

Results

In total, 14 providers participated: 10 were interviewed in phase I, and seven providers completed usability testing in phase II (3 providers participated in both phases). Most (90%) participants were interested in PGx-CDS systems to help improve medication efficacy and patient safety. Interviews yielded 11 themes sorted into two main categories: 1) health care providers’ views on PGx-CDS and 2) important design features for PGx-CDS. We organized these findings into guidance for PGx-CDS content and display. Usability testing of the PGx-CDS prototype showed high provider satisfaction.

Conclusion

This is one of the first studies to utilize a user-centered design approach to develop and assess a PGx-CDS interface prototype for Thiopurine Methyltransferase (TPMT). This study provides guidance for the development of a PGx-CDS, and particularly for biomarkers such as TPMT.

Integrating pharmacogenetic testing into primary care

Introduction

Pharmacogenetic (PGx) testing has greatly expanded due to enhanced understanding of the role of genes in drug response and advances in DNA-based testing technology development. As many primary care visits result in a prescription, the use of PGx testing may be particularly beneficial in this setting. However, integration of PGx testing may be limited as no uniform approach to delivery of tests has been established and providers are ill-prepared to integrate PGx testing into routine care.

Areas covered

In this paper, the readiness of primary care practitioners are reviewed as well as strategies to address these barriers based on published research and ongoing activities on education and implementation of PGx testing.

Expert Commentary

Widespread integration of PGx testing will warrant continued education and point-of-care decisional support. Primary care providers may also benefit from consultation services or team-based care with laboratory medicine specialists, pharmacists, and genetic counselors.

Integrating clinical decision support systems for pharmacogenomic testing into clinical routine - a scoping review of designs of user-system interactions in recent system development

Background

Pharmacogenomic clinical decision support systems (CDSS) have the potential to help overcome some of the barriers for translating pharmacogenomic knowledge into clinical routine. Before developing a prototype it is crucial for developers to know which pharmacogenomic CDSS features and user-system interactions have yet been developed, implemented and tested in previous pharmacogenomic CDSS efforts and if they have been successfully applied. We address this issue by providing an overview of the designs of user-system interactions of recently developed pharmacogenomic CDSS.

Methods

We searched PubMed for pharmacogenomic CDSS published between January 1, 2012 and November 15, 2016. Thirty-two out of 118 identified articles were summarized and included in the final analysis. We then compared the designs of user-system interactions of the 20 pharmacogenomic CDSS we had identified.

Results

Alerts are the most widespread tools for physician-system interactions, but need to be implemented carefully to prevent alert fatigue and avoid liabilities. Pharmacogenomic test results and override reasons stored in the local EHR might help communicate pharmacogenomic information to other internal care providers. Integrating patients into user-system interactions through patient letters and online portals might be crucial for transferring pharmacogenomic data to external health care providers. Inbox messages inform physicians about new pharmacogenomic test results and enable them to request pharmacogenomic consultations. Search engines enable physicians to compare medical treatment options based on a patient’s genotype.

Conclusions

Within the last 5 years, several pharmacogenomic CDSS have been developed. However, most of the included articles are solely describing prototypes of pharmacogenomic CDSS rather than evaluating them. To support the development of prototypes further evaluation efforts will be necessary. In the future, pharmacogenomic CDSS will likely include prediction models to identify patients who are suitable for preemptive genotyping.

Electronic supplementary material

The online version of this article (doi:10.1186/s12911-017-0480-y) contains supplementary material, which is available to authorized users.