Development of an Interoperable and Easily Transferable Clinical Decision Support System Deployment Platform: System Design and Development Study

Evaluation of an electronic health record Drug Interaction Customization Editor (DICE)

PURPOSE

Due to the low specificity of drug-drug interaction (DDI) warnings, hospitals and healthcare systems would benefit from the ability to customize alerts, thereby reducing the burden of alerts while simultaneously preventing harm. We developed a tool, called the Drug Interaction Customization Editor (DICE), as a prototype to identify features and functionality that could assist healthcare organizations in customizing DDI alerts.

METHODS

A team of pharmacists, physicians, and DDI experts identified attributes expected to be useful for filtering DDI warnings. A survey was sent to pharmacists with informatics responsibilities and other medication safety committee members to obtain their opinions about the tool. The survey asked participants to evaluate the 4 sections of the DICE tool (General, Medication, Patient, and Visit) on a scale ranging from 0 (not useful) to 100 (very useful). The survey provided an opportunity for participants to express their opinions on the overall usefulness of the DICE tool and to provide other comments.

RESULTS

The 50 survey respondents were mainly pharmacists (n = 47, 94%) with almost half (n = 23, 47%) having health information technology/informatics training. Most respondents (n = 33, 80%) were employed by organizations with over 350 beds. Respondents indicated the most useful features of the DICE tool were the ability to filter DDI warnings based on routes of administrations (mean [SD] rating scale score, 86.5 [21.6]), primary drug properties (85.7 [20.5]), patient attributes (85.6 [16.7]) and laboratory attributes (88.8 [18.0]). The overall impression of the DICE tool was rated at 82.8 (19.0), and when asked about the potential to reduce DDI alerts, respondents rated the tool at 83.7 (21.8).

CONCLUSION

The ability to customize DDI alerts using data elements currently within the electronic health records (EHRs) has the potential to decrease alert fatigue and override rates. This prototype DICE tool could be used by end users and vendors as a template for developing a more advanced DDI filtering tool within EHR systems.

Designing the User Interface of a Nitroglycerin Dose Titration Decision Support System: User-Centered Design Study

BACKGROUND

 Nurses adjust intravenous nitroglycerin infusions to provide acute relief for angina by manually increasing or decreasing the dosage. However, titration can pose challenges, as excessively high doses can lead to hypotension, and low doses may result in inadequate pain relief. Clinical decision support systems (CDSSs) that predict changes in blood pressure for nitroglycerin dose adjustments may assist nurses with titration.

OBJECTIVE

 This study aimed to design a user interface for a CDSS for nitroglycerin dose titration (Nitroglycerin Dose Titration Decision Support System [nitro DSS]).

METHODS

 A user-centered design (UCD) approach, consisting of an initial qualitative study with semistructured interviews to identify design specifications for prototype development, was used. This was followed by three iterative rounds of usability testing. Nurses with experience titrating nitroglycerin infusions in coronary care units participated.

RESULTS

 A total of 20 nurses participated, including 7 during the qualitative study and 15 during usability testing (2 nurses participated in both phases). Analysis of the qualitative data revealed four themes for the interface design to be (1) clear and consistent, (2) vigilant, (3) interoperable, and (4) reliable. The major elements of the final prototype included a feature for viewing the predicted and actual blood pressure over time to determine the reliability of the predictions, a drop-down option to report patient side effects, a feature to report reasons for not accepting the prediction, and a visual alert indicating any systolic blood pressure predictions below 90 mm Hg. Nurses' ratings on the questionnaires indicated excellent usability and acceptability of the final nitro DSS prototype.

CONCLUSION

 This study successfully applied a UCD approach to collaborate with nurses in developing a user interface for the nitro DSS that supports the clinical decision-making of nurses titrating nitroglycerin.

Development and Usability Testing of a Mobile App-Based Clinical Decision Support System for Delirium: Randomized Crossover Trial

Background

The 3-Minute Diagnostic Interview for Confusion Assessment Method-Defined Delirium (3D-CAM) is an instrument specially developed for the assessment of delirium in general wards, with high reported sensitivity and specificity. However, the use of the 3D-CAM by bedside nurses in routine practice showed relatively poor usability, with multiple human errors during assessment.

Objective

This study aimed to develop a mobile app-based delirium assessment tool based on the 3D-CAM and evaluate its usability among older patients by bedside nurses.

Methods

The Delirium Assessment Tool With Decision Support Based on the 3D-CAM (3D-DST) was developed to address existing issues of the 3D-CAM and optimize the assessment process. Following a randomized crossover design, questionnaires were used to evaluate the usability of the 3D-DST among older adults by bedside nurses. Meanwhile, the performances of both the 3D-DST and the 3D-CAM paper version, including the assessment completion rate, time required for completing the assessment, and the number of human errors made by nurses during assessment, were recorded, and their differences were compared.

Results

The 3D-DST included 3 assessment modules, 9 evaluation interfaces, and 16 results interfaces, with built-in reminders to guide nurses in completing the delirium assessment. In the usability testing, a total of 432 delirium assessments (216 pairs) on 148 older adults were performed by 72 bedside nurses with the 3D-CAM paper version and the 3D-DST. Compared to the 3D-CAM paper version, the mean usability score was significantly higher when using the 3D-DST (4.35 vs 3.40; P<.001). The median scores of the 6 domains of the satisfactory evaluation questionnaire for nurses using the 3D-CAM paper version and the 3D-DST were above 2.83 and 4.33 points, respectively (P<.001). The average time for completing the assessment reduced by 2.1 minutes (4.4 vs 2.3 min; P<.001) when the 3D-DST was used.

Conclusions

This study demonstrated that the 3D-DST significantly improved the efficiency of delirium assessment and was considered highly acceptable by bedside nurses.

Machine Learning-Enabled Clinical Information Systems Using Fast Healthcare Interoperability Resources Data Standards: Scoping Review

Background

Machine learning-enabled clinical information systems (ML-CISs) have the potential to drive health care delivery and research. The Fast Healthcare Interoperability Resources (FHIR) data standard has been increasingly applied in developing these systems. However, methods for applying FHIR to ML-CISs are variable.

Objective

This study evaluates and compares the functionalities, strengths, and weaknesses of existing systems and proposes guidelines for optimizing future work with ML-CISs.

Methods

Embase, PubMed, and Web of Science were searched for articles describing machine learning systems that were used for clinical data analytics or decision support in compliance with FHIR standards. Information regarding each system's functionality, data sources, formats, security, performance, resource requirements, scalability, strengths, and limitations was compared across systems.

Results

A total of 39 articles describing FHIR-based ML-CISs were divided into the following three categories according to their primary focus: clinical decision support systems (n=18), data management and analytic platforms (n=10), or auxiliary modules and application programming interfaces (n=11). Model strengths included novel use of cloud systems, Bayesian networks, visualization strategies, and techniques for translating unstructured or free-text data to FHIR frameworks. Many intelligent systems lacked electronic health record interoperability and externally validated evidence of clinical efficacy.

Conclusions

Shortcomings in current ML-CISs can be addressed by incorporating modular and interoperable data management, analytic platforms, secure interinstitutional data exchange, and application programming interfaces with adequate scalability to support both real-time and prospective clinical applications that use electronic health record platforms with diverse implementations.

The need for multimodal health data modeling: a practical approach for a federated-learning healthcare platform

Federated learning initiatives in healthcare are being developed to collaboratively train predictive models without the need to centralize sensitive personal data. GenoMed4All is one such project, with the goal of connecting European clinical and -omics data repositories on rare diseases through a federated learning platform. Currently, the consortium faces the challenge of a lack of well-established international datasets and interoperability standards for federated learning applications on rare diseases. This paper presents our practical approach to select and implement a Common Data Model (CDM) suitable for the federated training of predictive models applied to the medical domain, during the initial design phase of our federated learning platform. We describe our selection process, composed of identifying the consortium's needs, reviewing our functional and technical architecture specifications, and extracting a list of business requirements. We review the state of the art and evaluate three widely-used approaches (FHIR, OMOP and Phenopackets) based on a checklist of requirements and specifications. We discuss the pros and cons of each approach considering the use cases specific to our consortium as well as the generic issues of implementing a European federated learning healthcare platform. A list of lessons learned from the experience in our consortium is discussed, from the importance of establishing the proper communication channels for all stakeholders to technical aspects related to -omics data. For federated learning projects focused on secondary use of health data for predictive modeling, encompassing multiple data modalities, a phase of data model convergence is sorely needed to gather different data representations developed in the context of medical research, interoperability of clinical care software, imaging, and -omics analysis into a coherent, unified data model. Our work identifies this need and presents our experience and a list of actionable lessons learned for future work in this direction.