A framework and model for evaluating clinical decision support architectures

Clinical Decision Support Systems Used in Transplantation: Are They Tools for Success or an Unnecessary Gadget? A Systematic Review

Although clinical decision support systems (CDSSs) have been used since the 1970s for a wide variety of clinical tasks including optimization of medication orders, improved documentation, and improved patient adherence, to date, no systematic reviews have been carried out to assess their utilization and efficacy in transplant medicine. The aim of this study is to systematically review studies that utilized a CDSS and assess impact on patient outcomes. A total of 48 articles were identified as meeting the author-derived inclusion criteria, including tools for posttransplant monitoring, pretransplant risk assessment, waiting list management, immunosuppressant management, and interpretation of histopathology. Studies included 15 984 transplant recipients. Tools aimed at helping with transplant patient immunosuppressant management were the most common (19 studies). Thirty-four studies (85%) found an overall clinical benefit following the implementation of a CDSS in clinical practice. Although there are limitations to the existing literature, current evidence suggests that implementing CDSS in transplant clinical settings may improve outcomes for patients. Limited evidence was found using more advanced technologies such as artificial intelligence in transplantation, and future studies should investigate the role of these emerging technologies.

Evaluation in Life Cycle of Information Technology (ELICIT) framework: Supporting the innovation life cycle from business case assessment to summative evaluation

OBJECTIVE

Our objective was to develop an evaluation framework for electronic health record (EHR)-integrated innovations to support evaluation activities at each of four information technology (IT) life cycle phases: planning, development, implementation, and operation.

METHODS

The evaluation framework was developed based on a review of existing evaluation frameworks from health informatics and other domains (human factors engineering, software engineering, and social sciences); expert consensus; and real-world testing in multiple EHR-integrated innovation studies.

RESULTS

The resulting Evaluation in Life Cycle of IT (ELICIT) framework covers four IT life cycle phases and three measure levels (society, user, and IT). The ELICIT framework recommends 12 evaluation steps: (1) business case assessment; (2) stakeholder requirements gathering; (3) technical requirements gathering; (4) technical acceptability assessment; (5) user acceptability assessment; (6) social acceptability assessment; (7) social implementation assessment; (8) initial user satisfaction assessment; (9) technical implementation assessment; (10) technical portability assessment; (11) long-term user satisfaction assessment; and (12) social outcomes assessment.

DISCUSSION

Effective evaluation requires a shared understanding and collaboration across disciplines throughout the entire IT life cycle. In contrast with previous evaluation frameworks, the ELICIT framework focuses on all phases of the IT life cycle across the society, user, and IT levels. Institutions seeking to establish evaluation programs for EHR-integrated innovations could use our framework to create such shared understanding and justify the need to invest in evaluation.

CONCLUSION

As health care undergoes a digital transformation, it will be critical for EHR-integrated innovations to be systematically evaluated. The ELICIT framework can facilitate these evaluations.

Electronic clinical decision support for children with minor head trauma and intracranial injuries: a sociotechnical analysis

BACKGROUND

Current management of children with minor head trauma (MHT) and intracranial injuries is not evidence-based and may place some children at risk of harm. Evidence-based electronic clinical decision support (CDS) for management of these children may improve patient safety and decrease resource use. To guide these efforts, we evaluated the sociotechnical environment impacting the implementation of electronic CDS, including workflow and communication, institutional culture, and hardware and software infrastructure, among other factors.

METHODS

Between March and May, 2020 semi-structured qualitative focus group interviews were conducted to identify sociotechnical influences on CDS implementation. Physicians from neurosurgery, emergency medicine, critical care, and pediatric general surgery were included, along with information technology specialists. Participants were recruited from nine health centers in the United States. Focus group transcripts were coded and analyzed using thematic analysis. The final themes were then cross-referenced with previously defined sociotechnical dimensions.

RESULTS

We included 28 physicians and four information technology specialists in seven focus groups (median five participants per group). Five physicians were trainees and 10 had administrative leadership positions. Through inductive thematic analysis, we identified five primary themes: (1) clinical impact; (2) stakeholders and users; (3) tool content; (4) clinical practice integration; and (5) post-implementation evaluation measures. Participants generally supported using CDS to determine an appropriate level-of-care for these children. However, some had mixed feelings regarding how the tool could best be used by different specialties (e.g. use by neurosurgeons versus non-neurosurgeons). Feedback from the interviews helped refine the tool content and also highlighted potential technical and workflow barriers to address prior to implementation.

CONCLUSIONS

We identified key factors impacting the implementation of electronic CDS for children with MHT and intracranial injuries. These results have informed our implementation strategy and may also serve as a template for future efforts to implement health information technology in a multidisciplinary, emergency setting.

Low prevalence of clinical decision support to calculate caloric and fluid intake for infants in the neonatal intensive care unit

BACKGROUND

Clinical decision support (CDS) improves nutrition delivery for infants in the neonatal intensive care unit (NICU), however, the prevalence of CDS to support nutrition is unknown.

METHODS

Online surveys, with telephone and email validation of responses, were administered to NICU clinicians in the Children's Hospital Neonatal Consortium (CHNC). We determined and compared the availability of CDS to calculate calories and fluid received in the prior 24 h, stratified by enteral and parenteral intake, using McNemar's test.

RESULTS

Clinicians at all 34 CHNC hospitals responded with 98 of 108 (91%) surveys completed. NICUs have considerably less CDS to calculate enteral calories received than enteral fluid received (32% vs. 82%, p < 0.001) and less CDS to calculate parenteral calories received than parenteral fluid received (29% vs. 82%, p < 0.001).

DISCUSSION

Most CHNC NICUs are unable to reliably and consistently monitor caloric intake delivered to critically ill infants at risk for growth failure.

Developing a framework for evidence-based grading and assessment of predictive tools for clinical decision support

BACKGROUND

Clinical predictive tools quantify contributions of relevant patient characteristics to derive likelihood of diseases or predict clinical outcomes. When selecting predictive tools for implementation at clinical practice or for recommendation in clinical guidelines, clinicians are challenged with an overwhelming and ever-growing number of tools, most of which have never been implemented or assessed for comparative effectiveness. To overcome this challenge, we have developed a conceptual framework to Grade and Assess Predictive tools (GRASP) that can provide clinicians with a standardised, evidence-based system to support their search for and selection of efficient tools.

METHODS

A focused review of the literature was conducted to extract criteria along which tools should be evaluated. An initial framework was designed and applied to assess and grade five tools: LACE Index, Centor Score, Well's Criteria, Modified Early Warning Score, and Ottawa knee rule. After peer review, by six expert clinicians and healthcare researchers, the framework and the grading of the tools were updated.

RESULTS

GRASP framework grades predictive tools based on published evidence across three dimensions: 1) Phase of evaluation; 2) Level of evidence; and 3) Direction of evidence. The final grade of a tool is based on the highest phase of evaluation, supported by the highest level of positive evidence, or mixed evidence that supports a positive conclusion. Ottawa knee rule had the highest grade since it has demonstrated positive post-implementation impact on healthcare. LACE Index had the lowest grade, having demonstrated only pre-implementation positive predictive performance.

CONCLUSION

GRASP framework builds on widely accepted concepts to provide standardised assessment and evidence-based grading of predictive tools. Unlike other methods, GRASP is based on the critical appraisal of published evidence reporting the tools' predictive performance before implementation, potential effect and usability during implementation, and their post-implementation impact. Implementing the GRASP framework as an online platform can enable clinicians and guideline developers to access standardised and structured reported evidence of existing predictive tools. However, keeping GRASP reports up-to-date would require updating tools' assessments and grades when new evidence becomes available, which can only be done efficiently by employing semi-automated methods for searching and processing the incoming information.

Clinical decision support alert malfunctions: analysis and empirically derived taxonomy

Objective

To develop an empirically derived taxonomy of clinical decision support (CDS) alert malfunctions.

Materials and Methods

We identified CDS alert malfunctions using a mix of qualitative and quantitative methods: (1) site visits with interviews of chief medical informatics officers, CDS developers, clinical leaders, and CDS end users; (2) surveys of chief medical informatics officers; (3) analysis of CDS firing rates; and (4) analysis of CDS overrides. We used a multi-round, manual, iterative card sort to develop a multi-axial, empirically derived taxonomy of CDS malfunctions.

Results

We analyzed 68 CDS alert malfunction cases from 14 sites across the United States with diverse electronic health record systems. Four primary axes emerged: the cause of the malfunction, its mode of discovery, when it began, and how it affected rule firing. Build errors, conceptualization errors, and the introduction of new concepts or terms were the most frequent causes. User reports were the predominant mode of discovery. Many malfunctions within our database caused rules to fire for patients for whom they should not have (false positives), but the reverse (false negatives) was also common.

Discussion

Across organizations and electronic health record systems, similar malfunction patterns recurred. Challenges included updates to code sets and values, software issues at the time of system upgrades, difficulties with migration of CDS content between computing environments, and the challenge of correctly conceptualizing and building CDS.

Conclusion

CDS alert malfunctions are frequent. The empirically derived taxonomy formalizes the common recurring issues that cause these malfunctions, helping CDS developers anticipate and prevent CDS malfunctions before they occur or detect and resolve them expediently.

Multi-model-based interactive authoring environment for creating shareable medical knowledge

OBJECTIVE

Technologically integrated healthcare environments can be realized if physicians are encouraged to use smart systems for the creation and sharing of knowledge used in clinical decision support systems (CDSS). While CDSSs are heading toward smart environments, they lack support for abstraction of technology-oriented knowledge from physicians. Therefore, abstraction in the form of a user-friendly and flexible authoring environment is required in order for physicians to create shareable and interoperable knowledge for CDSS workflows. Our proposed system provides a user-friendly authoring environment to create Arden Syntax MLM (Medical Logic Module) as shareable knowledge rules for intelligent decision-making by CDSS.

METHODS AND MATERIALS

Existing systems are not physician friendly and lack interoperability and shareability of knowledge. In this paper, we proposed Intelligent-Knowledge Authoring Tool (I-KAT), a knowledge authoring environment that overcomes the above mentioned limitations. Shareability is achieved by creating a knowledge base from MLMs using Arden Syntax. Interoperability is enhanced using standard data models and terminologies. However, creation of shareable and interoperable knowledge using Arden Syntax without abstraction increases complexity, which ultimately makes it difficult for physicians to use the authoring environment. Therefore, physician friendliness is provided by abstraction at the application layer to reduce complexity. This abstraction is regulated by mappings created between legacy system concepts, which are modeled as domain clinical model (DCM) and decision support standards such as virtual medical record (vMR) and Systematized Nomenclature of Medicine - Clinical Terms (SNOMED CT). We represent these mappings with a semantic reconciliation model (SRM).

RESULTS

The objective of the study is the creation of shareable and interoperable knowledge using a user-friendly and flexible I-KAT. Therefore we evaluated our system using completeness and user satisfaction criteria, which we assessed through the system- and user-centric evaluation processes. For system-centric evaluation, we compared the implementation of clinical information modelling system requirements in our proposed system and in existing systems. The results suggested that 82.05% of the requirements were fully supported, 7.69% were partially supported, and 10.25% were not supported by our system. In the existing systems, 35.89% of requirements were fully supported, 28.20% were partially supported, and 35.89% were not supported. For user-centric evaluation, the assessment criterion was 'ease of use'. Our proposed system showed 15 times better results with respect to MLM creation time than the existing systems. Moreover, on average, the participants made only one error in MLM creation using our proposed system, but 13 errors per MLM using the existing systems.

CONCLUSION

We provide a user-friendly authoring environment for creation of shareable and interoperable knowledge for CDSS to overcome knowledge acquisition complexity. The authoring environment uses state-of-the-art decision support-related clinical standards with increased ease of use.

Requirements and validation of a prototype learning health system for clinical diagnosis

INTRODUCTION

Diagnostic error is a major threat to patient safety in the context of family practice. The patient safety implications are severe for both patient and clinician. Traditional approaches to diagnostic decision support have lacked broad acceptance for a number of well-documented reasons: poor integration with electronic health records and clinician workflow, static evidence that lacks transparency and trust, and use of proprietary technical standards hindering wider interoperability. The learning health system (LHS) provides a suitable infrastructure for development of a new breed of learning decision support tools. These tools exploit the potential for appropriate use of the growing volumes of aggregated sources of electronic health records.

METHODS

We describe the experiences of the TRANSFoRm project developing a diagnostic decision support infrastructure consistent with the wider goals of the LHS. We describe an architecture that is model driven, service oriented, constructed using open standards, and supports evidence derived from electronic sources of patient data. We describe the architecture and implementation of 2 critical aspects for a successful LHS: the model representation and translation of clinical evidence into effective practice and the generation of curated clinical evidence that can be used to populate those models, thus closing the LHS loop.

RESULTS/CONCLUSIONS

Six core design requirements for implementing a diagnostic LHS are identified and successfully implemented as part of this research work. A number of significant technical and policy challenges are identified for the LHS community to consider, and these are discussed in the context of evaluating this work: medico-legal responsibility for generated diagnostic evidence, developing trust in the LHS (particularly important from the perspective of decision support), and constraints imposed by clinical terminologies on evidence generation.

Data-driven knowledge acquisition, validation, and transformation into HL7 Arden Syntax

OBJECTIVE

The objective of this study is to help a team of physicians and knowledge engineers acquire clinical knowledge from existing practices datasets for treatment of head and neck cancer, to validate the knowledge against published guidelines, to create refined rules, and to incorporate these rules into clinical workflow for clinical decision support.

METHODS AND MATERIALS

A team of physicians (clinical domain experts) and knowledge engineers adapt an approach for modeling existing treatment practices into final executable clinical models. For initial work, the oral cavity is selected as the candidate target area for the creation of rules covering a treatment plan for cancer. The final executable model is presented in HL7 Arden Syntax, which helps the clinical knowledge be shared among organizations. We use a data-driven knowledge acquisition approach based on analysis of real patient datasets to generate a predictive model (PM). The PM is converted into a refined-clinical knowledge model (R-CKM), which follows a rigorous validation process. The validation process uses a clinical knowledge model (CKM), which provides the basis for defining underlying validation criteria. The R-CKM is converted into a set of medical logic modules (MLMs) and is evaluated using real patient data from a hospital information system.

RESULTS

We selected the oral cavity as the intended site for derivation of all related clinical rules for possible associated treatment plans. A team of physicians analyzed the National Comprehensive Cancer Network (NCCN) guidelines for the oral cavity and created a common CKM. Among the decision tree algorithms, chi-squared automatic interaction detection (CHAID) was applied to a refined dataset of 1229 patients to generate the PM. The PM was tested on a disjoint dataset of 739 patients, which gives 59.0% accuracy. Using a rigorous validation process, the R-CKM was created from the PM as the final model, after conforming to the CKM. The R-CKM was converted into four candidate MLMs, and was used to evaluate real data from 739 patients, yielding efficient performance with 53.0% accuracy.

CONCLUSION

Data-driven knowledge acquisition and validation against published guidelines were used to help a team of physicians and knowledge engineers create executable clinical knowledge. The advantages of the R-CKM are twofold: it reflects real practices and conforms to standard guidelines, while providing optimal accuracy comparable to that of a PM. The proposed approach yields better insight into the steps of knowledge acquisition and enhances collaboration efforts of the team of physicians and knowledge engineers.

A Framework for Usable and Effective Clinical Decision Support: Experience from the iCPR Randomized Clinical Trial

INTRODUCTION

The promise of Clinical Decision Support (CDS) has always been to transform patient care and improve patient outcomes through the delivery of timely and appropriate recommendations that are patient specific and, more often than not, are appropriately actionable. However, the users of CDS-providers-are frequently bombarded with inappropriate and inapplicable CDS that often are not informational, not integrated into the workflow, not patient specific, and that may present out of date and irrelevant recommendations.

METHODS

The integrated clinical prediction rule (iCPR) project was a randomized clinical trial (RCT) conducted to determine if a novel form of CDS, i.e., clinical prediction rules (CPRs), could be efficiently integrated into workflow and result in changes in outcomes (e.g., antibiotic ordering) when embedded within a commercial electronic health record (EHR). We use the lessons learned from the iCPR project to illustrate a framework for constructing usable, useful, and effective actionable CDS while employing off-the-shelf functionality in a production system. Innovations that make up the framework combine the following: (1) active and actionable decision support, (2) multiple rounds of usability testing with iterative development for user acceptance, (3) numerous context sensitive triggers, (4) dedicated training and support for users of the CDS tool for user adoption, and (5) support from clinical and administrative leadership. We define "context sensitive triggers" as being workflow events (i.e., context) that result in a CDS intervention.

DISCUSSION

Success of the framework can be measured by CDS adoption (i.e., intervention is being used), acceptance (compliance with recommendations), and clinical outcomes (where appropriate). This framework may have broader implications for the deployment of Health Information Technology (HIT).

RESULTS AND CONCLUSION

iCPR was well adopted(57.4% of users) and accepted (42.7% of users). Usability testing identified and fixed many issues before the iCPR RCT. The level of leadership support and clinical guidance for iCPR was key in establishing a culture of acceptance for both the tool and its recommendations contributing to adoption and acceptance. The dedicated training and support lead to the majority of the residents reporting a high level of comfort with both iCPR tools strep pharyngitis (64.4 percent) and pneumonia (62.7 percent) as well as a high likelihood of using the tools in the future. A surprising framework addition resulted from usability testing: context sensitive triggers.

Clinical Decision Support Systems (CDSS) for preventive management of COPD patients

BACKGROUND

The use of information and communication technologies to manage chronic diseases allows the application of integrated care pathways, and the optimization and standardization of care processes. Decision support tools can assist in the adherence to best-practice medicine in critical decision points during the execution of a care pathway.

OBJECTIVES

The objectives are to design, develop, and assess a clinical decision support system (CDSS) offering a suite of services for the early detection and assessment of chronic obstructive pulmonary disease (COPD), which can be easily integrated into a healthcare providers' work-flow.

METHODS

The software architecture model for the CDSS, interoperable clinical-knowledge representation, and inference engine were designed and implemented to form a base CDSS framework. The CDSS functionalities were iteratively developed through requirement-adjustment/development/validation cycles using enterprise-grade software-engineering methodologies and technologies. Within each cycle, clinical-knowledge acquisition was performed by a health-informatics engineer and a clinical-expert team.

RESULTS

A suite of decision-support web services for (i) COPD early detection and diagnosis, (ii) spirometry quality-control support, (iii) patient stratification, was deployed in a secured environment on-line. The CDSS diagnostic performance was assessed using a validation set of 323 cases with 90% specificity, and 96% sensitivity. Web services were integrated in existing health information system platforms.

CONCLUSIONS

Specialized decision support can be offered as a complementary service to existing policies of integrated care for chronic-disease management. The CDSS was able to issue recommendations that have a high degree of accuracy to support COPD case-finding. Integration into healthcare providers' work-flow can be achieved seamlessly through the use of a modular design and service-oriented architecture that connect to existing health information systems.

Investigation of Data Representation Issues in Computerizing Clinical Practice Guidelines in China

Objectives

Methods

Results

Conclusions

Enabling cross-platform clinical decision support through Web-based decision support in commercial electronic health record systems: proposal and evaluation of initial prototype implementations

Enabling clinical decision support (CDS) across multiple electronic health record (EHR) systems has been a desired but largely unattained aim of clinical informatics, especially in commercial EHR systems. A potential opportunity for enabling such scalable CDS is to leverage vendor-supported, Web-based CDS development platforms along with vendor-supported application programming interfaces (APIs). Here, we propose a potential staged approach for enabling such scalable CDS, starting with the use of custom EHR APIs and moving towards standardized EHR APIs to facilitate interoperability. We analyzed three commercial EHR systems for their capabilities to support the proposed approach, and we implemented prototypes in all three systems. Based on these analyses and prototype implementations, we conclude that the approach proposed is feasible, already supported by several major commercial EHR vendors, and potentially capable of enabling cross-platform CDS at scale.

Design of a Decision Support System for a Home Telehealth Application

A decision support system (DSS) that has been designed to manage patients using a home telehealth system is presented. The DSS has been developed to assist home telehealth clinical support staff with their workload, and to provide more effective communication between multiple home telehealth users. The three-tier system architecture that consists of a data layer; a business logic layer; and a front-end layer employs business processes and uses a rule engine for its logic and knowledge base. This paper discusses the design considerations involved in the construction of a DSS for the purpose of home telehealth, and illustrates how it may be developed using entirely open source software.

A task-based support architecture for developing point-of-care clinical decision support systems for the emergency department

OBJECTIVES

The purpose of this study was to create a task-based support architecture for developing clinical decision support systems (CDSSs) that assist physicians in making decisions at the point-of-care in the emergency department (ED). The backbone of the proposed architecture was established by a task-based emergency workflow model for a patient-physician encounter.

METHODS

The architecture was designed according to an agent-oriented paradigm. Specifically, we used the O-MaSE (Organization-based Multi-agent System Engineering) method that allows for iterative translation of functional requirements into architectural components (e.g., agents). The agent-oriented paradigm was extended with ontology-driven design to implement ontological models representing knowledge required by specific agents to operate.

RESULTS

The task-based architecture allows for the creation of a CDSS that is aligned with the task-based emergency workflow model. It facilitates decoupling of executable components (agents) from embedded domain knowledge (ontological models), thus supporting their interoperability, sharing, and reuse. The generic architecture was implemented as a pilot system, MET3-AE--a CDSS to help with the management of pediatric asthma exacerbation in the ED. The system was evaluated in a hospital ED.

CONCLUSIONS

The architecture allows for the creation of a CDSS that integrates support for all tasks from the task-based emergency workflow model, and interacts with hospital information systems. Proposed architecture also allows for reusing and sharing system components and knowledge across disease-specific CDSSs.

The Arden Syntax standard for clinical decision support: experiences and directions

Arden Syntax is a widely recognized standard for representing clinical and scientific knowledge in an executable format. It has a history that reaches back until 1989 and is currently maintained by the Health Level 7 (HL7) organization. We created a production-ready development environment, compiler, rule engine and application server for Arden Syntax. Over the course of several years, we have applied this Arden - Syntax - based CDS system in a wide variety of clinical problem domains, such as hepatitis serology interpretation, monitoring of nosocomial infections or the prediction of metastatic events in melanoma patients. We found the Arden Syntax standard to be very suitable for the practical implementation of CDS systems. Among the advantages of Arden Syntax are its status as an actively developed HL7 standard, the readability of the syntax, and various syntactic features such as flexible list handling. A major challenge we encountered was the technical integration of our CDS systems in existing, heterogeneous health information systems. To address this issue, we are currently working on incorporating the HL7 standard GELLO, which provides a standardized interface and query language for accessing data in health information systems. We hope that these planned extensions of the Arden Syntax might eventually help in realizing the vision of a global, interoperable and shared library of clinical decision support knowledge.

Construction of a drug safety assurance information system based on clinical genotyping

To capitalize on the vast potential of patient genetic information to aid in assuring drug safety, a substantial effort is needed in both the training of healthcare professionals and the operational enablement of clinical environments. Our research aims to satisfy these needs through the development of a drug safety assurance information system (GeneScription) based on clinical genotyping that utilizes patient-specific genetic information to predict and prevent adverse drug responses. In this paper, we present the motivations for this work, the algorithms at the heart of GeneScription, and a discussion of our system and its uses. We also describe our efforts to validate GeneScription through its evaluation by practicing pharmacists and pharmacy professors and its repeated use in training pharmacists. The positive assessment of the GeneScription software tool by these domain experts provides strong validation of the importance, accuracy, and effectiveness of GeneScription.

Computerization of workflows, guidelines, and care pathways: a review of implementation challenges for process-oriented health information systems

OBJECTIVE

There is a need to integrate the various theoretical frameworks and formalisms for modeling clinical guidelines, workflows, and pathways, in order to move beyond providing support for individual clinical decisions and toward the provision of process-oriented, patient-centered, health information systems (HIS). In this review, we analyze the challenges in developing process-oriented HIS that formally model guidelines, workflows, and care pathways.

METHODS

A qualitative meta-synthesis was performed on studies published in English between 1995 and 2010 that addressed the modeling process and reported the exposition of a new methodology, model, system implementation, or system architecture. Thematic analysis, principal component analysis (PCA) and data visualisation techniques were used to identify and cluster the underlying implementation 'challenge' themes.

RESULTS

One hundred and eight relevant studies were selected for review. Twenty-five underlying 'challenge' themes were identified. These were clustered into 10 distinct groups, from which a conceptual model of the implementation process was developed.

DISCUSSION AND CONCLUSION

We found that the development of systems supporting individual clinical decisions is evolving toward the implementation of adaptable care pathways on the semantic web, incorporating formal, clinical, and organizational ontologies, and the use of workflow management systems. These architectures now need to be implemented and evaluated on a wider scale within clinical settings.

Implementation of workflow engine technology to deliver basic clinical decision support functionality

Background

Workflow engine technology represents a new class of software with the ability to graphically model step-based knowledge. We present application of this novel technology to the domain of clinical decision support. Successful implementation of decision support within an electronic health record (EHR) remains an unsolved research challenge. Previous research efforts were mostly based on healthcare-specific representation standards and execution engines and did not reach wide adoption. We focus on two challenges in decision support systems: the ability to test decision logic on retrospective data prior prospective deployment and the challenge of user-friendly representation of clinical logic.

Results

We present our implementation of a workflow engine technology that addresses the two above-described challenges in delivering clinical decision support. Our system is based on a cross-industry standard of XML (extensible markup language) process definition language (XPDL). The core components of the system are a workflow editor for modeling clinical scenarios and a workflow engine for execution of those scenarios. We demonstrate, with an open-source and publicly available workflow suite, that clinical decision support logic can be executed on retrospective data. The same flowchart-based representation can also function in a prospective mode where the system can be integrated with an EHR system and respond to real-time clinical events. We limit the scope of our implementation to decision support content generation (which can be EHR system vendor independent). We do not focus on supporting complex decision support content delivery mechanisms due to lack of standardization of EHR systems in this area. We present results of our evaluation of the flowchart-based graphical notation as well as architectural evaluation of our implementation using an established evaluation framework for clinical decision support architecture.

Conclusions

We describe an implementation of a free workflow technology software suite (available at http://code.google.com/p/healthflow) and its application in the domain of clinical decision support. Our implementation seamlessly supports clinical logic testing on retrospective data and offers a user-friendly knowledge representation paradigm. With the presented software implementation, we demonstrate that workflow engine technology can provide a decision support platform which evaluates well against an established clinical decision support architecture evaluation framework. Due to cross-industry usage of workflow engine technology, we can expect significant future functionality enhancements that will further improve the technology's capacity to serve as a clinical decision support platform.

Development and evaluation of a comprehensive clinical decision support taxonomy: comparison of front-end tools in commercial and internally developed electronic health record systems

BACKGROUND

Clinical decision support (CDS) is a valuable tool for improving healthcare quality and lowering costs. However, there is no comprehensive taxonomy of types of CDS and there has been limited research on the availability of various CDS tools across current electronic health record (EHR) systems.

OBJECTIVE

To develop and validate a taxonomy of front-end CDS tools and to assess support for these tools in major commercial and internally developed EHRs.

STUDY DESIGN AND METHODS

We used a modified Delphi approach with a panel of 11 decision support experts to develop a taxonomy of 53 front-end CDS tools. Based on this taxonomy, a survey on CDS tools was sent to a purposive sample of commercial EHR vendors (n=9) and leading healthcare institutions with internally developed state-of-the-art EHRs (n=4).

RESULTS

Responses were received from all healthcare institutions and 7 of 9 EHR vendors (response rate: 85%). All 53 types of CDS tools identified in the taxonomy were found in at least one surveyed EHR system, but only 8 functions were present in all EHRs. Medication dosing support and order facilitators were the most commonly available classes of decision support, while expert systems (eg, diagnostic decision support, ventilator management suggestions) were the least common.

CONCLUSION

We developed and validated a comprehensive taxonomy of front-end CDS tools. A subsequent survey of commercial EHR vendors and leading healthcare institutions revealed a small core set of common CDS tools, but identified significant variability in the remainder of clinical decision support content.

The role of standardized data and terminological systems in computerized clinical decision support systems: literature review and survey

INTRODUCTION

Clinical decision support systems (CDSSs) should be seamlessly integrated with existing clinical information systems to enable automatic provision of advice at the time and place where decisions are made. It has been suggested that a lack of agreed data standards frequently hampers this integration. We performed a literature review to investigate whether CDSSs used standardized (i.e. coded or numerical) data and which terminological systems have been used to code data. We also investigated whether a lack of standardized data was considered an impediment for CDSS implementation.

METHODS

Articles reporting an evaluation of a CDSS that provided a computerized advice based on patient-specific data items were identified based on a former literature review on CDSS and on CDSS studies identified in AMIA's 'Year in Review'. Authors of these articles were contacted to check and complete the extracted data. A questionnaire among the authors of included studies was used to determine the obstacles in CDSS implementation.

RESULTS

We identified 77 articles published between 1995 and 2008. Twenty-two percent of the evaluated CDSSs used only numerical data. Fifty one percent of the CDSSs that used coded data applied an international terminology. The most frequently used international terminology were the ICD (International Classification of Diseases), used in 68% of the cases and LOINC (Logical Observation Identifiers Names and Codes) in 12% of the cases. More than half of the authors experienced barriers in CDSS implementation. In most cases these barriers were related to the lack of electronically available standardized data required to invoke or activate the CDSS.

CONCLUSION

Many CDSSs applied different terminological systems to code data. This diversity hampers the possibility of sharing and reasoning with data within different systems. The results of the survey confirm the hypothesis that data standardization is a critical success factor for CDSS development.

The state of the art in clinical knowledge management: an inventory of tools and techniques

PURPOSE

To explore the need for, and use of, high-quality, collaborative, clinical knowledge management (CKM) tools and techniques to manage clinical decision support (CDS) content.

METHODS

In order to better understand the current state of the art in CKM, we developed a survey of potential CKM tools and techniques. We conducted an exploratory study by querying a convenience sample of respondents about their use of specific practices in CKM.

RESULTS

The following tools and techniques should be priorities in organizations interested in developing successful computer-based provider order entry (CPOE) and CDS implementations: (1) a multidisciplinary team responsible for creating and maintaining the clinical content; (2) an external organizational repository of clinical content with web-based viewer that allows anyone in the organization to review it; (3) an online, collaborative, interactive, Internet-based tool to facilitate content development; (4) an enterprise-wide tool to maintain the controlled clinical terminology concepts. Even organizations that have been successfully using computer-based provider order entry with advanced clinical decision support features for well over 15 years are not using all of the CKM tools or practices that we identified.

CONCLUSIONS

If we are to further stimulate progress in the area of clinical decision support, we must continue to develop and refine our understanding and use of advanced CKM capabilities.

Clinical decision support capabilities of commercially-available clinical information systems

BACKGROUND

The most effective decision support systems are integrated with clinical information systems, such as inpatient and outpatient electronic health records (EHRs) and computerized provider order entry (CPOE) systems. Purpose The goal of this project was to describe and quantify the results of a study of decision support capabilities in Certification Commission for Health Information Technology (CCHIT) certified electronic health record systems.

METHODS

The authors conducted a series of interviews with representatives of nine commercially available clinical information systems, evaluating their capabilities against 42 different clinical decision support features.

RESULTS

Six of the nine reviewed systems offered all the applicable event-driven, action-oriented, real-time clinical decision support triggers required for initiating clinical decision support interventions. Five of the nine systems could access all the patient-specific data items identified as necessary. Six of the nine systems supported all the intervention types identified as necessary to allow clinical information systems to tailor their interventions based on the severity of the clinical situation and the user's workflow. Only one system supported all the offered choices identified as key to allowing physicians to take action directly from within the alert. Discussion The principal finding relates to system-by-system variability. The best system in our analysis had only a single missing feature (from 42 total) while the worst had eighteen.This dramatic variability in CDS capability among commercially available systems was unexpected and is a cause for concern.

CONCLUSIONS

These findings have implications for four distinct constituencies: purchasers of clinical information systems, developers of clinical decision support, vendors of clinical information systems and certification bodies.

SANDS: a service-oriented architecture for clinical decision support in a National Health Information Network

In this paper, we describe and evaluate a new distributed architecture for clinical decision support called SANDS (Service-oriented Architecture for NHIN Decision Support), which leverages current health information exchange efforts and is based on the principles of a service-oriented architecture. The architecture allows disparate clinical information systems and clinical decision support systems to be seamlessly integrated over a network according to a set of interfaces and protocols described in this paper. The architecture described is fully defined and developed, and six use cases have been developed and tested using a prototype electronic health record which links to one of the existing prototype National Health Information Networks (NHIN): drug interaction checking, syndromic surveillance, diagnostic decision support, inappropriate prescribing in older adults, information at the point of care and a simple personal health record. Some of these use cases utilize existing decision support systems, which are either commercially or freely available at present, and developed outside of the SANDS project, while other use cases are based on decision support systems developed specifically for the project. Open source code for many of these components is available, and an open source reference parser is also available for comparison and testing of other clinical information systems and clinical decision support systems that wish to implement the SANDS architecture. The SANDS architecture for decision support has several significant advantages over other architectures for clinical decision support. The most salient of these are: