Conceptual alignment of electronic health record data with guideline and workflow knowledge

A Community-of-Practice-based Evaluation Methodology for Knowledge Intensive Computational Methods and its Application to Multimorbidity Decision Support

OBJECTIVE

The study has dual objectives. Our first objective (1) is to develop a community-of-practice-based evaluation methodology for knowledge-intensive computational methods. We target a whitebox analysis of the computational methods to gain insight on their functional features and inner workings. In more detail, we aim to answer evaluation questions on (i) support offered by computational methods for functional features within the application domain; and (ii) in-depth characterizations of the underlying computational processes, models, data and knowledge of the computational methods. Our second objective (2) involves applying the evaluation methodology to answer questions (i) and (ii) for knowledge-intensive clinical decision support (CDS) methods, which operationalize clinical knowledge as computer interpretable guidelines (CIG); we focus on multimorbidity CIG-based clinical decision support (MGCDS) methods that target multimorbidity treatment plans.

MATERIALS AND METHODS

Our methodology directly involves the research community of practice in (a) identifying functional features within the application domain; (b) defining exemplar case studies covering these features; and (c) solving the case studies using their developed computational methods-research groups detail their solutions and functional feature support in solution reports. Next, the study authors (d) perform a qualitative analysis of the solution reports, identifying and characterizing common themes (or dimensions) among the computational methods. This methodology is well suited to perform whitebox analysis, as it directly involves the respective developers in studying inner workings and feature support of computational methods. Moreover, the established evaluation parameters (e.g., features, case studies, themes) constitute a re-usable benchmark framework, which can be used to evaluate new computational methods as they are developed. We applied our community-of-practice-based evaluation methodology on MGCDS methods.

RESULTS

Six research groups submitted comprehensive solution reports for the exemplar case studies. Solutions for two of these case studies were reported by all groups. We identified four evaluation dimensions: detection of adverse interactions, management strategy representation, implementation paradigms, and human-in-the-loop support.Based on our whitebox analysis, we present answers to the evaluation questions (i) and (ii) for MGCDS methods.

DISCUSSION

The proposed evaluation methodology includes features of illuminative and comparison-based approaches; focusing on understanding rather than judging/scoring or identifying gaps in current methods. It involves answering evaluation questions with direct involvement of the research community of practice, who participate in setting up evaluation parameters and solving exemplar case studies. Our methodology was successfully applied to evaluate six MGCDS knowledge-intensive computational methods. We established that, while the evaluated methods provide a multifaceted set of solutions with different benefits and drawbacks, no single MGCDS method currently provides a comprehensive solution for MGCDS.

CONCLUSION

We posit that our evaluation methodology, applied here to gain new insights into MGCDS, can be used to assess other types of knowledge-intensive computational methods and answer other types of evaluation questions. Our case studies can be accessed at our GitHub repository (https://github.com/william-vw/MGCDS).

What Role Can Process Mining Play in Recurrent Clinical Guidelines Issues? A Position Paper

In the age of Evidence-Based Medicine, Clinical Guidelines (CGs) are recognized to be an indispensable tool to support physicians in their daily clinical practice. Medical Informatics is expected to play a relevant role in facilitating diffusion and adoption of CGs. However, the past pioneering approaches, often fragmented in many disciplines, did not lead to solutions that are actually exploited in hospitals. Process Mining for Healthcare (PM4HC) is an emerging discipline gaining the interest of healthcare experts, and seems able to deal with many important issues in representing CGs. In this position paper, we briefly describe the story and the state-of-the-art of CGs, and the efforts and results of the past approaches of medical informatics. Then, we describe PM4HC, and we answer questions like how can PM4HC cope with this challenge? Which role does PM4HC play and which rules should be employed for the PM4HC scientific community?

Computer-interpretable clinical guidelines: a methodological review

Clinical practice guidelines (CPGs) aim to improve the quality of care, reduce unjustified practice variations and reduce healthcare costs. In order for them to be effective, clinical guidelines need to be integrated with the care flow and provide patient-specific advice when and where needed. Hence, their formalization as computer-interpretable guidelines (CIGs) makes it possible to develop CIG-based decision-support systems (DSSs), which have a better chance of impacting clinician behavior than narrative guidelines. This paper reviews the literature on CIG-related methodologies since the inception of CIGs, while focusing and drawing themes for classifying CIG research from CIG-related publications in the Journal of Biomedical Informatics (JBI). The themes span the entire life-cycle of CIG development and include: knowledge acquisition and specification for improved CIG design, including (1) CIG modeling languages and (2) CIG acquisition and specification methodologies, (3) integration of CIGs with electronic health records (EHRs) and organizational workflow, (4) CIG validation and verification, (5) CIG execution engines and supportive tools, (6) exception handling in CIGs, (7) CIG maintenance, including analyzing clinician's compliance to CIG recommendations and CIG versioning and evolution, and finally (8) CIG sharing. I examine the temporal trends in CIG-related research and discuss additional themes that were not identified in JBI papers, including existing themes such as overcoming implementation barriers, modeling clinical goals, and temporal expressions, as well as futuristic themes, such as patient-centric CIGs and distributed CIGs.

A healthcare management system for Turkey based on a service-oriented architecture

The current Turkish healthcare management system has a structure that is extremely inordinate, cumbersome and inflexible. Furthermore, this structure has no common point of view and thus has no interoperability and responds slowly to innovations. The purpose of this study is to show that using which methods can the Turkish healthcare management system provide a structure that could be more modern, more flexible and more quick to respond to innovations and changes taking advantage of the benefits given by a service-oriented architecture (SOA). In this paper, the Turkish healthcare management system is chosen to be examined since Turkey is considered as one of the Third World countries and the information architecture of the existing healthcare management system of Turkey has not yet been configured with SOA, which is a contemporary innovative approach and should provide the base architecture of the new solution. The innovation of this study is the symbiosis of two main integration approaches, SOA and Health Level 7 (HL7), for integrating divergent healthcare information systems. A model is developed which is based on SOA and enables obtaining a healthcare management system having the SSF standards (HSSP Service Specification Framework) developed by the framework of the HSSP (Healthcare Services Specification Project) under the leadership of HL7 and the Object Management Group.

Computerization of workflows, guidelines, and care pathways: a review of implementation challenges for process-oriented health information systems

OBJECTIVE

There is a need to integrate the various theoretical frameworks and formalisms for modeling clinical guidelines, workflows, and pathways, in order to move beyond providing support for individual clinical decisions and toward the provision of process-oriented, patient-centered, health information systems (HIS). In this review, we analyze the challenges in developing process-oriented HIS that formally model guidelines, workflows, and care pathways.

METHODS

A qualitative meta-synthesis was performed on studies published in English between 1995 and 2010 that addressed the modeling process and reported the exposition of a new methodology, model, system implementation, or system architecture. Thematic analysis, principal component analysis (PCA) and data visualisation techniques were used to identify and cluster the underlying implementation 'challenge' themes.

RESULTS

One hundred and eight relevant studies were selected for review. Twenty-five underlying 'challenge' themes were identified. These were clustered into 10 distinct groups, from which a conceptual model of the implementation process was developed.

DISCUSSION AND CONCLUSION

We found that the development of systems supporting individual clinical decisions is evolving toward the implementation of adaptable care pathways on the semantic web, incorporating formal, clinical, and organizational ontologies, and the use of workflow management systems. These architectures now need to be implemented and evaluated on a wider scale within clinical settings.

Facilitating pre-operative assessment guidelines representation using SNOMED CT

OBJECTIVE

To investigate whether SNOMED CT covers the terms used in pre-operative assessment guidelines, and if necessary, how the measured content coverage can be improved.

METHODS

Pre-operative assessment guidelines were retrieved from the websites of (inter)national anesthesia-related societies. The recommendations in the guidelines were rewritten to "IF condition THEN action" statements to facilitate data extraction. Terms were extracted from the IF-THEN statements and mapped to SNOMED CT. Content coverage was measured by using three scores: no match, partial match and complete match. Non-covered concepts were evaluated against the SNOMED CT editorial documentation.

RESULTS

From 6 guidelines, 133 terms were extracted, of which 71% (n=94) completely matched with SNOMED CT concepts. Disregarding the vague concepts in the included guidelines SNOMED CT's content coverage was 89%. Of the 39 non-completely covered concepts, 69% violated at least one of SNOMED CT's editorial principles or rules. These concepts were categorized based on four categories: non-reproducibility, classification-derived phrases, numeric ranges, and procedures categorized by complexity.

CONCLUSION

Guidelines include vague terms that cannot be well supported by terminological systems thereby hampering guideline-based decision support systems. This vagueness reduces the content coverage of SNOMED CT in representing concepts used in the pre-operative assessment guidelines. Formalization of the guidelines using SNOMED CT is feasible but to optimize this, first the vagueness of some guideline concepts should be resolved and a few currently missing but relevant concepts should be added to SNOMED CT.

Structured product labeling improves detection of drug-intolerance issues

OBJECTIVES

This study sought to assess the value of the Health Level 7/U.S. Food and Drug Administration Structured Product Labeling (SPL) drug knowledge representation standard and its associated terminology sources for drug-intolerance (allergy) decision support in computerized provider order entry (CPOE) systems.

DESIGN

The Regenstrief Institute CPOE drug-intolerance issue detection system and its knowledge base was compared with a method based on existing SPL label content enriched with knowledge sources used with SPL (NDF-RT/MeSH). Both methods were applied to a large set of drug-intolerance (allergy) records, drug orders, and medication dispensing records covering >50,000 patients over 30 years.

MEASUREMENTS

The number of drug-intolerance issues detected by both methods was counted, as well as the number of patients with issues, number of distinct drugs, and number of distinct intolerances. The difference between drug-intolerance issues detected or missed by either method was qualitatively analyzed.

RESULTS

Although <70% of terms were mapped to SPL, the new approach detected four times as many drug-intolerance issues on twice as many patients.

CONCLUSION

The SPL-based approach is more sensitive and suggests that mapping local dictionaries to SPL, and enhancing the depth and breadth of coverage of SPL content are worth accelerating. The study also highlights specificity problems known to trouble drug-intolerance decision support and suggests how terminology and methods of recording drug intolerances could be improved.

Computer-based execution of clinical guidelines: a review

PURPOSE

Clinical guidelines are useful tools to standardize and improve health care. The automation of the guideline execution process is a basic step towards its widespread use in medical centres. This paper presents an analysis and a comparison of eight systems that allow the enactment of clinical guidelines in a (semi) automatic fashion.

METHODS

This paper presents a review of the literature (2000-2007) collected from medical databases as well as international conferences in the medical informatics area.

RESULTS

Eight systems containing a guideline execution engine were selected. The language used to represent the guidelines as well as the architecture of these systems were compared. Different aspects have been assessed for each system, such as the integration with external elements or the coordination mechanisms used in the execution of clinical guidelines. Security and terminology issues complement the above study.

CONCLUSIONS

Although these systems could be beneficial for clinicians and patients, it is an ongoing research area, and they are not yet fully implemented and integrated into existing careflow management systems and hence used in daily practice in health care institutions.

IT support for clinical pathways—Lessons learned

Clinical pathways are an effective instrument to decrease undesired practice variability and improve clinician performance. IT-applications embedded into clinical routine work can help to increase pathway compliance. Successfully implementing such applications requires both a responsive IT infrastructure and a participatory and iterative design process aimed at achieving user acceptance and usability. Experiences from the implementation and iterative improvement of an online surgical pathway at Marburg University Medical Centre have shown that pathway conformance actually could be improved by the use of IT. An analysis of the iterative design process has shown that future pathway projects can benefit from the lessons learned during this project. Based on these lessons recommendations for developing well adapted interaction mechanisms are presented, aimed at improving process alignment. Our goal is to build up a library of tested reusable components to reduce the number of iterations for pathway implementation.

Runtime application of Hybrid-Asbru clinical guidelines

Clinical guidelines are a major tool in improving the quality of medical care. However, to support the automation of guideline-based care, several requirements must be filled, such as specification of the guidelines in a machine-interpretable format and a connection to an Electronic Patient Record (EPR). For several different reasons, it is beneficial to convert free-text guidelines gradually, through several intermediate representations, to a machine-interpretable format. It is also realistic to consider the case when an EPR is unavailable. We propose an innovative approach to the runtime application of intermediate-represented Hybrid-Asbru guidelines, with or without an available EPR. The new approach capitalizes on our extensive work on developing the Digital electronic Guideline Library (DeGeL) framework. The new approach was implemented as the Spock system. For evaluation, three guidelines were specified in an intermediate format and were applied to a set of simulated patient records designed to cover prototypical cases. In all cases, the Spock system produced the expected output, and did not produce an unexpected one. Thus, we have demonstrated the capability of the Spock system to apply guidelines encoded in the Hybrid-Asbru intermediate representation, when an EPR is not available.

Design and implementation of the GLIF3 guideline execution engine

We have developed the GLIF3 Guideline Execution Engine (GLEE) as a tool for executing guidelines encoded in the GLIF3 format. In addition to serving as an interface to the GLIF3 guideline representation model to support the specified functions, GLEE provides defined interfaces to electronic medical records (EMRs) and other clinical applications to facilitate its integration with the clinical information system at a local institution. The execution model of GLEE takes the "system suggests, user controls" approach. A tracing system is used to record an individual patient's state when a guideline is applied to that patient. GLEE can also support an event-driven execution model once it is linked to the clinical event monitor in a local environment. Evaluation has shown that GLEE can be used effectively for proper execution of guidelines encoded in the GLIF3 format. When using it to execute each guideline in the evaluation, GLEE's performance duplicated that of the reference systems implementing the same guideline but taking different approaches. The execution flexibility and generality provided by GLEE, and its integration with a local environment, need to be further evaluated in clinical settings. Integration of GLEE with a specific event-monitoring and order-entry environment is the next step of our work to demonstrate its use for clinical decision support. Potential uses of GLEE also include quality assurance, guideline development, and medical education.

Specifying workflow process requirements for an emergency medical service

Recent trends in healthcare delivery have led to a gradual shift in the conceptualisation of healthcare information systems towards supporting healthcare processes in a more direct way. The move towards integrated and managed care, which requires designing healthcare processes around patient needs and incorporating efficiency considerations has led to an increased interest in process-oriented healthcare information systems based on workflow technology. This means to actively deliver the tasks to be performed to the right persons at the right time with the necessary information and the application functions needed. Moreover, workflow technology promotes a component-oriented development whereby the process logic is separated from application logic. This paper presents an approach to capturing process logic requirements for healthcare workflow systems with a view to design a system that is easily adjustable to process changes and to evolving organizational structures at a reasonable cost.