Computer-interpretable clinical guidelines: a methodological review

Provision and evaluation of explanations within an automated planning-based approach to solving the multimorbidity problem

The multimorbidity problem involves the identification and mitigation of adverse interactions that occur when multiple computer interpretable guidelines are applied concurrently to develop a treatment plan for a patient diagnosed with multiple diseases. Solving this problem requires decision support approaches which are difficult to comprehend for physicians. As such, the rationale for treatment plans generated by these approaches needs to be provided.

OBJECTIVE

To develop an explainability component for an automated planning-based approach to the multimorbidity problem, and to assess the fidelity and interpretability of generated explanations using a clinical case study.

METHODS

The explainability component leverages the task-network model for representing computer interpretable guidelines. It generates post-hoc explanations composed of three aspects that answer why specific clinical actions are in a treatment plan, why specific revisions were applied, and how factors like medication cost, patient's adherence, etc. influence the selection of specific actions. The explainability component is implemented as part of MitPlan, where we revised our planning-based approach to support explainability. We developed an evaluation instrument based on the system causability scale and other vetted surveys to evaluate the fidelity and interpretability of its explanations using a two dimensional comparison study design.

RESULTS

The explainability component was implemented for MitPlan and tested in the context of a clinical case study. The fidelity and interpretability of the generated explanations were assessed using a physician-focused evaluation study involving 21 participants from two different specialties and two levels of experience. Results show that explanations provided by the explainability component in MitPlan are of acceptable fidelity and interpretability, and that the clinical justification of the actions in a treatment plan is important to physicians.

CONCLUSION

We created an explainability component that enriches an automated planning-based approach to solving the multimorbidity problem with meaningful explanations for actions in a treatment plan. This component relies on the task-network model to represent computer interpretable guidelines and as such can be ported to other approaches that also use the task-network model representation. Our evaluation study demonstrated that explanations that support a physician's understanding of the clinical reasons for the actions in a treatment plan are useful and important.

Making Science Computable Using Evidence-Based Medicine on Fast Healthcare Interoperability Resources: Standards Development Project

BACKGROUND

Evidence-based medicine (EBM) has the potential to improve health outcomes, but EBM has not been widely integrated into the systems used for research or clinical decision-making. There has not been a scalable and reusable computer-readable standard for distributing research results and synthesized evidence among creators, implementers, and the ultimate users of that evidence. Evidence that is more rapidly updated, synthesized, disseminated, and implemented would improve both the delivery of EBM and evidence-based health care policy.

OBJECTIVE

This study aimed to introduce the EBM on Fast Healthcare Interoperability Resources (FHIR) project (EBMonFHIR), which is extending the methods and infrastructure of Health Level Seven (HL7) FHIR to provide an interoperability standard for the electronic exchange of health-related scientific knowledge.

METHODS

As an ongoing process, the project creates and refines FHIR resources to represent evidence from clinical studies and syntheses of those studies and develops tools to assist with the creation and visualization of FHIR resources.

RESULTS

The EBMonFHIR project created FHIR resources (ie, ArtifactAssessment, Citation, Evidence, EvidenceReport, and EvidenceVariable) for representing evidence. The COVID-19 Knowledge Accelerator (COKA) project, now Health Evidence Knowledge Accelerator (HEvKA), took this work further and created FHIR resources that express EvidenceReport, Citation, and ArtifactAssessment concepts. The group is (1) continually refining FHIR resources to support the representation of EBM; (2) developing controlled terminology related to EBM (ie, study design, statistic type, statistical model, and risk of bias); and (3) developing tools to facilitate the visualization and data entry of EBM information into FHIR resources, including human-readable interfaces and JSON viewers.

CONCLUSIONS

EBMonFHIR resources in conjunction with other FHIR resources can support relaying EBM components in a manner that is interoperable and consumable by downstream tools and health information technology systems to support the users of evidence.

Use of Real-World FHIR Data Combined with Context-Sensitive Decision Modeling to Guide Sentinel Biopsy in Melanoma

Background: To support clinical decision-making at the point of care, the "best next step" based on Standard Operating Procedures (SOPs) and actual accurate patient data must be provided. To do this, textual SOPs have to be transformed into operable clinical algorithms and linked to the data of the patient being treated. For this linkage, we need to know exactly which data are needed by clinicians at a certain decision point and whether these data are available. These data might be identical to the data used within the SOP or might integrate a broader view. To address these concerns, we examined if the data used by the SOP is also complete from the point of view of physicians for contextual decision-making. Methods: We selected a cohort of 67 patients with stage III melanoma who had undergone adjuvant treatment and mainly had an indication for a sentinel biopsy. First, we performed a step-by-step simulation of the patient treatment along our clinical algorithm, which is based on a hospital-specific SOP, to validate the algorithm with the given Fast Healthcare Interoperability Resources (FHIR)-based data of our cohort. Second, we presented three different decision situations within our algorithm to 10 dermatooncologists, focusing on the concrete patient data used at this decision point. The results were conducted, analyzed, and compared with those of the pure algorithmic simulation. Results: The treatment paths of patients with melanoma could be retrospectively simulated along the clinical algorithm using data from the patients' electronic health records. The subsequent evaluation by dermatooncologists showed that the data used at the three decision points had a completeness between 84.6% and 100.0% compared with the data used by the SOP. At one decision point, data on "patient age (at primary diagnosis)" and "date of first diagnosis" were missing. Conclusions: The data needed for our decision points are available in the FHIR-based dataset. Furthermore, the data used at decision points by the SOP and hence the clinical algorithm are nearly complete compared with the data required by physicians in clinical practice. This is an important precondition for further research focusing on presenting decision points within a treatment process integrated with the patient data needed.

ROAD2H: Development and evaluation of an open-source explainable artificial intelligence approach for managing co-morbidity and clinical guidelines

Introduction

Clinical decision support (CDS) systems (CDSSs) that integrate clinical guidelines need to reflect real-world co-morbidity. In patient-specific clinical contexts, transparent recommendations that allow for contraindications and other conflicts arising from co-morbidity are a requirement. In this work, we develop and evaluate a non-proprietary, standards-based approach to the deployment of computable guidelines with explainable argumentation, integrated with a commercial electronic health record (EHR) system in Serbia, a middle-income country in West Balkans.

Methods

We used an ontological framework, the Transition-based Medical Recommendation (TMR) model, to represent, and reason about, guideline concepts, and chose the 2017 International global initiative for chronic obstructive lung disease (GOLD) guideline and a Serbian hospital as the deployment and evaluation site, respectively. To mitigate potential guideline conflicts, we used a TMR-based implementation of the Assumptions-Based Argumentation framework extended with preferences and Goals (ABA+G). Remote EHR integration of computable guidelines was via a microservice architecture based on HL7 FHIR and CDS Hooks. A prototype integration was developed to manage chronic obstructive pulmonary disease (COPD) with comorbid cardiovascular or chronic kidney diseases, and a mixed-methods evaluation was conducted with 20 simulated cases and five pulmonologists.

Results

Pulmonologists agreed 97% of the time with the GOLD-based COPD symptom severity assessment assigned to each patient by the CDSS, and 98% of the time with one of the proposed COPD care plans. Comments were favourable on the principles of explainable argumentation; inclusion of additional co-morbidities was suggested in the future along with customisation of the level of explanation with expertise.

Conclusion

An ontological model provided a flexible means of providing argumentation and explainable artificial intelligence for a long-term condition. Extension to other guidelines and multiple co-morbidities is needed to test the approach further.

Evaluation of clinical practice guideline-derived clinical decision support systems using a novel quality model

Over the last decade, clinical practice guidelines (CPGs) have become an important asset for daily life in healthcare organizations. Efficient management and digitization of CPGs help achieve organizational objectives and improve patient care and healthcare quality by reducing variability. However, digitizing CPGs is a difficult, complex task because they are usually expressed as text, and this often leads to the development of partial software solutions. At present, different research proposals and CPG-derived CDSS (clinical decision support system) do exist for managing CPG digitalization lifecycles (from modeling to deployment and execution), but they do not all provide full lifecycle support, making it more difficult to choose solutions or proposals that fully meet the needs of a healthcare organization. This paper proposes a method based on quality models to uniformly compare and evaluate technological tools, providing a rigorous method that uses qualitative and quantitative analysis of technological aspects. In addition, this paper also presents how this method has been instantiated to evaluate and compare CPG-derived CDSS by highlighting each phase of the CPG digitization lifecycle. Finally, discussion and analysis of currently available tools are presented, identifying gaps and limitations.

The Application of Computer Technology to Clinical Practice Guideline Implementation: A Scoping Review

Implementation of clinical practice guidelines (CPG) is a complex and challenging task. Computer technology, including artificial intelligence (AI), has been explored to promote the CPG implementation. This study has reviewed the main domains where computer technology and AI has been applied to CPG implementation. PubMed, Embase, Web of science, the Cochrane Library, China National Knowledge Infrastructure database, WanFang DATA, VIP database, and China Biology Medicine disc database were searched from inception to December 2021. Studies involving the utilization of computer technology and AI to promote the implementation of CPGs were eligible for review. A total of 10429 published articles were identified, 117 met the inclusion criteria. 21 (17.9%) focused on the utilization of AI techniques to classify or extract the relative content of CPGs, such as recommendation sentence, condition-action sentences. 47 (40.2%) focused on the utilization of computer technology to represent guideline knowledge to make it understandable by computer. 15 (12.8%) focused on the utilization of AI techniques to verify the relative content of CPGs, such as conciliation of multiple single-disease guidelines for comorbid patients. 34 (29.1%) focused on the utilization of AI techniques to integrate guideline knowledge into different resources, such as clinical decision support systems. We conclude that the application of computer technology and AI to CPG implementation mainly concentrated on the guideline content classification and extraction, guideline knowledge representation, guideline knowledge verification, and guideline knowledge integration. The AI methods used for guideline content classification and extraction were pattern-based algorithm and machine learning. In guideline knowledge representation, guideline knowledge verification, and guideline knowledge integration, computer techniques of knowledge representation were the most used.

A novel method for continuous measurements of clinical practice guideline adherence

Introduction

Clinical practice guidelines (hereafter 'guidelines') are crucial in providing evidence-based recommendations for physicians and multidisciplinary teams to make informed decisions regarding diagnostics and treatment in various diseases, including cancer. While guideline implementation has been shown to reduce (unwanted) variability and improve outcome of care, monitoring of adherence to guidelines remains challenging. Real-world data collected from cancer registries can provide a continuous source for monitoring adherence levels. In this work, we describe a novel structured approach to guideline evaluation using real-world data that enables continuous monitoring. This method was applied to endometrial cancer patients in the Netherlands and implemented through a prototype web-based dashboard that enables interactive usage and supports various analyses.

Method

The guideline under study was parsed into clinical decision trees (CDTs) and an information standard was drawn up. A dataset from the Netherlands Cancer Registry (NCR) was used and data items from both instruments were mapped. By comparing guideline recommendations with real-world data an adherence classification was determined. The developed prototype can be used to identify and prioritize potential topics for guideline updates.

Results

CDTs revealed 68 data items for recording in an information standard. Thirty-two data items from the NCR were mapped onto information standard data items. Four CDTs could sufficiently be populated with NCR data.

Conclusion

The developed methodology can evaluate a guideline to identify potential improvements in recommendations and the success of the implementation strategy. In addition, it is able to identify patient and disease characteristics that influence decision-making in clinical practice. The method supports a cyclical process of developing, implementing and evaluating guidelines and can be scaled to other diseases and settings. It contributes to a learning healthcare cycle that integrates real-world data with external knowledge.

Modelling clinical narrative as computable knowledge: The NICE computable implementation guidance project

Introduction

Translating narrative clinical guidelines to computable knowledge is a long-standing challenge that has seen a diverse range of approaches. The UK National Institute for Health and Care Excellence (NICE) Content Advisory Board (CAB) aims ultimately to (1) guide clinical decision support and other software developers to increase traceability, fidelity and consistency in supporting clinical use of NICE recommendations, (2) guide local practice audit and intervention to reduce unwarranted variation, (3) provide feedback to NICE on how future recommendations should be developed.

Objectives

The first phase of work was to explore a range of technical approaches to transition NICE toward the production of natively digital content.

Methods

Following an initial 'collaborathon' in November 2022, the NICE Computable Implementation Guidance project (NCIG) was established. We held a series of workstream calls approximately fortnightly, focusing on (1) user stories and trigger events, (2) information model and definitions, (3) horizon-scanning and output format. A second collaborathon was held in March 2023 to consolidate progress across the workstreams and agree residual actions to complete.

Results

While we initially focussed on technical implementation standards, we decided that an intermediate logical model was a more achievable first step in the journey from narrative to fully computable representation. NCIG adopted the WHO Digital Adaptation Kit (DAK) as a technology-agnostic method to model user scenarios, personae, processes and workflow, core data elements and decision-support logic. Further work will address indicators, such as prescribing compliance, and implementation in document templates for primary care patient record systems.

Conclusions

The project has shown that the WHO DAK, with some modification, is a promising approach to build technology-neutral logical specifications of NICE recommendations. Implementation of concurrent computable modelling by multidisciplinary teams during guideline development poses methodological and cultural questions that are complex but tractable given suitable will and leadership.

A prospective observational concordance study to evaluate computational model-driven clinical practice guidelines for Type 2 diabetes mellitus

BACKGROUND

Clinical Practice Guidelines (CPGs) provide healthcare professionals with performance and decision-making support during the treatment of patients. Sometimes, however, they are poorly implemented. The IDE4ICDS platform was developed and validated with CPGs for type 2 diabetes mellitus (T2DM).

OBJECTIVE

The main objective of this paper is to present the results of the clinical validation of the IDE4ICDS platform in a real clinical environment at two health clinics in the Andalusian Public Health System (SSPA) in the southern Spanish region of Andalusia.

METHODS

National and international knowledge sources on T2DM were selected and reviewed and used to define a diabetes CPG model on the IDE4ICDS platform. Once the diabetes CPG was configured and deployed, it was validated. A total of 506 patients were identified as meeting the inclusion criteria, of whom 130 could be recruited and 89 attended the appointment.

RESULTS

A concordance analysis was performed with the kappa value. Overall agreement between the recommendations provided by the system and those recorded in each patient's EHR was good (0.61 - 0.80) with a total kappa index of 0.701, leading to the conclusion that the system provided appropriate recommendations for each patient and was therefore well-functioning.

CONCLUSIONS

A series of possible improvements were identified based on the limitations for the recovery of variables related to the quality of these recolected variables, the detection of duplicate recommendations based on different input variables for the same patient, and clinical usability, such as the capacity to generate reports based on the recommendations generated. Nevertheless, the project resulted in the IDE4ICDS platform: a Clinical Decision Support System (CDSS) capable of providing appropriate recommendations for improving the management and quality of patient care and optimizing health outcomes. The result of this validation is a safe and effective pathway for developing and adopting digital transformation at the regional scale of the use of biomedical knowledge in real healthcare.

An Evaluation Framework for a Novel Process to Codevelop Written and Computable Guidelines

Clinical practice guidelines (CPGs) support individual and population health by translating new, evidence-based knowledge into recommendations for health practice. CPGs can be provided as computable, machine-readable guidelines that support the translation of recommendations into shareable, interoperable clinical decision support and other digital tools (eg, quality measures, case reports, care plans). Interdisciplinary collaboration among guideline developers and health information technology experts can facilitate the translation of written guidelines into computable ones. The benefits of interdisciplinary work include a focus on the needs of end-users who apply guidelines in practice through clinic decision support systems as part of the Centers for Disease Control and Prevention's (CDC's) Adapting Clinical Guidelines for the Digital Age (ACG) initiative, a group of interdisciplinary experts proposed a process to facilitate the codevelopment of written and computable CPGs, referred to as the "integrated process (IP)."1 This paper presents a framework for evaluating the IP based on a combination of vetted evaluation models and expert opinions. This framework combines 3 types of evaluations: process, product, and outcomes. These evaluations assess the value of interdisciplinary expert collaboration in carrying out the IP, the quality, usefulness, timeliness, and acceptance of the guideline, and the guideline's health impact, respectively. A case study is presented that illustrates application of the framework.

An Integrated Process for Co-Developing and Implementing Written and Computable Clinical Practice Guidelines

The goal of this article is to describe an integrated parallel process for the co-development of written and computable clinical practice guidelines (CPGs) to accelerate adoption and increase the impact of guideline recommendations in clinical practice. From February 2018 through December 2021, interdisciplinary work groups were formed after an initial Kaizen event and using expert consensus and available literature, produced a 12-phase integrated process (IP). The IP includes activities, resources, and iterative feedback loops for developing, implementing, disseminating, communicating, and evaluating CPGs. The IP incorporates guideline standards and informatics practices and clarifies how informaticians, implementers, health communicators, evaluators, and clinicians can help guideline developers throughout the development and implementation cycle to effectively co-develop written and computable guidelines. More efficient processes are essential to create actionable CPGs, disseminate and communicate recommendations to clinical end users, and evaluate CPG performance. Pilot testing is underway to determine how this IP expedites the implementation of CPGs into clinical practice and improves guideline uptake and health outcomes.

Computer-interpretable guidelines: electronic tools to enhance the utility of thyroid nodule clinical practice guidelines and risk stratification tools

Clinicians seeking guidance for evaluating and managing thyroid nodules currently have several resources. The principal ones are narrative clinical guidelines and clinical risk calculators. This paper will review the strengths and weaknesses of both. The paper will introduce a concept of computer interpretable guideline, a novel way of transforming narrative guidelines in to a clinical decision support tool that can provide patient specific recommendations at the point of care. The paper then describes an experience of developing an interactive web based computer interpretable guideline for thyroid nodule management, called Thyroid Nodule Management App (TNAPP). The advantages of this approach and the potential barriers for widespread adaptation are discussed.

Using graph rewriting to operationalize medical knowledge for the revision of concurrently applied clinical practice guidelines

Clinical practice guidelines (CPGs) are patient management tools that synthesize medical knowledge into an actionable format. CPGs are disease specific with limited applicability to the management of complex patients suffering from multimorbidity. For the management of these patients, CPGs need to be augmented with secondary medical knowledge coming from a variety of knowledge repositories. The operationalization of this knowledge is key to increasing CPGs' uptake in clinical practice. In this work, we propose an approach to operationalizing secondary medical knowledge inspired by graph rewriting. We assume that the CPGs can be represented as task network models, and provide an approach for representing and applying codified medical knowledge to a specific patient encounter. We formally define revisions that model and mitigate adverse interactions between CPGs and we use a vocabulary of terms to instantiate these revisions. We demonstrate the application of our approach using synthetic and clinical examples. We conclude by identifying areas for future work with the vision of developing a theory of mitigation that will facilitate the development of comprehensive decision support for the management of multimorbid patients.

Pros and cons of streamlining and use of computerised clinical decision support systems to future-proof oncological multidisciplinary team meetings

Introduction

Nowadays nearly every patient with cancer is discussed in a multidisciplinary team meeting (MDTM) to determine an optimal treatment plan. The growth in the number of patients to be discussed is unsustainable. Streamlining and use of computerised clinical decision support systems (CCDSSs) are two major ways to restructure MDTMs. Streamlining is the process of selecting the patients who need to be discussed and in which type of MDTM. Using CCDSSs, patient data is automatically loaded into the minutes and a guideline-based treatment proposal is generated. We aimed to identify the pros and cons of streamlining and CCDSSs.

Methods

Semi-structured interviews were conducted with Dutch MDTM participants. With purposive sampling we maximised variation in participants' characteristics. Interview data were thematically analysed.

Results

Thirty-five interviews were analysed. All interviewees agreed on the need to change the current MDTM workflow. Streamlining suggestions were thematised based on standard and complex cases and the location of the MDTM (i.e. local, regional or nationwide). Interviewees suggested easing the pressure on MDTMs by discussing standard cases briefly, not at all, or outside the MDTM with only two to three specialists. Complex cases should be discussed in tumour-type-specific regional MDTMs and highly complex cases by regional/nationwide expert teams. Categorizing patients as standard or complex was found to be the greatest challenge of streamlining. CCDSSs were recognised as promising, although none of the interviewees had made use of them. The assumed advantage was their capacity to generate protocolised treatment proposals based on automatically uploaded patient data, to unify treatment proposals and to facilitate research. However, they were thought to limit the freedom to deviate from the treatment advice.

Conclusion

To make oncological MDTMs sustainable, methods of streamlining should be developed and introduced. Physicians still have doubts about the value of CCDSSs.

A process mining approach for clinical guidelines compliance: real-world application in rectal cancer

In the era of evidence-based medicine, several clinical guidelines were developed, supporting cancer management from diagnosis to treatment and aiming to optimize patient care and hospital resources. Nevertheless, individual patient characteristics and organizational factors may lead to deviations from these standard recommendations during clinical practice. In this context, process mining in healthcare constitutes a valid tool to evaluate conformance of real treatment pathways, extracted from hospital data warehouses as event log, to standard clinical guidelines, translated into computer-interpretable formats. In this study we translate the European Society of Medical Oncology guidelines for rectal cancer treatment into a computer-interpretable format using Pseudo-Workflow formalism (PWF), a language already employed in pMineR software library for Process Mining in Healthcare. We investigate the adherence of a real-world cohort of rectal cancer patients treated at Fondazione Policlinico Universitario A. Gemelli IRCCS, data associated with cancer diagnosis and treatment are extracted from hospital databases in 453 patients diagnosed with rectal cancer. PWF enables the easy implementation of guidelines in a computer-interpretable format and visualizations that can improve understandability and interpretability of physicians. Results of the conformance checking analysis on our cohort identify a subgroup of patients receiving a long course treatment that deviates from guidelines due to a moderate increase in radiotherapy dose and an addition of oxaliplatin during chemotherapy treatment. This study demonstrates the importance of PWF to evaluate clinical guidelines adherence and to identify reasons of deviations during a treatment process in a real-world and multidisciplinary setting.

A Community-of-Practice-based Evaluation Methodology for Knowledge Intensive Computational Methods and its Application to Multimorbidity Decision Support

OBJECTIVE

The study has dual objectives. Our first objective (1) is to develop a community-of-practice-based evaluation methodology for knowledge-intensive computational methods. We target a whitebox analysis of the computational methods to gain insight on their functional features and inner workings. In more detail, we aim to answer evaluation questions on (i) support offered by computational methods for functional features within the application domain; and (ii) in-depth characterizations of the underlying computational processes, models, data and knowledge of the computational methods. Our second objective (2) involves applying the evaluation methodology to answer questions (i) and (ii) for knowledge-intensive clinical decision support (CDS) methods, which operationalize clinical knowledge as computer interpretable guidelines (CIG); we focus on multimorbidity CIG-based clinical decision support (MGCDS) methods that target multimorbidity treatment plans.

MATERIALS AND METHODS

Our methodology directly involves the research community of practice in (a) identifying functional features within the application domain; (b) defining exemplar case studies covering these features; and (c) solving the case studies using their developed computational methods-research groups detail their solutions and functional feature support in solution reports. Next, the study authors (d) perform a qualitative analysis of the solution reports, identifying and characterizing common themes (or dimensions) among the computational methods. This methodology is well suited to perform whitebox analysis, as it directly involves the respective developers in studying inner workings and feature support of computational methods. Moreover, the established evaluation parameters (e.g., features, case studies, themes) constitute a re-usable benchmark framework, which can be used to evaluate new computational methods as they are developed. We applied our community-of-practice-based evaluation methodology on MGCDS methods.

RESULTS

Six research groups submitted comprehensive solution reports for the exemplar case studies. Solutions for two of these case studies were reported by all groups. We identified four evaluation dimensions: detection of adverse interactions, management strategy representation, implementation paradigms, and human-in-the-loop support.Based on our whitebox analysis, we present answers to the evaluation questions (i) and (ii) for MGCDS methods.

DISCUSSION

The proposed evaluation methodology includes features of illuminative and comparison-based approaches; focusing on understanding rather than judging/scoring or identifying gaps in current methods. It involves answering evaluation questions with direct involvement of the research community of practice, who participate in setting up evaluation parameters and solving exemplar case studies. Our methodology was successfully applied to evaluate six MGCDS knowledge-intensive computational methods. We established that, while the evaluated methods provide a multifaceted set of solutions with different benefits and drawbacks, no single MGCDS method currently provides a comprehensive solution for MGCDS.

CONCLUSION

We posit that our evaluation methodology, applied here to gain new insights into MGCDS, can be used to assess other types of knowledge-intensive computational methods and answer other types of evaluation questions. Our case studies can be accessed at our GitHub repository (https://github.com/william-vw/MGCDS).

Representation of evidence-based clinical practice guideline recommendations on FHIR

BACKGROUND

Various formalisms have been developed to represent clinical practice guideline recommendations in a computer-interpretable way. However, none of the existing formalisms leverage the structured and computable information that emerge from the evidence-based guideline development process. Thus, we here propose a FHIR-based format that uses computer-interpretable representations of the knowledge artifacts that emerge during the process of evidence-based guideline development to directly serve as the basis of evidence-based recommendations.

METHODS

We identified the information required to represent evidence-based clinical practice guideline recommendations and reviewed the knowledge artifacts emerging during the evidence-based guideline development process. We then conducted a consensus-based design process with domain experts to develop an information model for guideline recommendation representation that is structurally aligned to the evidence-based guideline recommendation development process and a corresponding representation based on FHIR resources developed for evidence-based medicine (EBMonFHIR). The resulting recommendations were modelled and represented in conformance with the FHIR Clinical Guidelines (CPG-on-FHIR) implementation guide.

RESULTS

The information model of evidence-based clinical guideline recommendations and its EBMonFHIR-/CPG-on-FHIR-based representation contain the clinical contents of individual guideline recommendations, a set of metadata for the recommendations, the ratings for the recommendations (e.g., strength of recommendation, certainty of overall evidence), the ratings of certainty of evidence for individual outcomes (e.g., risk of bias) and links to the underlying evidence (systematic reviews based on primary studies). We created profiles and an implementation guide for all FHIR resources required to represent the knowledge artifacts generated during evidence-based guideline development and their re-use as the basis for recommendations and used the profiles to implement an exemplary clinical guideline recommendation.

CONCLUSIONS

The FHIR implementation guide presented here can be used to directly link the evidence assessment process of evidence-based guideline recommendation development, i.e. systematic reviews and evidence grading, and the underlying evidence from primary studies to the resulting guideline recommendations. This not only allows the evidence on which recommendations are based on to be evaluated transparently and critically, but also enables guideline developers to leverage computable evidence in a more direct way to facilitate the generation of computer-interpretable guideline recommendations.

A model-driven transformation approach for the modelling of processes in clinical practice guidelines

Clinical Practice Guidelines (CPGs) include recommendations aimed at optimising patient care, informed by a review of the available clinical evidence. To achieve their potential benefits, CPG should be readily available at the point of care. This can be done by translating CPG recommendations into one of the languages for Computer-Interpretable Guidelines (CIGs). This is a difficult task for which the collaboration of clinical and technical staff is crucial. However, in general CIG languages are not accessible to non-technical staff. We propose to support the modelling of CPG processes (and hence the authoring of CIGs) based on a transformation, from a preliminary specification in a more accessible language into an implementation in a CIG language. In this paper, we approach this transformation following the Model-Driven Development (MDD) paradigm, in which models and transformations are key elements for software development. To demonstrate the approach, we implemented and tested an algorithm for the transformation from the BPMN language for business processes to the PROforma CIG language. This implementation uses transformations defined in the ATLAS Transformation Language. Additionally, we conducted a small experiment to assess the hypothesis that a language such as BPMN can facilitate the modelling of CPG processes by clinical and technical staff.

Monitoring hybrid process specifications with conflict management: An automata-theoretic approach

Complexity of medical treatments can vary from prescribing medicine for a specific ailment to managing a complex set of simultaneous medical issues. In the latter case, doctors are assisted by clinical guidelines which outline standard medical procedures, tests, treatments, etc. To facilitate the use of such guidelines, they can be digitized as processes and adopted in complex process engines offering additional help to health providers such as decision support while monitoring active treatments so as to detect flaws in treatment procedures and suggest possible reactions on them. For example, a patient may present symptoms of multiple diseases simultaneously (requiring multiple clinical guidelines to be followed), while also being allergic to some often-used drugs (requiring additional constraints to be respected). This can easily lead to treating a patient based on a set of process specifications which are not fully compatible with each other. While a scenario like that commonly occurs in practice, research in that direction has thus far given little consideration to how to specify multiple clinical guidelines and how to automatically combine their specifications in the context of the monitoring task. In our previous work (Alman et al., 20222), we presented a conceptual framework for handling the above cases in the context of monitoring. In this paper, we present the algorithms necessary for implementing key components of this conceptual framework. More specifically, we provide formal languages for representing clinical guideline specifications and formalize a solution for monitoring the interplay of such specifications expressed as a combination of (data-aware) Petri nets and temporal logic rules. The proposed solution seamlessly handles combination of the input process specifications and provides both early conflict detection and decision support during process execution. We also discuss a proof-of-concept implementation of our approach and present the results of extensive scalability experiments.

Construction and application of Chinese breast cancer knowledge graph based on multi-source heterogeneous data

The knowledge graph is a critical resource for medical intelligence. The general medical knowledge graph tries to include all diseases and contains much medical knowledge. However, it is challenging to review all the triples manually. Therefore the quality of the knowledge graph can not support intelligence medical applications. Breast cancer is one of the highest incidences of cancer at present. It is urgent to improve the efficiency of breast cancer diagnosis and treatment through artificial intelligence technology and improve the postoperative health status of breast cancer patients. This paper proposes a framework to construct a breast cancer knowledge graph from heterogeneous data resources in response to this demand. Specifically, this paper extracts knowledge triple from clinical guidelines, medical encyclopedias and electronic medical records. Furthermore, the triples from different data resources are fused to build a breast cancer knowledge graph (BCKG). Experimental results demonstrate that BCKG can support knowledge-based question answering, breast cancer postoperative follow-up and healthcare, and improve the quality and efficiency of breast cancer diagnosis, treatment and management.

APPRAISE-RS: Automated, updated, participatory, and personalized treatment recommender systems based on GRADE methodology

Purpose

Clinical practice guidelines (CPGs) have become fundamental tools for evidence-based medicine (EBM). However, CPG suffer from several limitations, including obsolescence, lack of applicability to many patients, and limited patient participation. This paper presents APPRAISE-RS, which is a methodology that we developed to overcome these limitations by automating, extending, and iterating the methodology that is most commonly used for building CPGs: the GRADE methodology.

Method

APPRAISE-RS relies on updated information from clinical studies and adapts and automates the GRADE methodology to generate treatment recommendations. APPRAISE-RS provides personalized recommendations because they are based on the patient's individual characteristics. Moreover, both patients and clinicians express their personal preferences for treatment outcomes which are considered when making the recommendation (participatory). Rule-based system approaches are used to manage heuristic knowledge.

Results

APPRAISE-RS has been implemented for attention deficit hyperactivity disorder (ADHD) and tested experimentally on 28 simulated patients. The resulting recommender system (APPRAISE-RS/TDApp) shows a higher degree of treatment personalization and patient participation than CPGs, while recommending the most frequent interventions in the largest body of evidence in the literature (EBM). Moreover, a comparison of the results with four blinded psychiatrist prescriptions supports the validation of the proposal.

Conclusions

APPRAISE-RS is a valid methodology to build recommender systems that manage updated, personalized and participatory recommendations, which, in the case of ADHD includes at least one intervention that is identical or very similar to other drugs prescribed by psychiatrists.

Supporting physicians in the coordination of distributed execution of CIGs to treat comorbid patients

Clinical Practice Guidelines (CPGs) encode the "best" medical practices to treat patients affected by a specific disease and are widely used in the medical practice. Starting from the '90s', several Computer-Interpretable Guideline (CIG) systems have been devised to provide physicians with CPG-based decision support. CPGs (and CIGs) are devoted to provide evidence-based recommendations for one specific disease. In order to support the treatment of patients affected by multiple diseases (i.e., comorbid patients), challenging additional tasks have to be addressed, such as (i) the detection of the interactions between CIG actions, (ii) their management, and, finally, (iii) the "merge" or conciliation of the CIGs. Several CIG approaches have been recently extended in order to face (at least one of) such challenging problems, and one of them is GLARE. However, besides the solutions to tasks (i)-(iii) above, the "run-time" support to physicians treating a comorbid patient requires additional capabilities, to support the distribution of the management of interactions and of the execution of CIGs among different physicians. In this paper, we propose a general framework, based on GLARE and GLARE-SSCPM, to provide such additional capabilities.

Business Process Model and Notation and openEHR Task Planning for Clinical Pathway Standards in Infections: Critical Analysis

Background

Clinical pathways (CPs) are usually expressed by means of workflow formalisms, providing health care personnel with an easy-to-understand, high-level conceptual model of medical steps in specific patient conditions, thereby improving overall health care process quality in clinical practice. From a standardized perspective, the business process model and notation (BPMN), a widely spread general-purpose process formalism, has been used for conceptual modeling in clinical domains, mainly because of its easy-to-use graphical notation, facilitating the common understanding and communication of the parties involved in health care. However, BPMN is not particularly oriented toward the peculiarities of complex clinical processes such as infection diagnosis and treatment, in which time plays a critical role, which is why much of the BPMN clinical-oriented research has revolved around how to extend the standard to address these special needs. The shift from an agnostic, general-purpose BPMN notation to a natively clinical-oriented notation such as openEHR Task Planning (TP) could constitute a major step toward clinical process improvement, enhancing the representation of CPs for infection treatment and other complex scenarios.

Objective

Our work aimed to analyze the suitability of a clinical-oriented formalism (TP) to successfully represent typical process patterns in infection treatment, identifying domain-specific improvements to the standard that could help enhance its modeling capabilities, thereby promoting the widespread adoption of CPs to improve medical practice and overall health care quality.

Methods

Our methodology consisted of 4 major steps: identification of key features of infection CPs through literature review, clinical guideline analysis, and BPMN extensions; analysis of the presence of key features in TP; modeling of relevant process patterns of catheter-related bloodstream infection as a case study; and analysis and proposal of extensions in view of the results.

Results

We were able to easily represent the same logic applied in the extended BPMN-based process models in our case study using out-of-the-box standard TP primitives. However, we identified possible improvements to the current version of TP to allow for simpler conceptual models of infection CPs and possibly of other complex clinical scenarios.

Conclusions

Our study showed that the clinical-oriented TP specification is able to successfully represent the most complex catheter-related bloodstream infection process patterns depicted in our case study and identified possible extensions that can help increase its adequacy for modeling infection CPs and possibly other complex clinical conditions.

System for Context-Specific Visualization of Clinical Practice Guidelines (GuLiNav): Concept and Software Implementation

Background

Clinical decision support systems often adopt and operationalize existing clinical practice guidelines leading to higher guideline availability, increased guideline adherence, and data integration. Most of these systems use an internal state-based model of a clinical practice guideline to derive recommendations but do not provide the user with comprehensive insight into the model.

Objective

Here we present a novel approach based on dynamic guideline visualization that incorporates the individual patient’s current treatment context.

Methods

We derived multiple requirements to be fulfilled by such an enhanced guideline visualization. Using business process and model notation as the representation format for computer-interpretable guidelines, a combination of graph-based representation and logical inferences is adopted for guideline processing. A context-specific guideline visualization is inferred using a business rules engine.

Results

We implemented and piloted an algorithmic approach for guideline interpretation and processing. As a result of this interpretation, a context-specific guideline is derived and visualized. Our implementation can be used as a software library but also provides a representational state transfer interface. Spring, Camunda, and Drools served as the main frameworks for implementation. A formative usability evaluation of a demonstrator tool that uses the visualization yielded high acceptance among clinicians.

Conclusions

The novel guideline processing and visualization concept proved to be technically feasible. The approach addresses known problems of guideline-based clinical decision support systems. Further research is necessary to evaluate the applicability of the approach in specific medical use cases.

Toward a Symbolic AI Approach to the WHO/ACSM Physical Activity & Sedentary Behavior Guidelines

The World Health Organization and the American College of Sports Medicine have released guidelines on physical activity and sedentary behavior, as part of an effort to reduce inactivity worldwide. However, to date, there is no computational model that can facilitate the integration of these recommendations into health solutions (e.g., digital coaches). In this paper, we present an operational and machine-readable model that represents and is able to reason about these guidelines. To this end, we adopted a symbolic AI approach that combines two paradigms of research in knowledge representation and reasoning: ontology and rules. Thus, we first present HeLiFit, a domain ontology implemented in OWL, which models the main entities that characterize the definition of physical activity, as defined per guidance. Then, we describe HeLiFit-Rule, a set of rules implemented in the RDFox Rule language, which can be used to represent and reason with these recommendations in concrete real-world applications. Furthermore, to ensure a high level of syntactic/semantic interoperability across different systems, our framework is also compliant with the FHIR standard. Through motivating scenarios that highlight the need for such an implementation, we finally present an evaluation of our model that provides results that are both encouraging in terms of the value of our solution and also provide a basis for future work.

A scoping review of knowledge authoring tools used for developing computerized clinical decision support systems

Objective

Clinical Knowledge Authoring Tools (CKATs) are integral to the computerized Clinical Decision Support (CDS) development life cycle. CKATs enable authors to generate accurate, complete, and reliable digital knowledge artifacts in a relatively efficient and affordable manner. This scoping review aims to compare knowledge authoring tools and derive the common features of CKATs.

Materials and Methods

We performed a keyword-based literature search, followed by a snowball search, to identify peer-reviewed publications describing the development or use of CKATs. We used PubMed and Embase search engines to perform the initial search (n = 1579). After removing duplicate articles, nonrelevant manuscripts, and not peer-reviewed publication, we identified 47 eligible studies describing 33 unique CKATs. The reviewed CKATs were further assessed, and salient characteristics were extracted and grouped as common CKAT features.

Results

Among the identified CKATs, 55% use an open source platform, 70% provide an application programming interface for CDS system integration, and 79% provide features to validate/test the knowledge. The majority of the reviewed CKATs describe the flow of information, offer a graphical user interface for knowledge authors, and provide intellisense coding features (94%, 97%, and 97%, respectively). The composed list of criteria for CKAT included topics such as simulating the clinical setting, validating the knowledge, standardized clinical models and vocabulary, and domain independence. None of the reviewed CKATs met all common criteria.

Conclusion

Our scoping review highlights the key specifications for a CKAT. The CKAT specification proposed in this review can guide CDS authors in developing more targeted CKATs.

Decision support for comorbid conditions via execution-time integration of clinical guidelines using transaction-based semantics and temporal planning

In case of comorbidity, i.e., multiple medical conditions, Clinical Decision Support Systems (CDSS) should issue recommendations based on all relevant disease-related Clinical Practice Guidelines (CPG). However, treatments from multiple comorbid CPG often interact adversely (e.g., drug-drug interactions) or introduce operational inefficiencies (e.g., redundant scans). A common solution is the a-priori integration of computerized CPG, which involves integration decisions such as discarding, replacing or delaying clinical tasks (e.g., treatments) to avoid adverse interactions or inefficiencies. We argue this insufficiently deals with execution-time events: as the patient's health profile evolves, acute conditions occur, and real-time delays take place, new CPG integration decisions will often be needed, and prior ones may need to be reverted or undone. Any realistic CPG integration effort needs to further consider temporal aspects of clinical tasks-these are not only restricted by temporal constraints from CPGs (e.g., sequential relations, task durations) but also by CPG integration efforts (e.g., avoid treatment overlap). This poses a complex execution-time challenge and makes it difficult to determine an up-to-date, optimal comorbid care plan. We present a solution for dynamic integration of CPG in response to evolving health profiles and execution-time events. CPG integration policies are formulated by clinical experts for coping with comorbidity at execution-time, with clearly defined integration semantics that build on Description and Transaction Logics. A dynamic planning approach reconciles temporal constraints of CPG tasks at execution-time based on their importance, and continuously updates an optimal task schedule.

Conversion of a colorectal cancer guideline into clinical decision trees with assessment of validity

Objective

The interpretation and clinical application of guidelines can be challenging and time-consuming, which may result in noncompliance to guidelines. The aim of this study was to convert the Dutch guideline for colorectal cancer (CRC) into decision trees and subsequently implement decision trees in an online decision support environment to facilitate guideline application.

Methods

The recommendations of the Dutch CRC guidelines (published in 2014) were translated into decision trees consisting of decision nodes, branches and leaves that represent data items, data item values and recommendations, respectively. Decision trees were discussed with experts in the field and published as interactive open access decision support software (available at www.oncoguide.nl). Decision tree validation and a concordance analysis were performed using consecutive reports (January 2016–January 2017) from CRC multidisciplinary tumour boards (MTBs) at Amsterdam University Medical Centers, location AMC.

Results

In total, we developed 34 decision trees driven by 101 decision nodes based on the guideline recommendations. Decision trees represented recommendations for diagnostics (n = 1), staging (n = 10), primary treatment (colon: n = 1, rectum: n = 5, colorectal: n = 9), pathology (n = 4) and follow-up (n = 3) and included one overview decision tree for optimal navigation. We identified several guideline information gaps and areas of inconclusive evidence. A total of 158 patients’ MTB reports were eligible for decision tree validation and resulted in treatment recommendations in 80% of cases. The concordance rate between decision tree treatment recommendations and MTB advices was 81%. Decision trees reported in 22 out of 24 non-concordant cases (92%) that no guideline recommendation was available.

Conclusions

We successfully converted the Dutch CRC guideline into decision trees and identified several information gaps and areas of inconclusive evidence, the latter being the main cause of the observed disagreement between decision tree recommendations and MTB advices. Decision trees may contribute to future strategies to optimize quality of care for CRC patients.

A computerized decision support system (CDSS) for antibiotic prescription in primary care-Antibioclic: implementation, adoption and sustainable use in the era of extended antimicrobial resistance

OBJECTIVES

To describe the implementation and use of a computerized decision support system (CDSS) for antibiotic prescription in primary care in France (Antibioclic). The CDSS targets 37 infectious diseases and has been freely available on a website since 2011.

METHODS

Description and implementation of the architecture of a CDSS for antibiotic prescription in general practice. Analysis of the queries made between 2012 and 2018 on the CDSS by GPs. Analysis of two cross-sectional studies of users in 2014 and 2019.

RESULTS

The number of queries increased from a median of 796/day [IQR, 578-989] in 2012 to 11 125/day [5592-12 505] in 2018. Unique users increased from 414/day [245-494] in 2012 to 5365/day [2891-5769] in 2018. Time taken to make a query was 2 min [1.9-2.1]. Among 3 542 347 queries in 2018, 78% were for adults. Six situations accounted for ≥50% of queries: cystitis; acute otitis media; acute sinusitis; community-acquired pneumonia; sore throat; and pyelonephritis. Queries concerned pathologies for which antibiotic prescription was necessary (64%), was conditional on additional clinical steps (34%) or was not recommended (2%). Most users (81%) were GPs, with median age of 38 years [31-52] and 58% were female. Among the 4016 GPs who responded to the surveys, the vast majority (96%) reported using the CDSS during the consultation, with 24% systematically using Antibioclic to initiate an antibiotic course and 93% having followed the CDSS recommendation for the latest prescription. Most GPs were comfortable using the CDSS in front of a patient.

CONCLUSIONS

Antibioclic has been adopted and is widely used in primary care in France. Its interoperability could allow its adaptation and implementation in other countries.

Assumption-based argumentation with preferences and goals for patient-centric reasoning with interacting clinical guidelines

A paramount, yet unresolved issue in personalised medicine is that of automated reasoning with clinical guidelines in multimorbidity settings. This entails enabling machines to use computerised generic clinical guideline recommendations and patient-specific information to yield patient-tailored recommendations where interactions arising due to multimorbidities are resolved. This problem is further complicated by patient management desiderata, in particular the need to account for patient-centric goals as well as preferences of various parties involved. We propose to solve this problem of automated reasoning with interacting guideline recommendations in the context of a given patient by means of computational argumentation. In particular, we advance a structured argumentation formalism ABA+G (short for Assumption-Based Argumentation with Preferences (ABA+) and Goals) for integrating and reasoning with information about recommendations, interactions, patient’s state, preferences and prioritised goals. ABA+G combines assumption-based reasoning with preferences and goal-driven selection among reasoning outcomes. Specifically, we assume defeasible applicability of guideline recommendations with the general goal of patient well-being, resolve interactions (conflicts and otherwise undesirable situations) among recommendations based on the state and preferences of the patient, and employ patient-centered goals to suggest interaction-resolving, goal-importance maximising and preference-adhering recommendations. We use a well-established Transition-based Medical Recommendation model for representing guideline recommendations and identifying interactions thereof, and map the components in question, together with the given patient’s state, prioritised goals, and preferences over actions, to ABA+G for automated reasoning. In this, we follow principles of patient management and establish corresponding theoretical properties as well as illustrate our approach in realistic personalised clinical reasoning scenaria.

American Association of Clinical Endocrinology And Associazione Medici Endocrinologi Thyroid Nodule Algorithmic Tool

OBJECTIVE

The first edition of the American Association of Clinical Endocrinology/American College of Endocrinology/Associazione Medici Endocrinologi Guidelines for the Diagnosis and Management of Thyroid Nodules was published in 2006 and updated in 2010 and 2016. The American Association of Clinical Endocrinology/American College of Endocrinology/Associazione Medici Endocrinologi multidisciplinary thyroid nodules task force was charged with developing a novel interactive electronic algorithmic tool to evaluate thyroid nodules.

METHODS

The Thyroid Nodule App (termed TNAPP) was based on the updated 2016 clinical practice guideline recommendations while incorporating recent scientific evidence and avoiding unnecessary diagnostic procedures and surgical overtreatment. This manuscript describes the algorithmic tool development, its data requirements, and its basis for decision making. It provides links to the web-based algorithmic tool and a tutorial.

RESULTS

TNAPP and TI-RADS were cross-checked on 95 thyroid nodules with histology-proven diagnoses.

CONCLUSION

TNAPP is a novel interactive web-based tool that uses clinical, imaging, cytologic, and molecular marker data to guide clinical decision making to evaluate and manage thyroid nodules. It may be used as a heuristic tool for evaluating and managing patients with thyroid nodules. It can be adapted to create registries for solo practices, large multispecialty delivery systems, regional and national databases, and research consortiums. Prospective studies are underway to validate TNAPP to determine how it compares with other ultrasound-based classification systems and whether it can improve the care of patients with clinically significant thyroid nodules while reducing the substantial burden incurred by those who do not benefit from further evaluation and treatment.

A review of AI and Data Science support for cancer management

INTRODUCTION

Thanks to improvement of care, cancer has become a chronic condition. But due to the toxicity of treatment, the importance of supporting the quality of life (QoL) of cancer patients increases. Monitoring and managing QoL relies on data collected by the patient in his/her home environment, its integration, and its analysis, which supports personalization of cancer management recommendations. We review the state-of-the-art of computerized systems that employ AI and Data Science methods to monitor the health status and provide support to cancer patients managed at home.

OBJECTIVE

Our main objective is to analyze the literature to identify open research challenges that a novel decision support system for cancer patients and clinicians will need to address, point to potential solutions, and provide a list of established best-practices to adopt.

METHODS

We designed a review study, in compliance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, analyzing studies retrieved from PubMed related to monitoring cancer patients in their home environments via sensors and self-reporting: what data is collected, what are the techniques used to collect data, semantically integrate it, infer the patient's state from it and deliver coaching/behavior change interventions.

RESULTS

Starting from an initial corpus of 819 unique articles, a total of 180 papers were considered in the full-text analysis and 109 were finally included in the review. Our findings are organized and presented in four main sub-topics consisting of data collection, data integration, predictive modeling and patient coaching.

CONCLUSION

Development of modern decision support systems for cancer needs to utilize best practices like the use of validated electronic questionnaires for quality-of-life assessment, adoption of appropriate information modeling standards supplemented by terminologies/ontologies, adherence to FAIR data principles, external validation, stratification of patients in subgroups for better predictive modeling, and adoption of formal behavior change theories. Open research challenges include supporting emotional and social dimensions of well-being, including PROs in predictive modeling, and providing better customization of behavioral interventions for the specific population of cancer patients.

Quality Improvement in Otolaryngology-Head and Neck Surgery: Developing Registry-Enabled Quality Measures From Guidelines for Cerumen Impaction and Allergic Rhinitis Through a Transparent and Systematic Process

BACKGROUND AND SIGNIFICANCE

Quality measurement can drive improvement in clinical care and allow for easy reporting of quality care by clinicians, but creating quality measures is a time-consuming and costly process. ECRI (formerly Emergency Care Research Institute) has pioneered a process to support systematic translation of clinical practice guidelines into electronic quality measures using a transparent and reproducible pathway. This process could be used to augment or support the development of electronic quality measures of the American Academy of Otolaryngology-Head and Neck Surgery Foundation (AAO-HNSF) and others as the Centers for Medicare and Medicaid Services transitions from the Merit-Based Incentive Payment System (MIPS) to the MIPS Value Pathways for quality reporting.

METHODS

We used a transparent and reproducible process to create electronic quality measures based on recommendations from 2 AAO-HNSF clinical practice guidelines (cerumen impaction and allergic rhinitis). Steps of this process include source material review, electronic content extraction, logic development, implementation barrier analysis, content encoding and structuring, and measure formalization. Proposed measures then go through the standard publication process for AAO-HNSF measures.

RESULTS

The 2 guidelines contained 29 recommendation statements, of which 7 were translated into electronic quality measures and published. Intermediate products of the guideline conversion process facilitated development and were retained to support review, updating, and transparency. Of the 7 initially published quality measures, 6 were approved as 2018 MIPS measures, and 2 continued to demonstrate a gap in care after a year of data collection.

CONCLUSION

Developing high-quality, registry-enabled measures from guidelines via a rigorous reproducible process is feasible. The streamlined process was effective in producing quality measures for publication in a timely fashion. Efforts to better identify gaps in care and more quickly recognize recommendations that would not translate well into quality measures could further streamline this process.

OpenClinical.net: Artificial intelligence and knowledge engineering at the point of care

Objective

OpenClinical.net is a way of disseminating clinical guidelines to improve quality of care whose distinctive feature is to combine the benefits of clinical guidelines and other human-readable material with the power of artificial intelligence to give patient-specific recommendations. A key objective is to empower healthcare professionals to author, share, critique, trial and revise these ‘executable’ models of best practice.

Design

OpenClinical.net Alpha (www.openclinical.net) is an operational publishing platform that uses a class of artificial intelligence techniques called knowledge engineering to capture human expertise in decision-making, care planning and other cognitive skills in an intuitive but formal language called PROforma.3 PROforma models can be executed by a computer to yield patient-specific recommendations, explain the reasons and provide supporting evidence on demand.

Results

PROforma has been validated in a wide range of applications in diverse clinical settings and specialties, with trials published in high impact peer-reviewed journals. Trials have included patient workup and risk assessment; decision support (eg, diagnosis, test and treatment selection, prescribing); adaptive care pathways and care planning. The OpenClinical software platform presently supports authoring, testing, sharing and maintenance. OpenClinical’s open-access, open-source repository Repertoire currently carries approximately 50+ diverse examples (https://openclinical.net/index.php?id=69).

Conclusion

OpenClinical.net is a showcase for a PROforma-based approach to improving care quality, safety, efficiency and better patient experience in many kinds of routine clinical practice. This human-centred approach to artificial intelligence will help to ensure that it is developed and used responsibly and in ways that are consistent with professional priorities and public expectations.

MitPlan: A planning approach to mitigating concurrently applied clinical practice guidelines

As the population ages, patients' complexity and the scope of their care is increasing. Over 60% of the population is 65 years of age or older and suffers from multi-morbidity, which is associated with two times as many patient-physician encounters. Yet clinical practice guidelines (CPGs) are developed to treat a single disease. To reconcile these two competing issues, previously we developed a framework for mitigation, i.e., identifying and addressing adverse interactions in multi-morbid patients managed according to multiple CPGs. That framework relies on first-order logic (FOL) to represent CPGs and secondary medical knowledge and FOL theorem proving to establish valid patient management plans. In the work presented here, we leverage our earlier research and simplify the mitigation process by representing it as a planning problem using the Planning Domain Definition Language (PDDL). This new framework, called MitPlan, identifies and addresses adverse interactions using durative planning actions that embody clinical actions (including medication administration and patient testing), supports a physician-defined length of planning horizons, and optimizes plans based on patient preferences and action costs. It supports a variety of criteria when developing management plans, including the total cost of prescribed treatment and the cost of the revisions to be introduced. The solution to MitPlan's planning problem is a sequence of timed actions that are easy to interpret when creating a management plan. We demonstrate MitPlan's capabilities using illustrative and clinical case studies.

Acquiring guideline-enabled data driven clinical knowledge model using formally verified refined knowledge acquisition method

BACKGROUND AND OBJECTIVE

Validation and verification are the critical requirements for the knowledge acquisition method of the clinical decision support system (CDSS). After acquiring the medical knowledge from diverse sources, the rigorous validation and formal verification process are required before creating the final knowledge model. Previously, we have proposed a hybrid knowledge acquisition method with the support of a rigorous validation process for acquiring medical knowledge from clinical practice guidelines (CPGs) and patient data for the treatment of oral cavity cancer. However, due to lack of formal verification process, it involves various inconsistencies in knowledge relevant to the formalism of knowledge, conformance to CPGs, quality of knowledge, and complexities of knowledge acquisition artifacts.

METHODS

This paper presents the refined knowledge acquisition (ReKA) method, which uses the Z formal verification process. The ReKA method adopts the verification method and explores the mechanism of theorem proving using the Z notation. It enhances a hybrid knowledge acquisition method to thwart the inconsistencies using formal verification.

RESULTS

ReKA adds a set of nine additional criteria to be used to have a final valid refined clinical knowledge model. These criteria ensure the validity of the final knowledge model concerning formalism of knowledge, conformance to GPGs, quality of the knowledge, usage of stringent conditions and treatment plans, and inconsistencies possibly resulting from the complexities. Evaluation, using four medical knowledge acquisition scenarios, shows that newly added knowledge in CDSS due to the additional criteria by the ReKA method always produces a valid knowledge model. The final knowledge model was also evaluated with 1229 oral cavity patient cases, which outperformed with an accuracy of 72.57% compared to a similar approach with an accuracy of 69.7%. Furthermore, the ReKA method identified a set of decision paths (about 47.8%) in the existing approach, which results in a final knowledge model with low quality, non-conformed from standard CPGs.

CONCLUSION

ReKA refined the hybrid knowledge acquisition method by discovering the missing steps in the current validation process at the acquisition stage. As a formally proven method, it always yields a valid knowledge model having high quality, supporting local practices, and influenced by standard CPGs. Furthermore, the final knowledge model obtained from ReKA also preserves the performance such as the accuracy of the individual source knowledge models.

Prototype Development of an Expert System of Computerized Clinical Guidelines for COVID-19 Diagnosis and Management in Saudi Arabia

The increasing number of COVID-19 patients has increased health care professionals' workloads, making the management of dynamic patient information in a timely and comprehensive manner difficult and sometimes impossible. Compounding this problem is a lack of health care professionals and trained medical staff to handle the increased number of patients. Although Saudi Arabia has recently improved the quality of its health services, there is still no suitable intelligent system that can help health practitioners follow the clinical guidelines and automated risk assessment and treatment plan remotely, which would allow for the effective follow-up of patients of COVID-19. The proposed system includes five sub-systems: an information management system, a knowledge-based expert system, adaptive learning, a notification and follow-up system, and a mobile tracker system. This study shows that, to control epidemics, there is a method to overcome the shortage of specialists in the management of infections in Saudi Arabia, both today and in the future. The availability of computerized clinical guidance and an up-to-date knowledge base play a role in Saudi health organizations, which may not have to constantly train their physician staff and may no longer have to rely on international experts, since the expert system can offer clinicians all the information necessary to treat their patients.

XML Data and Knowledge-Encoding Structure for a Web-Based and Mobile Antenatal Clinical Decision Support System: Development Study

Background

Displeasure with the functionality of clinical decision support systems (CDSSs) is considered the primary challenge in CDSS development. A major difficulty in CDSS design is matching the functionality to the desired and actual clinical workflow. Computer-interpretable guidelines (CIGs) are used to formalize medical knowledge in clinical practice guidelines (CPGs) in a computable language. However, existing CIG frameworks require a specific interpreter for each CIG language, hindering the ease of implementation and interoperability.

Objective

This paper aims to describe a different approach to the representation of clinical knowledge and data. We intended to change the clinician’s perception of a CDSS with sufficient expressivity of the representation while maintaining a small communication and software footprint for both a web application and a mobile app. This approach was originally intended to create a readable and minimal syntax for a web CDSS and future mobile app for antenatal care guidelines with improved human-computer interaction and enhanced usability by aligning the system behavior with clinical workflow.

Methods

We designed and implemented an architecture design for our CDSS, which uses the model-view-controller (MVC) architecture and a knowledge engine in the MVC architecture based on XML. The knowledge engine design also integrated the requirement of matching clinical care workflow that was desired in the CDSS. For this component of the design task, we used a work ontology analysis of the CPGs for antenatal care in our particular target clinical settings.

Results

In comparison to other common CIGs used for CDSSs, our XML approach can be used to take advantage of the flexible format of XML to facilitate the electronic sharing of structured data. More importantly, we can take advantage of its flexibility to standardize CIG structure design in a low-level specification language that is ubiquitous, universal, computationally efficient, integrable with web technologies, and human readable.

Conclusions

Our knowledge representation framework incorporates fundamental elements of other CIGs used in CDSSs in medicine and proved adequate to encode a number of antenatal health care CPGs and their associated clinical workflows. The framework appears general enough to be used with other CPGs in medicine. XML proved to be a language expressive enough to describe planning problems in a computable form and restrictive and expressive enough to implement in a clinical system. It can also be effective for mobile apps, where intermittent communication requires a small footprint and an autonomous app. This approach can be used to incorporate overlapping capabilities of more specialized CIGs in medicine.

Towards a goal-oriented methodology for clinical-guideline-based management recommendations for patients with multimorbidity: GoCom and its preliminary evaluation

Patients with chronic multimorbidity are becoming more common as life expectancy increases, making it necessary for physicians to develop complex management plans. We are looking at the patient management process as a goal-attainment problem. Hence, our aim is to develop a goal-oriented methodology for providing decision support for managing patients with multimorbidity continuously, as the patient's health state is progressing and new goals arise (e.g., treat ulcer, prevent osteoporosis). Our methodology allows us to detect and mitigate inconsistencies among guideline recommendations stemming from multiple clinical guidelines, while consulting medical ontologies and terminologies and relying on patient information standards. This methodology and its implementation as a decision-support system, called GoCom, starts with computer-interpretable clinical guidelines (CIGs) for single problems that are formalized using the PROforma CIG language. We previously published the architecture of the system as well as a CIG elicitation guide for enriching PROforma tasks with properties referring to vocabulary codes of goals and physiological effects of management plans. In this paper, we provide a formalization of the conceptual model of GoCom that generates, for each morbidity of the patient, a patient-specific goal tree that results from the PROforma engine's enactment of the CIG with the patient's data. We also present the "Controller" algorithm that drives the GoCom system. Given a new problem that a patient develops, the Controller detects inconsistencies among goals pertaining to different comorbid problems and consults the CIGs to generate alternative non-conflicted and goal-oriented management plans that address the multiple goals simultaneously. In this stage of our research, the inconsistencies that can be detected are of two types - starting vs. stopping medications that belong to the same medication class hierarchy, and detecting opposing physiological effect goals that are specified in concurrent CIGs (e.g., decreased blood pressure vs. increased blood pressure). However, the design of GoCom is modular and generic and allows the future introduction of additional interaction detection and mitigation strategies. Moreover, GoCom generates explanations of the alternative non-conflicted management plans, based on recommendations stemming from the clinical guidelines and reasoning patterns. GoCom's functionality was evaluated using three cases of multimorbidity interactions that were checked by our three clinicians. Usefulness was evaluated with two studies. The first evaluation was a pilot study with ten 6th year medical students and the second evaluation was done with 27 6th medical students and interns. The participants solved complex realistic cases of multimorbidity patients: with and without decision-support, two cases in the first evaluation and 6 cases in the second evaluation. Use of GoCom increased completeness of the patient management plans produced by the medical students from 0.44 to 0.71 (P-value of 0.0005) in the first evaluation, and from 0.31 to 0.78 (P-value < 0.0001) in the second evaluation. Correctness in the first evaluation was very high with (0.98) or without the system (0.91), with non-significant difference (P-value ≥ 0.17). In the second evaluation, use of GoCom increased correctness from 0.68 to 0.83 (P-value of 0.001). In addition, GoCom's explanation and visualization were perceived as useful by the vast majority of participants. While GoCom's detection of goal interactions is currently limited to detection of starting vs. stopping the same medication or medication subclasses and detecting conflicting physiological effects of concurrent medications, the evaluation demonstrated potential of the system for improving clinical decision-making for multimorbidity patients.

Clinical decision support systems for chronic diseases: A Systematic literature review

A Clinical Decision Support System (CDSS) aims to assist physicians, nurses and other professionals in decision-making related to the patient's clinical condition. CDSSs deal with pertinent and critical data, and special care should be taken in their design to ensure the development of usable, secure and reliable tools.

OBJECTIVE

This paper aims to investigate existing literature dealing with the development process of CDSSs for monitoring chronic diseases, analysing their functionalities and characteristics, and the software engineering representation in their design.

METHODS

A systematic literature review (SLR) is conducted to analyse the literature on CDSSs for monitoring chronic diseases and the application of software engineering techniques in their design.

RESULTS

Fourteen included studies revealed that the most addressed disease was diabetes (42.8%) and the most commonly proposed approach was diagnostic (85.7%). Regarding data sources, the studies show a predominance on the use of databases (85.7%), with other data sources such as sensors (42.8%) and self-report (28.6%) also being considered. Analysing the representation for engineering techniques, we found Behaviour diagrams (42.8%) to be the most frequent, closely followed by Structural diagrams (35.7%) and others (78.6%) being largely mentioned. Some studies also approached the requirement specification (21.4%). The most common target evaluation was the performance of the system (64.2%) and the most common metric was accuracy (57.1%).

CONCLUSION

We conclude that software engineering, in its completeness, has scarce representation in studies focused on the development of CDSSs for chronic diseases.