Using clinical reasoning ontologies to make smarter clinical decision support systems: a systematic review and data synthesis

Towards a knowledge-based decision support system to foster the return to work of wheelchair users

Accidents at work may force workers to face abrupt changes in their daily life: one of the most impactful accident cases consists of the worker remaining in a wheelchair. Return To Work (RTW) of wheelchair users in their working age is still challenging, encompassing the expertise of clinical and rehabilitation personnel and social workers to match the workers' residual capabilities with job requirements. This work describes a novel and prototypical knowledge-based Decision Support System (DSS) that matches workers' residual capabilities with job requirements, thus helping vocational therapists and clinical personnel in the RTW decision-making process for WUs. The DSS leverages expert knowledge in the form of ontologies to represent the International Classification of Functioning, Disability, and Health (ICF) and the Occupational Information Network (O*NET). These taxonomies enable both workers' health conditions and job requirements formalization, which are processed to assess the suitability of a job depending on a worker's condition. Consequently, the DSS suggests a list of jobs a wheelchair user can still perform, exploiting his/her residual abilities at their best. The manuscript describes the theoretical approach and technological foundations of such DSS, illustrating its development, its output metric, and application. The developed solution was tested with real wheelchair users' health conditions provided by the Italian National Institute for Insurance against Accidents at Work. The feasibility of an approach based on objective data was thus demonstrated, providing a novel point of view in the critical process of decision-making during RTW.

A knowledge-based decision support system to support family doctors in personalizing type-2 diabetes mellitus medical nutrition therapy

BACKGROUND

Type-2 Diabetes Mellitus (T2D) is a growing concern worldwide, and family doctors are called to help diabetic patients manage this chronic disease, also with Medical Nutrition Therapy (MNT). However, MNT for Diabetes is usually standardized, while it would be much more effective if tailored to the patient. There is a gap in patient-tailored MNT which, if addressed, could support family doctors in delivering effective recommendations. In this context, decision support systems (DSSs) are valuable tools for physicians to support MNT for T2D patients - as long as DSSs are transparent to humans in their decision-making process. Indeed, the lack of transparency in data-driven DSS might hinder their adoption in clinical practice, thus leaving family physicians to adopt general nutrition guidelines provided by the national healthcare systems.

METHOD

This work presents a prototypical ontology-based clinical Decision Support System (OnT2D- DSS) aimed at assisting general practice doctors in managing T2D patients, specifically in creating a tailored dietary plan, leveraging clinical expert knowledge. OnT2D-DSS exploits clinical expert knowledge formalized as a domain ontology to identify a patient's phenotype and potential comorbidities, providing personalized MNT recommendations for macro- and micro-nutrient intake. The system can be accessed via a prototypical interface.

RESULTS

Two preliminary experiments are conducted to assess both the quality and correctness of the inferences provided by the system and the usability and acceptance of the OnT2D-DSS (conducted with nutrition experts and family doctors, respectively).

CONCLUSIONS

Overall, the system is deemed accurate by the nutrition experts and valuable by the family doctors, with minor suggestions for future improvements collected during the experiments.

Ontology-based decision support systems for diabetes nutrition therapy: A systematic literature review

Diabetes is a non-communicable disease that has reached epidemic proportions, affecting 537 million people globally. Artificial Intelligence can support patients or clinicians in diabetes nutrition therapy - the first medical therapy in most cases of Type 1 and Type 2 diabetes. In particular, ontology-based recommender and decision support systems can deliver a computable representation of experts' knowledge, thus delivering patient-tailored nutritional recommendations or supporting clinical personnel in identifying the most suitable diet. This work proposes a systematic literature review of the domain ontologies describing diabetes in such systems, identifying their underlying conceptualizations, the users targeted by the systems, the type(s) of diabetes tackled, and the nutritional recommendations provided. This review also delves into the structure of the domain ontologies, highlighting several aspects that may hinder (or foster) their adoption in recommender and decision support systems for diabetes nutrition therapy. The results of this review process allow to underline how recommendations are formulated and the role of clinical experts in developing domain ontologies, outlining the research trends characterizing this research area. The results also allow for identifying research directions that can foster a preeminent role for clinical experts and clinical guidelines in a cooperative effort to make ontologies more interoperable - thus enabling them to play a significant role in the decision-making processes about diabetes nutrition therapy.

Clinical decision support system in emergency telephone triage: A scoping review of technical design, implementation and evaluation

OBJECTIVES

Emergency department overcrowding could be improved by upstream telephone triage. Emergency telephone triage aims at managing and orientating adequately patients as early as possible and distributing limited supply of staff and materials. This complex task could be improved with the use of Clinical decision support systems (CDSS). The aim of this scoping review was to identify literature gaps for the future development and evaluation of CDSS for Emergency telephone triage.

MATERIALS AND METHODS

We present here a scoping review of CDSS designed for emergency telephone triage, and compared them in terms of functional characteristics, technical design, health care implementation and methodologies used for evaluation, following the PRISMA-ScR guidelines.

RESULTS

Regarding design, 19 CDSS were retrieved: 12 were knowledge based CDSS (decisional algorithms built according to guidelines or clinical expertise) and 7 were data driven (statistical, machine learning, or deep learning models). Most of them aimed at assisting nurses or non-medical staff by providing patient orientation and/or severity/priority assessment. Eleven were implemented in real life, and only three were connected to the Electronic Health Record. Regarding evaluation, CDSS were assessed through various aspects: intrinsic characteristics, impact on clinical practice or user apprehension. Only one pragmatic trial and one randomized controlled trial were conducted.

CONCLUSION

This review highlights the potential of a hybrid system, user tailored, flexible, connected to the electronic health record, which could work with oral, video and digital data; and the need to evaluate CDSS on intrinsic characteristics and impact on clinical practice, iteratively at each distinct stage of the IT lifecycle.

Heuristics used for evaluating the usability of mobile health applications: A systematic literature review

Objective

Mobile health applications hold immense potential for enhancing health outcomes. Usability is one of the main factors for the adoption and use of mobile health applications. However, despite the growing importance of mHealth applications, clear standards for their evaluation remain elusive. The present study aimed to determine heuristics for the usability evaluation of health-related applications.

Methods

We systematically searched multiple databases for relevant papers published between January 2008 and April 2021. Articles were reviewed, and data were extracted and categorized from those meeting inclusion criteria by two authors independently. Heuristics were identified based on statements, words, and concepts expressed in the studies. These heuristics were first mapped to Nielsen's heuristics based on their differences or similarities. The remaining heuristics that were very important for mobile applications were categorized into new heuristics.

Results

Seventeen studies met the eligibility criteria. Seventy-nine heuristics were extracted from the papers. After combining the items with the same concepts and removing irrelevant items based on the exclusion criteria, 20 heuristics remained. Common heuristics such as "Visibility of system status" and "Flexibility and efficiency of use" were categorized into 10 previously established heuristics and new heuristics like "Navigation" and "User engagement" were recognized as new ones.

Conclusions

In our study, we have meticulously identified 20 heuristics that hold promise for evaluating and designing mHealth applications. These heuristics can be used by the researchers for the development of robust tools for heuristic evaluation. These tools, when adapted or tailored for health domain applications, have the potential to significantly enhance the quality of mHealth applications. Ultimately, this improvement in quality translates to enhanced patient safety.

Protocol Registration

(10.17605/OSF.IO/PZJ7H).

21st century (clinical) decision support in nursing and allied healthcare. Developing a learning health system: a reasoned design of a theoretical framework

In this paper, we present a framework for developing a Learning Health System (LHS) to provide means to a computerized clinical decision support system for allied healthcare and/or nursing professionals. LHSs are well suited to transform healthcare systems in a mission-oriented approach, and is being adopted by an increasing number of countries. Our theoretical framework provides a blueprint for organizing such a transformation with help of evidence based state of the art methodologies and techniques to eventually optimize personalized health and healthcare. Learning via health information technologies using LHS enables users to learn both individually and collectively, and independent of their location. These developments demand healthcare innovations beyond a disease focused orientation since clinical decision making in allied healthcare and nursing is mainly based on aspects of individuals' functioning, wellbeing and (dis)abilities. Developing LHSs depends heavily on intertwined social and technological innovation, and research and development. Crucial factors may be the transformation of the Internet of Things into the Internet of FAIR data & services. However, Electronic Health Record (EHR) data is in up to 80% unstructured including free text narratives and stored in various inaccessible data warehouses. Enabling the use of data as a driver for learning is challenged by interoperability and reusability.To address technical needs, key enabling technologies are suitable to convert relevant health data into machine actionable data and to develop algorithms for computerized decision support. To enable data conversions, existing classification and terminology systems serve as definition providers for natural language processing through (un)supervised learning.To facilitate clinical reasoning and personalized healthcare using LHSs, the development of personomics and functionomics are useful in allied healthcare and nursing. Developing these omics will be determined via text and data mining. This will focus on the relationships between social, psychological, cultural, behavioral and economic determinants, and human functioning.Furthermore, multiparty collaboration is crucial to develop LHSs, and man-machine interaction studies are required to develop a functional design and prototype. During development, validation and maintenance of the LHS continuous attention for challenges like data-drift, ethical, technical and practical implementation difficulties is required.

Identifying the data elements and functionalities of clinical decision support systems to administer medication for neonates and pediatrics: a systematic literature review

BACKGROUND

Patient safety is a central healthcare policy worldwide. Adverse drug events (ADE) are among the main threats to patient safety. Children are at a higher risk of ADE in each stage of medication management process. ADE rate is high in the administration stage, as the final stage of preventing medication errors in pediatrics and neonates. The most effective way to reduce ADE rate is using medication administration clinical decision support systems (MACDSSs). The present study reviewed the literature on MACDSS for neonates and pediatrics. It identified and classified the data elements that mapped onto the Fast Healthcare Interoperability Resources (FHIR) standard and the functionalities of these systems to guide future research.

METHODS

PubMed/ MEDLINE, Embase, CINAHL, and ProQuest databases were searched from 1995 to June 31, 2021. Studies that addressed developing or applying medication administration software for neonates and pediatrics were included. Two authors reviewed the titles, abstracts, and full texts. The quality of eligible studies was assessed based on the level of evidence. The extracted data elements were mapped onto the FHIR standard.

RESULTS

In the initial search, 4,856 papers were identified. After removing duplicates, 3,761 titles, and abstracts were screened. Finally, 56 full-text papers remained for evaluation. The full-text review of papers led to the retention of 10 papers which met the eligibility criteria. In addition, two papers from the reference lists were included. A total number of 12 papers were included for analysis. Six papers were categorized as high-level evidence. Only three papers evaluated their systems in a real environment. A variety of data elements and functionalities could be observed. Overall, 84 unique data elements were extracted from the included papers. The analysis of reported functionalities showed that 18 functionalities were implemented in these systems.

CONCLUSION

Identifying the data elements and functionalities as a roadmap by developers can significantly improve MACDSS performance. Though many CDSSs have been developed for different medication processes in neonates and pediatrics, few have actually evaluated MACDSSs in reality. Therefore, further research is needed on the application and evaluation of MACDSSs in the real environment.

PROTOCOL REGISTRATION

(dx.doi.org/10.17504/protocols.io.bwbwpape).

Construction and effectiveness evaluation of a knowledge-based infectious disease monitoring and decision support system

To improve the hospital's ability to proactively detect infectious diseases, a knowledge-based infectious disease monitoring and decision support system was established based on real medical records and knowledge rules. The effectiveness of the system was evaluated using interrupted time series analysis. In the system, a monitoring and alert rule library for infectious diseases was generated by combining infectious disease diagnosis guidelines with literature and a real medical record knowledge map. The system was integrated with the electronic medical record system, and doctors were provided with various types of real-time warning prompts when writing medical records. The effectiveness of the system's alerts was analyzed from the perspectives of false positive rates, rule accuracy, alert effectiveness, and missed case rates using interrupted time series analysis. Over a period of 12 months, the system analyzed 4,497,091 medical records, triggering a total of 12,027 monitoring alerts. Of these, 98.43% were clinically effective, while 1.56% were invalid alerts, mainly owing to the relatively rough rules generated by the guidelines leading to several false alarms. In addition, the effectiveness of the system's alerts, distribution of diagnosis times, and reporting efficiency of doctors were analyzed. 89.26% of infectious disease cases could be confirmed and reported by doctors within 5 min of receiving the alert, and 77.6% of doctors could complete the filling of 33 items of information within 2 min, which is a reduction in time compared to the past. The timely reminders from the system reduced the rate of missed cases by doctors; the analysis using interrupted time series method showed an average reduction of 4.4037% in the missed-case rate. This study proposed a knowledge-based infectious disease decision support system based on real medical records and knowledge rules, and its effectiveness was verified. The system improved the management of infectious diseases, increased the reliability of decision-making, and reduced the rate of underreporting.

Identifying barriers and facilitators to successful implementation of computerized clinical decision support systems in hospitals: a NASSS framework-informed scoping review

BACKGROUND

Successful implementation and utilization of Computerized Clinical Decision Support Systems (CDSS) in hospitals is complex and challenging. Implementation science, and in particular the Nonadoption, Abandonment, Scale-up, Spread and Sustainability (NASSS) framework, may offer a systematic approach for identifying and addressing these challenges. This review aimed to identify, categorize, and describe barriers and facilitators to CDSS implementation in hospital settings and map them to the NASSS framework. Exploring the applicability of the NASSS framework to CDSS implementation was a secondary aim.

METHODS

Electronic database searches were conducted (21 July 2020; updated 5 April 2022) in Ovid MEDLINE, Embase, Scopus, PyscInfo, and CINAHL. Original research studies reporting on measured or perceived barriers and/or facilitators to implementation and adoption of CDSS in hospital settings, or attitudes of healthcare professionals towards CDSS were included. Articles with a primary focus on CDSS development were excluded. No language or date restrictions were applied. We used qualitative content analysis to identify determinants and organize them into higher-order themes, which were then reflexively mapped to the NASSS framework.

RESULTS

Forty-four publications were included. These comprised a range of study designs, geographic locations, participants, technology types, CDSS functions, and clinical contexts of implementation. A total of 227 individual barriers and 130 individual facilitators were identified across the included studies. The most commonly reported influences on implementation were fit of CDSS with workflows (19 studies), the usefulness of the CDSS output in practice (17 studies), CDSS technical dependencies and design (16 studies), trust of users in the CDSS input data and evidence base (15 studies), and the contextual fit of the CDSS with the user's role or clinical setting (14 studies). Most determinants could be appropriately categorized into domains of the NASSS framework with barriers and facilitators in the "Technology," "Organization," and "Adopters" domains most frequently reported. No determinants were assigned to the "Embedding and Adaptation Over Time" domain.

CONCLUSIONS

This review identified the most common determinants which could be targeted for modification to either remove barriers or facilitate the adoption and use of CDSS within hospitals. Greater adoption of implementation theory should be encouraged to support CDSS implementation.

Semantically enabling clinical decision support recommendations

BACKGROUND

Clinical decision support systems have been widely deployed to guide healthcare decisions on patient diagnosis, treatment choices, and patient management through evidence-based recommendations. These recommendations are typically derived from clinical practice guidelines created by clinical specialties or healthcare organizations. Although there have been many different technical approaches to encoding guideline recommendations into decision support systems, much of the previous work has not focused on enabling system generated recommendations through the formalization of changes in a guideline, the provenance of a recommendation, and applicability of the evidence. Prior work indicates that healthcare providers may not find that guideline-derived recommendations always meet their needs for reasons such as lack of relevance, transparency, time pressure, and applicability to their clinical practice.

RESULTS

We introduce several semantic techniques that model diseases based on clinical practice guidelines, provenance of the guidelines, and the study cohorts they are based on to enhance the capabilities of clinical decision support systems. We have explored ways to enable clinical decision support systems with semantic technologies that can represent and link to details in related items from the scientific literature and quickly adapt to changing information from the guidelines, identifying gaps, and supporting personalized explanations. Previous semantics-driven clinical decision systems have limited support in all these aspects, and we present the ontologies and semantic web based software tools in three distinct areas that are unified using a standard set of ontologies and a custom-built knowledge graph framework: (i) guideline modeling to characterize diseases, (ii) guideline provenance to attach evidence to treatment decisions from authoritative sources, and (iii) study cohort modeling to identify relevant research publications for complicated patients.

CONCLUSIONS

We have enhanced existing, evidence-based knowledge by developing ontologies and software that enables clinicians to conveniently access updates to and provenance of guidelines, as well as gather additional information from research studies applicable to their patients' unique circumstances. Our software solutions leverage many well-used existing biomedical ontologies and build upon decades of knowledge representation and reasoning work, leading to explainable results.

An Ontology-Based Approach to Improving Medication Appropriateness in Older Patients: Algorithm Development and Validation Study

Background: Inappropriate medication in older patients with multimorbidity results in a greater risk of adverse drug events. Clinical decision support systems (CDSSs) are intended to improve medication appropriateness. One approach to improving CDSSs is to use ontologies instead of relational databases. Previously, we developed OntoPharma-an ontology-based CDSS for reducing medication prescribing errors. Objective: The primary aim was to model a domain for improving medication appropriateness in older patients (chronic patient domain). The secondary aim was to implement the version of OntoPharma containing the chronic patient domain in a hospital setting. Methods: A 4-step process was proposed. The first step was defining the domain scope. The chronic patient domain focused on improving medication appropriateness in older patients. A group of experts selected the following three use cases: medication regimen complexity, anticholinergic and sedative drug burden, and the presence of triggers for identifying possible adverse events. The second step was domain model representation. The implementation was conducted by medical informatics specialists and clinical pharmacists using Protégé-OWL (Stanford Center for Biomedical Informatics Research). The third step was OntoPharma-driven alert module adaptation. We reused the existing framework based on SPARQL to query ontologies. The fourth step was implementing the version of OntoPharma containing the chronic patient domain in a hospital setting. Alerts generated from July to September 2022 were analyzed. Results: We proposed 6 new classes and 5 new properties, introducing the necessary changes in the ontologies previously created. An alert is shown if the Medication Regimen Complexity Index is ≥40, if the Drug Burden Index is ≥1, or if there is a trigger based on an abnormal laboratory value. A total of 364 alerts were generated for 107 patients; 154 (42.3%) alerts were accepted. Conclusions: We proposed an ontology-based approach to provide support for improving medication appropriateness in older patients with multimorbidity in a scalable, sustainable, and reusable way. The chronic patient domain was built based on our previous research, reusing the existing framework. OntoPharma has been implemented in clinical practice and generates alerts, considering the following use cases: medication regimen complexity, anticholinergic and sedative drug burden, and the presence of triggers for identifying possible adverse events.

Using graph rewriting to operationalize medical knowledge for the revision of concurrently applied clinical practice guidelines

Clinical practice guidelines (CPGs) are patient management tools that synthesize medical knowledge into an actionable format. CPGs are disease specific with limited applicability to the management of complex patients suffering from multimorbidity. For the management of these patients, CPGs need to be augmented with secondary medical knowledge coming from a variety of knowledge repositories. The operationalization of this knowledge is key to increasing CPGs' uptake in clinical practice. In this work, we propose an approach to operationalizing secondary medical knowledge inspired by graph rewriting. We assume that the CPGs can be represented as task network models, and provide an approach for representing and applying codified medical knowledge to a specific patient encounter. We formally define revisions that model and mitigate adverse interactions between CPGs and we use a vocabulary of terms to instantiate these revisions. We demonstrate the application of our approach using synthetic and clinical examples. We conclude by identifying areas for future work with the vision of developing a theory of mitigation that will facilitate the development of comprehensive decision support for the management of multimorbid patients.

Ontologies Applied in Clinical Decision Support System Rules: Systematic Review

BACKGROUND

Clinical decision support systems (CDSSs) are important for the quality and safety of health care delivery. Although CDSS rules guide CDSS behavior, they are not routinely shared and reused.

OBJECTIVE

Ontologies have the potential to promote the reuse of CDSS rules. Therefore, we systematically screened the literature to elaborate on the current status of ontologies applied in CDSS rules, such as rule management, which uses captured CDSS rule usage data and user feedback data to tailor CDSS services to be more accurate, and maintenance, which updates CDSS rules. Through this systematic literature review, we aim to identify the frontiers of ontologies used in CDSS rules.

METHODS

The literature search was focused on the intersection of ontologies; clinical decision support; and rules in PubMed, the Association for Computing Machinery (ACM) Digital Library, and the Nursing & Allied Health Database. Grounded theory and PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 guidelines were followed. One author initiated the screening and literature review, while 2 authors validated the processes and results independently. The inclusion and exclusion criteria were developed and refined iteratively.

RESULTS

CDSSs were primarily used to manage chronic conditions, alerts for medication prescriptions, reminders for immunizations and preventive services, diagnoses, and treatment recommendations among 81 included publications. The CDSS rules were presented in Semantic Web Rule Language, Jess, or Jena formats. Despite the fact that ontologies have been used to provide medical knowledge, CDSS rules, and terminologies, they have not been used in CDSS rule management or to facilitate the reuse of CDSS rules.

CONCLUSIONS

Ontologies have been used to organize and represent medical knowledge, controlled vocabularies, and the content of CDSS rules. So far, there has been little reuse of CDSS rules. More work is needed to improve the reusability and interoperability of CDSS rules. This review identified and described the ontologies that, despite their limitations, enable Semantic Web technologies and their applications in CDSS rules.

[Ontologies Applied in Clinical Decision Support Systems for Diabetes]

A clinical decision support system (CDSS) integrated with electronic health records helps physicians at the grassroots make patient-appropriate and evidence-based treatment decisions and improves the efficiency of diagnosis and treatment. Furthermore, using ontologies to build up the medical knowledge base and patient data for CDSS enhances the automation and transparency of the reasoning process of CDSS and helps generate interpretable and accurate treatment recommendations. Herein, we reviewed the relevant ontologies in the field of diabetes treatment and the progress and challenges concerning ontology-based CDSSs. Firstly, we elaborated on the current status and challenges of diabetes treatment in China, highlighting the urgent need to improve the efficiency and quality of medical services. Then, we presented background information about ontologies and gave an overview of the framework, methodology, and features of using ontologies to construct CDSS. After that, we reviewed the ontologies and instances of ontology-based CDSS in the field of diabetes treatment in China and abroad and summarized their construction methods and features. Last but not the least, we discussed the future prospects of the field, suggesting that integrating evidence-based medicine with ontologies to build a reliable clinical recommendation system should be the current focus of CDSS development.

OntoPharma: ontology based clinical decision support system to reduce medication prescribing errors

Background

Clinical decision support systems (CDSS) have been shown to reduce medication errors. However, they are underused because of different challenges. One approach to improve CDSS is to use ontologies instead of relational databases. The primary aim was to design and develop OntoPharma, an ontology based CDSS to reduce medication prescribing errors. Secondary aim was to implement OntoPharma in a hospital setting.

Methods

A four-step process was proposed. (1) Defining the ontology domain. The ontology scope was the medication domain. An advisory board selected four use cases: maximum dosage alert, drug-drug interaction checker, renal failure adjustment, and drug allergy checker. (2) Implementing the ontology in a formal representation. The implementation was conducted by Medical Informatics specialists and Clinical Pharmacists using Protégé-OWL. (3) Developing an ontology-driven alert module. Computerised Physician Order Entry (CPOE) integration was performed through a REST API. SPARQL was used to query ontologies. (4) Implementing OntoPharma in a hospital setting. Alerts generated between July 2020/ November 2021 were analysed.

Results

The three ontologies developed included 34,938 classes, 16,672 individuals and 82 properties. The domains addressed by ontologies were identification data of medicinal products, appropriateness drug data, and local concepts from CPOE. When a medication prescribing error is identified an alert is shown. OntoPharma generated 823 alerts in 1046 patients. 401 (48.7%) of them were accepted.

Conclusions

OntoPharma is an ontology based CDSS implemented in clinical practice which generates alerts when a prescribing medication error is identified. To gain user acceptance OntoPharma has been designed and developed by a multidisciplinary team. Compared to CDSS based on relational databases, OntoPharma represents medication knowledge in a more intuitive, extensible and maintainable manner.

How are ontologies implemented to represent clinical practice guidelines in clinical decision support systems: protocol for a systematic review

BACKGROUND

Clinical practice guidelines are statements which are based on the best available evidence, and their goal is to improve the quality of patient care. Integrating clinical practice guidelines into computer systems can help physicians reduce medical errors and help them to have the best possible practice. Guideline-based clinical decision support systems play a significant role in supporting physicians in their decisions. Meantime, system errors are the most critical concerns in designing decision support systems that can affect their performance and efficacy. A well-developed ontology can be helpful in this matter. The proposed systematic review will specify the methods, components, language of rules, and evaluation methods of current ontology-driven guideline-based clinical decision support systems.

METHODS

This review will identify literature through searching MEDLINE (via Ovid), PubMed, EMBASE, Cochrane Library, CINAHL, ScienceDirect, IEEEXplore, and ACM Digital Library. Gray literature, reference lists, and citing articles of the included studies will be searched. The quality of the included studies will be assessed by the mixed methods appraisal tool (MMAT-version 2018). At least two independent reviewers will perform the screening, quality assessment, and data extraction. A third reviewer will resolve any disagreements. Proper data analysis will be performed based on the type of system and ontology engineering evaluation data.

DISCUSSION

The study will provide evidence regarding applying ontologies in guideline-based clinical decision support systems. The findings of this systematic review will be a guide for decision support system designers and developers, technologists, system providers, policymakers, and stakeholders. Ontology builders can use the information in this review to build well-structured ontologies for personalized medicine.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42018106501.

Documentation of hospitalization risk factors in electronic health records (EHRs): a qualitative study with home healthcare clinicians

OBJECTIVE

To identify the risk factors home healthcare (HHC) clinicians associate with patient deterioration and understand how clinicians respond to and document these risk factors.

METHODS

We interviewed multidisciplinary HHC clinicians from January to March of 2021. Risk factors were mapped to standardized terminologies (eg, Omaha System). We used directed content analysis to identify risk factors for deterioration. We used inductive thematic analysis to understand HHC clinicians' response to risk factors and documentation of risk factors.

RESULTS

Fifteen HHC clinicians identified a total of 79 risk factors that were mapped to standardized terminologies. HHC clinicians most frequently responded to risk factors by communicating with the prescribing provider (86.7% of clinicians) or following up with patients and caregivers (86.7%). HHC clinicians stated that a majority of risk factors can be found in clinical notes (ie, care coordination (53.3%) or visit (46.7%)).

DISCUSSION

Clinicians acknowledged that social factors play a role in deterioration risk; but these factors are infrequently studied in HHC. While a majority of risk factors were represented in the Omaha System, additional terminologies are needed to comprehensively capture risk. Since most risk factors are documented in clinical notes, methods such as natural language processing are needed to extract them.

CONCLUSION

This study engaged clinicians to understand risk for deterioration during HHC. The results of our study support the development of an early warning system by providing a comprehensive list of risk factors grounded in clinician expertize and mapped to standardized terminologies.

The use of cognitive task analysis in clinical and health services research — a systematic review

Background

At times, clinical case complexity and different types of uncertainty present challenges to less experienced clinicians or the naive application of clinical guidelines where this may not be appropriate. Cognitive task analysis (CTA) methods are used to elicit, document and transfer tacit knowledge about how experts make decisions.

Methods

We conducted a methodological review to describe the use of CTA methods in understanding expert clinical decision-making. We searched MEDLINE, EMBASE and PsycINFO from inception to 2019 for primary research studies which described the use of CTA methods to understand how qualified clinicians made clinical decisions in real-world clinical settings.

Results

We included 81 articles (80 unique studies) from 13 countries, published from 1993 to 2019, most commonly from surgical and critical care settings. The most common aims were to understand expert decision-making in particular clinical scenarios, using expert decision-making in the development of training programmes, understanding whether decision support tools were warranted and understanding procedural variability and error identification or reduction. Critical decision method (CDM) and CTA interviews were most frequently used, with hierarchical task analysis, task knowledge structures, think-aloud protocols and other methods less commonly used. Studies used interviews, observation, think-aloud exercises, surveys, focus groups and a range of more CTA-specific methodologies such as the systematic human error reduction and prediction approach. Researchers used CTA methods to investigate routine/typical (n = 64), challenging (n = 13) or more uncommon, rare events and anomalies (n = 3).

Conclusions

In conclusion, the elicitation of expert tacit knowledge using CTA has seen increasing use in clinical specialties working under challenging time pressures, complexity and uncertainty. CTA methods have great potential in the development, refinement, modification or adaptation of complex interventions, clinical protocols and practice guidelines.

Registration

PROSPERO ID CRD42019128418.

Supplementary Information

The online version contains supplementary material available at 10.1186/s40814-022-01002-6.

When BERT meets Bilbo: a learning curve analysis of pretrained language model on disease classification

Background

Natural language processing (NLP) tasks in the health domain often deal with limited amount of labeled data due to high annotation costs and naturally rare observations. To compensate for the lack of training data, health NLP researchers often have to leverage knowledge and resources external to a task at hand. Recently, pretrained large-scale language models such as the Bidirectional Encoder Representations from Transformers (BERT) have been proven to be a powerful way of learning rich linguistic knowledge from massive unlabeled text and transferring that knowledge to downstream tasks. However, previous downstream tasks often used training data at such a large scale that is unlikely to obtain in the health domain. In this work, we aim to study whether BERT can still benefit downstream tasks when training data are relatively small in the context of health NLP.

Method

We conducted a learning curve analysis to study the behavior of BERT and baseline models as training data size increases. We observed the classification performance of these models on two disease diagnosis data sets, where some diseases are naturally rare and have very limited observations (fewer than 2 out of 10,000). The baselines included commonly used text classification models such as sparse and dense bag-of-words models, long short-term memory networks, and their variants that leveraged external knowledge. To obtain learning curves, we incremented the amount of training examples per disease from small to large, and measured the classification performance in macro-averaged \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$F_{1}$$\end{document}F1 score.

Results

On the task of classifying all diseases, the learning curves of BERT were consistently above all baselines, significantly outperforming them across the spectrum of training data sizes. But under extreme situations where only one or two training documents per disease were available, BERT was outperformed by linear classifiers with carefully engineered bag-of-words features.

Conclusion

As long as the amount of training documents is not extremely few, fine-tuning a pretrained BERT model is a highly effective approach to health NLP tasks like disease classification. However, in extreme cases where each class has only one or two training documents and no more will be available, simple linear models using bag-of-words features shall be considered.

An interview study with medical scribes on how their work may alleviate clinician burnout through delegated health IT tasks

OBJECTIVES

To understand how medical scribes' work may contribute to alleviating clinician burnout attributable directly or indirectly to the use of health IT.

MATERIALS AND METHODS

Qualitative analysis of semistructured interviews with 32 participants who had scribing experience in a variety of clinical settings.

RESULTS

We identified 7 categories of clinical tasks that clinicians commonly choose to offload to medical scribes, many of which involve delegated use of health IT. These range from notes-taking and computerized data entry to foraging, assembling, and tracking information scattered across multiple clinical information systems. Some common characteristics shared among these tasks include: (1) time-consuming to perform; (2) difficult to remember or keep track of; (3) disruptive to clinical workflow, clinicians' cognitive processes, or patient-provider interactions; (4) perceived to be low-skill "clerical" work; and (5) deemed as adding no value to direct patient care.

DISCUSSION

The fact that clinicians opt to "outsource" certain clinical tasks to medical scribes is a strong indication that performing these tasks is not perceived to be the best use of their time. Given that a vast majority of healthcare practices in the US do not have the luxury of affording medical scribes, the burden would inevitably fall onto clinicians' shoulders, which could be a major source for clinician burnout.

CONCLUSIONS

Medical scribes help to offload a substantial amount of burden from clinicians-particularly with tasks that involve onerous interactions with health IT. Developing a better understanding of medical scribes' work provides useful insights into the sources of clinician burnout and potential solutions to it.

Prediction of Bladder Cancer Treatment Side Effects Using an Ontology-Based Reasoning for Enhanced Patient Health Safety

Predicting potential cancer treatment side effects at time of prescription could decrease potential health risks and achieve better patient satisfaction. This paper presents a new approach, founded on evidence-based medical knowledge, using as much information and proof as possible to help a computer program to predict bladder cancer treatment side effects and support the oncologist’s decision. This will help in deciding treatment options for patients with bladder malignancies. Bladder cancer knowledge is complex and requires simplification before any attempt to represent it in a formal or computerized manner. In this work we rely on the capabilities of OWL ontologies to seamlessly capture and conceptualize the required knowledge about this type of cancer and the underlying patient treatment process. Our ontology allows case-based reasoning to effectively predict treatment side effects for a given set of contextual information related to a specific medical case. The ontology is enriched with proofs and evidence collected from online biomedical research databases using “web crawlers”. We have exclusively designed the crawler algorithm to search for the required knowledge based on a set of specified keywords. Results from the study presented 80.3% of real reported bladder cancer treatment side-effects prediction and were close to really occurring adverse events recorded within the collected test samples when applying the approach. Evidence-based medicine combined with semantic knowledge-based models is prominent in generating predictions related to possible health concerns. The integration of a diversity of knowledge and evidence into one single integrated knowledge-base could dramatically enhance the process of predicting treatment risks and side effects applied to bladder cancer oncotherapy.

A Service-oriented Framework for Developing Personalized Patient Care Plans for COVID-19

The COVID-19 pandemic has identified weaknesses and stresses in the existing healthcare and governance system, even in the most developed countries. Given the scale and scope of the pandemic, existing healthcare systems are heavily resource constrained, and home-based isolation has been considered as a potential first step for reducing both the disease spread and the stress on the healthcare system. However, the needs and requirements of home-based isolation are extremely unique for each patient, depending on their medical condition and comorbidities, family responsibilities, and environmental constraints. Therefore, it is necessary to develop personalized patient care plans to ensure that the needs of each patient are appropriately met. In this paper we propose a service oriented framework that allows dynamic composition and management of such plans assuming existence of an appropriate knowledge base and availability of web-services interfaces of the underlying systems of caregivers and service providers. We develop a prototype implementation to show the feasibility of the proposed framework and discuss the challenges/issues in deploying such a system in practice.

Three ways of knowing: the integration of clinical expertise, evidence-based medicine, and artificial intelligence in assisted reproductive technologies

Decision-making in fertility care is on the cusp of a significant frameshift. Online tools to integrate artificial intelligence into the decision-making process across all aspects of ART are rapidly emerging. These tools have the potential to improve outcomes and transition decision-making from one based on traditional provider centric assessments toward a hybrid triad of expertise, evidence, and algorithmic data analytics using AI. We can look forward to a time when AI will be the third part of a provider's tool box to complement expertise and medical literature to enable ever more accurate predictions and outcomes in ART. In their fully integrated format, these tools will be part of a digital fertility ecosystem of analytics embedded within an EMR. To date, the impact of AI on ART outcomes is inconclusive. No prospective studies have shown clear cut benefit or cost reductions over current practices, but we are very early in the process of developing and evaluating these tools. We owe it to ourselves to begin to examine these AI-driven analytics and develop a very clear idea about where we can and should go before we roll these tools into clinical care. Thoughtful scrutiny is essential lest we find ourselves in a position of trying to modulate and modify after entry of these tools into our clinics and patient care. The purpose of this commentary is to highlight the evolution and impact AI has had in other fields relevant to the fertility sector and describe a vision for applications within ART that could improve outcomes, reduce costs, and positively impact clinical care.

Formal representation of patients' care context data: the path to improving the electronic health record

Objective

To develop a collection of concept-relationship-concept tuples to formally represent patients’ care context data to inform electronic health record (EHR) development.

Materials and Methods

We reviewed semantic relationships reported in the literature and developed a manual annotation schema. We used the initial schema to annotate sentences extracted from narrative note sections of cardiology, urology, and ear, nose, and throat (ENT) notes. We audio recorded ENT visits and annotated their parsed transcripts. We combined the results of each annotation into a consolidated set of concept-relationship-concept tuples. We then compared the tuples used within and across the multiple data sources.

Results

We annotated a total of 626 sentences. Starting with 8 relationships from the literature, we annotated 182 sentences from 8 inpatient consult notes (initial set of tuples = 43). Next, we annotated 232 sentences from 10 outpatient visit notes (enhanced set of tuples = 75). Then, we annotated 212 sentences from transcripts of 5 outpatient visits (final set of tuples = 82). The tuples from the visit transcripts covered 103 (74%) concepts documented in the notes of their respective visits. There were 20 (24%) tuples used across all data sources, 10 (12%) used only in inpatient notes, 15 (18%) used only in visit notes, and 7 (9%) used only in the visit transcripts.

Conclusions

We produced a robust set of 82 tuples useful to represent patients’ care context data. We propose several applications of our tuples to improve EHR navigation, data entry, learning health systems, and decision support.

Capturing Clinician Reasoning in Electronic Health Records: An Exploratory Study of Under-Treated Essential Hypertension

Monitoring response to antihypertensive medications is a frequent reason for outpatient visits. Blood pressure (BP) is often documented as elevated, but no change in medication occurs (Medication Non-adjustment or MNA). We studied the frequency of MNA, reasons for non-adjustment, how reasons (including reasons for patient nonadherence) were documented, and whether they could be represented in a clinical care context ontology. We examined 129 visit notes with MNA occurring in 80 cases (59%). We coded MNA as Conscious Maintenance (patient adherent but clinician continues therapy for stated reason), Nonadherence (clinician attributes BP elevation to patient nonadherence), and Finding Not Addressed (clinician does not indicate reasoning for MNA). We characterized Conscious Maintenance with 11 subcodes and Nonadherence with 6 subcodes. Our ontology successfully represented relationships between concepts and reasoning, supporting the feasibility of formal representation of clinical care contexts for patient care, decision support and research.

The anatomy of clinical documentation: an assessment and classification of narrative note sections format and content

Introduction. We systematically analyzed the most commonly used narrative note formats and content found in primary and specialty care visit notes to inform future research and electronic health record (EHR) development. Methods. We extracted data from the history of present illness (HPI) and impression and plan (IP) sections of 80 primary and specialty care visit notes. Two authors iteratively classified the format of the sections and compared the size of each section and the overall note size between primary and specialty care notes. We then annotated the content of these sections to develop a taxonomy of types of data communicated in the narrative note sections. Results. Both HPI and IP were significantly longer in primary care when compared to specialty care notes (HPI: n = 187 words, SD[130] vs. n = 119 words, SD [53]; p = 0.004 / IP: n = 270 words, SD [145] vs. n = 170 words, SD [101]; p < 0.001). Although we did not find a significant difference in the overall note size between the two groups, the proportion of HPI and IP content in relation to the total note size was significantly higher in primary care notes (40%, SD [13] vs. 28%, SD [11]; p < 0.001). We identified five combinations of format of HPI + IP sections respectively: (A) story + list with categories; (B) story + story; (C) list without categories + list with categories; (D) list with categories + list with categories; and (E) list with categories + story. HPI and IP content was significantly smaller in combination C compared to combination A (-172 words, [95% Conf. -326, -17.89]; p = 0.02). We identified seven taxa representing 45 different types of data: finding/condition documented (n = 14), intervention documented (n = 9), general descriptions and definitions (n = 7), temporal information (n = 6), reasons and justifications (n = 4), participants and settings (n = 4), and clinical documentation (n = 1). Conclusion. We identified commonly used narrative note section formats and developed a taxonomy of narrative note content to help researchers to tailor their efforts and design more efficient clinical documentation systems.