A Systematic Literature Review of Agents Applied in Healthcare

The Case Manager: An Agent Controlling the Activation of Knowledge Sources in a FHIR-Based Distributed Reasoning Environment

BACKGROUND

Within the CAPABLE project the authors developed a multi-agent system that relies on a distributed architecture. The system provides cancer patients with coaching advice and supports their clinicians with suitable decisions based on clinical guidelines.

OBJECTIVES

As in many multi-agent systems we needed to coordinate the activities of all agents involved. Moreover, since the agents share a common blackboard where all patients' data are stored, we also needed to implement a mechanism for the prompt notification of each agent upon addition of new information potentially triggering its activation.

METHODS

The communication needs have been investigated and modeled using the HL7-FHIR (Health Level 7-Fast Healthcare Interoperability Resources) standard to ensure proper semantic interoperability among agents. Then a syntax rooted in the FHIR search framework has been defined for representing the conditions to be monitored on the system blackboard for activating each agent.

RESULTS

The Case Manager (CM) has been implemented as a dedicated component playing the role of an orchestrator directing the behavior of all agents involved. Agents dynamically inform the CM about the conditions to be monitored on the blackboard, using the syntax we developed. The CM then notifies each agent whenever any condition of interest occurs. The functionalities of the CM and other actors have been validated using simulated scenarios mimicking the ones that will be faced during pilot studies and in production.

CONCLUSION

The CM proved to be a key facilitator for properly achieving the required behavior of our multi-agent system. The proposed architecture may also be leveraged in many clinical contexts for integrating separate legacy services, turning them into a consistent telemedicine framework and enabling application reusability.

A survey on agents applications in healthcare: Opportunities, challenges and trends

BACKGROUND AND OBJECTIVE

The agent abstraction is a powerful one, developed decades ago to represent crucial aspects of artificial intelligence research. The meaning has transformed over the years and now there are different nuances across research communities. At its core, an agent is an autonomous computational entity capable of sensing, acting, and capturing interactions with other agents and its environment. This review examines how agent-based techniques have been implemented and evaluated in a specific and very important domain, i.e. healthcare research.

METHODS

We survey key areas of agent-based research in healthcare, e.g. individual and collective behaviours, communicable and non-communicable diseases, and social epidemiology. We propose a systematic search and critical review of relevant recent works, introduced by an exploratory network analysis.

RESULTS

Network analysis enables to devise out 5 main research clusters, the most active authors, and 4 main research topics.

CONCLUSIONS

Our findings support discussion of some future directions for increasing the value of agent-based approaches in healthcare.

Exploring the influence of a user-specific explainable virtual advisor on health behaviour change intentions

Virtual advisors (VAs) are being utilised almost in every service nowadays from entertainment to healthcare. To increase the user's trust in these VAs and encourage the users to follow their advice, they should have the capability of explaining their decisions, particularly, when the decision is vital such as health advice. However, the role of an explainable VA in health behaviour change is understudied. There is evidence that people tend to change their intentions towards health behaviour when the persuasion message is linked to their mental state. Thus, this study explores this link by introducing an explainable VA that provides explanation according to the user's mental state (beliefs and goals) rather than the agent's mental state as commonly utilised in explainable agents. It further explores the influence of different explanation patterns that refer to beliefs, goals, or beliefs&goals on the user's behaviour change. An explainable VA was designed to advise undergraduate students how to manage their study-related stress by motivating them to change certain behaviours. With 91 participants, the VA was evaluated and the results revealed that user-specific explanation could significantly encourage behaviour change intentions and build good user-agent relationship. Small differences were found between the three types of explanation patterns.

Improving the Survival Time of Multiagents in Social Dilemmas through Neurotransmitter-Based Deep Q-Learning Model of Emotions

In multiagent systems, social dilemmas often arise whenever there is a competition over the limited resources. The major challenge is to establish cooperation among intelligent virtual agents for solving the situations of social dilemmas. In humans, personality and emotions are the primary factors that lead them toward a cooperative environment. To make agents cooperate, they have to become more like humans, that is, believable. Therefore, we hypothesize that emotions according to the personality give birth to believability, and if believability is introduced into agents through emotions, it improves their survival rate in social dilemma situations. The existing researches have introduced different computational models to introduce emotions in virtual agents, but they lack emotions through neurotransmitters. We have proposed a neurotransmitters-based deep Q-learning computational model in multiagents that is a suitable choice for emotion modeling and, hence, believability. The proposed model regulates the agents' emotions by controlling the virtual neurotransmitters (dopamine and oxytocin) according to the agent's personality. The personality of the agent is introduced using OCEAN model. To evaluate the proposed system, we simulated a survival scenario with limited food resources in different experiments. These experiments vary the number of selfish agents (higher neuroticism personality trait) and the selfless agents (higher agreeableness personality trait). Experimental results show that by adding the selfless agents in the scenario, the agents develop cooperation, and their collective survival time increases. Thus, to resolve the social dilemma problems in virtual agents, we can make agents believable through the proposed neurotransmitter-based emotional model. This proposed work may help in developing nonplayer characters (NPCs) in games.

A Recommendation System Based on AI for Storing Block Data in the Electronic Health Repository

The proliferation of wearable sensors that record physiological signals has resulted in an exponential growth of data on digital health. To select the appropriate repository for the increasing amount of collected data, intelligent procedures are becoming increasingly necessary. However, allocating storage space is a nuanced process. Generally, patients have some input in choosing which repository to use, although they are not always responsible for this decision. Patients are likely to have idiosyncratic storage preferences based on their unique circumstances. The purpose of the current study is to develop a new predictive model of health data storage to meet the needs of patients while ensuring rapid storage decisions, even when data is streaming from wearable devices. To create the machine learning classifier, we used a training set synthesized from small samples of experts who exhibited correlations between health data and storage features. The results confirm the validity of the machine learning methodology.

In Search of Embodied Conversational and Explainable Agents for Health Behaviour Change and Adherence

Conversational agents offer promise to provide an alternative to costly and scarce access to human health providers. Particularly in the context of adherence to treatment advice and health behavior change, they can provide an ongoing coaching role to motivate and keep the health consumer on track. Due to the recognized importance of face-to-face communication and establishment of a therapist-patient working alliance as the biggest single predictor of adherence, our review focuses on embodied conversational agents (ECAs) and their use in health and well-being interventions. The article also introduces ECAs who provide explanations of their recommendations, known as explainable agents (XAs), as a way to build trust and enhance the working alliance towards improved behavior change. Of particular promise, is work in which XAs are able to engage in conversation to learn about their user and personalize their recommendations based on their knowledge of the user and then tailor their explanations to the beliefs and goals of the user to increase relevancy and motivation and address possible barriers to increase intention to perform the healthy behavior.

The Clinical Decision Support System AMPEL for Laboratory Diagnostics: Implementation and Technical Evaluation

Background

Laboratory results are of central importance for clinical decision making. The time span between availability and review of results by clinicians is crucial to patient care. Clinical decision support systems (CDSS) are computational tools that can identify critical values automatically and help decrease treatment delay.

Objective

With this work, we aimed to implement and evaluate a CDSS that supports health care professionals and improves patient safety. In addition to our experiences, we also describe its main components in a general manner to make it applicable to a wide range of medical institutions and to empower colleagues to implement a similar system in their facilities.

Methods

Technical requirements must be taken into account before implementing a CDSS that performs laboratory diagnostics (labCDSS). These can be planned within the functional components of a reactive software agent, a computational framework for such a CDSS.

Results

We present AMPEL (Analysis and Reporting System for the Improvement of Patient Safety through Real-Time Integration of Laboratory Findings), a labCDSS that notifies health care professionals if a life-threatening medical condition is detected. We developed and implemented AMPEL at a university hospital and regional hospitals in Germany (University of Leipzig Medical Center and the Muldental Clinics in Grimma and Wurzen). It currently runs 5 different algorithms in parallel: hypokalemia, hypercalcemia, hyponatremia, hyperlactatemia, and acute kidney injury.

Conclusions

AMPEL enables continuous surveillance of patients. The system is constantly being evaluated and extended and has the capacity for many more algorithms. We hope to encourage colleagues from other institutions to design and implement similar CDSS using the theory, specifications, and experiences described in this work.

A Review of Agent-Based Programming for Multi-Agent Systems

Intelligent and autonomous agents is a subarea of symbolic artificial intelligence where these agents decide, either reactively or proactively, upon a course of action by reasoning about the information that is available about the world (including the environment, the agent itself, and other agents). It encompasses a multitude of techniques, such as negotiation protocols, agent simulation, multi-agent argumentation, multi-agent planning, and many others. In this paper, we focus on agent programming and we provide a systematic review of the literature in agent-based programming for multi-agent systems. In particular, we discuss both veteran (still maintained) and novel agent programming languages, their extensions, work on comparing some of these languages, and applications found in the literature that make use of agent programming.

Interactions of Approach and Avoidance Job Crafting and Work Engagement: A Comparison between Employees Affected and Not Affected by Organizational Changes

Job crafting describes proactive employee behaviors to improve the design of their work and working conditions, and to adapt their job to better suit their abilities and needs. During organizational changes, employees may use job crafting to adjust to the changes in their work and protect their well-being and motivation, i.e., work engagement. However, research shows that although the effects of job crafting strategies that expand the design of work (approach job crafting) have been positive on work engagement, the effects of job crafting strategies that diminish the scope of work (avoidance job crafting) have often been negative. This study investigated the effects of the interactions between different job crafting strategies on work engagement, an aspect that has not thus far been studied. Specifically, we hypothesized that avoidance job crafting is not harmful for work engagement when it is conducted in combination with approach job crafting, particularly during times of organizational change. A two-wave, 18-month follow-up study was conducted among public sector workers who either experienced (n = 479) or did not experience (n = 412) changes in their work. Latent moderated structural equation modeling revealed that avoidance job crafting did not reduce work engagement when combined with approach job crafting behaviors. Moreover, job crafting best benefited work engagement when it was combined with these opposing strategies. However, job crafting was beneficial for work engagement only among employees who were affected by organizational changes, that is, among employees whose job design had changed. Practically, organizations implementing changes could encourage proactive job redesign approaches among their employees—particularly both approach and avoidance types of job crafting strategies.

Rapid health data repository allocation using predictive machine learning

Health-related data is stored in a number of repositories that are managed and controlled by different entities. For instance, Electronic Health Records are usually administered by governments. Electronic Medical Records are typically controlled by health care providers, whereas Personal Health Records are managed directly by patients. Recently, Blockchain-based health record systems largely regulated by technology have emerged as another type of repository. Repositories for storing health data differ from one another based on cost, level of security and quality of performance. Not only has the type of repositories increased in recent years, but the quantum of health data to be stored has increased. For instance, the advent of wearable sensors that capture physiological signs has resulted in an exponential growth in digital health data. The increase in the types of repository and amount of data has driven a need for intelligent processes to select appropriate repositories as data is collected. However, the storage allocation decision is complex and nuanced. The challenges are exacerbated when health data are continuously streamed, as is the case with wearable sensors. Although patients are not always solely responsible for determining which repository should be used, they typically have some input into this decision. Patients can be expected to have idiosyncratic preferences regarding storage decisions depending on their unique contexts. In this paper, we propose a predictive model for the storage of health data that can meet patient needs and make storage decisions rapidly, in real-time, even with data streaming from wearable sensors. The model is built with a machine learning classifier that learns the mapping between characteristics of health data and features of storage repositories from a training set generated synthetically from correlations evident from small samples of experts. Results from the evaluation demonstrate the viability of the machine learning technique used.

Agent-oriented Decision Support System for Business Processes Management with Genetic Algorithm Optimization: an Application in Healthcare

Agent-based approaches have been known to be appropriate as systems and methods in medical administration in recent years. The increased attention to processes led to the recent growth of Business Process Management discipline, which quite exclusively adopt discrete-event modeling and simulation. This paper proposes a medical agent-oriented decision support system to integrate the achievements from management science, agent-based modeling, and artificial intelligence. In particular, we performed a practical application concerning a hospital emergency department medical system. We adopt the widely used multi-agent programmable modeling environment NetLogo. First, we demonstrated the ability to perform a clear representation of healthcare processes where agents (i.e., patients and hospital staff) operate in a 3D environment. This model allows performing a traditional what-if scenario analysis. Second, we explore how performing intelligent management of patients by applying genetic algorithms to find the criteria for the selection process of the subjects in the admission procedure. The results are encouraging towards a more extensive application of agent-oriented methodologies in healthcare management.

An intelligent real-time scheduler for out-patient clinics: A multi-agent system model

Scheduling of resources and patients are crucial in outpatient clinics, particularly when the patient demand is high and patient arrivals are random. Generally, outpatient clinic systems are push systems where scheduling is based on average demand prediction and is considered for long term (monthly or bimonthly). Often, planning and actual scenario vary due to uncertainty and variability in demand and this mismatch results in prolonged waiting times and under-utilization of resources. In this article, we model an outpatient clinics as a multi-agent system and propose an intelligent real-time scheduler that schedules patients and resources based on the actual status of departments. Two algorithms are implemented: one for resource scheduling that is based on predictive demand and the other is patient scheduling which performs path optimization depending on the actual status of departments. In order to match resources with stochastic demand, a coordination mechanism is developed that reschedules the resources in the outpatient clinics in real time through auction-bidding procedures. First, a simulation study of intelligent real-time scheduler is carried out followed by implementation of the same in an outpatient clinic of Aravind Eye Hospital, Madurai, India. This hospital has huge patient demand and the patient arrivals are random. The results show that the intelligent real-time scheduler improved the performance measures like waiting time, cycle time, and utilization significantly compared to scheduling of resources and patients in isolation. By scheduling resources and patients, based on system status and demand, the outpatient clinic system becomes a pull system. This scheduler transforms outpatient clinics from open loop system to closed-loop system.

Context-Aware Systems for Chronic Disease Patients: Scoping Review

Background

Context-aware systems, also known as context-sensitive systems, are computing applications designed to capture, interpret, and use contextual information and provide adaptive services according to the current context of use. Context-aware systems have the potential to support patients with chronic conditions; however, little is known about how such systems have been utilized to facilitate patient work.

Objective

This study aimed to characterize the different tasks and contexts in which context-aware systems for patient work were used as well as to assess any existing evidence about the impact of such systems on health-related process or outcome measures.

Methods

A total of 6 databases (MEDLINE, EMBASE, CINAHL, ACM Digital, Web of Science, and Scopus) were scanned using a predefined search strategy. Studies were included in the review if they focused on patients with chronic conditions, involved the use of a context-aware system to support patients’ health-related activities, and reported the evaluation of the systems by the users. Studies were screened by independent reviewers, and a narrative synthesis of included studies was conducted.

Results

The database search retrieved 1478 citations; 6 papers were included, all published from 2009 onwards. The majority of the papers were quasi-experimental and involved pilot and usability testing with a small number of users; there were no randomized controlled trials (RCTs) to evaluate the efficacy of a context-aware system. In the included studies, context was captured using sensors or self-reports, sometimes involving both. Most studies used a combination of sensor technology and mobile apps to deliver personalized feedback. A total of 3 studies examined the impact of interventions on health-related measures, showing positive results.

Conclusions

The use of context-aware systems to support patient work is an emerging area of research. RCTs are needed to evaluate the effectiveness of context-aware systems in improving patient work, self-management practices, and health outcomes in chronic disease patients.

Using Health Chatbots for Behavior Change: A Mapping Study

This study conducts a mapping study to survey the landscape of health chatbots along three research questions: What illnesses are chatbots tackling? What patient competences are chatbots aimed at? Which chatbot technical enablers are of most interest in the health domain? We identify 30 articles related to health chatbots from 2014 to 2018. We analyze the selected articles qualitatively and extract a triplet <technicalEnablers, competence, illness> for each of them. This data serves to provide a first overview of chatbot-mediated behavior change on the health domain. Main insights include: nutritional disorders and neurological disorders as the main illness areas being tackled; "affect" as the human competence most pursued by chatbots to attain change behavior; and "personalization" and "consumability" as the most appreciated technical enablers. On the other hand, main limitations include lack of adherence to good practices to case-study reporting, and a deeper look at the broader sociological implications brought by this technology.

BDI personal medical assistant agents: The case of trauma tracking and alerting

Personal assistant agents can have an important role in healthcare as a smart technology to support physicians in their daily work, helping to tackle the increasing complexity of their task environment. In this paper we present and discuss a personal medical assistant agent technology for trauma documentation and management, based on the Belief-Desire-Intention (BDI) architecture. The purpose of the personal assistant agent is twofold: to assist the Trauma Team in doing precision tracking during a trauma resuscitation, so as to (automatically) produce an accurate documentation of the trauma, and to generate alerts at real-time, to be eventually displayed either on smart-glasses or room-display.

Indexing the Event Calculus: Towards practical human-readable Personal Health Systems

Personal Health Systems (PHS) are mobile solutions tailored to monitoring patients affected by chronic non communicable diseases. In general, a patient affected by a chronic disease can generate large amounts of events: for example, in Type 1 Diabetic patients generate several glucose events per day, ranging from at least 6 events per day (under normal monitoring) to 288 per day when wearing a continuous glucose monitor (CGM) that samples the blood every 5 minutes for several days. Just by itself, without considering other physiological parameters, it would be impossible for medical doctors to individually and accurately follow every patient, highlighting the need of simple approaches towards querying physiological time series. Achieving this with current technology is not an easy task, as on one hand it cannot be expected that medical doctors have the technical knowledge to query databases and on the other hand these time series include thousands of events, which requires to re-think the way data is indexed. Anyhow, handling data streams efficiently is not enough. Domain experts' knowledge must be explicitly included into PHSs in a way that it can be easily readed and modified by medical staffs. Logic programming represents the perfect programming paradygm to accomplish this task. In this work, an Event Calculus-based reasoning framework to standardize and express domain-knowledge in the form of monitoring rules is suggested, and applied to three different use cases. However, if online monitoring has to be achieved, the reasoning performance must improve dramatically. For this reason, three promising mechanisms to index the Event Calculus Knowledge Base are proposed. All of them are based on different types of tree indexing structures: k-d trees, interval trees and red-black trees. The paper then compares and analyzes the performance of the three indexing techniques, by computing the time needed to check different type of rules (and eventually generating alerts), when the number of recorded events (e.g. values of physiological parameters) increases. The results show that customized jREC performs much better when the event average inter-arrival time is little compared to the checked rule time-window. Instead, where the events are more sparse, the use of k-d trees with standard EC is advisable. Finally, the Multi-Agent paradigm helps to wrap the various components of the system: the reasoning engines represent the agent minds, and the sensors are its body. The said agents have been developed in MAGPIE, a mobile event based Java agent platform.

Simulation of patient flow in multiple healthcare units using process and data mining techniques for model identification

INTRODUCTION

An approach to building a hybrid simulation of patient flow is introduced with a combination of data-driven methods for automation of model identification. The approach is described with a conceptual framework and basic methods for combination of different techniques. The implementation of the proposed approach for simulation of the acute coronary syndrome (ACS) was developed and used in an experimental study.

METHODS

A combination of data, text, process mining techniques, and machine learning approaches for the analysis of electronic health records (EHRs) with discrete-event simulation (DES) and queueing theory for the simulation of patient flow was proposed. The performed analysis of EHRs for ACS patients enabled identification of several classes of clinical pathways (CPs) which were used to implement a more realistic simulation of the patient flow. The developed solution was implemented using Python libraries (SimPy, SciPy, and others).

RESULTS

The proposed approach enables more a realistic and detailed simulation of the patient flow within a group of related departments. An experimental study shows an improved simulation of patient length of stay for ACS patient flow obtained from EHRs in Almazov National Medical Research Centre in Saint Petersburg, Russia.

CONCLUSION

The proposed approach, methods, and solutions provide a conceptual, methodological, and programming framework for the implementation of a simulation of complex and diverse scenarios within a flow of patients for different purposes: decision making, training, management optimization, and others.

The Emergence of Openness in Open-Source Projects: The Case of OpenEhR

The meaning of openness in open source is both intrinsically unstable and dynamic, and tends to fluctuate with time and context. We draw on a very particular open-source project primarily concerned with building rigorous clinical concepts to be used in electronic health records called openEHR. openEHR explains how openness is a concept that is purposely engaged with, and how, in this process of engagement, the very meaning of open matures and evolves within the project. Drawing on rich longitudinal data related to openEHR we theorise the evolving nature of openness and how this idea emerges through two intertwined processes of maturation and metamorphosis. While metamorphosis allows us to trace and interrogate the mutational evolution in openness, maturation analyses the small, careful changes crafted to build a very particular understanding of openness. Metamorphosis is less managed and controlled, whereas maturation is representative of highly precise work carried out in controlled form. Both processes work together in open-source projects and reinforce each other. Our study reveals that openness emerges and evolves in open-source projects where it can be understood to mean rigour; ability to participate; open implementation; and an open process. Our work contributes to a deepening in the theorisation of what it means to be an open-source project. The multiple and co-existing meanings of ‘open’ imply that open-source projects evolve in nonlinear ways where each critical meaning of openness causes a reflective questioning by the community of its continued status and existence.

Simulation Modelling in Healthcare: An Umbrella Review of Systematic Literature Reviews

BACKGROUND

Numerous studies examine simulation modelling in healthcare. These studies present a bewildering array of simulation techniques and applications, making it challenging to characterise the literature.

OBJECTIVE

The aim of this paper is to provide an overview of the level of activity of simulation modelling in healthcare and the key themes.

METHODS

We performed an umbrella review of systematic literature reviews of simulation modelling in healthcare. Searches were conducted of academic databases (JSTOR, Scopus, PubMed, IEEE, SAGE, ACM, Wiley Online Library, ScienceDirect) and grey literature sources, enhanced by citation searches. The articles were included if they performed a systematic review of simulation modelling techniques in healthcare. After quality assessment of all included articles, data were extracted on numbers of studies included in each review, types of applications, techniques used for simulation modelling, data sources and simulation software.

RESULTS

The search strategy yielded a total of 117 potential articles. Following sifting, 37 heterogeneous reviews were included. Most reviews achieved moderate quality rating on a modified AMSTAR (A Measurement Tool used to Assess systematic Reviews) checklist. All the review articles described the types of applications used for simulation modelling; 15 reviews described techniques used for simulation modelling; three reviews described data sources used for simulation modelling; and six reviews described software used for simulation modelling. The remaining reviews either did not report or did not provide enough detail for the data to be extracted.

CONCLUSION

Simulation modelling techniques have been used for a wide range of applications in healthcare, with a variety of software tools and data sources. The number of reviews published in recent years suggest an increased interest in simulation modelling in healthcare.

Embodied Conversational Agents for the Detection and Prevention of Suicidal Behaviour: Current Applications and Open Challenges

Embodied conversational agents (ECAs) are advanced computational interactive interfaces designed with the aim to engage users in the continuous and long-term use of a background application. The advantages and benefits of these agents have been exploited in several e-health systems. One of the medical domains where ECAs are recently applied is to support the detection of symptoms, prevention and treatment of mental health disorders. As ECAs based applications are increasingly used in clinical psychology, and due that one fatal consequence of mental health problems is the commitment of suicide, it is necessary to analyse how current ECAs in this clinical domain support the early detection and prevention of risk situations associated with suicidality. The present work provides and overview of the main features implemented in the ECAs to detect and prevent suicidal behaviours through two scenarios: ECAs acting as virtual counsellors to offer immediate help to individuals in risk; and ECAs acting as virtual patients for learning/training in the identification of suicide behaviours. A literature review was performed to identify relevant studies in this domain during the last decade, describing the main characteristics of the implemented ECAs and how they have been evaluated. A total of six studies were included in the review fulfilling the defined search criteria. Most of the experimental studies indicate promising results, though these types of ECAs are not yet commonly used in routine practice. The identification of some open challenges for the further development of ECAs within this domain is also discussed.