Integrating Laboratory Testing Results at Point-of-Care in Hospital@Home Care Settings: A FHIR-Based Approach

The care model Hospital@Home offers hospital-level treatment at home, aiming to alleviate hospital strain and enhance patient comfort. Despite its potential, integrating digital health solutions into this care model still remains limited. This paper proposes a concept for integrating laboratory testing at the Point of Care (POC) into Hospital@Home models to improve efficiency and interoperability.

METHODS

Using the HL7 FHIR standard and cloud infrastructure, we developed a concept for direct transmission of laboratory data collected at POC. Requirements were derived from literature and discussions with a POC testing device producer. An architecture for data exchange was developed based on these requirements.

RESULTS

Our concept enables access to laboratory data collected at POC, facilitating efficient data transfer and enhancing interoperability. A hypothetical scenario demonstrates the concept's feasibility and benefits, showcasing improved patient care and streamlined processes in Hospital@Home settings.

CONCLUSIONS

Integration of POC data into Hospital@Home models using the HL7 FHIR standard and cloud infrastructure offers potential to enhance patient care and streamline processes. Addressing challenges such as data security and privacy is crucial for its successful implementation into practice.

A Comprehensive Framework for Hospital@Home Care Models

Hospital@home is a healthcare approach, where patients receive active treatment from health professionals in their own home for conditions that would normally necessitate a hospital stay.

OBJECTIVE

To develop a framework of relevant features for describing hospital@home care models.

METHODS

The framework was developed based on a literature review and thematic analysis. We considered 42 papers describing hospital@home care approaches. Extracted features were grouped and aggregated in a framework.

RESULTS

The framework consists of nine dimensions: Persons involved, target patient population, service delivery, intended outcome, first point of contact, technology involved, quality, and data collection. The framework provides a comprehensive list of required roles, technologies and service types.

CONCLUSION

The framework can act as a guide for researchers to develop new technologies or interventions to improve hospital@home, particularly in areas such as tele-health, wearable technology, and patient self-management tools. Healthcare providers can use the framework as a guide or blueprint for building or expanding upon their hospital@home services.

Potential of Large Language Models in Health Care: Delphi Study

BACKGROUND

A large language model (LLM) is a machine learning model inferred from text data that captures subtle patterns of language use in context. Modern LLMs are based on neural network architectures that incorporate transformer methods. They allow the model to relate words together through attention to multiple words in a text sequence. LLMs have been shown to be highly effective for a range of tasks in natural language processing (NLP), including classification and information extraction tasks and generative applications.

OBJECTIVE

The aim of this adapted Delphi study was to collect researchers' opinions on how LLMs might influence health care and on the strengths, weaknesses, opportunities, and threats of LLM use in health care.

METHODS

We invited researchers in the fields of health informatics, nursing informatics, and medical NLP to share their opinions on LLM use in health care. We started the first round with open questions based on our strengths, weaknesses, opportunities, and threats framework. In the second and third round, the participants scored these items.

RESULTS

The first, second, and third rounds had 28, 23, and 21 participants, respectively. Almost all participants (26/28, 93% in round 1 and 20/21, 95% in round 3) were affiliated with academic institutions. Agreement was reached on 103 items related to use cases, benefits, risks, reliability, adoption aspects, and the future of LLMs in health care. Participants offered several use cases, including supporting clinical tasks, documentation tasks, and medical research and education, and agreed that LLM-based systems will act as health assistants for patient education. The agreed-upon benefits included increased efficiency in data handling and extraction, improved automation of processes, improved quality of health care services and overall health outcomes, provision of personalized care, accelerated diagnosis and treatment processes, and improved interaction between patients and health care professionals. In total, 5 risks to health care in general were identified: cybersecurity breaches, the potential for patient misinformation, ethical concerns, the likelihood of biased decision-making, and the risk associated with inaccurate communication. Overconfidence in LLM-based systems was recognized as a risk to the medical profession. The 6 agreed-upon privacy risks included the use of unregulated cloud services that compromise data security, exposure of sensitive patient data, breaches of confidentiality, fraudulent use of information, vulnerabilities in data storage and communication, and inappropriate access or use of patient data.

CONCLUSIONS

Future research related to LLMs should not only focus on testing their possibilities for NLP-related tasks but also consider the workflows the models could contribute to and the requirements regarding quality, integration, and regulations needed for successful implementation in practice.

Large Language Model-Based Evaluation of Medical Question Answering Systems: Algorithm Development and Case Study

BACKGROUND

Healthcare systems are increasingly resource constrained, leaving less time for important patient-provider interactions. Conversational agents (CAs) could be used to support the provision of information and to answer patients' questions. However, information must be accessible to a variety of patient populations, which requires understanding questions expressed at different language levels.

METHODS

This study describes the use of Large Language Models (LLMs) to evaluate predefined medical content in CAs across patient populations. These simulated populations are characterized by a range of health literacy. The evaluation framework includes both fully automated and semi-automated procedures to assess the performance of a CA.

RESULTS

A case study in the domain of mammography shows that LLMs can simulate questions from different patient populations. However, the accuracy of the answers provided varies depending on the level of health literacy.

CONCLUSIONS

Our scalable evaluation framework enables the simulation of patient populations with different health literacy levels and helps to evaluate domain specific CAs, thus promoting their integration into clinical practice. Future research aims to extend the framework to CAs without predefined content and to apply LLMs to adapt medical information to the specific (health) literacy level of the user.

Analysis of Critical Incident Reports Using Natural Language Processing

A Critical Incident Reporting System (CIRS) collects anecdotal reports from employees, which serve as a vital source of information about incidents that could potentially harm patients.

OBJECTIVES

To demonstrate how natural language processing (NLP) methods can help in retrieving valuable information from such incident data.

METHODS

We analyzed frequently occurring terms and sentiments as well as topics in data from the Swiss National CIRRNET database from 2006 to 2023 using NLP and BERTopic modelling.

RESULTS

We grouped the topics into 10 major themes out of which 6 are related to medication. Overall, they reflect the global trends in adverse events in healthcare (surgical errors, venous thromboembolism, falls). Additionally, we identified errors related to blood testing, COVID-19, handling patients with diabetes and pediatrics. 40-50% of the messages are written in a neutral tone, 30-40% in a negative tone.

CONCLUSION

The analysis of CIRS messages using text analysis tools helped in getting insights into common sources of critical incidents in Swiss healthcare institutions. In future work, we want to study more closely the relations, for example between sentiment and topics.

Transformer Models in Healthcare: A Survey and Thematic Analysis of Potentials, Shortcomings and Risks

Large Language Models (LLMs) such as General Pretrained Transformer (GPT) and Bidirectional Encoder Representations from Transformers (BERT), which use transformer model architectures, have significantly advanced artificial intelligence and natural language processing. Recognized for their ability to capture associative relationships between words based on shared context, these models are poised to transform healthcare by improving diagnostic accuracy, tailoring treatment plans, and predicting patient outcomes. However, there are multiple risks and potentially unintended consequences associated with their use in healthcare applications. This study, conducted with 28 participants using a qualitative approach, explores the benefits, shortcomings, and risks of using transformer models in healthcare. It analyses responses to seven open-ended questions using a simplified thematic analysis. Our research reveals seven benefits, including improved operational efficiency, optimized processes and refined clinical documentation. Despite these benefits, there are significant concerns about the introduction of bias, auditability issues and privacy risks. Challenges include the need for specialized expertise, the emergence of ethical dilemmas and the potential reduction in the human element of patient care. For the medical profession, risks include the impact on employment, changes in the patient-doctor dynamic, and the need for extensive training in both system operation and data interpretation.

Digital Interventions and Their Unexpected Outcomes - Time for Digitalovigilance?

The application of digital interventions in healthcare beyond research has been translated in the development of software as a medical device. Along with corresponding regulations for medical devices, there is a need for assessing adverse events to conduct post-market surveillance and to appropriately label digital health interventions to ensure proper use and patient safety. To date unexpected consequences of digital health interventions are neglected or ignored, or at least remain undescribed in literature. This paper is intended to raise awareness across the research community about these upcoming challenges. We recommend that - together with developing a new research field of digitalovigilance - a systematic assessment and monitoring of adverse events and unexpected interactions be included in clinical trials, along with the reporting of such events and the conduct of meta-analyses on critical aspects.

Large language model-based information extraction from free-text radiology reports: a scoping review protocol

INTRODUCTION

Radiological imaging is one of the most frequently performed diagnostic tests worldwide. The free-text contained in radiology reports is currently only rarely used for secondary use purposes, including research and predictive analysis. However, this data might be made available by means of information extraction (IE), based on natural language processing (NLP). Recently, a new approach to NLP, large language models (LLMs), has gained momentum and continues to improve performance of IE-related tasks. The objective of this scoping review is to show the state of research regarding IE from free-text radiology reports based on LLMs, to investigate applied methods and to guide future research by showing open challenges and limitations of current approaches. To our knowledge, no systematic or scoping review of IE from radiology reports based on LLMs has been published. Existing publications are outdated and do not comprise LLM-based methods.

METHODS AND ANALYSIS

This protocol is designed based on the JBI Manual for Evidence Synthesis, chapter 11.2: 'Development of a scoping review protocol'. Inclusion criteria and a search strategy comprising four databases (PubMed, IEEE Xplore, Web of Science Core Collection and ACM Digital Library) are defined. Furthermore, we describe the screening process, data charting, analysis and presentation of extracted data.

ETHICS AND DISSEMINATION

This protocol describes the methodology of a scoping literature review and does not comprise research on or with humans, animals or their data. Therefore, no ethical approval is required. After the publication of this protocol and the conduct of the review, its results are going to be published in an open access journal dedicated to biomedical informatics/digital health.

Assessing the Potential Risks of Digital Therapeutics (DTX): The DTX Risk Assessment Canvas

MOTIVATION

Digital therapeutics (DTX), i.e., health interventions that are provided through digital means, are increasingly available for use; in some countries, physicians can even prescribe selected DTX following a reimbursement by health insurances. This results in an increasing need for methodologies to consider and monitor DTX's negative consequences, their risks to patient safety, and possible adverse events. However, it is completely unknown which aspects should be subject to surveillance given the missing experiences with the tools and their negative impacts.

OBJECTIVE

Our aim is to develop a tool-the DTX Risk Assessment Canvas-that enables researchers, developers, and practitioners to reflect on the negative consequences of DTX in a participatory process.

METHOD

Taking the well-established business model canvas as a starting point, we identified relevant aspects to be considered in a risk assessment of a DTX. The aspects or building blocks of the canvas were constructed in a two-way process: first, we defined the aspects relevant for discussing and reflecting on how a DTX might bring negative consequences and risks for its users by considering ISO/TS 82304-2, the scientific literature, and by reviewing existing DTX and their listed adverse effects. The resulting aspects were grouped into thematic blocks and the canvas was created. Second, six experts in health informatics and mental health provided feedback and tested the understandability of the initial canvas by individually applying it to a DTX of their choice. Based on their feedback, the canvas was modified.

RESULTS

The DTX Risk Assessment Canvas is organized into 15 thematic blocks which are in turn grouped into three thematic groups considering the DTX itself, the users of the DTX, and the effects of the DTX. For each thematic block, questions have been formulated to guide the user of the canvas in reflecting on the single aspects. Conclusions: The DTX Risk Assessment Canvas is a tool to reflect the negative consequences and risks of a DTX by discussing different thematic blocks that together constitute a comprehensive interpretation of a DTX regarding possible risks. Applied during the DTX design and development phase, it can help in implementing countermeasures for mitigation or means for their monitoring.

How Can Transformer Models Shape Future Healthcare: A Qualitative Study

Transformer models have been successfully applied to various natural language processing and machine translation tasks in recent years, e.g. automatic language understanding. With the advent of more efficient and reliable models (e.g. GPT-3), there is a growing potential for automating time-consuming tasks that could be of particular benefit in healthcare to improve clinical outcomes. This paper aims at summarizing potential use cases of transformer models for future healthcare applications. Precisely, we conducted a survey asking experts on their ideas and reflections for future use cases. We received 28 responses, analyzed using an adapted thematic analysis. Overall, 8 use case categories were identified including documentation and clinical coding, workflow and healthcare services, decision support, knowledge management, interaction support, patient education, health management, and public health monitoring. Future research should consider developing and testing the application of transformer models for such use cases.

Personalized Digital Solutions for Mental Health

INTRODUCTION

Mental health is one of the major global concerns in the field of healthcare. The emergence of digital solutions is proving to be a great aid for individuals suffering from mental health disorders. These solutions are particularly useful and effective when they are personalized. The objective of this paper is to understand the personalization factors and the methods that have been used to collect information to personalize the digital mental health solutions.

METHODS

This paper builds on a previous review that analyzed the personalization of digital solutions in mHealth, and expands on the extracted information for the specific case of mental health.

RESULTS

Ten mental health digital solutions have been analyzed. The paper focuses on targeted conditions, personalization factors and the methods used for collecting personalization factors.

DISCUSSION

The analyzed mental health digital solutions cover a wide range of health conditions. It is remarkable that most articles do not explicitly mention the factors used to personalize the solution. Among the solutions that mention them, there is a great diversity of factors utilized, such as age, gender, user preferences, and subjective behavior. The authors point out the methods for obtaining data to personalize the solutions, including in-app questionnaires, self-reports, and usage data of the solutions.

CONCLUSIONS

The analysis of current mental health digital solutions emphasizes the need to create guidelines for designing personalized digital solutions for mental health.

Coadaptation Between Smart Technologies and Older Adults Over Time: Protocol for a Scoping Review

BACKGROUND

The Internet of Things (IoT) has gained significant attention due to advancements in technology and has potential applications in meeting the needs of an aging population. Smart technologies, a subset of IoT, can support older adults in aging in place, promoting independent living and improving their quality of life. However, there is a lack of research on how older adults and smart technologies coadapt over time to maximize their benefits and sustain adoption.

OBJECTIVE

We will aim to comprehensively review and analyze the existing scientific literature pertaining to the coadaptation between smart technologies and older adults. The primary focus will be to investigate the extent and nature of this coadaptation process and explore how older adults and technology coevolve over time to enhance older adults' experience with technology.

METHODS

This scoping review will follow the methodology outlined in the Joanna Briggs Institute Reviewer's Manual and adhere to the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analysis Extension for Scoping Reviews) guidelines for reporting. Peer-reviewed articles will be searched in databases like Ovid MEDLINE, OVID Embase, PEDro, OVID PsycINFO, EBSCO CINAHL, the Cochrane Library, Scopus, IEEE Xplore, Web of Science, and Global Index Medicus. The research team will create a data extraction form covering study characteristics, participant characteristics, underlying models and frameworks, research findings, implications for technology coadaptation, and any identified study limitations. A directed content analysis approach will be used, incorporating the Selection, Optimization, and Compensation framework and Sex- and Gender-Based Analysis Plus theoretical framework.

RESULTS

The results of this study are expected in January 2024.

CONCLUSIONS

This scoping review endeavors to present a thorough overview of the available evidence concerning how smart technologies interact with older adults over an extended period. The insights gained from this review will lay the groundwork for a research program that explores how older adults adapt to and use smart technologies throughout their lives, ultimately leading to improved user satisfaction and experience and facilitating aging in place with tailored support and user-centered design principles.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/51129.

Designing personalised mHealth solutions: An overview

INTRODUCTION

Mobile health, or mHealth, is based on mobile information and communication technologies and provides solutions for empowering individuals to participate in healthcare. Personalisation techniques have been used to increase user engagement and adherence to interventions delivered as mHealth solutions. This study aims to explore the current state of personalisation in mHealth, including its current trends and implementation.

MATERIALS AND METHODS

We conducted a review following PRISMA guidelines. Four databases (PubMed, ACM Digital Library, IEEE Xplore, and APA PsycInfo) were searched for studies on mHealth solutions that integrate personalisation. The retrieved papers were assessed for eligibility and useful information regarding integrated personalisation techniques.

RESULTS

Out of the 1,139 retrieved studies, 62 were included in the narrative synthesis. Research interest in the personalisation of mHealth solutions has increased since 2020. mHealth solutions were mainly applied to endocrine, nutritional, and metabolic diseases; mental, behavioural, or neurodevelopmental diseases; or the promotion of healthy lifestyle behaviours. Its main purposes are to support disease self-management and promote healthy lifestyle behaviours. Mobile applications are the most prevalent technological solution. Although several design models, such as user-centred and patient-centred designs, were used, no specific frameworks or models for personalisation were followed. These solutions rely on behaviour change theories, use gamification or motivational messages, and personalise the content rather than functionality. A broad range of data is used for personalisation purposes. There is a lack of studies assessing the efficacy of these solutions; therefore, further evidence is needed.

DISCUSSION

Personalisation in mHealth has not been well researched. Although several techniques have been integrated, the effects of using a combination of personalisation techniques remain unclear. Although personalisation is considered a persuasive strategy, many mHealth solutions do not employ it.

CONCLUSIONS

Open research questions concern guidelines for successful personalisation techniques in mHealth, design frameworks, and comprehensive studies on the effects and interactions among multiple personalisation techniques.

Defining and Scoping Participatory Health Informatics: An eDelphi Study

BACKGROUND

Health care has evolved to support the involvement of individuals in decision making by, for example, using mobile apps and wearables that may help empower people to actively participate in their treatment and health monitoring. While the term "participatory health informatics" (PHI) has emerged in literature to describe these activities, along with the use of social media for health purposes, the scope of the research field of PHI is not yet well defined.

OBJECTIVE

This article proposes a preliminary definition of PHI and defines the scope of the field.

METHODS

We used an adapted Delphi study design to gain consensus from participants on a definition developed from a previous review of literature. From the literature we derived a set of attributes describing PHI as comprising 18 characteristics, 14 aims, and 4 relations. We invited researchers, health professionals, and health informaticians to score these characteristics and aims of PHI and their relations to other fields over three survey rounds. In the first round participants were able to offer additional attributes for voting.

RESULTS

The first round had 44 participants, with 28 participants participating in all three rounds. These 28 participants were gender-balanced and comprised participants from industry, academia, and health sectors from all continents. Consensus was reached on 16 characteristics, 9 aims, and 6 related fields.

DISCUSSION

The consensus reached on attributes of PHI describe PHI as a multidisciplinary field that uses information technology and delivers tools with a focus on individual-centered care. It studies various effects of the use of such tools and technology. Its aims address the individuals in the role of patients, but also the health of a society as a whole. There are relationships to the fields of health informatics, digital health, medical informatics, and consumer health informatics.

CONCLUSION

We have proposed a preliminary definition, aims, and relationships of PHI based on literature and expert consensus. These can begin to be used to support development of research priorities and outcomes measurements.

Exploring the Evolution of Social Media in Mental Health Interventions: A Mapping Review

BACKGROUND

With the rise of social media, social media use for delivering mental health interventions has become increasingly popular. However, there is no comprehensive overview available on how this field developed over time.

OBJECTIVES

The objective of this paper is to provide an overview over time of the use of social media for delivering mental health interventions. Specifically, we examine which mental health conditions and target groups have been targeted, and which social media channels or tools have been used since this topic first appeared in research.

METHODS

To provide an overview of the use of social media for mental health interventions, we conducted a search for studies in four databases (PubMed; ACM Digital Library; PsycInfo; and CINAHL) and two trial registries (Clinicaltrials.gov; and Cochranelibrary.com). A sample of representative keywords related to mental health and social media was used for that search. Automatic text analysis methods (e.g., BERTopic analysis, word clouds) were applied to identify topics, and to extract target groups and types of social media.

RESULTS

A total of 458 studies were included in this review (n=228 articles, and n=230 registries). Anxiety and depression were the most frequently mentioned conditions in titles of both articles and registries. BERTopic analysis identified depression and anxiety as the main topics, as well as several addictions (including gambling, alcohol, and smoking). Mental health and women's research were highlighted as the main targeted topics of these studies. The most frequently targeted groups were "adults" (39.5%) and "parents" (33.4%). Facebook, WhatsApp, messenger platforms in general, Instagram, and forums were the most frequently mentioned tools in these interventions.

CONCLUSIONS

We learned that research interest in social media-based interventions in mental health is increasing, particularly in the last two years. A variety of tools have been studied, and trends towards forums and Facebook show that tools allowing for more content are preferred for mental health interventions. Future research should assess which social media tools are best suited in terms of clinical outcomes. Additionally, we conclude that natural language processing tools can help in studying trends in research on a particular topic.

How Participatory Health Informatics Catalyzes One Digital Health

OBJECTIVE

To identify links between Participatory Health Informatics (PHI) and the One Digital Health framework (ODH) and to show how PHI could be used as a catalyst or contributor to ODH.

METHODS

We have analyzed the addressed topics within the ODH framework in previous IMIA Yearbook contributions from our working group during the last 10 years. We have matched main themes with the ODH's framework three perspectives (individual health and wellbeing, population and society, and ecosystem).

RESULTS

PHI catalysts ODH individual health and wellbeing perspective by providing a more comprehensive view on human health, attitudes, and relations between human health and animal health. Integration of specific behavior change techniques or gamification strategies in digital solutions are effective to change behaviors which address the P5 paradigm. PHI supports the population and society perspective through the engagement of the various stakeholders in healthcare. At the same time, PHI might increase a risk for health inequities due to technologies inaccessible to all equally and challenges associated with this. PHI is a catalyst for the ecosystem perspective by contributing data into the digital health data ecosystem allowing for analysis of interrelations between the various data which in turn might provide links among all components of the healthcare ecosystem.

CONCLUSION

Our results suggest that PHI can and will involve topics relating to ODH. As the ODH concept crystalizes and becomes increasingly influential, its themes will permeate and become embedded in PHI even more. We look forward to these developments and co-evolution of the two frameworks.

The performance of serious games for enhancing attention in cognitively impaired older adults

Attention, which is the process of noticing the surrounding environment and processing information, is one of the cognitive functions that deteriorate gradually as people grow older. Games that are used for other than entertainment, such as improving attention, are often referred to as serious games. This study examined the effectiveness of serious games on attention among elderly individuals suffering from cognitive impairment. A systematic review and meta-analyses of randomized controlled trials were carried out. A total of 10 trials ultimately met all eligibility criteria of the 559 records retrieved. The synthesis of very low-quality evidence from three trials, as analyzed in a meta-study, indicated that serious games outperform no/passive interventions in enhancing attention in cognitively impaired older adults (P < 0.001). Additionally, findings from two other studies demonstrated that serious games are more effective than traditional cognitive training in boosting attention among cognitively impaired older adults. One study also concluded that serious games are better than traditional exercises in enhancing attention. Serious games can enhance attention in cognitively impaired older adults. However, given the low quality of the evidence, the limited number of participants in most studies, the absence of some comparative studies, and the dearth of studies included in the meta-analyses, the results remain inconclusive. Thus, until the aforementioned limitations are rectified in future research, serious games should serve as a supplement, rather than a replacement, to current interventions.

Towards Safe Conversational Agents in Healthcare

Conversational agents (CA) are becoming very popular to deliver digital health interventions. These dialog-based systems are interacting with patients using natural language which might lead to misunderstandings and misinterpretations. To avoid patient harm, safety of health CA has to be ensured. This paper raises awareness on safety when developing and distributing health CA. For this purpose, we identify and describe facets of safety and make recommendations for ensuring safety in health CA. We distinguish three facets of safety: 1) system safety, 2) patient safety, and 3) perceived safety. System safety comprises data security and privacy which has to be considered when selecting technologies and developing the health CA. Patient safety is related to risk monitoring and risk management, to adverse events and content accuracy. Perceived safety concerns a user's perception of the level of danger and user's level of comfort during the use. The latter can be supported when data security is guaranteed and relevant information on the system and its capabilities are provided.

How to Design Successful Participatory Design Workshops for Digital Health Solutions?

Participatory design (PD) is increasingly used to support design and development of digital health solutions. The involves representatives of future user groups and experts to collect their needs and preferences and ensure easy to use and useful solutions. However, reflections and experiences with PD in designing digital health solutions are rarely reported. The objective of this paper is to collect those experiences including lessons learnt and moderator experiences, and to identify challenges. For this purpose, we conducted a multiple case study to explore the skill development process required to successfully design a solution in the three cases. From the results, we derived good practice guidelines to support designing successful PD workshops. They include adapting the workshop activities and material to the vulnerable participant group and considering their environment and previous experiences, planning sufficient time for preparation and supporting the activities with appropriate material. We conclude that PD workshop results are perceived as useful for designing digital health solutions, but careful design is very relevant.

What Do Autistic People Discuss on Twitter? An Approach Using BERTopic Modelling

Social media provide easy ways to autistic individuals to communicate and to make their voices heard. The objective of this paper is to identify the main themes that are being discussed by autistic people on Twitter. We collected a sample of tweets containing the hashtag #ActuallyAutistic during the period 10/02/2022 and 14/09/2022. To identify the most discussed topics, BERTopic modelling was applied. We manually grouped the detected topics into 6 major themes using inductive content analysis: 1) General aspects of autism and experiences of autistic individuals; 2) Autism awareness, pride and funding; 3) Interventions, mostly related to Applied Behavior Analysis; 4) Reactions and expressions; 5) Everyday life as an autistic (lifelong condition, work, housing…); and 6) Symbols and characteristics. The majority of tweets were presenting general aspects and experiences as autistic individuals; raising awareness; and about their dissatisfaction with some interventions. The identification of autistic individuals' main discussion themes could help to develop meaningful public health agendas and research involving and addressed to autistic individuals.

Exploring Obese Adults' Preferences for a Physical Activity Chatbot: Qualitative Study

Social media chatbots could help increase obese adults' physical activity behaviour. The study aims to explore obese adults' preferences for a physical activity chatbot. Individual- and focus group interviews will be conducted in 2023. Identified preferences will inform the development of a chatbot that motivates obese adults to increase their physical activity. The interview guide was tested in a pilot interview.

Fading Fatigue - A Self-Management App for Supporting Long-COVID Patients with Fatigue

Fatigue is the most prevalent Long-COVID symptom. Individuals who are affected have to learn to organize and manage daily activities according to the subjectively perceived energy reserves. Our objective was to develop an application, Fading Fatigue, that supports patients in their energy management, in particular after an initial therapy guided by health professionals. Fading Fatigue was developed in an iterative approach and implemented as a client-server application. Interviews and a literature search were conducted to identify limitations and challenges of the current treatment. Fading Fatigue offers several tools for energy management: a daily energy planner, a documentation aid for well-being and a progress view. Future work should study usability. Inclusion of additional features increasing the adherence such as providing feedback could be considered.

How Does Shortage of Health IT Professionals Impact on the Digital Health Transformation?

BACKGROUND

The need for software suppliers to react swiftly to the plethora of application requests and constantly shifting market requirements is one of the major problems facing the health IT business in the context of digital health transformation. This can only be achieved when the necessary staff and resources are available.

OBJECTIVES

The objective of this work is to identify challenges health IT companies are confronted with related to personnel capacities and skilled workers.

METHODS

Using a questionnaire distributed through newsletters and social media among representatives of software companies and hospitals we collected information on current hurdles of health software providers and their strategies to overcome these in order to address the demands of the digital health transformation.

RESULTS

The main findings of the survey are that scarce resources in software development are among the reasons for not achieving strategic goals on time in the health IT sector and for not being able to react flexibly to market changes. A strategy to overcome missing expert knowledge and own resources without free capacity is to hire external resources.

CONCLUSIONS

With the ever-changing landscape of digital health, it is essential to have skilled workers with knowledge on the peculiarities of clinical workflows. The existing shortage of skilled workers leads to a reduction of innovative power in the health IT sector, potentially slowing down the digital health transformation.

Designing a Digital Medical Interview Assistant for Radiology

Radiologists rarely interact with the patients whose radiological images they are reviewing due to time and resource constraints. However, relevant information about the patient's medical history could improve reporting performance and quality. In this work, our objective was to collect requirements for a digital medical interview assistant (DMIA) that collects the medical history from patients by means of a conversational agent and structures as well as provides the collected data to radiologists. Requirements were gathered based on a narrative literature review, a patient questionnaire and input from a radiologist. Based on these results, a system architecture for the DMIA was developed. 37 functional and 17 non-functional requirements were identified. The resulting architecture comprises five components, namely Chatbot, Natural language processing (NLP), Administration, Content Definition and Workflow Engine. To be able to quickly adapt the chatbot content according to the information needs of a specific radiological examination, there is a need for developing a sustainable process for the content generation that considers standardized data modelling as well as rewording of clinical language into consumer health vocabulary understandable to a diverse patient user group.

Framework for Guiding the Development of High-Quality Conversational Agents in Healthcare

Evaluating conversational agents (CAs) that are supposed to be applied in healthcare settings and ensuring their quality is essential to avoid patient harm and ensure efficacy of the CA-delivered intervention. However, a guideline for a standardized quality assessment of health CAs is still missing. The objective of this work is to describe a framework that provides guidance for development and evaluation of health CAs. In previous work, consensus on categories for evaluating health CAs has been found. In this work, we identify concrete metrics, heuristics, and checklists for these evaluation categories to form a framework. We focus on a specific type of health CA, namely rule-based systems that are based on written input and output, have a simple personality without any kind of embodiment. First, we identified relevant metrics, heuristics, and checklists to be linked to the evaluation categories through a literature search. Second, five experts judged the metrics regarding their relevance to be considered within evaluation and development of health CAs. The final framework considers nine aspects from a general perspective, five aspects from a response understanding perspective, one aspect from a response generation perspective, and three aspects from an aesthetics perspective. Existing tools and heuristics specifically designed for evaluating CAs were linked to these evaluation aspects (e.g., Bot usability scale, design heuristics for CAs); tools related to mHealth evaluation were adapted when necessary (e.g., aspects from the ISO technical specification for mHealth Apps). The resulting framework comprises aspects to be considered not only as part of a system evaluation, but already during the development. In particular, aspects related to accessibility or security have to be addressed in the design phase (e.g., which input and output options are provided to ensure accessibility?) and have to be verified after the implementation phase. As a next step, transfer of the framework to other types of health CAs has to be studied. The framework has to be validated by applying it during health CA design and development.

Sentiment analysis of clinical narratives: A scoping review

A clinical sentiment is a judgment, thought or attitude promoted by an observation with respect to the health of an individual. Sentiment analysis has drawn attention in the healthcare domain for secondary use of data from clinical narratives, with a variety of applications including predicting the likelihood of emerging mental illnesses or clinical outcomes. The current state of research has not yet been summarized. This study presents results from a scoping review aiming at providing an overview of sentiment analysis of clinical narratives in order to summarize existing research and identify open research gaps. The scoping review was carried out in line with the PRISMA-ScR (Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews) guideline. Studies were identified by searching 4 electronic databases (e.g., PubMed, IEEE Xplore) in addition to conducting backward and forward reference list checking of the included studies. We extracted information on use cases, methods and tools applied, used datasets and performance of the sentiment analysis approach. Of 1,200 citations retrieved, 29 unique studies were included in the review covering a period of 8 years. Most studies apply general domain tools (e.g. TextBlob) and sentiment lexicons (e.g. SentiWordNet) for realizing use cases such as prediction of clinical outcomes; others proposed new domain-specific sentiment analysis approaches based on machine learning. Accuracy values between 71.5-88.2% are reported. Data used for evaluation and test are often retrieved from MIMIC databases or i2b2 challenges. Latest developments related to artificial neural networks are not yet fully considered in this domain. We conclude that future research should focus on developing a gold standard sentiment lexicon, adapted to the specific characteristics of clinical narratives. Efforts have to be made to either augment existing or create new high-quality labeled data sets of clinical narratives. Last, the suitability of state-of-the-art machine learning methods for natural language processing and in particular transformer-based models should be investigated for their application for sentiment analysis of clinical narratives.

Correction: Wearable Artificial Intelligence for Anxiety and Depression: Scoping Review

[This corrects the article DOI: 10.2196/42672.].

Correction: Wearable Artificial Intelligence for Anxiety and Depression: Scoping Review (Preprint).. [DOI: 10.2196/preprints.46233] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

UNSTRUCTURED

 

Wearable Artificial Intelligence for Anxiety and Depression: Scoping Review

BACKGROUND

Anxiety and depression are the most common mental disorders worldwide. Owing to the lack of psychiatrists around the world, the incorporation of artificial intelligence (AI) into wearable devices (wearable AI) has been exploited to provide mental health services.

OBJECTIVE

This review aimed to explore the features of wearable AI used for anxiety and depression to identify application areas and open research issues.

METHODS

We searched 8 electronic databases (MEDLINE, PsycINFO, Embase, CINAHL, IEEE Xplore, ACM Digital Library, Scopus, and Google Scholar) and included studies that met the inclusion criteria. Then, we checked the studies that cited the included studies and screened studies that were cited by the included studies. The study selection and data extraction were carried out by 2 reviewers independently. The extracted data were aggregated and summarized using narrative synthesis.

RESULTS

Of the 1203 studies identified, 69 (5.74%) were included in this review. Approximately, two-thirds of the studies used wearable AI for depression, whereas the remaining studies used it for anxiety. The most frequent application of wearable AI was in diagnosing anxiety and depression; however, none of the studies used it for treatment purposes. Most studies targeted individuals aged between 18 and 65 years. The most common wearable device used in the studies was Actiwatch AW4 (Cambridge Neurotechnology Ltd). Wrist-worn devices were the most common type of wearable device in the studies. The most commonly used category of data for model development was physical activity data, followed by sleep data and heart rate data. The most frequently used data set from open sources was Depresjon. The most commonly used algorithm was random forest, followed by support vector machine.

CONCLUSIONS

Wearable AI can offer great promise in providing mental health services related to anxiety and depression. Wearable AI can be used by individuals for the prescreening assessment of anxiety and depression. Further reviews are needed to statistically synthesize the studies' results related to the performance and effectiveness of wearable AI. Given its potential, technology companies should invest more in wearable AI for the treatment of anxiety and depression.

Digital health as an enabler for hospital@home: A rising trend or just a vision?

Background

Hospital@home is a model of healthcare, where healthcare professionals actively treat patients in their homes for conditions that may otherwise require hospitalization. Similar models of care have been implemented in jurisdictions around the world over the past few years. However, there are new developments in health informatics including digital health and participatory health informatics that may have an impact on hospital@home approaches.

Objectives

This study aims to identify the current state of implementation of emerging concepts into the hospital@home research and models of care; to identify strengths and weaknesses, opportunities, and threats associated with the models of care; and to suggest a research agenda.

Methods

We employed two research methodologies, namely, a literature review and a SWOT (strengths, weaknesses, opportunities, and threats) analysis. The literature from the last 10 years was collected from PubMed using the search string "hospital at home" OR "care at home" OR "patient at home." Relevant information was extracted from the included articles.

Results

Title and abstract review were conducted on 1,371 articles. The full-text review was conducted on 82 articles. Data were extracted from 42 articles that met our review criteria. Most of the studies originated from the United States and Spain. Several medical conditions were considered. The use of digital tools and technologies was rarely reported. In particular, innovative approaches such as wearables or sensor technologies were rarely used. The current landscape of hospital@home models of care simply delivers hospital care in the patient's home. Tools or approaches from taking a participatory health informatics design approach involving a range of stakeholders (such as patients and their caregivers) were not reported in the literature reviewed. In addition, emerging technologies supporting mobile health applications, wearable technologies, and remote monitoring were rarely discussed.

Conclusion

There are multiple benefits and opportunities associated with hospital@home implementations. There are also threats and weaknesses associated with the use of this model of care. Some weaknesses could be addressed by using digital health and wearable technologies to support patient monitoring and treatment at home. Employing a participatory health informatics approach to design and implementation could help to ensure the acceptance of such care models.

Classification of user queries according to a hierarchical medical procedure encoding system using an ensemble classifier

The Swiss classification of surgical interventions (CHOP) has to be used in daily practice by physicians to classify clinical procedures. Its purpose is to encode the delivered healthcare services for the sake of quality assurance and billing. For encoding a procedure, a code of a maximal of 6-digits has to be selected from the classification system, which is currently realized by a rule-based system composed of encoding experts and a manual search in the CHOP catalog. In this paper, we will investigate the possibility of automatic CHOP code generation based on a short query to enable automatic support of manual classification. The wide and deep hierarchy of CHOP and the differences between text used in queries and catalog descriptions are two apparent obstacles for training and deploying a learning-based algorithm. Because of these challenges, there is a need for an appropriate classification approach. We evaluate different strategies (multi-class non-terminal and per-node classifications) with different configurations so that a flexible modular solution with high accuracy and efficiency can be provided. The results clearly show that the per-node binary classification outperforms the non-terminal multi-class classification with an F1-micro measure between 92.6 and 94%. The hierarchical prediction based on per-node binary classifiers achieved a high exact match by the single code assignment on the 5-fold cross-validation. In conclusion, the hierarchical context from the CHOP encoding can be employed by both classifier training and representation learning. The hierarchical features have all shown improvement in the classification performances under different configurations, respectively: the stacked autoencoder and training examples aggregation using true path rules as well as the unified vocabulary space have largely increased the utility of hierarchical features. Additionally, the threshold adaption through Bayesian aggregation has largely increased the vertical reachability of the per node classification. All the trainable nodes can be triggered after the threshold adaption, while the F1 measures at code levels 3–6 have been increased from 6 to 89% after the threshold adaption.

What Can We Learn from Quality Requirements in ISO/TS 82304-2 for Evaluating Conversational Agents in Healthcare?

Evaluating conversational agents (CA) that are supposed to be applied in healthcare and ensuring their quality is essential to avoid patient harm. However, most researchers only study usability and use the CA in clinical trials before conducting such careful evaluation. In previous work, consensus on metrics for evaluating healthcare CA have been found. However, the metrics are still too generic to form an evaluation framework. In this work, we try to link the ISO technical specification ISO/TS 82304-2 Quality Requirements for Health and Wellness Apps to the set of metrics to come a step closer towards an evaluation framework. We identify three links between ISO requirements and the set of metrics, namely accessibility, usability, and security. Although the technical specification rather lists aspects to be considered during development instead of concrete metrics for studying the quality, we can link to some aspects that are also of interest for health CA evaluation. For example, measuring the readability for ensuring accessibility or implementing the Web Content Accessibility Guidelines are two aspects of relevance for health CA.

A Review of Artificial Intelligence and Robotics in Transformed Health Ecosystems

Health care is shifting toward become proactive according to the concept of P5 medicine-a predictive, personalized, preventive, participatory and precision discipline. This patient-centered care heavily leverages the latest technologies of artificial intelligence (AI) and robotics that support diagnosis, decision making and treatment. In this paper, we present the role of AI and robotic systems in this evolution, including example use cases. We categorize systems along multiple dimensions such as the type of system, the degree of autonomy, the care setting where the systems are applied, and the application area. These technologies have already achieved notable results in the prediction of sepsis or cardiovascular risk, the monitoring of vital parameters in intensive care units, or in the form of home care robots. Still, while much research is conducted around AI and robotics in health care, adoption in real world care settings is still limited. To remove adoption barriers, we need to address issues such as safety, security, privacy and ethical principles; detect and eliminate bias that could result in harmful or unfair clinical decisions; and build trust in and societal acceptance of AI.

Can We Do Better than Gesturing? Requirements for a Digital Communication Aid to Support Non-Verbal Communication in Paediatric Emergency Care

Providing urgent and emergency care to migrant children is often hampered or delayed. Reasons for this are language barriers when children, and their care givers, don't speak any of the languages commonly spoken in Switzerland, which include German, French, Italian, and English. By a participatory design process, we want to develop a novel image-based digital communication aid tailored to the needs of migrant patients and nurses within Swiss paediatric clinics.

Does Enrichment of Clinical Texts by Ontology Concepts Increases Classification Accuracy?

In the medical domain, multiple ontologies and terminology systems are available. However, existing classification and prediction algorithms in the clinical domain often ignore or insufficiently utilize semantic information as it is provided in those ontologies. To address this issue, we introduce a concept for augmenting embeddings, the input to deep neural networks, with semantic information retrieved from ontologies. To do this, words and phrases of sentences are mapped to concepts of a medical ontology aggregating synonyms in the same concept. A semantically enriched vector is generated and used for sentence classification. We study our approach on a sentence classification task using a real world dataset which comprises 640 sentences belonging to 22 categories. A deep neural network model is defined with an embedding layer followed by two LSTM layers and two dense layers. Our experiments show, classification accuracy without content enriched embeddings is for some categories higher than without enrichment. We conclude that semantic information from ontologies has potential to provide a useful enrichment of text. Future research will assess to what extent semantic relationships from the ontology can be used for enrichment.

Social Media, Digital Health Literacy, and Digital Ethics in the Light of Health Equity

OBJECTIVE

Social media is used in the context of healthcare, for example in interventions for promoting health. Since social media are easily accessible they have potential to promote health equity. This paper studies relevant factors impacting on health equity considered in social media interventions.

METHODS

We searched for literature to identify potential relevant factors impacting on health equity considered in social media interventions. We included studies that reported examples of health interventions using social media, focused on health equity, and analyzed health equity factors of social media. We identified Information about health equity factors and targeted groups.

RESULTS

We found 17 relevant articles. Factors impacting on health equity reported in the included papers were extracted and grouped into three categories: digital health literacy, digital ethics, and acceptability.

CONCLUSIONS

Literature shows that it is likely that digital technologies will increase health inequities associated with increased age, lower level of educational attainment, and lower socio-economic status. To address this challenge development of social media interventions should consider participatory design principles, visualization, and theories of social sciences.

Usability Assessment of Conversational Agents in Healthcare: A Literature Review

Conversational agents (CA) are chatbot-based systems supporting the interaction with users through text, speech, or other modalities. They are used in an increasing number of medical use cases. Even though usability is considered a prerequisite for the success of mHealth apps using CA, there is still no standard procedure to study usability of health CA. In this paper, we report the results from a systematic literature review aiming at identifying study designs, tools, and metrics used to assess usability in health CA. We searched three bibliographic databases (PubMed, Scopus, IEEE Xplore) for papers reporting on CA in healthcare to extract information on the usability assessment of those CA. From 273 retrieved results, we included 66 papers for full text review. 34 of them reported on usability assessments. A broad range of tools is used (e.g. SUS, UEQ), but also individual questionnaires are exploited. The examined studies use scenario-based setups but assess also real-world usage. Exploratory setups are rarely reported. Due to the differences in the study designs and assessment tools, it is impossible to compare usability among CA. Thus, we recommend to develop a standardised procedure that can be always applied and which can be enriched by assessments needed for evaluating usability of CA-specific features.

Usability Testing of a Social Media Chatbot for Increasing Physical Activity Behavior

Digital interventions for increasing physical activity behavior have shown great potential, especially those with social media. Chatbots, also known as conversational agents, have emerged in healthcare in relation to digital interventions and have proven effective in promoting physical activity among adults. The study’s objective is to explore users’ experiences with a social media chatbot. The concept and the prototype development of the social media chatbot MYA were realized in three steps: requirement analysis, concept development, and implementation. MYA’s design includes behavior change techniques effective in increasing physical activity through digital interventions. Participants in a usability study answered a survey with the Chatbot Usability Questionnaire (CUQ), which is comparable to the Systems Usability Scale. The mean CUQ score was below 68, the benchmark for average usability. The highest mean CUQ score was 64.5 for participants who thought MYA could help increase their physical activity behavior. The lowest mean CUQ score was 40.6 for participants aged between 50 and 69 years. Generally, MYA was considered to be welcoming, very easy to use, realistic, engaging, and informative. However, some technical issues were identified. A good and diversified user experience promotes prolonged chatbot use. Addressing identified issues will enhance users’ interaction with MYA.

Classifying Numbers from EEG Data - Which Neural Network Architecture Performs Best?

This paper presents a comparison of deep learning models for classifying P300 events, i.e., event-related potentials of the brain triggered during the human decision-making process. The evaluated models include CNN, (Bi | Deep | CNN-) LSTM, ConvLSTM, LSTM + Attention. The experiments were based on a large publicly available EEG dataset of school-age children conducting the "Guess the number"-experiment. Several hyperparameter choices were experimentally investigated resulting in 30 different models included in the comparison. Ten models with good performance on the validation data set were also automatically optimized with Grid Search. Monte Carlo Cross Validation was used to test all models on test data with 30 iterations. The best performing model was the Deep LSTM with an accuracy of 77.1% followed by the baseline (CNN) 76.1%. The significance test using a 5x2 cross validation paired t-test demonstrated that no model was significantly better than the baseline. We recommend experimenting with other architectures such as Inception, ResNet and Graph Convolutional Network.

Participatory Development of an Image-Based Communication Aid for Migrant Patients and Emergency Nurses

Language barriers hamper or delay delivery of urgent and emergency care to migrant children when they or their parents don't speak any of the languages commonly spoken in Switzerland. In such situations, nurses often fall back to use ad hoc communication aids, including translation apps and visual dictionaries, to collect information about a patient's medical history. In this paper, we report on the participatory design process for a novel image-based communication aid. It is specifically tailored to the needs of migrant patients and nurses within Swiss pediatric clinics. We collected requirements in surveys and in-depth interviews with pediatric nurses. A prototype app was developed and tested with users in a scenario-based usability test. The results clearly show that the images developed, especially for symptoms, accidents or nutrition and excretion, are well comprehensible for triage and anamnesis. In contrast, a temporal classification or chronological occurrence of health incidents is difficulty to express with images.

Intervention Platform for Action Observation and Motor Imagery Training After Stroke: Usability Test

Action observation (AO) and motor imagery (MI) are considered as promising therapeutic approaches in the rehabilitation of patients after a stroke (PaS). Observing and mentally rehearsing motor movements stimulate the motor system in the brain and result in a positive effect on movement execution. To support patients in the early rehabilitation phase after a stroke, ANIMATE, a digital health intervention platform was developed. The platform guides the user through 6 activities of daily living by observing and imagining the corresponding movements. We conducted a scenario-based usability test with 9 PaS at a rehabilitation centre to identify existing usability issues. PaS found the app easy to use and they could interact with it without problems. Although they judged the app as useful, they stated to be not willing to use the app on a regular basis. Including features for customising ANIMATE regarding the individual rehabilitation goals and needs of PaS, as well as personalisation could help in increasing the motivation to use and the benefits of the platform.

Digital Medical Interview Assistant for Radiology: Opportunities and Challenges

BACKGROUND

Personal contact between radiologists and their patients is scarce due to time constraints and logistical reasons which impacts on patient knowledgeability and satisfaction, but also on examination and diagnostic quality.

OBJECTIVE

We illuminate medical history interviews from a radiologist's perspective and discuss its impact on the diagnostic quality. Based on these insights, we develop a digital medical interview assistant (DMIA) for radiology that is intended to collect information helping in improving radiological diagnostics.

METHODS

Conditions, issues, problems in the radiological examination process are assessed to collect requirements and to specify questions for a digital medical history interview.

RESULTS

A DMIA with conversational user interface is developed using the scripting language RiveScript. It is accessible through a social media messenger (Telegram messenger). An initial assessment of usability demonstrates a good usability.

CONCLUSION

To overcome the information gap in radiology, a DMIA can simulate an assessment interview. It is still necessary to remove existing barriers in interaction with the DMIA for example by facilitating data entry options.

Implementation of Cognitive Behavioral Therapy in e-Mental Health Apps: Literature Review

Background

To address the matter of limited resources for treating individuals with mental disorders, e–mental health has gained interest in recent years. More specifically, mobile health (mHealth) apps have been suggested as electronic mental health interventions accompanied by cognitive behavioral therapy (CBT).

Objective

This study aims to identify the therapeutic aspects of CBT that have been implemented in existing mHealth apps and the technologies used. From these, we aim to derive research gaps that should be addressed in the future.

Methods

Three databases were screened for studies on mHealth apps in the context of mental disorders that implement techniques of CBT: PubMed, IEEE Xplore, and ACM Digital Library. The studies were independently selected by 2 reviewers, who then extracted data from the included studies. Data on CBT techniques and their technical implementation in mHealth apps were synthesized narratively.

Results

Of the 530 retrieved citations, 34 (6.4%) studies were included in this review. mHealth apps for CBT exploit two groups of technologies: technologies that implement CBT techniques for cognitive restructuring, behavioral activation, and problem solving (exposure is not yet realized in mHealth apps) and technologies that aim to increase user experience, adherence, and engagement. The synergy of these technologies enables patients to self-manage and self-monitor their mental state and access relevant information on their mental illness, which helps them cope with mental health problems and allows self-treatment.

Conclusions

There are CBT techniques that can be implemented in mHealth apps. Additional research is needed on the efficacy of the mHealth interventions and their side effects, including inequalities because of the digital divide, addictive internet behavior, lack of trust in mHealth, anonymity issues, risks and biases for user groups and social contexts, and ethical implications. Further research is also required to integrate and test psychological theories to improve the impact of mHealth and adherence to the e–mental health interventions.

Evaluation Metrics for Health Chatbots: A Delphi Study

BACKGROUND

 In recent years, an increasing number of health chatbots has been published in app stores and described in research literature. Given the sensitive data they are processing and the care settings for which they are developed, evaluation is essential to avoid harm to users. However, evaluations of those systems are reported inconsistently and without using a standardized set of evaluation metrics. Missing standards in health chatbot evaluation prevent comparisons of systems, and this may hamper acceptability since their reliability is unclear.

OBJECTIVES

 The objective of this paper is to make an important step toward developing a health-specific chatbot evaluation framework by finding consensus on relevant metrics.

METHODS

 We used an adapted Delphi study design to verify and select potential metrics that we retrieved initially from a scoping review. We invited researchers, health professionals, and health informaticians to score each metric for inclusion in the final evaluation framework, over three survey rounds. We distinguished metrics scored relevant with high, moderate, and low consensus. The initial set of metrics comprised 26 metrics (categorized as global metrics, metrics related to response generation, response understanding and aesthetics).

RESULTS

 Twenty-eight experts joined the first round and 22 (75%) persisted to the third round. Twenty-four metrics achieved high consensus and three metrics achieved moderate consensus. The core set for our framework comprises mainly global metrics (e.g., ease of use, security content accuracy), metrics related to response generation (e.g., appropriateness of responses), and related to response understanding. Metrics on aesthetics (font type and size, color) are less well agreed upon-only moderate or low consensus was achieved for those metrics.

CONCLUSION

 The results indicate that experts largely agree on metrics and that the consensus set is broad. This implies that health chatbot evaluation must be multifaceted to ensure acceptability.

Social Media Chatbot for Increasing Physical Activity: Usability Study

Fully automated self-help interventions integrated with social media chatbots could serve as highly cost-effective physical activity promotion tools for a large population. We have developed MYA, a Telegram-based chatbot for increasing physical activity. The objective of this study was to assess the usability of MYA. To identify usability issues, we recruited volunteers and asked them to interact with MYA and to answer the Chatbot Usability Questionnaire. Thirty volunteers participated in the study, 83.3% agreed MYA was welcoming during initial setup and 63.3% agreed MYA was very easy to use. MYA was perceived as realistic and engaging, easy to navigate, and its responses were useful, appropriate, and informative (all 53.3%). However, 63.3% of respondents agreed MYA failed to recognize most of their inputs, and 43.3% claimed it would be easy to get confused when using MYA. Although the results are encouraging, it remains unclear if a social media chatbot can motivate people to increase their physical activity. MYA has the potential to do that, with improvements in functionalities like challenge personalization. The efficacy of these approaches should be studied in a clinical trial.

Security, privacy, and healthcare-related conversational agents: a scoping review

Health chatbots interview patients and collect health data. This process makes demands on data security and data privacy. To identify how and to what extent security and privacy are considered in current health chatbots. We conducted a scoping review by searching three bibliographic databases (PubMed, ACM Digital Library, IEEExplore) for papers reporting on chatbots in healthcare. We extracted which, how, and where data is stored by health chatbots and identified which external services have access to the data. Out of 1026 retrieved papers, we included 70 studies in the qualitative synthesis. Most papers report on chatbots that collect and process personal health data, usually in the context of mental health coaching applications. The majority did not provide any information regarding security or privacy aspects. We were able to determine limitations in literature and identified concrete challenges, including data access and usage of (third-party) services, data storage, data security methods, use case peculiarities and data privacy, as well as legal requirements. Data privacy and security in health chatbots are still underresearched and related information is underrepresented in scientific literature. By addressing the five key challenges in future, the transfer of theoretical solutions into practice can be facilitated.

Defining participatory health informatics - a scoping review

Healthcare has been shifting toward individuals participating in decision-making and empowered to be active in their treatment, and health monitoring. The term "participatory health informatics" (PHI) started to appear in literature. A clear definition of PHI is missing, and facets of PHI still have to be shaped. The objective of this paper is to offer a definition of PHI considering themes and technologies that make healthcare participatory. We searched Pubmed, ACM Digital Library, IEEE Xplore, EMBASE, and conference proceedings for articles that reported about use of information technology or informatics in the context of PHI. We performed qualitative synthesis and reported summary statistics. 39 studies were eligible after screening 382 titles and abstracts and reviewing 82 full texts. The top 5 person-centered key themes related to PHI included empowerment, decision-making, informed patient, collaboration, and disease management. Finally, we propose to define PHI as multidisciplinary field that uses information technology as provided through the web, smartphones, or wearables to increase participation of individuals in their care process and to enable them in self-care and shared decision-making. Goals to be achieved through PHI include maintaining health and well-being; improving the healthcare system and health outcomes; sharing experiences; achieving life goals; and self-education.

Role of Participatory Health Informatics in Detecting and Managing Pandemics: Literature Review

OBJECTIVES

Using participatory health informatics (PHI) to detect disease outbreaks or learn about pandemics has gained interest in recent years. However, the role of PHI in understanding and managing pandemics, citizens' role in this context, and which methods are relevant for collecting and processing data are still unclear, as is which types of data are relevant. This paper aims to clarify these issues and explore the role of PHI in managing and detecting pandemics.

METHODS

Through a literature review we identified studies that explore the role of PHI in detecting and managing pandemics. Studies from five databases were screened: PubMed, CINAHL (Cumulative Index to Nursing and Allied Health Literature), IEEE Xplore, ACM (Association for Computing Machinery) Digital Library, and Cochrane Library. Data from studies fulfilling the eligibility criteria were extracted and synthesized narratively.

RESULTS

Out of 417 citations retrieved, 53 studies were included in this review. Most research focused on influenza-like illnesses or COVID-19 with at least three papers on other epidemics (Ebola, Zika or measles). The geographic scope ranged from global to concentrating on specific countries. Multiple processing and analysis methods were reported, although often missing relevant information. The majority of outcomes are reported for two application areas: crisis communication and detection of disease outbreaks.

CONCLUSIONS

For most diseases, the small number of studies prevented reaching firm conclusions about the utility of PHI in detecting and monitoring these disease outbreaks. For others, e.g., COVID-19, social media and online search patterns corresponded to disease patterns, and detected disease outbreak earlier than conventional public health methods, thereby suggesting that PHI can contribute to disease and pandemic monitoring.