Methods of usability testing in the development of eHealth applications: A scoping review

Designing and Developing an eHealth Program for Patients With Persistent Physical Symptoms: Usability Study

BACKGROUND

Patients with persistent physical symptoms presenting in primary care are often affected by multiple symptoms and reduced functioning. The medical and societal costs of these patients are high, and there is a need for new interventions tailored to both the patients and health care system.

OBJECTIVE

This study aimed to examine the usability of an unguided, self-help treatment program, "My Symptoms," developed to assist patients and general practitioners in symptom management.

METHODS

In all, 11 users (4 patients with persistent physical symptoms and 7 laypeople) participated in web-based thinking-aloud interviews involving the performance of predefined tasks in the program. Thematic analysis was used to categorize the severity of usability issues. General usability heuristics were cross-referenced with the usability issues.

RESULTS

The analysis identified important usability issues related to functionality, navigation, and content. The study shows how therapeutic knowledge in some cases was lost in the translation of face-to-face therapy to a digital format. The user testing helped uncover how the functionality of the digital elements and general navigation of the program played a huge part in locating and accessing the needed treatment. Examples of redesign to mediate the therapeutic value in the digital format involving health care professionals, web developers, and users are provided. The study also highlights the differences of involving patients and laypeople in the interviews.

CONCLUSIONS

Taking the experience of common symptoms as a point of departure, patients and laypeople contributed to finding usability issues on program functionality, navigation, and content to improve the program and make the treatment more accessible to users.

Designing a Future eHealth Service for Posthospitalization Self-management Support in Long-term Illness: Qualitative Interview Study

BACKGROUND

For patients with noncommunicable diseases (NCDs; eg, heart failure [HF] and colorectal cancer [CRC]), eHealth interventions could meet their posthospital discharge needs and strengthen their ability to self-manage. However, inconclusive evidence exists regarding how to design eHealth services to meet the complex needs of patients. To foster patient acceptability and ensure the successful development and implementation of eHealth solutions, it is beneficial to include different stakeholders (ie, patients and health care professionals) in the design and development phase of such services. The involvement of different stakeholders could contribute to ensuring feasible, acceptable, and usable solutions and that eHealth services are developed in response to users' supportive care needs when transitioning to home after hospitalization. This study is the first step of a larger complex intervention study aimed at meeting the postdischarge needs of 2 NCD populations.

OBJECTIVE

This study aimed to explore the perspectives of patients with HF and CRC and health care professionals on patient self-management needs following hospital discharge and investigate how a future nurse-assisted eHealth service could be best designed to foster patient acceptability, support self-management, and smooth the transition from hospital to home.

METHODS

A qualitative, explorative, and descriptive approach was used. We conducted 38 semistructured interviews with 10 patients with HF, 9 patients surgically treated for CRC with curative intent, 6 registered nurses recruited as nurse navigators of a planned eHealth service, and 13 general practitioners experienced in HF and CRC treatment and follow-up care. Patients were recruited conveniently from HF and CRC outpatient clinics, and the nurses were recruited from the cardiology and gastro-surgical departments at a university hospital in the southwest of Norway. The general practitioners were recruited from primary care in surrounding municipalities. Semistructured interview guides were used for data collection, and the data were analyzed using thematic analysis.

RESULTS

In total, 3 main themes were derived from the data analysis: expecting information, reassurance, and guidance when using eHealth for HF and CRC self-management; expecting eHealth to be comprehensible, supportive, and knowledge promoting; and recognizing both the advantages and disadvantages of eHealth for HF and CRC self-management. The data generated from this interview study depicted the diverse needs for self-management support of patients with CRC and HF after hospital discharge. In addition, valuable suggestions were identified regarding the design and content of the eHealth service. However, participants described both possible advantages and disadvantages of a remote eHealth service.

CONCLUSIONS

This study is the first step in the development of an eHealth service for posthospitalization self-management support for long-term illnesses. It concerns patients' supportive care needs and user requirements of an eHealth service. The findings of this study may add value to the planning and development of eHealth interventions for patients with NCDs.

A Comprehensive Digital Self-care Support System for Older Adults With Multiple Chronic Conditions: Development, Feasibility, and Usability Testing of myHESTIA

The objective of this mixed methods study is to evaluate the need for a comprehensive digital self-care support system (CDSSS) for older adults with multiple chronic conditions (MCC) and to examine whether such a system can be developed to enable daily capture of self-care data. The 3-phase study involved Phase-1: user needs assessment and prototype development; Phase-2: preliminary user evaluation of the prototype; and Phase-3: 4-week small group usability and feasibility testing of the tracking component of the prototype. Results of Phase-1 show the need for a CDSSS. Phase-2 results demonstrate interest among older adults in using such a CDSSS and Phase-3 findings show that older adults found the tracking component of the system easy to use for capturing daily inputs. Overall, the findings show that it is feasible to design a CDSSS for older adults with MCC in a way that is usable and functional for them.

Usability of Graphical User Interfaces With Semiautomatic Delabeling Feature to Improve Drug Allergy Documentation

BACKGROUND

The quality of allergy documentation in electronic health records is frequently poor.

OBJECTIVE

To compare the usability of 3 graphical user interfaces (GUIs) for drug allergy documentation.

METHODS

Physicians tested 3 GUIs by means of 5 fictional drug allergy scenarios: the current GUI (GUI 0), using mainly free-text, and 2 new coded versions (GUI 1 and GUI 2) asking information on allergen category, specific allergen, symptom(s), symptom onset, timing of initial reaction, and diagnosis status with a semiautomatic delabeling feature. Satisfaction was measured by the System Usability Scale questionnaire, efficiency by time to complete the tasks, and effectiveness by a task completion score. Posttest interviews provided more in-depth qualitative feedback.

RESULTS

Thirty physicians from 7 different medical specialties and with varying degrees of experience participated. The mean System Usability Scale scores for GUI 1 (77.25, adjective rating "Good") and GUI 2 (78.42, adjective rating "Good") were significantly higher than for GUI 0 (56.58, adjective rating "OK") (Z, 6.27, P_adj < .001 and Z, 6.62, P_adj < .001, respectively). There was no significant difference in task time between GUIs. Task completion scores of GUI 1 and GUI 2 were higher than for GUI 0 (Z, 9.59, P_adj < .001 and Z, 11.87, P_adj < .001, respectively). Quantitative and qualitative findings were combined to propose a GUI 3 with high usability.

CONCLUSIONS

The usability and quality of allergy documentation was higher for the newly developed coded GUIs with a semiautomatic delabeling feature without being more time-consuming.

A bridge from uncertainty to understanding: The meaning of symptom management digital health technology during cancer treatment

Objective

Digital health technology is valued as a tool to provide person-centred care and improve health outcomes amongst people with cancer and their family caregivers. Although the evidence to date shows encouraging effectiveness, there is limited knowledge regarding the lived experience and personal meaning of using supportive technology during cancer treatment. The aim of this study was to explore the lived experiences of people with colorectal cancer receiving chemotherapy using digital health symptom management technology and their family caregivers.

Methods

A longitudinal and multi-perspective interpretative phenomenological analytical approach was adopted including three people with newly diagnosed colorectal cancer and four family caregivers.

Findings

Three superordinate themes and related subthemes were identified. The first theme (The 3 Cs of symptom management technology) centred on the continuity of care that participants felt while using the technology. The second theme (Digital health technology as a psychosocial support) offered insights into the psychological benefits using technology incurred as they navigated their cancer diagnosis including sense of control and psychological safety. The final theme (Impact of digital health technology on family caregivers) details the supportive effect the technology had on family caregivers' role, responsibilities and well-being during the cancer experience.

Conclusion

Digital health technology can act as a bridge from uncertainty to an understanding regarding a cancer diagnosis and its treatment. Digital health technology can support peoples' understanding of cancer and enhance self-management practices, while being a psychological support in navigating the uncertain and often worrying period of receiving cancer treatment.

Fact or Fiction? The Development and Evaluation of a Tobacco Virtual Health Tool

The virtual setting is an important setting for health promotion as individuals increasingly go online for health information and support. Yet, users can have difficulty finding valid, trustworthy, and user-friendly health information online. In 2022, we launched an interactive Fact or Fiction Tobacco virtual health tool. The virtual health tool uses evidence-informed tailored content to engage users and refer them to local tobacco cessation resources. The present paper describes the development, user testing, and evaluation of this tool. The Fact or Fiction virtual health tool was designed by tobacco cessation and health marketing experts and informed by health behaviour theories of change. The tool captures data on who is seeking health information, the user's stage of readiness to quit tobacco products, and whether they act by accessing referred resources. In 2021, we conducted two phases of user testing prior to marketing the tool publicly. After 7 weeks of marketing, we collected data on user interactions with the tool and evaluated the reach of the tool. Results from user testing found the tool to be engaging, easy to use, and quick to complete. Adaptations were made to simplify and condense text and include additional animations. During the first seven weeks of the tool being live, it reached 2306 users, and 38.7% of those users were current or occasional tobacco users. Users were classified based on their intention to quit. Bivariate analysis found that the tool was successful in driving tobacco users towards action as 21.2% tobacco users who were looking to quit and 8.8% of tobacco users who were not looking to quit clicked on local tobacco cessation resources. This virtual health tool is reaching the targeted population and providing tailored information needed at each stage of the continuum of health behaviour change. Among tobacco users looking to quit, this virtual health tool acts as a quick referral to local tobacco cessation resources.

Assessing the Acceptability of Home Blood Monitoring for Patients With Cancer Who Are Receiving Systemic Anticancer Therapy From a Patient, Caregiver, and Clinician Perspective: Focus Group and Interview Study

BACKGROUND

Regular blood testing is an integral part of systemic anticancer therapy delivery. Blood tests are required before every administration of treatment to ensure that a patient is sufficiently well to receive it. Blood testing is burdensome for patients as they require either an extra visit within 48 hours of planned administration of treatment or a significantly long visit if performed on the day of treatment. The additional time for appointments can have a significant impact on the quality of life of someone who is living with cancer. In the United Kingdom, the COVID-19 pandemic created unprecedented disruption to the delivery of cancer care. Face-to-face hospital visits were reduced, resulting in the need to develop more innovative ways of working to minimize treatment interruptions. This led to significant uptake of digital technologies, with new models of care rapidly deployed across the UK health service to meet these challenges.

OBJECTIVE

This study aimed to explore the acceptability of a point-of-care home blood monitoring device for people with cancer who are receiving systemic anticancer therapy, which is being developed in response to the increased need for remote care for patients with cancer.

METHODS

Qualitative focus groups and semistructured interviews were conducted with patients (23/47, 49%), caregivers (6/47, 13%), and health care professionals (18/47, 38%) over a 19-month time frame from May 2019 to December 2020. Data were analyzed using framework analysis guided by the Unified Theory of Acceptance and Use of Technology model.

RESULTS

Analysis identified 4 overarching themes: performance expectancy, effort expectancy, social influence, and facilitating conditions.

CONCLUSIONS

This study found that patients with cancer, their caregivers, and health care professionals had positive perceptions about home blood monitoring. Although they are often considered synonymously, self-testing and self-management are not mutually exclusive, and this study illustrated some disparity in opinions regarding patient self-management. Home blood monitoring has the potential to provide patients with cancer with a convenient option for blood monitoring. It would minimize hospital attendances, decrease late treatment deferrals, and provide prompt recognition of cancer treatment toxicities, thus enhancing the existing nurse-led protocols and clinical pathways. Home blood monitoring would create a long-term sustainable transformation for the delivery of cancer care, using digital health to act as a facilitator to address a pertinent issue regarding improving the efficiency of hospital resources and increasing the delivery of personalized patient care. Further studies are needed to determine how and where home blood monitoring would fit within clinical pathways, in a way that is robust and equitable.

Web version of the protocol of the orofacial myofunctional evaluation with scores: usability and learning

PURPOSE

The Orofacial Myofunctional Evaluation with Scores (OMES) protocol has been validated and used in clinical practice and research. The goals of this study were to develop, analyze and improve a version of OMES for the Web and to investigate the relationship between the usability judgments and the prior experience of the evaluators and whether using the interface promotes learning, as shown by the task completion time (TCT).

METHODS

Study steps: 1) inspection of the prototype by the team; 2) evaluation of usability by three experienced speech-language pathologists (SLPs); and 3) evaluation of its usability by 12 SLPs with varying levels of experience in the use of OMES. Participants answered the Heuristic evaluation (HE), the Computer System Usability Questionnaire (CSUQ), and expressed free comments. The TCT was recorded.

RESULTS

The OMES-Web reached excellent usability levels, and the participants were highly satisfied. The correlations between the participants' experience and the HE and CSUQ scores were not significant. The TCT decreased significantly throughout the tasks.

CONCLUSION

OMES-Web meets the usability criteria, and participants feel satisfied with the system regardless of their level of experience. The fact that it is easy to learn favors its adoption by professionals.

[Development, Evaluation and Implementation of Digital Health Interventions (Part 1) - Discussion Paper of the Digital Health Working Group of the German Network for Health Services Research (DNVF)]

The development and application of digital interventions in health-related topics are gaining momentum in health service research. Digital interventions are often complex and need to be evaluated and implemented in complex settings. Due to their characteristics, this poses methodological challenges for health services research that have to be identified and addressed. Hence, the Working Group on Digital Health of the German Network for Health Services Research (DNVF) has prepared a discussion paper. This paper discusses methodological, practical and theoretical challenges associated with the development and evaluation of digital interventions from the perspective of health services research. Possible solutions are suggested and future research needs to address these methodological challenges are identified.

Patient-facing digital tools for delivering genetic services: a systematic review

This study systematically reviewed the literature on the impact of digital genetics tools on patient care and system efficiencies. MEDLINE and Embase were searched for articles published between January 2010 and March 2021. Studies evaluating the use of patient-facing digital tools in the context of genetic service delivery were included. Two reviewers screened and extracted patient-reported and system-focused outcomes from each study. Data were synthesised using a descriptive approach. Of 3226 unique studies identified, 87 were included. A total of 70 unique digital tools were identified. As a result of using digital tools, 84% of studies reported a positive outcome in at least one of the following patient outcomes: knowledge, psychosocial well-being, behavioural/management changes, family communication, decision-making or level of engagement. Digital tools improved workflow and efficiency for providers and reduced the amount of time they needed to spend with patients. However, we identified a misalignment between study purpose and patient-reported outcomes measured and a lack of tools that encompass the entire genetic counselling and testing trajectory. Given increased demand for genetic services and the shift towards virtual care, this review provides evidence that digital tools can be used to efficiently deliver patient-centred care. Future research should prioritise development, evaluation and implementation of digital tools that can support the entire patient trajectory across a range of clinical settings. PROSPERO registration numberCRD42020202862.

Usability evaluation of mobile phone technologies for capturing cancer patient-reported outcomes and physical functions

Background

By eliminating the requirement for participants to make frequent visits to research sites, mobile phone applications ("apps") may help to decentralize clinical trials. Apps may also be an effective mechanism for capturing patient-reported outcomes and other endpoints, helping to optimize patient care during and outside of clinical trials.

Objectives

We report on the usability of Digital BioMarkers for Clinical Impact (DigiBioMarC™ (DBM)), a novel smartphone-based app used by cancer patients in conjunction with a wearable device (Apple Watch®). DBM is designed to collect patient-reported outcomes and record physical functions.

Methods

In a fully decentralized "bring-your-own-device" smartphone study, we enrolled 54 cancer patient and caregiver dyads from Kaiser Permanente Northern California (KPNC) from October 2020 through March 2021. Patients used the app for at least 28 days, completed weekly questionnaires about their symptoms, physical functions, and mood, and performed timed physical tasks. Usability was determined through a subset of the Mobile App Rating Scale (MARS), the full System Usability Scale (SUS), the Net Promoter Score (NPS), and semi-structured interviews.

Results

We obtained usability survey data from 50 of 54 patients. Median responses to the selected MARS questions and the mean SUS scores indicated above average usability. The NPS from the semi-structured interviews at the end of the study was 24, indicating a favorable score.

Conclusions

Cancer patients reported above average usability for the DBM app. Qualitative analyses indicated that the app was easy to use and helpful. Future work will emphasize implementing further patient recommendations and evaluating the app's clinical efficacy in multiple settings.

A framework for design and usability testing of telerehabilitation system for adults with chronic diseases: A panoramic scoping review

Objective

This scoping review aimed to identify the design and usability testing of a telerehabilitation (TR) system, and its characteristics and functionalities that are best-suited for rehabilitating adults with chronic diseases.

Methods

Searches were conducted in PubMed, EBSCO, Web of Science, and Cochrane library for studies published between January 2017 and December 2022. We followed the Joanna Briggs Institute guidelines and the framework by Arksey and O'Malley. Screening was undertaken by two reviewers, and data extraction was undertaken by the first author. Then, the data were further reviewed and discussed thoroughly with the team members.

Results

A total of 31 results were identified, with the core criteria of developing and testing a telerehabilitation system, including a mobile app for cardiovascular diseases, cancer, diabetes, and chronic respiratory disorders. All developed systems resulted from multidisciplinary teams and employed mixed-methods research. We proposed the "input-process-output" framework that identified phases of both system design and usability testing. Through system design, we reported the use of user-centered design, iterative design, users' needs and characteristics, theory underpinning development, and the expert panel in 64%, 75%, 86%, 82%, and 71% of the studies, respectively. We recorded the application of moderated usability testing, unmoderated testing (1), and unmoderated testing (2) in 74%, 63%, and 15% of the studies, respectively. The identified design and testing activities produced a matured system, a high-fidelity prototype, and a released system in 81.5%, 15%, and 3.5%, respectively.

Conclusion

This review provides a framework for TR system design and testing for a wide range of chronic diseases that require prolonged management through remote monitoring using a mobile app. The identified "input-process-output" framework highlights the inputs, design, development, and improvement as components of the system design. It also identifies the "moderated-unmoderated" model for conducting usability testing. This review illustrates characteristics and functionalities of the TR systems and healthcare professional roles.

One Website to Gather them All: Usability Testing of the New German SKin Cancer INFOrmation (SKINFO) Website-A Mixed-methods Approach

Skin cancer patients increasingly search the internet to acquire disease-related information. However, information on the internet may be misleading. Recently, SKINFO has been launched, a website exclusively created for German-speaking skin cancer patients providing information as well as additional resources of verified quality. Here, we describe the results of the first usability test of SKINFO using a mixed-methods approach. Ten adult patients with skin cancer were recruited for usability testing in the skin cancer units of the University Hospitals of Erlangen and Dresden, Germany. Testing consisted of three different scenarios where patients were asked to find specific information on the SKINFO website guided by the think-aloud method. Descriptive analysis and content analyses were performed. All patients would recommend SKINFO and appreciated its content, design, and structure. Think-aloud analysis identified the topics layout, navigation, and content and structure which would benefit from refinement. Major criticism included the navigation through the website, and the desire for more specific information addressing patients' relatives and the latest, up-to-date information. Overall, usability testing showed that the unique web-based information platform has the potential to support patients coping with skin cancer and thus strengthen informed decision-making.

Physician preference for receiving machine learning predictive results: A cross-sectional multicentric study

Artificial intelligence (AI) algorithms are transforming several areas of the digital world and are increasingly being applied in healthcare. Mobile apps based on predictive machine learning models have the potential to improve health outcomes, but there is still no consensus on how to inform doctors about their results. The aim of this study was to investigate how healthcare professionals prefer to receive predictions generated by machine learning algorithms. A systematic search in MEDLINE, via PubMed, EMBASE and Web of Science was first performed. We developed a mobile app, RandomIA, to predict the occurrence of clinical outcomes, initially for COVID-19 and later expected to be expanded to other diseases. A questionnaire called System Usability Scale (SUS) was selected to assess the usability of the mobile app. A total of 69 doctors from the five regions of Brazil tested RandomIA and evaluated three different ways to visualize the predictions. For prognostic outcomes (mechanical ventilation, admission to an intensive care unit, and death), most doctors (62.9%) preferred a more complex visualization, represented by a bar graph with three categories (low, medium, and high probability) and a probability density graph for each outcome. For the diagnostic prediction of COVID-19, there was also a majority preference (65.4%) for the same option. Our results indicate that doctors could be more inclined to prefer receiving detailed results from predictive machine learning algorithms.

Usability of the REHOME Solution for the Telerehabilitation in Neurological Diseases: Preliminary Results on Motor and Cognitive Platforms

The progressive aging of the population and the consequent growth of individuals with neurological diseases and related chronic disabilities, will lead to a general increase in the costs and resources needed to ensure treatment and care services. In this scenario, telemedicine and e-health solutions, including remote monitoring and rehabilitation, are attracting increasing interest as tools to ensure the sustainability of the healthcare system or, at least, to support the burden for health care facilities. Technological advances in recent decades have fostered the development of dedicated and innovative Information and Communication Technology (ICT) based solutions, with the aim of complementing traditional care and treatment services through telemedicine applications that support new patient and disease management strategies. This is the background for the REHOME project, whose technological solution, presented in this paper, integrates innovative methodologies and devices for remote monitoring and rehabilitation of cognitive, motor, and sleep disorders associated with neurological diseases. One of the primary goals of the project is to meet the needs of patients and clinicians, by ensuring continuity of treatment from healthcare facilities to the patient's home. To this end, it is important to ensure the usability of the solution by elderly and pathological individuals. Preliminary results of usability and user experience questionnaires on 70 subjects recruited in three experimental trials are presented here.

Usability evaluation of mHealth apps for elderly individuals: a scoping review

BACKGROUND

Usability is a key factor affecting the acceptance of mobile health applications (mHealth apps) for elderly individuals, but traditional usability evaluation methods may not be suitable for use in this population because of aging barriers. The objectives of this study were to identify, explore, and summarize the current state of the literature on the usability evaluation of mHealth apps for older adults and to incorporate these methods into the appropriate evaluation stage.

METHODS

Electronic searches were conducted in 10 databases. Inclusion criteria were articles focused on the usability evaluation of mHealth apps designed for older adults. The included studies were classified according to the mHealth app usability evaluation framework, and the suitability of evaluation methods for use among the elderly was analyzed.

RESULTS

Ninety-six articles met the inclusion criteria. Research activity increased steeply after 2013 (n = 92). Satisfaction (n = 74) and learnability (n = 60) were the most frequently evaluated critical measures, while memorability (n = 13) was the least evaluated. The ratios of satisfaction, learnability, operability, and understandability measures were significantly related to the different stages of evaluation (P < 0.05). The methods used for usability evaluation were questionnaire (n = 68), interview (n = 36), concurrent thinking aloud (n = 25), performance metrics (n = 25), behavioral observation log (n = 14), screen recording (n = 3), eye tracking (n = 1), retrospective thinking aloud (n = 1), and feedback log (n = 1). Thirty-two studies developed their own evaluation tool to assess unique design features for elderly individuals.

CONCLUSION

In the past five years, the number of studies in the field of usability evaluation of mHealth apps for the elderly has increased rapidly. The mHealth apps are often used as an auxiliary means of self-management to help the elderly manage their wellness and disease. According to the three stages of the mHealth app usability evaluation framework, the critical measures and evaluation methods are inconsistent. Future research should focus on selecting specific critical measures relevant to aging characteristics and adapting usability evaluation methods to elderly individuals by improving traditional tools, introducing automated evaluation tools and optimizing evaluation processes.

A Pilot Study Investigating the Use of Mobile Technology for Coordinating Educational Plans in Inclusive Settings

Good coordination among school staff and families leads to increased learning quality and academic success for students with special education needs and disabilities (SEND). This pilot study aims to investigate the use of mobile technology for the coordination of therapy and learning for students with SEND. This study first follows a participatory design methodology to identify the key design principles required to inform the design of a coordination mobile app for special education. Then, a mobile app (IEP-Connect) is designed and implemented with the aim of facilitating information sharing between different parties involved in the intervention of students with SEND. The proposed app uses the Individualized Educational Plan (IEP) as the focal point of coordination. The evaluation of the app focused on students with autism spectrum disorder (ASD) as their learning requires sharing information from different distributed sources. Results from the usability study revealed that the app has “good” usability and that participants were satisfied with the use of the app for recording and sharing IEP information. The results of this study provide an understanding of the ways in which a coordination app for special education could be made easy and rewarding to use.

Performance and usability evaluation of a mobile health data capture application in clinical cancer trials follow-up

•

A mobile health data capture application can improve quality of data collection in clinical cancer trials.

•

The use of a mobile health data capture application optimizes the efficiency and effectivity of a trial participant’s visit to a clinic.

•

A mobile health data capture application relieves participants from instant recall on possible health changes during visits.

•

The use of a mobile health data capture application helps a healthcare provider in preparing a participant’s visit.

Mobile health data capture applications (mHDA’s) may improve communication between healthcare providers and patients. However, there is limited literature about the use of mHDA’s facilitating clinical trials. In this study, the effectiveness of an application, supporting follow-up visits of cancer trial participants was investigated. Twenty participants were provided with an e-questionnaire via the mHDA. Participants rated the usability of the application as high performing (mean Systems Usability Scale 87 points). The research team rated the mHDA as highly applicable and efficient in preparing visits. Anamnesis, physical examination and agreement on further policy were performed within an average of 31 min.

Usability, acceptability and feasibility of a novel technology with visual guidance with video and audio recording during newborn resuscitation: a pilot study

OBJECTIVE

Inadequate adherence to resuscitation for non-crying infants will have poor outcome and thus rationalise a need for real-time guidance and quality improvement technology. This study assessed the usability, feasibility and acceptability of a novel technology of real-time visual guidance, with sound and video recording during resuscitation.

SETTING

A public hospital in Nepal.

DESIGN

A cross-sectional design.

INTERVENTION

The technology has an infant warmer with light, equipped with a tablet monitor, NeoBeat and upright bag and mask. The tablet records resuscitation activities, ventilation sound, heart rate and display time since birth. Healthcare providers (HCPs) were trained on the technology before piloting.

DATA COLLECTION AND ANALYSIS

HCPs who had at least 8 weeks of experience using the technology completed a questionnaire on usability, feasibility and acceptability (ranged 1-5 scale). Overall usability score was calculated (ranged 1-100 scale).

RESULTS

Among the 30 HCPs, 25 consented to the study. The usability score was good with the mean score (SD) of 68.4% (10.4). In terms of feasibility, the participants perceived that they did not receive adequate support from the hospital administration for use of the technology, mean score (SD) of 2.44 (1.56). In terms of acceptability, the information provided in the monitor, that is, time elapsed from birth was easy to understand with mean score (SD) of 4.60 (0.76).

CONCLUSION

The study demonstrates reasonable usability, feasibility and acceptability of a technological solution that records audio visual events during resuscitation and provides visual guidance to improve care.

Evaluation of the nation-wide implementation of ALS home monitoring & coaching: an e-health innovation for personalized care for patients with motor neuron disease

BACKGROUND

To improve the care for patients with motor neuron disease an e-health innovation for continuous monitoring of disease progression and patients' well-being (ALS H&C) was implemented in 10 multidisciplinary rehabilitation settings. The first aim was to evaluate the implementation of ALS H&C by assessing several implementation outcomes, technology acceptance and usability of the innovation according to the end users. The secondary aim was to explore differences in these outcomes between the teams with sustainable and unsustainable implementation.

METHODS

The chosen implementation strategy was a combination of the implementation process model by Grol & Wensing and a participatory action research approach. In three meetings with multidisciplinary project groups the innovation was introduced, the expected barriers/facilitators identified, and action plans to resolve each barrier developed. After a 3-month pilot phase, patients and their healthcare providers were asked to complete an online evaluation survey to assess implementation outcomes, based on Proctor's evaluation framework (i.e., acceptability, feasibility, fidelity, sustainability). Telemedicine technology acceptance was assessed according the technology acceptance model of Chau, and user experiences with the System Usability Scale (SUS). Implementation outcomes of teams with sustainable implementation (continuation after completion of the pilot phase) and unsustainable implementation (discontinuation after the pilot phase) were compared.

RESULTS

The implementation outcomes from the patients' perspective (N = 71) were positive; they found ALS H&C to be an acceptable and feasible care concept. Patients' technology acceptance was high, with positive attitudes towards ALS H&C, and positive views on perceived technology control, usefulness, and ease of use. Patients rated their satisfaction with the (web) app on a scale from 1 (not satisfied at all) to 10 (very satisfied) with a 7.0 (median; IQR 1.0). Healthcare providers (N = 76) also found ALS H&C acceptable and appropriate as well, but were less positive about the feasibility and usability of ALS H&C (mean SUS 58.8 [SD 11.3]). ALS H&C has largely been implemented as intended and the implementation was sustainable in 7 teams. Teams who discontinued ALS H&C after the pilot phase (N = 2) had more fidelity issues.

CONCLUSIONS

A participatory action research approach supported by theoretical approaches used in implementation science led to a sustainable implementation of ALS H&C in 7 of the participating teams. To improve implementation success, additional implementation strategies to increase feasibility, usability and fidelity are necessary.

TRIAL REGISTRATION

Trial NL8542 registered at Netherlands Trial Register (trialregister.nl) on 15th April 2020.

Design and development of an eHealth intervention to support self-Management in people with musculoskeletal Disorders - ‘eHealth: It’s TIME’: a study protocol

Background: Musculoskeletal disorders (MSDs) are a leading cause of global morbidity, with the burden expected to increase in the near future. Self-management, with the support of healthcare professionals, is recommended for many MSDs. However, frequent clinical contact is not feasible. Previous research has highlighted the need for a co-designed eHealth-mediated self-management follow-up support intervention which integrates remote monitoring and behavioural change. Thus, the current study aims to develop and design a user-centred, eHealth-mediated self-management support prototype for people with MSDs. Methods: A three-step, iterative system development cycle will be utilised to develop and design the “eHealth: It’s TIME prototype”. The three-step process will include creating website features and content using two sequential focus groups with people with MSDs (n = 6 – 8); heuristic testing using the 10 heuristic principles of Nielsen (n = 5); and usability testing through in-person 60-minute interviews with people with MSDs (n = 3 – 5) and musculoskeletal physiotherapists (n = 3 – 5). Conclusion: The eHealth: It’s TIME prototype will be a systematically developed, follow-up self-management support intervention guided by behavioural change theory and the preferences of end users.

A Short Digital Food Frequency Questionnaire (DIGIKOST-FFQ) Assessing Dietary Intake and Other Lifestyle Factors Among Norwegians: Qualitative Evaluation With Focus Group Interviews and Usability Testing

Background

In-person dietary counseling and interventions have shown promising results in changing habits toward healthier lifestyles, but they are costly to implement in large populations. Developing digital tools to assess individual dietary intake and lifestyle with integrated personalized feedback systems may help overcome this challenge. We developed a short digital food frequency questionnaire, known as the DIGIKOST-FFQ, to assess diet and other lifestyle factors based on the Norwegian Food-Based Dietary Guidelines. The DIGIKOST-FFQ includes a personalized feedback system, the DIGIKOST report, that benchmarks diet and lifestyle habits. We used qualitative focus group interviews and usability tests to test the feasibility and usability of the DIGIKOST application.

Objective

We aimed to explore attitudes, perceptions, and challenges in completing the DIGIKOST-FFQ. We also investigated perceptions and understanding of the personalized feedback in the DIGIKOST report and the technical flow and usability of the DIGIKOST-FFQ and the DIGIKOST report.

Methods

Healthy individuals and cancer survivors were invited to participate in the focus group interviews. The transcripts were analyzed using thematic analysis. Another group of healthy individuals completed the usability testing, which was administered individually by a moderator and 2 observers. The results were analyzed based on predefined assignments and discussion with the participants about the interpretation of the DIGIKOST report and technical flow of the DIGIKOST-FFQ.

Results

A total of 20 individuals participated in the focus group interviews, divided into 3 groups of healthy individuals and 3 groups of cancer survivors. Each group consisted of 3 to 4 individuals. Five main themes were investigated: (1) completion time (on average 19.1, SD 8.3, minutes, an acceptable duration), (2) layout (participants reported the DIGIKOST-FFQ was easy to navigate and had clear questions but presented challenges in reporting dietary intake, sedentary time, and physical activity in the last year), (3) questions (the introductory questions on habitual intake worked well), (4) pictures (the pictures were very helpful, but some portion sizes were difficult to differentiate and adding weight in grams would have been helpful), and (5) motivation (users were motivated to obtain personalized feedback). Four individuals participated in the usability testing. The results showed that the users could seamlessly log in, give consent, fill in the DIGIKOST-FFQ, and receive, print, and read the DIGIKOST report. However, parts of the report were perceived as difficult to interpret.

Conclusions

The DIGIKOST-FFQ was overall well received by participants, who found it feasible to use; however, some adjustments with regard to reporting dietary intake and lifestyle habits were suggested. The DIGIKOST report with personalized feedback was the main motivation to complete the questionnaire. The results from the usability testing revealed a need for adjustments and updates to make the report easier to read.

Design, Development, and Evaluation of the Blood Collection Management Workstation

Purpose

To design and develop a blood collection management workstation with high usability to reduce the risk of preanalytical errors and improve patient safety.

Methods

A five-phase mobile application development lifecycle model (MADLC) and experience-based co-design (EBCD) were used for design and development. Subsequently, the blood collection management workstation was evaluated using the Chinese System Usability Scale (SUS) in a general ward setting from January to June 2021.

Results

It was used on 2593 in-patients who underwent phlebotomy with 12,378 tubes being labeled. The rate of errors and meantime for blood sampling were decreased compared with the same period in the previous year. A total of 14 nurses agreed to participate in the evaluation, and the overall raw SUS score was 69.26 ± 10.39, which indicated above average results.

Conclusion

The blood collection management workstation has shown the potential to decrease errors and improve working efficiency in a clinical setting. The study also identified some weaknesses, which will be amended in the future.

Comparing the Usability of the Web-Based 24-h Dietary Recall R24W and ASA24-Canada-2018 among French-Speaking Adults from Québec

Automated, self-administered, Web-based 24-h dietary recall tools are increasingly available for nutrition research in different settings, particularly in epidemiological studies and national surveys because of their practicality and efficiency. However, the usability of different 24-h dietary recall tools must be assessed and compared for use in specific populations as it is a major driver of the response rate and retention of participants. The primary aim of this study was to compare the usability of two validated, self-administered, web-based 24-h dietary recall tools available for the Canadian population: the R24W and the 2018 Canadian version of the ASA24. The R24W was developed in French for primary use in the province of Québec, Canada while the ASA24 was developed in English for primary use in the USA and recently adapted and translated for use in French-speaking Canadian adults. Whether the R24W and the ASA24-Canada-2018 yield similar nutritional data was also tested. In this randomized crossover study, 48 women and 20 men (mean age of 35 ± 14 years; range: 19−79 years) recruited in the province of Quebec completed the R24W and the ASA24-Canada-2018 in French twice on each occasion. Participants also completed the System Usability Scale (SUS), a reliable and valid scale giving a global view of subjective assessments of usability. Mean SUS score as well as mean dietary intakes of energy, nutrients and food groups generated by each tool were compared using mixed model analyses for repeated measures. On a scale of 0 to 100, the mean SUS scores (±SD) for the R24W and the ASA24-Canada-2018 were 81 ± 2 and 58 ± 2, respectively (p < 0.0001). 84% of participants stated that they would prefer to use the R24W if they were invited to complete additional 24-h dietary recalls. No significant difference was found between the R24W and the ASA24-Canada-2018 for the intake of energy, proteins, lipids, saturated fatty acids, carbohydrates, fibers, sodium and vegetables and fruits. In sum, while the R24W and the ASA24-Canada-2018 generate comparable self-reported dietary intake data, the R24W showed a better usability than the ASA24-Canada-2018 in a sample of French-speaking adults from the province of Quebec.

Measuring the Usability of eHealth Solutions for Patients With Parkinson Disease: Observational Study

Background

Parkinson disease (PD) is a neurodegenerative disorder with a variety of motor and nonmotor symptoms. Many of these symptoms can be monitored by eHealth solutions, including smartphone apps, wearable sensors, and camera systems. The usability of such systems is a key factor in long-term use, but not much is known about the predictors of successful use and preferable methods to assess usability in patients with PD.

Objective

This study tested methods to assess usability and determined prerequisites for successful use in patients with PD.

Methods

We performed comprehensive usability assessments with 18 patients with PD using a mixed methods usability battery containing the System Usability Scale, a rater-based evaluation of device-specific tasks, and qualitative interviews. Each patient performed the usability battery with 2 of 3 randomly assigned devices: a tablet app, wearable sensors, and a camera system. The usability battery was administered at the beginning and at the end of a 4-day testing period. Between usability batteries, the systems were used by the patients during 3 sessions of motor assessments (wearable sensors and camera system) and at the movement disorder ward (tablet app).

Results

In this study, the rater-based evaluation of tasks discriminated the best between the 3 eHealth solutions, whereas subjective modalities such as the System Usability Scale were not able to distinguish between the systems. Successful use was associated with different clinical characteristics for each system: eHealth literacy and cognitive function predicted successful use of the tablet app, and better motor function and lower age correlated with the independent use of the camera system. The successful use of the wearable sensors was independent of clinical characteristics. Unfortunately, patients who were not able to use the devices well provided few improvement suggestions in qualitative interviews.

Conclusions

eHealth solutions should be developed with a specific set of patients in mind and subsequently tested in this cohort. For a complete picture, usability assessments should include a rater-based evaluation of task performance, and there is a need to develop strategies to circumvent the underrepresentation of poorly performing patients in qualitative usability research.

How Digital Therapeutics Are Urging the Need for a Paradigm Shift: From Evidence-Based Health Care to Evidence-Based Well-being

A scientific paradigm consists of a set of shared rules, beliefs, values, methods, and instruments for addressing scientific problems. Currently, health care embraces the paradigm of evidence-based health care (EBH). This paradigm prompts health care institutions to base decisions on the best available evidence, which is commonly generated in large-scale randomized controlled trials. We illustrate the application of EBH via the evaluation of drugs. We show how EBH is challenged when it is applied to the evaluation of digital therapeutics, which refers to technology and data to prevent, manage, or treat a medical disorder or disease. We conclude that amid the growing application of digital therapeutics, the paradigm of EBH is challenged in four domains: population, intervention, comparison, outcome. In the second part of this viewpoint, we argue for a paradigm shift in health care so we can optimally evaluate and implement digital therapeutics, and we sketch out the contours of this novel paradigm. We address the need for considering design in health care and evaluation processes, studying user values so that health care can move from a focus on health to well-being, focusing on individual experiences rather than the average, addressing the need for evaluation in authentic use contexts, and stressing the need for continuous evaluation of the dynamic relations between users, context, and digital therapeutics. We conclude that the transition from EBH toward evidence-based well-being would improve the successful implementation of digital technologies in health care.

Learning experience design of an mHealth self-management intervention for adolescents with type 1 diabetes

Type 1 diabetes (T1D) is a lifelong and chronic condition that can cause severely compromised health. The T1D treatment regimen is complex, and is a particular challenge for adolescents, who frequently experience a number of treatment adherence barriers (e.g., forgetfulness, planning and organizational challenges, stress). Diabetes Journey is a gamified mHealth program designed to improve T1D self-management through a specific focus on decreasing adherence barriers and improving executive functioning skills for adolescents. Grounded in situativity theory and guided by a sociotechnical-pedagogical usability framework, Diabetes Journey was designed, developed, and evaluated using a learning experience design approach. This approach applied design thinking methods within a Successive Approximation Model design process. Iterative design and formative evaluation were conducted across three design phases, and improvements were implemented following each phase. Findings from the user testing phase indicate Diabetes Journey is a user-friendly mHealth program with high usability that holds promise for enhancing adolescents' T1D self-management. Implications for future designers and researchers are discussed regarding the social dimension of the sociotechnical-pedagogical usability framework. An extension to the framework is proposed to extend the social dimension to include socio-cultural and contextual considerations when designing mHealth applications. Consideration of the pedagogical and sociocultural dimensions of learning is imperative when developing psychoeducational interventions.

Eliciting Information Needs of Child Patients: Adapting the Kano Model to the Design of mHealth Applications

BACKGROUND

 Health care services are increasingly being digitized, but extant literature shows that digital technologies and applications are often developed without careful consideration of user needs. Research is needed to identify and investigate best-in-class methods to support user-centered design of mHealth applications.

OBJECTIVES

 The article investigates how the Kano model can be adapted and used for the purpose of eliciting child patients' information needs during the design phase of mHealth application development. The aim is to demonstrate its applicability for collecting and analyzing patient-centered data that are key to designing technology-supported solutions for health management.

METHODS

 The article is based on a mixed-methods case study, which includes interviews with 21 patients aged 6 to 18. Structured interviews are analyzed based on prescriptions of the Kano model. Semi-structured interviews about child patients' information needs are analyzed thematically.

RESULTS

 The results demonstrate several improvements to the Kano model that take into account the difficulties of effectively communicating with child patients. The combination of two types of interviews offers unique insights into the what, how, and why of patients' needs. Adaptation of the Kano model, simplification of response options, and participation of child patients' parents in interviews facilitate data collection.

CONCLUSIONS

 The article shows how the Kano model can be adapted to provide an effective means of eliciting child patients' needs. Adapting the model by combining structured and semi-structured interviews makes it a powerful tool in designing mHealth applications.

Designing, Developing, Evaluating, and Implementing a Smartphone-Delivered, Rule-Based Conversational Agent (DISCOVER): Development of a Conceptual Framework

BACKGROUND

Conversational agents (CAs), also known as chatbots, are computer programs that simulate human conversations by using predetermined rule-based responses or artificial intelligence algorithms. They are increasingly used in health care, particularly via smartphones. There is, at present, no conceptual framework guiding the development of smartphone-based, rule-based CAs in health care. To fill this gap, we propose structured and tailored guidance for their design, development, evaluation, and implementation.

OBJECTIVE

The aim of this study was to develop a conceptual framework for the design, evaluation, and implementation of smartphone-delivered, rule-based, goal-oriented, and text-based CAs for health care.

METHODS

We followed the approach by Jabareen, which was based on the grounded theory method, to develop this conceptual framework. We performed 2 literature reviews focusing on health care CAs and conceptual frameworks for the development of mobile health interventions. We identified, named, categorized, integrated, and synthesized the information retrieved from the literature reviews to develop the conceptual framework. We then applied this framework by developing a CA and testing it in a feasibility study.

RESULTS

The Designing, Developing, Evaluating, and Implementing a Smartphone-Delivered, Rule-Based Conversational Agent (DISCOVER) conceptual framework includes 8 iterative steps grouped into 3 stages, as follows: design, comprising defining the goal, creating an identity, assembling the team, and selecting the delivery interface; development, including developing the content and building the conversation flow; and the evaluation and implementation of the CA. They were complemented by 2 cross-cutting considerations-user-centered design and privacy and security-that were relevant at all stages. This conceptual framework was successfully applied in the development of a CA to support lifestyle changes and prevent type 2 diabetes.

CONCLUSIONS

Drawing on published evidence, the DISCOVER conceptual framework provides a step-by-step guide for developing rule-based, smartphone-delivered CAs. Further evaluation of this framework in diverse health care areas and settings and for a variety of users is needed to demonstrate its validity. Future research should aim to explore the use of CAs to deliver health care interventions, including behavior change and potential privacy and safety concerns.

Mixed methods evaluation to explore participant experiences of a pilot randomized trial to facilitate self-management of people living with stroke: Inspiring virtual enabled resources following vascular events (iVERVE)

INTRODUCTION

Despite digital health tools being popular for supporting self-management of chronic diseases, little research has been undertaken on stroke. We developed and pilot tested, using a randomized controlled design, a multicomponent digital health programme, known as Inspiring Virtual Enabled Resources following Vascular Events (iVERVE), to improve self-management after stroke. The 4-week trial incorporated facilitated person-centred goal setting, with those in the intervention group receiving electronic messages aligned to their goals, versus limited administrative messages for the control group. In this paper, we describe the participant experience of the various components involved with the iVERVE trial.

METHODS

Mixed method design: satisfaction surveys (control and intervention) and a focus group interview (purposively selected intervention participants). Experiences relating to goal setting and overall trial satisfaction were obtained from intervention and control participants, with feedback on the electronic message component from intervention participants. Inductive thematic analysis was used for interview data and open-text responses, and closed questions were summarized descriptively. Triangulation of data allowed participants' perceptions to be explored in depth.

RESULTS

Overall, 27/54 trial participants completed the survey (13 intervention: 52%; 14 control: 48%); and 5/8 invited participants in the intervention group attended the focus group. Goal setting: The approach was considered comprehensive, with the involvement of health professionals in the process helpful in developing realistic, meaningful and person-centred goals. Electronic messages (intervention): Messages were perceived as easy to understand (92%), and the frequency of receipt was considered appropriate (11/13 survey; 4/5 focus group). The content of messages was considered motivational (62%) and assisted participants to achieve their goals (77%). Some participants described the benefits of receiving messages as a 'reminder' to act. Overall trial satisfaction: Messages were acceptable for educating about stroke (77%). Having options for short message services or email to receive messages was considered important. Feedback on the length of the intervention related to specific goals, and benefits of receiving the programme earlier after stroke was expressed.

CONCLUSION

The participant experience has indicated acceptance and utility of iVERVE. Feedback from this evaluation is invaluable to inform refinements to future Phase II and III trials, and wider research in the field.

PATIENT OR PUBLIC CONTRIBUTION

Two consumer representatives sourced from the Stroke Foundation (Australia) actively contributed to the design of the iVERVE programme. In this study, participant experiences directly contributed to the further development of the iVERVE intervention and future trial design.

Development of an online cancer data collection and processing tool for population-based cancer registries in a low-resource setting: The CanDCap experience from Golestan, Iran

BACKGROUND

Golestan Population-based Cancer Registry (GPCR) with more than 15-years experiences developed an in-house online software called Cancer Data Collection and Processing (CanDCap) to improve its data collection operations from the conventional offline method to new online method. We aimed to report the methods and framework that GPCR applied to design and implementation of the CanDCap.

METHODS

CanDCap was designed based on International Agency for Research on Cancer (IARC) protocols and standards and according to the GPCR workflow. CanDCap has two parts including a web-based online part for data collection and a windows-based part for data processing consisting of quality control and deduplication of repeated records. Questionnaire for User Interface Satisfaction (QUIS) was used in order to assess user interaction satisfaction.

RESULTS

CanDCap was implemented in 2018 and could improve the quality of the GPCR data during its first three years of activity (2018-2020), during which about 9,000 records were registered. The coverage for optional items including national ID, father name, address and telephone number were improved from 23 %, 32 %, 83 % and 82 % in conventional offline method (2015-2017) to 83 %, 81 %, 87 %, and 90 % after using the CanDCap (2018-2020), respectively. The timeliness was also improved from 4 years to 2 years. Overall, user interaction satisfaction was acceptable (7.8 out of 9).

CONCLUSION

CanDCap could resulted in improvement in data quality and timeliness of the GPCR as a cancer registry unit with limited resources. It has the potential to be considered as a model for population-based cancer registries in lower-resource settings.

Usability Testing of the Kidney Score Platform to Enhance Communication About Kidney Disease in Primary Care Settings: Qualitative Think-Aloud Study

Background

Patient awareness of chronic kidney disease (CKD) is low in part due to suboptimal testing for CKD among those at risk and lack of discussions about kidney disease between patients and clinicians. To bridge these gaps, the National Kidney Foundation developed the Kidney Score Platform, which is a web-based series of tools that includes resources for health care professionals as well as an interactive, dynamic patient-facing component that includes a brief questionnaire about risk factors for kidney disease, individualized assessment of risk for developing CKD, and self-management tools to manage one’s kidney disease.

Objective

The aim of this study is to perform usability testing of the patient component of the Kidney Score platform among veterans with and at risk for kidney disease and among clinicians working as primary care providers in Veterans Affairs administration.

Methods

Think-aloud exercises were conducted, during which participants (veterans and clinicians) engaged with the platform while verbalizing their thoughts and making their perceptions, reasonings, and decision points explicit. A usability facilitator observed participants’ behaviors and probed selectively to clarify their comprehension of the tool’s instructions, content, and overall functionality. Thematic analysis on the audio-recording transcripts was performed, focusing on positive attributes, negative comments, and areas that required facilitator involvement.

Results

Veterans (N=18) were 78% (14/18) male with a mean age of 58.1 years. Two-thirds (12/18) were of non-White race/ethnicity, 28% (5/18) had laboratory evidence of CKD without a formal diagnosis, and 50% (9/18) carried a diagnosis of hypertension or diabetes. Clinicians (N=19) were 29% (5/17) male, 30% (5/17) of non-White race/ethnicity, and had a mean of 17 (range 4-32) years of experience. Veterans and clinicians easily navigated the online tool and appreciated the personalized results page as well as the inclusion of infographics to deliver key educational messages. Three major themes related to content and communication about risk for CKD emerged from the think-aloud exercises: (1) tension between lay and medical terminology when discussing kidney disease and diagnostic tests, (2) importance of linking general information to concrete self-management actions, and (3) usefulness of the tool as an adjunct to the office visit to prepare for patient-clinician communication. Importantly, these themes were consistent among interviews involving both veterans and clinicians.

Conclusions

Veterans and clinicians both thought that the Kidney Score Platform would successfully promote communication and discussion about kidney disease in primary care settings. Tension between using medical terminology that is used regularly by clinicians versus lay terminology to promote CKD awareness was a key challenge, and knowledge of this can inform the development of future CKD educational materials.

The Effect of Mental Health Intervention before English Test on College Students' English Test Scores

It is necessary to pay attention to and study the physical and mental health development of college students. On the basis of statistical analysis of the questionnaire, this paper studies the impact of pretest mental health intervention on college students' English test scores, summarizes the problems and characteristics of college students' mental health, and explores the reasons for psychological problems. Based on the research results and related survey data of the mental health status of college students by scholars, this paper believes that these reasons mainly include students' personal reasons, school and social environment pressure, and insufficient family support factors. The data analysis shows that in the relationship between English education and mental health in colleges and universities, the amount of mental health intervention of students' extracurricular exercise has a stable and low negative correlation with the scores of each factor of SCL-90 (Symptom Checklist 90) and is negatively correlated with the scores of each subscale. The experimental results show that there are extremely significant differences in academic performance (F = 5.084, P < 0.01), and there is a certain linear relationship between students' mental health level and students' English exercise feeling. The lower the fatigue score, the higher the mental health level of the students, which effectively promotes the mental health development of college students.

Rapid systematic review on developing web-based interventions to support people affected by cancer

OBJECTIVE

To systematically identify and explore the existing evidence to inform the development of web-based interventions to support people affected by cancer (PABC).

DESIGN

A rapid review design was employed in accordance with the guidance produced by the Cochrane Rapid Reviews Methods Group and reported using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses checklist. A rapid review was chosen due to the need for a timely evidence synthesis to underpin the subsequent development of a digital resource (Shared Lives: Cancer) as part of an ongoing funded project.

METHODS AND OUTCOMES

Keyword searches were performed in MEDLINE to identify peer-reviewed literature that reported primary data on the development of web-based interventions designed to support PABC. The review included peer-reviewed studies published in English with no limits set on publication date or geography. Key outcomes included any primary data that reported on the design, usability, feasibility, acceptability, functionality and user experience of web-based resource development.

RESULTS

Ten studies were identified that met the pre-specified eligibility criteria. All studies employed an iterative, co-design approach underpinned by either quantitative, qualitative or mixed methods. The findings were grouped into the following overarching themes: (1) exploring current evidence, guidelines and theory, (2) identifying user needs and preferences and (3) evaluating the usability, feasibility and acceptability of resources. Resources should be informed by the experiences of a wide range of end-users taking into consideration current guidelines and theory early in the design process. Resource design and content should be developed around the user's needs and preferences and evaluated through usability, feasibility or acceptability testing using quantitative, qualitative or mixed methods.

CONCLUSION

The findings of this rapid review provide novel methodological insights into the approaches used to design web-based interventions to support PABC. Our findings have the potential to inform and guide researchers when considering the development of future digital health resources.

TRIAL REGISTRATION NUMBER

The review protocol was registered on the Open Science Framework (https://osf.io/ucvsz).

Combined Expert and User-Driven Usability Assessment of Trauma Decision Support Systems Improves User-Centered Design

BACKGROUND

Trauma clinical decision support systems improve adherence with evidence-based practice but suffer from poor usability and the lack of a user-centered design. The objective of this study was to compare the effectiveness of user and expert-driven usability testing methods to detect usability issues in a rib fracture clinical decision support system and identify guiding principles for trauma clinical decision support systems.

METHODS

A user-driven and expert-driven usability investigation was conducted using a clinical decision support system developed for patients with rib fractures. The user-driven usability evaluation was as follows: 10 clinicians were selected for simulation-based usability testing using snowball sampling, and each clinician completed 3 simulations using a video-conferencing platform. End-users participated in a novel team-based approach that simulated realistic clinical workflows. The expert-driven heuristic evaluation was as follows: 2 usability experts conducted a heuristic evaluation of the clinical decision support system using 10 common usability heuristics. Usability issues were identified, cataloged, and ranked for severity using a 4-level ordinal scale. Thematic analysis was utilized to categorize the identified usability issues.

RESULTS

Seventy-nine usability issues were identified; 63% were identified by experts and 48% by end-users. Notably, 58% of severe usability issues were identified by experts alone. Only 11% of issues were identified by both methods. Five themes were identified that could guide the design of clinical decision support systems-transparency, functionality and integration into workflow, automated and noninterruptive, flexibility, and layout and appearance. Themes were preferentially identified by different methods.

CONCLUSION

We found that a dual-method usability evaluation involving usability experts and end-users drastically improved detection of usability issues over single-method alone. We identified 5 themes to guide trauma clinical decision support system design. Performing usability testing via a remote video-conferencing platform facilitated multi-site involvement despite a global pandemic.

Evaluating barriers and facilitators to the uptake of mHealth apps in cancer care using the Consolidated Framework for Implementation Research (CFIR): A scoping literature review (Preprint)

BACKGROUND

Mobile health (mHealth) solutions have proven to be effective in a wide range of patient outcomes and have proliferated over time. However, a persistent challenge of digital health technologies, including mHealth, is that they are characterized by early dropouts in clinical practice and struggle to be used outside experimental settings or on larger scales.

OBJECTIVE

This study aimed to explore barriers and enablers to the uptake of mHealth solutions used by patients with cancer undergoing treatment, using a theory-guided implementation science model, that is, the Consolidated Framework for Implementation Research (CFIR).

METHODS

A scoping literature review was conducted using PubMed (MEDLINE), Web of Science, and ScienceDirect databases in March 2022. We selected studies that analyzed the development, evaluation, and implementation of mHealth solutions for patients with cancer that were used in addition to the standard of care. Only empirical designs (eg, randomized controlled trials, observational studies, and qualitative studies) were considered. First, information on the study characteristics, patient population, app functionalities, and study outcomes was extracted. Then, the CFIR model was used as a practical tool to guide data collection and interpretation of evidence on mHealth uptake.

RESULTS

Overall, 91 papers were included in the data synthesis. The selected records were mostly randomized controlled trials (26/91, 29%) and single-arm, noncomparative studies (52/91, 57%). Most of the apps (42/73, 58%) were designed for both patients and clinicians and could be used to support any type of cancer (29/73, 40%) and a range of oncological treatments. Following the CFIR scheme (intervention, outer setting, inner setting, individuals, process), multistakeholder co-design, codevelopment, and testing of mHealth interventions were identified as key enablers for later uptake. A variety of external drivers emerged, although the most relevant outer incentive fostering mHealth use was addressing patient needs. Among organizational factors likely to influence technology uptake, interoperability was the most prominent, whereas other providers' dimensions such as managerial attitudes or organizational culture were not systematically discussed. Technology-related impediments that could hamper the use of mHealth at the individual level were considered least often.

CONCLUSIONS

The hype surrounding mHealth in cancer care is hindered by several factors that can affect its use in real world and nonexperimental settings. Compared with the growing evidence on mHealth efficacy, knowledge to inform the uptake of mHealth solutions in clinical cancer care is still scarce. Although some of our findings are supported by previous implementation research, our analysis elaborates on the distinguishing features of mHealth apps and provides an integrated perspective on the factors that should be accounted for implementation efforts. Future syntheses should liaise these dimensions with strategies observed in successful implementation initiatives.

System Usability Scale Benchmarking for Digital Health Apps: Meta-analysis

Background

The System Usability Scale (SUS) is a widely used scale that has been used to quantify the usability of many software and hardware products. However, the SUS was not specifically designed to evaluate mobile apps, or in particular digital health apps (DHAs).

Objective

The aim of this study was to examine whether the widely used SUS distribution for benchmarking (mean 68, SD 12.5) can be used to reliably assess the usability of DHAs.

Methods

A search of the literature was performed using the ACM Digital Library, IEEE Xplore, CORE, PubMed, and Google Scholar databases to identify SUS scores related to the usability of DHAs for meta-analysis. This study included papers that published the SUS scores of the evaluated DHAs from 2011 to 2021 to get a 10-year representation. In total, 117 SUS scores for 114 DHAs were identified. R Studio and the R programming language were used to model the DHA SUS distribution, with a 1-sample, 2-tailed t test used to compare this distribution with the standard SUS distribution.

Results

The mean SUS score when all the collected apps were included was 76.64 (SD 15.12); however, this distribution exhibited asymmetrical skewness (–0.52) and was not normally distributed according to Shapiro-Wilk test (P=.002). The mean SUS score for “physical activity” apps was 83.28 (SD 12.39) and drove the skewness. Hence, the mean SUS score for all collected apps excluding “physical activity” apps was 68.05 (SD 14.05). A 1-sample, 2-tailed t test indicated that this health app SUS distribution was not statistically significantly different from the standard SUS distribution (P=.98).

Conclusions

This study concludes that the SUS and the widely accepted benchmark of a mean SUS score of 68 (SD 12.5) are suitable for evaluating the usability of DHAs. We speculate as to why physical activity apps received higher SUS scores than expected. A template for reporting mean SUS scores to facilitate meta-analysis is proposed, together with future work that could be done to further examine the SUS benchmark scores for DHAs.

The Assessment of a Personalized Nutrition Tool (eNutri) in Germany: Pilot Study on Usability Metrics and Users’ Experiences

Background

To address the epidemic burden of diet-related diseases, adequate dietary intake assessments are needed to determine the actual nutrition intake of a population. In this context, the eNutri web app has been developed, providing online automated personalized dietary advice, based on nutritional information recorded via an integrated and validated food frequency questionnaire (FFQ). Originally developed for a British population and their dietary habits, the eNutri tool has specifically been adapted to the German population, taking into account national eating habits and dietary recommendations.

Objective

The primary aim of this study is to evaluate the system usability and users’ experience and feedback on the eNutri app in a small-scale preliminary study. The secondary aim is to investigate the efficacy of personalized nutrition (PN) recommendations versus general dietary advice in altering eating habits.

Methods

The app was piloted for 4 weeks by 106 participants from across Germany divided into a PN group and a control group. The groups differed according to the degree of personalization of dietary recommendations obtained.

Results

An overall System Usability Scale (SUS) score of 78.4 (SD 12.2) was yielded, indicating an above average user experience. Mean completion time of the FFQ was 26.7 minutes (SD 10.6 minutes). Across subgroups (age, sex, device screen sizes) no differences in SUS or completion time were found, indicating an equal performance for all users independent of the assigned experimental group. Participants’ feedback highlighted the need for more personalized dietary advice for controls, while personalized nutritional recommendations improved the awareness of healthy eating behavior. Further improvements to the eNutri app were suggested by the app users.

Conclusions

In total, the eNutri app has proven to be a suitable instrument to capture the dietary habits of a German population sample. Regarding functionality, system usability, and handling, direct user feedback was quite positive. Nutritional advice given was rated ambivalent, pointing to several weaknesses in the eNutri app, minimizing the system’s full potential. A higher level of personalization within nutritional advice subjectively improved the app’s usability. The insights gained will be used as a basis to further develop and improve this digital diet assessment tool.

Remote Moderator and Observer Experiences and Decision-making During Usability Testing of a Web-Based Empathy Training Portal: Content Analysis

Background

COVID-19 restrictions severely curtailed empirical endeavors that involved in-person interaction, such as usability testing sessions for technology development. Researchers and developers found themselves using web-based moderation for usability testing. Skilled remote moderators and observers are fundamental in this approach. However, to date, more empirical work is needed that captures the perceptions and support needs of moderators and observers in testing situations.

Objective

The aim of this paper was to identify remote moderator and observer participant experiences and their use of certain tools to capture feedback of users as they interact with the web browser application.

Methods

This research is part of a broader study on an educational web browser application for nursing students to learn perspective taking and enhance their perceptual understanding of a dialogue partner’s thoughts and feelings. The broader study used a quantitative and think-aloud qualitative problem-discovery usability study design. This case study explored written accounts of the remote moderator and observer participants regarding their roles, experiences, and reactions to the testing protocol and their suggestions for improved techniques and strategies for conducting remote usability testing. Content analysis was used to analyze participants’ experiences in the usability testing sessions.

Results

We collected data from 1 remote moderator and 2 remote observers. Five themes were identified: dealing with personal stressors, dealing with user anxiety, maintaining social presence, ethical response to the study protocol, and communication during sessions. The participants offered recommendations for the design of future remote testing activities as well as evidence-informed training materials for usability project personnel.

Conclusions

This study’s findings contribute to a growing body of endeavors to understand human-computer interaction and its impact on remote moderator and observer roles. As technology rapidly advances, more remote usability testing will occur where the knowledge gleaned in this study can have an impact. Recommendations based on moderator and observer participant perspectives identify the need for more evidence-informed training materials for their roles that focus on web-based interpersonal communication skills, execution of user testing protocols, troubleshooting technology and test user issues, proficiency in web conferencing platforms, behavior analysis and feedback technologies, and time management.

Performance and usability testing of an automated tool for detection of peripheral artery disease using electronic health records

Peripheral artery disease (PAD) is a common cardiovascular disorder that is frequently underdiagnosed, which can lead to poorer outcomes due to lower rates of medical optimization. We aimed to develop an automated tool to identify undiagnosed PAD and evaluate physician acceptance of a dashboard representation of risk assessment. Data were derived from electronic health records (EHR). We developed and compared traditional risk score models to novel machine learning models. For usability testing, primary and specialty care physicians were recruited and interviewed until thematic saturation. Data from 3168 patients with PAD and 16,863 controls were utilized. Results showed a deep learning model that utilized time engineered features outperformed random forest and traditional logistic regression models (average AUCs 0.96, 0.91 and 0.81, respectively), P < 0.0001. Of interviewed physicians, 75% were receptive to an EHR-based automated PAD model. Feedback emphasized workflow optimization, including integrating risk assessments directly into the EHR, using dashboard designs that minimize clicks, and providing risk assessments for clinically complex patients. In conclusion, we demonstrate that EHR-based machine learning models can accurately detect risk of PAD and that physicians are receptive to automated risk detection for PAD. Future research aims to prospectively validate model performance and impact on patient outcomes.

Early Detection and Case Management of Skin Diseases with a Mobile Health Application 'eSkinHealth': Protocol for a Mixed-Methods Pilot Study in Côte d'Ivoire

BACKGROUND

The prevalence of skin diseases is extremely high in sub-Saharan Africa, among which are skin neglected tropical diseases (skin NTDs) that could lead to life-long disabilities and deformities if not diagnosed and treated early. To achieve early detection and early treatment of these skin diseases, we developed a mobile health application (mHealth app): 'eSkinHealth.'

OBJECTIVE

This paper outlines a protocol for evaluating the effect of our eSkinHealth app in the early detection and effective management of skin diseases in Côte d'Ivoire.

METHODS

A mixed-methods pilot trial will be conducted in Côte d'Ivoire and will consist of 3 phases: phase 1, the development and improvement of the eSkinHealth app; phase 2, a pilot trial to evaluate the usability of the eSkinHealth app for local medical staff in Côte d'Ivoire; and phase 3, a pilot trial to evaluate the effectiveness of early detection and case management of targeted skin NTDs (Buruli ulcer, leprosy, yaws, and lymphatic filariasis) with the eSkinHealth app in Côte d'Ivoire. The pilot study will be implemented as a 2-arm trial with local healthcare providers and patients with skin NTDs over a 3-month follow-up period. The local healthcare providers will be assigned to an intervention group receiving the eSkinHealth app to be used in their daily practices or a control group. Training will be provided on the usage and implementation of the app and diagnostic pipeline to the intervention group only, while both groups will receive training on skin diseases. Our primary outcome is to evaluate the early detection and effective management of skin diseases using the eSkinHealth app in Côte d'Ivoire by number of cases diagnosed and managed. Additionally, we will evaluate the eSkinHealth app with validated questionnaires and in-depth interviews. Procedures of our methods have been reviewed and approved by the IRB of the Ministry of Health, Côte d'Ivoire and by Tulane University in 2021.

RESULTS

This study was funded in 2021. We started enrolment of patients in February 2022, and data collection is currently underway. We expect the first results to be submitted for publication in 2023.

CONCLUSIONS

Our 'eSkinHealth' is a field-adapted platform that could both provide direct diagnostic and management assistance to health workers in remote settings. The study will provide evidence for the usability and the effectiveness of the eSkinHealth app to improve the early detection and case management of skin NTDs in Côte d'Ivoire; and further, are expected to contribute to knowledge on mobile health approaches in the control of skin NTDs.

CLINICALTRIAL

2020-2054 (Clinicaltrials.gov).

International Survey on Frailty Assessment in Patients with Cancer

Background

Frailty negatively affects the outcomes of patients with cancer, and its assessment might vary widely in the real world. The objective of this study was to explore awareness and use of frailty screening tools among the ONCOassist healthcare professionals (HCPs) users.

Materials and Methods

We sent 2 emails with a cross-sectional 15-item survey in a 3-week interval between April and May 2021. Differences in the awareness and use of tools according to respondents’ continents, country income, and job types were investigated.

Results

Seven hundred thirty-seven HCPs from 91 countries (81% physicians, 13% nurses, and 5% other HCPs) completed the survey. Three hundred and eighty-five (52%) reported assessing all or the majority of their patients; 518 (70%) at baseline and before starting a new treatment. Three hundred and four (43%) HCPs were aware of performance status (PS) scores only, 309 (42%) age/frailty/comorbidity (AFC) screening, and 102 (14%) chemotoxicity predictive tools. Five hundred and thirty-seven (73%) reported using tools; 423 (57%) just PS, 237 (32%) AFC, and 60 (8%) chemotoxicity ones. Reasons for tools non-use (485 responders) were awareness (70%), time constraints (28%), and uselessness (2%). There were significant differences in awareness and use of screening tools among different continents, country income, job types, and medical specialties (P < .001 for all comparisons).

Conclusion

Among selected oncology HCPs, there is still a worldwide lack of knowledge and usage of frailty screening tools, which may differ according to their geography, country income, and education. Targeted initiatives to raise awareness and education are needed to implement frailty assessment in managing patients with cancer.

ICT-Based Individualized Training of Institutionalized Individuals With Dementia. Evaluation of Usability and Trends Toward the Effectiveness of the InCoPE-App

Physical activity interventions can alleviate the course of disease for individuals with dementia (IWD) who have been extraordinarily affected by the COVID-19 pandemic. Information and Communication Technology (ICT) provides new opportunities not only to mitigate negative effects of the pandemic but also to sustainably improve everyday life of IWD in nursing homes. The aim of the present study was to evaluate the ICT-based InCoPE-App, which was used to assess physical and cognitive performance and deliver individualized exercise for IWD, with regard to 1) user experience of nursing assistants, and 2) trends toward the effectiveness of the intervention on physical and cognitive performance of IWD. An 18-week individualized multidomain intervention (2 × 60 min/session) was delivered to an intervention group (IG; n = 10, mean age 88.4 ± 5.6, 70% female) by nursing assistants (n = 10, mean age 56.1 ± 10.4, 90% female) using the InCoPE-App. A control group (CG; n = 3, mean age 87.3 ± 3.5, 100% female) received conventional treatment. User experience was assessed among nursing assistants by different questionnaires, i.e., PSSUQ and ISONORM 9241/110-S for usability, and AttrakDiff2 for pragmatic (PQ), hedonic quality-identity and stimulation (HQI and HQS), and attractiveness (ATT). Trends toward the effectiveness of the intervention were assessed using MMSE (global cognitive function), FICSIT-4 (balance), 6MWT and TUG (mobility), and m30CST (function of lower limbs). Usability of the InCoPE-App was rated as high by nursing assistants (mean ± SD; overall PSSUQ 2.11 ± 0.75; overall ISONORM 9241/110-S 1.90 ± 0.88; ATT 1.86 ± 1.01; PQ 1.79 ± 1.03; HQI 1.8 ± 0.79; and HQS 1.37 ± 0.69). Dropout was high in the total sample (36.7%). Trends toward the effectiveness were observed within IG in nine IWD who showed positive or neutral trends in at least two physical performance outcomes. Seven participants had positive or neutral trends in the FICSIT-4, seven participants in m30CST, and four and seven participants in 6MWT and TUG, respectively. In conclusion, the InCoPE-App has good nursing assistant-rated usability, whereas training effects and intervention adherence were rather low most likely due to COVID-19 restrictions. Single-subject research revealed more positive than negative trends in IG of IWD. Further research is needed to evaluate feasibility, suitability, and effectiveness of the InCoPE-App.

Examining the Feasibility of an Application-Based Patient-Reported Outcome Monitoring for Breast Cancer Patients: A Pretest for the PRO B Study

In preparation for the PRO B study which aims to examine the effects of an app-based intensified patient-reported outcome (PRO) monitoring for metastatic breast cancer patients, prior assessment of its feasibility was carried out. Sixteen breast cancer patients visiting the breast cancer unit at Charité were recruited and downloaded an app connected to an ePRO system. They received electronic questionnaires on two occasions (baseline and the following week) and were subsequently contacted for a semi-structured phone interview for evaluation. Eleven participants answered at least one questionnaire. Some participants did not receive any or only a part of the questionnaires due to technical problems with the app. Participants who completed the evaluation questionnaire (n = 6) were overall satisfied with the weekly PRO questionnaire. All interviewed (n = 11) participants thought it was feasible to answer the PRO questionnaires on a weekly basis for one year, as planned in the PRO B study. The pretest revealed a need for major technical adjustments to the app because push notifications about the receipt of new questionnaires were not displayed on some smartphone models. Due to the low number of participants, generalization of the findings is limited to our specific context and study. Nevertheless, we could conclude that if technical aspects of the app were improved, the PRO B study could be implemented as planned. The ePRO questionnaire was considered feasible and adequate from the patients’ perspectives.

Chinese Version of the Mobile Health App Usability Questionnaire: Translation, Adaptation, and Validation Study

Background

The usability of mobile health (mHealth) apps needs to be effectively evaluated before they are officially approved to be used to deliver health interventions. To this end, the mHealth App Usability Questionnaire (MAUQ) has been designed and proved valid and reliable in assessing the usability of mHealth apps. However, this English questionnaire needs to be translated into other languages, adapted, and validated before being utilized to evaluate the usability of mHealth apps.

Objective

This study aims to improve, further adapt, and validate the Chinese version of the MAUQ (C-MAUQ; interactive for patients) on Left-handed Doctor, one of the most popular “reaching out to patients” interactive mHealth apps with chatbot function in China, to test the reliability and cross-cultural adaptability of the questionnaire.

Methods

The MAUQ (interactive for patients) has been translated into Chinese and validated for its reliability on Good Doctor, one of the most influential “reaching out to patients” mHealth apps without chatbot function in China. After asking for the researchers’ approval to use this Chinese version, we adjusted and further adapted the C-MAUQ by checking it against the original English version and improving its comprehensibility, readability, idiomaticity, and cross-cultural adaptability. Following a trial survey completed by 50 respondents on wenjuanxing, the most popular online questionnaire platform in China, the improved version of the C-MAUQ (I-C-MAUQ) was finally used to evaluate the usability of Left-handed Doctor through an online questionnaire survey (answered by 322 participants) on wenjuanxing, to test its internal consistency, reliability, and validity.

Results

The I-C-MAUQ still retained the 21 items and 3 dimensions of the original MAUQ: 8 items for usability and satisfaction, 6 items for system information arrangement, and 7 items for efficiency. The translation problems in the C-MAUQ, including (1) redundancy, (2) incompleteness, (3) misuse of parts of speech, (4) choice of inappropriate words, (5) incomprehensibility, and (6) cultural difference–induced improper translation, were improved. As shown in the analysis of data obtained through the online survey, the I-C-MAUQ had a better internal consistency (ie, the correlation coefficient between the score of each item and the total score of the questionnaire determined within the range of 0.861-0.938; P<.01), reliability (Cronbach α=.988), and validity (Kaiser–Meyer–Olkin=0.973), compared with the C-MAUQ. It was effectively used to test the usability of Left-handed Doctor, eliciting over 80% of informants’ positive attitudes toward this mHealth app.

Conclusions

The I-C-MAUQ is highly reliable and valid for Left-handed Doctor, and suitable for testing the usability of interactive mHealth apps used by patients in China. This finding further confirms the cross-cultural validity, reliability, and adaptability of the MAUQ. We identified certain factors influencing the perceived usability of mHealth apps, including users’ age, gender, education, profession, and possibly previous experience with mHealth apps and the chatbot function of such apps. Most notably, we found a wider acceptance of this new technology among young Chinese female college students who were more engaged in the interaction with health care chatbots. The age-, gender-, and profession-induced preference for new digital health interventions in China aligns with the findings in other similar studies in America and Malaysia. This preference identifies areas for further research on the social, cultural, and gender adaptation of health technologies.

Development and validation of a health information system for assistance and research in gestational trophoblast disease

BACKGROUND

Gestational Trophoblastic Disease (GTD) comprises pathological forms of placental trophoblastic tissue proliferation. When benign, they present with hydatidiform moles, and when malignant, they are called Gestational Trophoblastic Neoplasia. With the growth of the practice of digital health, allied to updated therapeutic approaches, the Outpatient Clinic for Gestational Trophoblastic Disease has built a Health Information System (HIS), contributing to the teaching-learning binomial, as well as to self-care.

METHODS

This is a cross-sectional and blind technological assessment research for developing SIS-Mola (Website for the medical team and the Application "MolaApp" aimed at patients with GTD). We used the Praxis management approach to manage the application creation project. In the tasks involving real-time chat, a WebSocket layer was created and hosted together with the project's web services, which use the Arch Linux operating system. For the evaluations, we provided questionnaires developed based on the System Usability Scale (SUS), to determine the degree of user satisfaction, with objective questions on the Likert scale. We invited 28 participants for the evaluations, among ABDTG specialist physicians, doctors from the DTG Outpatient Clinic team, and the patients. The study was systematized according to the rules of treatment and follow-up in treating the disease.

RESULTS

The tests were conducted from November 2021 to February 2022. The responses obtained on a Likert scale indicated reliability and credibility to the HIS, since the total usability score, measured by the ten questions of the SUS instrument, had a mean of 81.1 (clinicians), 80 (patients) and median of 77.5 for both groups. The sample was characterized according to the variables: age, gender, education, computer knowledge, and profession.

CONCLUSION

Developing a HIS in the GTD Outpatient Clinic met the objectives regarding the rules of treatment and follow-up of patients. With these digital tools, it is possible to obtain data about the patient's health, sending information through exams performed and appropriate treatments. The connectivity capacity allows agile care, saving time, costs and solving the displacement problem. The TICs generate natural efficiency for the organization in the flow of service and the formation of a database, improving the quality of the assistance.

Patients' Experiences of Web-Based Access to Electronic Health Records in Finland: Cross-sectional Survey

Background

Patient portals that provide access to electronic health records offer a means for patients to better understand and self-manage their health. Yet, patient access to electronic health records raises many concerns among physicians, and little is known about the use practices and experiences of patients who access their electronic health records via a mature patient portal that has been available for citizens for over five years.

Objective

We aimed to identify patients’ experiences using a national patient portal to access their electronic health records. In particular, we focused on understanding usability-related perceptions and the benefits and challenges of reading clinical notes written by health care professionals.

Methods

Data were collected from 3135 patient users of the Finnish My Kanta patient portal through a web-based survey in June 2021 (response rate: 0.7%). Patients received an invitation to complete the questionnaire when they logged out of the patient portal. Respondents were asked to rate the usability of the patient portal, and the ratings were used to calculate approximations of the System Usability Scale score. Patients were also asked about the usefulness of features, and whether they had discussed the notes with health professionals. Open-ended questions were used to ask patients about their experiences of the benefits and challenges related to reading health professionals’ notes.

Results

Overall, patient evaluations of My Kanta were positive, and its usability was rated as good (System Usability Scale score approximation: mean 72.7, SD 15.9). Patients found the portal to be the most useful for managing prescriptions and viewing the results of examinations and medical notes. Viewing notes was the most frequent reason (978/3135, 31.2%) for visiting the portal. Benefits of reading the notes mentioned by patients included remembering and understanding what was said by health professionals and the instructions given during an appointment, the convenience of receiving information about health and care, the capability to check the accuracy of notes, and using the information to support self-management. However, there were challenges related to difficulty in understanding medical terminology, incorrect or inadequate notes, missing notes, and usability.

Conclusions

Patients actively used medical notes to receive information to follow professionals' instructions to take care of their health, and patient access to electronic health records can support self-management. However, for the benefits to be realized, improvements in the quality and availability of medical professionals’ notes are necessary. Providing a standard information structure could help patients find the information they need. Furthermore, linking notes to vocabularies and other information sources could also improve the understandability of medical terminology; patient agency could be supported by allowing them to add comments to their notes, and patient trust of the system could be improved by allowing them to control the visibility of the professionals’ notes.

Virtual respiratory therapy delivered through a smartphone app: a mixed-methods randomised usability study

INTRODUCTION

A new smartphone app (QUT Inspire) has been developed to detect inspiratory sound and deliver virtual incentive spirometry (ISy), a respiratory therapy technique used in postoperative recuperation, management of some chronic conditions and with potential applications in SARS-CoV-2 rehabilitation. The aim of this study was to compare the usability of this new app with a clinical ISy device as measured by effectiveness, efficiency and satisfaction.

METHODS

In this mixed-methods randomised usability study, healthy volunteers (aged 39.2±12.2 years, n=24) compared inspirations using the QUT Inspire app and a Triflo II clinical ISy device. A post-test questionnaire and a semi-structured interview explored dimensions of usability regarding the new app.

RESULTS

The duration of inspirations performed using the QUT Inspire app (7.3±2.0 s) were comparable with use of the Triflo II ISy device (7.5±2.3 s). No artefacts arising from the order of device testing were identified. App users held their phones adjacent but not proximal to their mouths (13.6±6.4 cm), notwithstanding instructions to keep the phone less than 5 cm away for optimal breath sound detection. The use of onscreen text or video instructional materials did not result in a significant reduction in this distance. Participants reported clear preferences for the app (100%, n=24) to motivate persistence with repeated inspirations. App gamification features such as a timer (75%, n=18) and breath counter (83.3%, n=20) were well regarded. Analysis of semi-structured interviews identified four main themes arising from this study: visual reward from responsive app animations, clinical look and feel influencing credibility, perceived effort affecting engagement and selective adoption of gamification features.

CONCLUSION

This study demonstrates that a virtual ISy app can be effective, efficient and have high satisfaction. Improvements informed by this research include use of additional phone sensors to optimise sound detection and minimising the distance that phones are held from the user's mouth. Further research in randomised controlled trials are needed to evaluate performance of this app in clinical contexts where ISy is currently employed.

Achievement of Developmental Milestones Recorded in Real Time: A Mobile App-Based Study

OBJECTIVE

To clarify the characteristics of milestone achievement of Japanese infants using data assessed and recorded in real time via a Web-based app by caregivers.

STUDY DESIGN

Between 2014 and 2019, the achievement of developmental milestones of 16 627 Japanese infants were recorded via the mobile app Papatto Ikuji along with pertinent information including sex, birth date, anthropometric measurements, and information on feeding. The milestones consisted of 20 items belonging to 4 domains: personal-social, fine motor-adaptative, language, and gross motor. The distribution of age at milestone achievement was compared with 4 conventional normative data. Subgroup analyses according to sex, season of birth, and feeding methods were performed with restricted mean survival time analysis.

RESULTS

Age distributions of milestone achievement were generally similar in the app-based and conventional data. The exception was "Smile responsively," for which the age at achievement seemed higher in the app-based data. Subgroup analyses showed female-dominant sex differences in the achievement of person-social and language milestones. The results also showed seasonality in achievement of gross motor milestones, with earlier achievement in infants born in winter compared with those born in summer.

CONCLUSIONS

We describe mobile app-based data on developmental milestone achievements that were recorded in real time. The app-based data were generally comparable with conventional normative data, suggesting that its potential to assess the development of children in real time. This could be used complementarily with the current well-child visits at scheduled time points.