A Psychometric Tool for Evaluating Executive Functions in Parkinson's Disease

Recently, there has been an increasing interest in using 360° virtual-reality video for an ecologically valid assessment of executive functioning in the neurologic population. In this framework, we have developed the EXecutive-functions Innovative Tool (EXIT 360°), an original 360°-based instrument for a multicomponent, ecologically valid evaluation of executive functioning in Parkinson’s Disease (PD). This work aimed to test the usability and user experience of EXIT 360° in patients with PD (PwPD). Twenty-seven PwPD and twenty-seven healthy controls underwent an evaluation that involved: (1) usability assessment by the System Usability Scale and (2) evaluation of user experience using the ICT—Sense of Presence and User Experience Questionnaire. Results showed a satisfactory level of usability for patients (mean = 76.94 ± 9.18) and controls (mean = 80 ± 11.22), with good scores for usability and learnability. Regarding user experience, patients provided a positive overall impression of the tool, evaluating it as attractive, enjoyable, activating, and funny. Moreover, EXIT 360° showed good pragmatic (e.g., efficient, fast, clear) and hedonic quality (e.g., exciting, interesting, and creative). Finally, PwPD considered EXIT 360° as an original tool with high ecological validity (mean = 4.29 ± 0.61), spatial presence (mean = 3.11 ± 0.83) and engagement (mean = 3.43 ± 0.54) without relevant adverse effects. Technological expertise had no impact on performance. Overall, EXIT 360° appeared to be a usable, easy-to-learn, engaging, and innovative instrument for PD. Further studies will be conducted to deepen its efficacy in distinguishing between healthy subjects and patients with executive dysfunctions.

Parents' Experience With a Mobile Health Intervention to Influence Human Papillomavirus Vaccination Decision Making: Mixed Methods Study

BACKGROUND

Human papillomavirus (HPV)-attributed cancers are preventable, yet HPV vaccination rates severely lag behind other adolescent vaccinations. HPVcancerFree (HPVCF) is a mobile health (mHealth) intervention developed to influence parental HPV vaccination decision making by raising awareness of HPV, reducing HPV vaccination barriers, and enabling HPV vaccination scheduling and reminders through a smartphone app. Evaluating the user experience of mHealth interventions is a vital component in assessing their quality and success but tends to be underreported in mHealth intervention evaluation.

OBJECTIVE

We aimed to evaluate the user experience of HPVCF, an HPV cancer prevention app designed for a pediatric clinic network, using mixed methods data collected from log files, survey measures, and qualitative feedback.

METHODS

Study data were evaluated from parents in a large US pediatric clinic network using HPVCF in the treatment study condition of a group randomized controlled trial. Log data captured HPVCF retention and use. Postintervention rating scales and items assessed HPVCF utility, usefulness, understandability, appeal, credibility, and perceived impact. Overall quality was evaluated using the user version of the Mobile Application Rating Scale (uMars). Open-ended responses assessed parent recommendations for HPVCF enhancement.

RESULTS

The 98 parents were mainly female (n=94, 96%), 41 (5.67) years of age, college educated (n=55, 56%), and White and non-Hispanic (n=55, 56%) and had private health insurance for their children (n=75, 77%). Parents used HPVCF 197 times, with the average visit duration approximating 3.5 minutes. The uMARS app quality score was positively skewed (4.2/5.0). Mean ratings were highest for information (4.46 [SD 0.53]) and lowest for engagement (3.74 [SD 0.69]). In addition, of 95 parents, 45 (47%) rated HPVCF as helpful in HPV vaccination decision making and 16 (17%) attributed HPV vaccine initiation to HPVCF. Parents reported that HPVCF increased their awareness (84/95, 88%), knowledge (84/95, 88%), and HPV vaccination intentions (64/95, 67%). Most of the 98 parents rated the 4 HPVCF components as useful (72-92 [73%-94%]). Parents also agreed that HPVCF is clear (86/95, 91%), accurate (86/95, 91%), and more helpful than other HPV vaccine information they had received (89/95, 94%) and that they would recommend it to others (81/95, 85%). In addition, parents suggested ways to increase awareness and engagement with the app, along with opportunities to enhance the content and functionality.

CONCLUSIONS

HPVCF was well received by parents and performed well on indicators of quality, usefulness, utility, credibility, and perceived impact. This study contributes a multimethod and multimeasure evaluation to the growing body of literature focused on assessing the user experience of patient-focused technology-mediated applications for HPV education.

Measuring User-Perceived Characteristics for Banking Services: Proposing a Methodology

With the continuous technological enhancement of banking services, customers can avail of better, more secure services which present improved opportunities and convenience. Of the many methods available to perform banking operations, customers commonly use traditional banking, online banking, and mobile banking. Each of these existing methods has advantages and limitations that affect customer experience, trust, satisfaction, and continued intention to use such services. In this study, an attempt was made to develop and fit a model to evaluate and measure the effect of perceived characteristics on banking services. To this end, a questionnaire was administered to 91 participants in Korea to investigate their experiences in the three types of services: offline banking (traditional banking), online banking, and automated teller machines (ATM). The factor design for evaluating the user experience through the perceived characteristics of the banking system was performed by conducting exploratory and confirmatory factor analyses. The proposed model exhibited validity and reliability to evaluate the user experience in the banking system. The results obtained can help banking specialists and professionals increase the level of customers’ trust, loyalty, and intention to use their services.

An mHealth App for Fibromyalgia-like Post-COVID-19 Syndrome: Protocol for the Analysis of User Experience and Clinical Data

Background

Post–COVID-19 syndrome, also referred as “long covid,” describes persisting symptoms after SARS-CoV-2 infection, including myalgia, fatigue, respiratory, or neurological symptoms. Objective symptoms are often lacking, thus resembling a fibromyalgia-like syndrome. Digital therapeutics have shown efficiency in similar chronic disorders such as fibromyalgia, offering specific disease monitoring and interventions such as cognitive behavioral therapy or physical and respiratory exercise guidance.

Objective

This protocol aims to study the requirements and features of a new mobile health (mHealth) app among patients with fibromyalgia-like post–COVID-19 syndrome in a clinical trial.

Methods

We created a web application prototype for the post–COVID-19 syndrome called “POCOS,” as a web-based rehabilitation tool aiming to improve clinical outcomes. Patients without organ damage or ongoing inflammation will be included in the study. App use will be assessed through user experience questionnaires, focus groups, and clinical data analysis. Subsequently, we will analyze cross-sectional and longitudinal clinical data.

Results

The developed mHealth app consists of a clinically adapted app interface with a simplified patient-reported outcome assessment, monitoring of medical interventions, and disease activity as well as web-based instructions for specific physical and respiratory exercises, stress reduction, and lifestyle instructions. The enrollment of participants is expected to be carried out in November 2021.

Conclusions

User experience plays an important role in digital therapeutics and needs to be clinically tested to allow further improvement. We here describe this process for a new app for the treatment of the fibromyalgia-like post–COVID-19 syndrome and discuss the relevance of the potential outcomes such as natural disease course and disease phenotypes.

International Registered Report Identifier (IRRID)

PRR1-10.2196/32193

Comparing the Use of DynaMed and UpToDate by Physician Trainees in Clinical Decision-Making: A Randomized Crossover Trial

BACKGROUND

Costs vary substantially among electronic medical knowledge resources used for clinical decision support, warranting periodic assessment of institution-wide adoption.

OBJECTIVES

To compare two medical knowledge resources, UpToDate and DynaMed Plus, regarding accuracy and time required to answer standardized clinical questions and user experience.

METHODS

A crossover trial design was used, wherein physicians were randomized to first use one of the two medical knowledge resources to answer six standardized questions. Following use of each resource, they were surveyed regarding their user experience. The percentage of accurate answers and time required to answer each question were recorded. The surveys assessed ease of use, enjoyment using the resource, quality of information, and ability to assess level of evidence. Tests of carry-over effects were performed. Themes were identified within open-ended survey comments regarding overall user experience.

RESULTS

Among 26 participating physicians, accuracy of answers differed by 4 percentage points or less. For all but one question, there were no significant differences in the time required for completion. Most participants felt both resources were easy to use, contained high quality of information, and enabled assessment of the level of evidence. A greater proportion of participants endorsed enjoyment of use with UpToDate (23/26, 88%) compared with DynaMed Plus (16/26, 62%). Themes from open-ended comments included interface/information presentation, coverage of clinical topics, search functions, and utility for clinical decision-making. The majority (59%) of open-ended comments expressed an overall preference for UpToDate, compared with 19% preferring DynaMed Plus.

CONCLUSION

DynaMed Plus is noninferior to UpToDate with respect to ability to achieve accurate answers, time required for answering clinical questions, ease of use, quality of information, and ability to assess level of evidence. However, user experience was more positive with UpToDate. Future studies of electronic medical knowledge resources should continue to emphasize evaluation of usability and user experience.

The Impact of Mental Health Care Provider Website Transgender and Nonbinary Affirmation on Site User Experience for Transgender/Nonbinary and Cisgender People

Transgender and gender nonbinary (TGNB) individuals use provider information, such as websites, to ascertain whether providers are affirming of TGNB identities. Yet, providers may be unsure about what level of affirmation is sufficient to be perceived as affirming and may be concerned that cisgender people may have negative reactions to website material that is TGNB affirming. The present study undertook a user experience (UX) investigation of TGNB and cisgender participants' UX of mental health provider websites. Participants (310 cisgender, 224 TGNB) were recruited online. Participants viewed one of five mock provider websites. Results of a multivariate analysis of covariance (MANCOVA) indicated that TGNB had more positive evaluations of the three more affirming sites, compared to the less affirming sites, along UX domains related to likelihood of going to that provider, Mental Health Help-Seeking Attitudes, positive emotional UX, and negative emotional UX. Regardless of gender identity, participants rated the three more affirming sites as more welcoming. The results support the use of TGNB affirmation in the UX of TGNB individuals viewing provider websites and further suggest that TGNB affirmation does not detract from the UX of cisgender people. Implications for affirming practice with TGNB populations are reviewed.

User Experience, Engagement, and Popularity in Mental Health Apps: Secondary Analysis of App Analytics and Expert App Reviews

Background

User experience and engagement are critical elements of mental health apps’ abilities to support users. However, work examining the relationships among user experience, engagement, and popularity has been limited. Understanding how user experience relates to engagement with and popularity of mental health apps can demonstrate the relationship between subjective and objective measures of app use. In turn, this may inform efforts to develop more effective and appealing mental health apps and ensure that they reach wide audiences.

Objective

We aimed to examine the relationship among subjective measures of user experience, objective measures of popularity, and engagement in mental health apps.

Methods

We conducted a preregistered secondary data analysis in a sample of 56 mental health apps. To measure user experience, we used expert ratings on the Mobile App Rating Scale (MARS) and consumer ratings from the Apple App Store and Google Play. To measure engagement, we acquired estimates of monthly active users (MAU) and user retention. To measure app popularity, we used download count, total app revenue, and MAU again.

Results

MARS total score was moderately positively correlated with app-level revenue (Kendall rank [T]=0.30, P=.002), MAU (T=0.39, P<.001), and downloads (T=0.41, P<.001). However, the MARS total score and each of its subscales (Engagement, Functionality, Aesthetics, and Information) showed extremely small correlations with user retention 1, 7, and 30 days after downloading. Furthermore, the total MARS score only correlated with app store rating at T=0.12, which, at P=.20, did not meet our threshold for significance.

Conclusions

More popular mental health apps receive better ratings of user experience than less popular ones. However, user experience does not predict sustained engagement with mental health apps. Thus, mental health app developers and evaluators need to better understand user experience and engagement, as well as to define sustained engagement, what leads to it, and how to create products that achieve it. This understanding might be supported by better collaboration between industry and academic teams to advance a science of engagement.

Interaction Design Patterns for Augmented Reality Fitting Rooms

In this work, we explore the role of augmented reality as a meta-user interface, with particular reference to its applications for interactive fitting room systems and the impact on the related shopping experience. Starting from literature and existing systems, we synthesized a set of nine interaction design patterns to develop AR fitting rooms and to support the shopping experience. The patterns were evaluated through a focus group with possible stakeholders with the aim of evaluating and envisioning the effects on the shopping experience. The focus group analysis shows as a result that the shopping experience related to an AR fitting room based on the proposed patterns is influenced by three main factors, namely: the perception of the utility, the ability to generate interest and curiosity, and the perceived comfort of the interaction and environment in which the system is installed. As a further result, the study shows that the patterns can successfully support these factors, but some elements that emerged from the focus group should be more investigated and taken into consideration by the designers.

Usability, Acceptability, and Satisfaction of a Wearable Activity Tracker in Older Adults: Observational Study in a Real-Life Context in Northern Portugal

BACKGROUND

The use of activity trackers has significantly increased over the last few years. This technology has the potential to improve the levels of physical activity and health-related behaviors in older adults. However, despite the potential benefits, the rate of adoption remains low among older adults. Therefore, understanding how technology is perceived may potentially offer insight to promote its use.

OBJECTIVE

This study aimed to (1) assess acceptability, usability, and user satisfaction with the Xiaomi Mi Band 2 in Portuguese community-dwelling older adults in a real-world context; (2) explore the mediating effect of the usability on the relationship between user characteristics and satisfaction; and (3) examine the moderating effect of user characteristics on the relationship between usability and user satisfaction.

METHODS

Older adults used the Xiaomi Mi Band 2 over 15 days. The user experience was evaluated through the Technology Acceptance Model 3, System Usability Scale, and User Satisfaction Evaluation Questionnaire. An integrated framework for usability and user satisfaction was used to explore user experience. Statistical data analysis included descriptive data analysis, reliability analysis, confirmatory factor analysis, and mediation and moderation analyses.

RESULTS

A sample of 110 older adults with an average age of 68.41 years (SD 3.11) completed the user experience questionnaires. Mean user acceptance was very high-perceived ease of use: 6.45 (SD 0.78); perceptions of external control: 6.74 (SD 0.55); computer anxiety: 6.85 (SD 0.47); and behavioral intention: 6.60 (SD 0.97). The usability was excellent with an average score of 92.70 (SD 10.73), and user satisfaction was classified as a good experience 23.30 (SD 2.40). The mediation analysis confirmed the direct positive effect of usability on satisfaction (β=.530; P<.01) and the direct negative effect of depression on usability (β=-.369; P<.01). Lastly, the indirect effect of usability on user satisfaction was higher in individuals with lower Geriatric Depression Scale levels.

CONCLUSIONS

Findings demonstrate that the Xiaomi Mi Band 2 is suitable for older adults. Furthermore, the results confirmed usability as a determinant of satisfaction with the technology and extended the existing knowledge about wearable activity trackers in older adults.

Smart Assistive Technology for Cooking for People With Cognitive Impairments Following a Traumatic Brain Injury: User Experience Study

Background

User experience (UX), including usability, should be formally assessed multiple times throughout the development process to optimize the acceptability and integration of a new technology before implementing it within the home environment of people living with cognitive impairments.

Objective

The aim of this study is to identify UX issues, notably usability issues, and factors to consider for the future implementation of the COOK (Cognitive Orthosis for Cooking) within the home of individuals with traumatic brain injury (TBI) to identify modifications to improve the technology.

Methods

This study comprised two rounds of UX evaluations, including extensive usability testing, which were completed in a laboratory context: 3 sessions with 5 experts and, after improvement of COOK, 2 sessions with 10 participants with TBI. Each session included the use of scenarios and questionnaires on UX and usability.

Results

Both rounds demonstrated good usability outcomes and hedonic qualities. Various usability issues were identified by participants, such as navigation inconsistencies, technical bugs, and the need for more feedback. Factors to consider in the future implementation of COOK were also mentioned by participants with TBI, including environmental (eg, space available and presence of pets) and personal factors (eg, level of comfort with technology, presence of visual deficits, and preferences).

Conclusions

By evaluating UX, including usability, various times throughout the development process and including experts and end users, our research team was able to develop a technology that was perceived as usable, pleasant, and well-designed. This research is an example of how and when people with cognitive impairments (ie, people with TBI) can be involved in evaluating the UX of new technology.

Users' Experiences With the NoHoW Web-Based Toolkit With Weight and Activity Tracking in Weight Loss Maintenance: Long-term Randomized Controlled Trial

Background

Digital behavior change interventions (DBCIs) offer a promising channel for providing health promotion services. However, user experience largely determines whether they are used, which is a precondition for effectiveness.

Objective

The primary aim of this study is to evaluate user experiences with the NoHoW Toolkit (TK)—a DBCI that targets weight loss maintenance—over a 12-month period by using a mixed methods approach and to identify the main strengths and weaknesses of the TK and the external factors affecting its adoption. The secondary aim is to objectively describe the measured use of the TK and its association with user experience.

Methods

An 18-month, 2×2 factorial randomized controlled trial was conducted. The trial included 3 intervention arms receiving an 18-week active intervention and a control arm. The user experience of the TK was assessed quantitatively through electronic questionnaires after 1, 3, 6, and 12 months of use. The questionnaires also included open-ended items that were thematically analyzed. Focus group interviews were conducted after 6 months of use and thematically analyzed to gain deeper insight into the user experience. Log files of the TK were used to evaluate the number of visits to the TK, the total duration of time spent in the TK, and information on intervention completion.

Results

The usability level of the TK was rated as satisfactory. User acceptance was rated as modest; this declined during the trial in all the arms, as did the objectively measured use of the TK. The most appreciated features were weekly emails, graphs, goal setting, and interactive exercises. The following 4 themes were identified in the qualitative data: engagement with features, decline in use, external factors affecting user experience, and suggestions for improvements.

Conclusions

The long-term user experience of the TK highlighted the need to optimize the technical functioning, appearance, and content of the DBCI before and during the trial, similar to how a commercial app would be optimized. In a trial setting, the users should be made aware of how to use the intervention and what its requirements are, especially when there is more intensive intervention content.

Trial Registration

ISRCTN Registry ISRCTN88405328; https://www.isrctn.com/ISRCTN88405328

International Registered Report Identifier (IRRID)

RR2-10.1136/bmjopen-2019-029425

Continuous User Experience Monitoring of a Patient-Completed Preoperative Assessment System in the United Kingdom: Cross-sectional Study

BACKGROUND

Anesthetic preoperative assessment (POA) is now a common part of the surgical care pathway, and guidelines support its routine use. MyPreOp (Ultramed Ltd) is a web-based POA system that enables remote assessments. Usability is a key factor in the success of digital health solutions.

OBJECTIVE

This study aims to assess the usability of the MyPreOp system through patient feedback, investigate the amount of time it took for patients to complete the POA questionnaire and the factors that influenced completion time, and explore the effect on completion times of implementing a validated eHealth usability scale, as compared to using a simple but unvalidated usability evaluation scale, and to test the feasibility of administering a more detailed usability evaluation scale in a staggered manner so as not to unduly increase completion times.

METHODS

In this cross-sectional study, anonymized data sets were extracted from the MyPreOp system. The participants were adults (aged ≥18 years), scheduled for nonurgent surgical procedures performed in hospitals in the United Kingdom, who gave consent for their anonymized data to be analyzed. Data collected included age, gender, American Society of Anesthesiology (ASA) physical classification status, and completion time. Two user experience evaluations were used: in Phase 1, 2 questions asking about overall experience and ease of use, and in Phase 2, a previously validated usability questionnaire, with its 20 questions equally distributed among 5 succeeding patient cohorts. There were 2593 respondents in total (Phase 1: n=1193; Phase 2: n=1400). The median age of the participants was 46 years, and 1520 (58.62%) of the 2593 respondents were female. End points measured were the median completion times in Phase I and Phase II. The data were collected by extracting a subset of records from the database and exported to a spreadsheet for analysis (Excel, Microsoft Corporation). The data were analyzed for differences in completion times between Phase I and Phase II, as well as for differences between age groups, genders, and ASA classifications.

RESULTS

MyPreOp scored well in usability in both phases. In Phase 1, 81.64% (974/1193) of respondents had a good or better experience, and 93.8% (1119/1193) found it easy to use. The usability rating in Phase 2 was 4.13 out of a maximum of 5, indicating high usability. The median completion time was 40.4 minutes. The implementation of the longer usability evaluation scale in Phase 2 did not negatively impact the completion times. Age and ASA physical status were found to be moderately associated with increased completion times.

CONCLUSIONS

MyPreOp rates high in both user experience and usability. The method of dividing the questionnaire into 5 blocks is valid and does not negatively affect completion times. Further research into the factors affecting completion time is recommended.

A post-phenomenological analysis of using menstruation tracking apps for the management of premenstrual syndrome

Objectives

Menstruation tracking digital applications (MTA) are a popular technology, yet there is a lacuna of research on how women use this technology for the management of PMS. Theoretical frameworks for understanding users' experiences are also underdeveloped in this nascent field. The objectives of the study were therefore twofold, to propose a theoretical framework for understanding women's use of MTA and apply it to the analysis of users' experiences in the management of PMS.

Method

A novel theoretical framework was proposed, informed by post-phenomenology, postfeminist healthism, feminist new materialism and digital health technologies as public pedagogy. This framework focuses analytic attention on affective relationships between subjectivity, bodily sensations, digital technology, and discourse. It was used to structure the analysis of five in-depth timeline interviews with women in Aotearoa New Zealand who experienced benefits from using MTA to manage PMS symptoms.

Results

Three pedagogical relationships were identified: a pedagogy of empowerment, where users learnt to control, predict and manage their PMS symptoms in line with healthism; a pedagogy of appreciation, where users learnt to understand their menstruating bodies as amazing, a valued part of them, and awe-inspiring that radically overturned past internalised stigma; and an 'untrustworthy teacher' who eroded this affirmative learning through inaccuracy, positioning users in dis-preferred categories, or being 'creepy'.

Conclusions

MTA offers huge possibilities for challenging menstrual stigma that need to be nurtured, developed, and protected; and there are benefits for analysing MTA within wider scholarship on postfeminist healthism.

Bio-physiological-signals-based VR cybersickness detection

With the gradual maturity of virtual reality (VR) technology in recent years, VR industry is in a trend of rapid growth, providing new possibilities for content design. Although VR technology has been able to provide users with excellent immersive experience, side effects that affect the user experience still exist, especially the cybersickness. It would cause extreme physical discomfort to the users and the discontinuation of use. Many researchers have tried to find the inducement of cybersickness and to detect and limit the occurrence of this symptom, but most of the current detection and analysis methods rely on subjective questionnaires to collect users’ posterior states, such as dizziness, nausea, cold sweats, disorientation, eyestrain and so on. There is no mature real-time cybersickness detection system for VR developers to evaluate the susceptibility of their products to cybersickness so far, which has hindered the adoption of VR to some extent. The purpose of this study is to implement the real-time monitoring of cybersickness using physiological sensors to measure data and quantify the influence factors of cybersickness through deep learning model. Besides, we have developed a VR experimental platform and passive navigation task to induce user cybersickness. During the experiment, to train the LSTM Attention neural network model, we collected the user’s real-time physiological signals, including skin electrical activity (EDA) and electrocardiogram (ECG), as well as the position and bone rotation data of the users’ virtual avatar. The model can detect the level of users’ cybersickness in real-time during VR experience. And the model has been verified by the fivefold cross-validation that the average accuracy of 96.85% was achieved for classification of cybersickness level, showing great performance compared with other relevant studies. The results show the feasibility of accurate classification of cybersickness using the model we proposed. Also the model can provide reference for VR researchers and developers to improve the user experience.

Expansion in speech time can restore comprehension in a simultaneously speaking bilingual robot

Introduction: In this study, the development of a social robot, capable of giving speech simultaneously in more than one language was in mind. However, the negative effect of background noise on speech comprehension is well-documented in previous works. This deteriorating effect is more highlighted when the background noise has speech-like properties. Hence, the presence of speech as the background noise in a simultaneously speaking bilingual robot can be fatal for the speech comprehension of each person listening to the robot. Methods: To improve speech comprehension and consequently, user experience in the intended bilingual robot, the effect of time expansion on speech comprehension in a multi-talker speech scenario was investigated. Sentence recognition, speech comprehension, and subjective evaluation tasks were implemented in the study. Results: The obtained results suggest that a reduced speech rate, leading to an expansion in the speech time, in addition to increased pause duration in both the target and background speeches can lead to statistically significant improvement in both sentence recognition, and speech comprehension of participants. More interestingly, participants got a higher score in the time-expanded multi-talker speech than in the standard-speed single-talker speech in the speech comprehension and, in the sentence recognition task. However, this positive effect could not be attributed merely to the time expansion, as we could not repeat the same positive effect in a time-expanded single-talker speech. Discussion: The results obtained in this study suggest a facilitating effect of the presence of the background speech in a simultaneously speaking bilingual robot provided that both languages are presented in a time-expanded manner. The implications of such a simultaneously speaking robot are discussed.

Trends in… Modernization at the Nation's Medical Library

This manuscript describes the ongoing efforts to meet ever-changing patron needs by modernizing operations, infrastructure, workforce, and customer service at the National Library of Medicine (NLM). Charged with collecting, organizing, preserving, and disseminating biomedical and life sciences information to the public, the NLM constantly adapts to evolving technologies and scientific advancements in order to better fulfill its mission. Historical context and an overview of operational decision-making offer a window into the workings of our nation's medical library.

User Testing to Improve Retrieval and Comprehension of Information in Guidelines to Improve Medicines Safety

OBJECTIVE

The aim of the study was to investigate the effectiveness of user testing for improving healthcare professionals' retrieval and comprehension of information in medicines guidelines.

METHODS

The United Kingdom's Injectable Medicines Guide was selected as a case study. This gives guidance to nurses on preparing and administering intravenous medicines on hospital wards, in line with standard UK practice. Three rounds of user testing were completed with 10 hospital nurses per round, using the Injectable Medicines Guide for voriconazole and aminophylline. Participants used the guidelines to answer 17 questions related to the administration of these medicines. Answers were scored for "finding" and "understanding" the required information. Semistructured interviews explored participants' opinions of guideline content, design, and wording, with responses analyzed thematically. The guidelines were revised between rounds.

RESULTS

In round 1, 8 of 17 questions were answered correctly by all participants. Participants had difficulty with dose, dilution, administration rate, and adverse effects questions. Revisions included new subsections and increased calculation support. In round 2, 14 of 17 questions were answered correctly by all participants. Difficulty persisted with dose and administration rate questions and further revisions made. In round 3, 15 of 17 questions were answered correctly by all participants. Across all rounds, participants considered appropriate subheadings and information order as important for fast location of information. Specific, detailed, and practical instructions were perceived as important to improve understandability and usefulness.

CONCLUSIONS

Key information in medicines guidelines may not be found and/or understood by healthcare professionals. User testing increased information retrieval and comprehension and could have an important role in improving the safety of medicines use.

Health workers' experience of a digital health intervention implemented in peri-urban communities in Karachi, Pakistan

Objective

Digital health interventions (DHIs) have the potential to improve access and quality of care in low-middle-income countries. The aim of this study was to assess the acceptability, usability and aesthetics of a DHI by frontline workers in peri-urban community settings in Karachi, Pakistan.

Methods

A mixed-methods study was carried out in peri-urban field sites in Karachi, Pakistan, where maternal and childcare services are provided through front-line care providers using a DHI. These workers include community health workers, midwives, and physicians who were using the DHI for at least six months. For quantitative data, a questionnaire regarding the module design and interface, technical difficulty, and appropriate utilisation was assessed using a 5-point Likert scale. For qualitative data, focus group discussions (FGDs) based on experiences regarding operability, design, its effect on work efficiency and the provision of beneficial health services were conducted.

Results

There were 93 respondents for the quantitative questionnaire who reported high satisfaction (>85%) with the DHI in many themes including content quality, aesthetics and ease of use. Participants were least satisfied with service quality (45% satisfaction only) due to issues related to data sync and network connections in these areas. During the FGDs, the workers stated that the DHI helped them with accessing previous data and providing quality health care services to the community.

Conclusion

Although frontline workers reported a few technical difficulties while using the DHI, the majority reported that it was acceptable, had user-friendly features and was beneficial in their work processes.

Psychometric properties of the Telepsych User Experience Scale (TUES)

PURPOSE

To develop and psychometrically evaluate a brief measure of the telemental health experience among people receiving psychiatric and psychological care-the Telepsych User Experience Scale (TUES).

METHODS

The TUES was administered at 6 months to 364 study participants who screened positive for posttraumatic stress disorder and/or bipolar disorder and used telepsych services. The factor structure of the TUES was examined using exploratory and confirmatory factor analyses. Psychometric testing of the final scale examined (1) internal reliability, (2) criterion validity, (3) convergent validity, and (4) test-retest reliability using omega coefficients, negative binomial regression, and correlations, respectively. A week after the 12-month follow-up survey was completed, a retest was administered to 29 patients to assess reliability.

FINDINGS

Factor analytic methods indicated a single latent factor (user experience) and correlated error variance (method effect of item wording) for 2 items. To enhance clinical utility, we removed the 2 negatively worded items, resulting in a 5-item scale. Confirmatory analyses indicated excellent fit of the final model, which retained the best performing items from each hypothesized construct. The TUES demonstrated evidence of internal consistency (omega = 0.88-0.90), convergent validity, (r = 0.58), and criterion validity through telepsych engagement (incidence rate ratio = 1.19, P < .001), though test-retest reliability was unacceptable (r = 0.41).

CONCLUSION

The TUES is a pragmatic instrument with evidence of validity and internal reliability. Replication is necessary, but this initial psychometric evaluation suggests the TUES is a promising, brief yet comprehensive measure of telemental health user experience with clinical populations in rural settings.

Improving User Experience of Virtual Health Assistants: Scoping Review

Background

Virtual assistants can be used to deliver innovative health programs that provide appealing, personalized, and convenient health advice and support at scale and low cost. Design characteristics that influence the look and feel of the virtual assistant, such as visual appearance or language features, may significantly influence users’ experience and engagement with the assistant.

Objective

This scoping review aims to provide an overview of the experimental research examining how design characteristics of virtual health assistants affect user experience, summarize research findings of experimental research examining how design characteristics of virtual health assistants affect user experience, and provide recommendations for the design of virtual health assistants if sufficient evidence exists.

Methods

We searched 5 electronic databases (Web of Science, MEDLINE, Embase, PsycINFO, and ACM Digital Library) to identify the studies that used an experimental design to compare the effects of design characteristics between 2 or more versions of an interactive virtual health assistant on user experience among adults. Data were synthesized descriptively. Health domains, design characteristics, and outcomes were categorized, and descriptive statistics were used to summarize the body of research. Results for each study were categorized as positive, negative, or no effect, and a matrix of the design characteristics and outcome categories was constructed to summarize the findings.

Results

The database searches identified 6879 articles after the removal of duplicates. We included 48 articles representing 45 unique studies in the review. The most common health domains were mental health and physical activity. Studies most commonly examined design characteristics in the categories of visual design or conversational style and relational behavior and assessed outcomes in the categories of personality, satisfaction, relationship, or use intention. Over half of the design characteristics were examined by only 1 study. Results suggest that empathy and relational behavior and self-disclosure are related to more positive user experience. Results also suggest that if a human-like avatar is used, realistic rendering and medical attire may potentially be related to more positive user experience; however, more research is needed to confirm this.

Conclusions

There is a growing body of scientific evidence examining the impact of virtual health assistants’ design characteristics on user experience. Taken together, data suggest that the look and feel of a virtual health assistant does affect user experience. Virtual health assistants that show empathy, display nonverbal relational behaviors, and disclose personal information about themselves achieve better user experience. At present, the evidence base is broad, and the studies are typically small in scale and highly heterogeneous. Further research, particularly using longitudinal research designs with repeated user interactions, is needed to inform the optimal design of virtual health assistants.

The Application of Smart Medical Care in Community Health Maintenance from User Experience: A Preliminary Study in College Students

The aim of this study was to explore the students' perception of the development of smart healthcare, especially the experience of using 3C products in the maintenance of personal health. This helps us understand the capacity health managers should be endowed with in the modern era of smart medicine. And, we unraveled the college students' subjective feelings about using APP in health. Our results provide references for incorporating health maintenance platform resources into basic healthcare curriculums.

Remote Patient Monitoring Program for COVID-19 Patients Following Hospital Discharge: A Cross-Sectional Study

Background: The COVID-19 pandemic created an urgent need to act to reduce the spread of the virus and alleviate congestion from healthcare services, protect healthcare providers, and help them maintain satisfactory quality and safety of care. Remote COVID-19 monitoring platforms emerged as potential solutions. Objective: The purpose of this study was to evaluate the capacity and contribution of two different platforms used to remotely monitor patients with COVID-19 to maintain quality, safety, and patient engagement in care, as well as their acceptability, usefulness, and user-friendliness from the user's perspective. The first platform is focused on telecare phone calls (Telecare-Covid), and the second is a telemonitoring app (CareSimple-Covid). Methods: We performed a cross-sectional study. The data were collected through a phone survey from May to August 2020. Data were analyzed using descriptive statistics and t-test analysis. Participants' responses and comments on open-ended questions were analyzed using content analysis to identify certain issues and challenges and potential avenues for improving the platforms. Results: Fifty one patients participated in the study. Eighteen participants used the CareSimple-Covid platform and 33 participants used the Telecare-Covid platform. Overall, the satisfaction rate for quality and safety of care for the two platforms was 80%. Over 88% of the users on each platform considered the platforms' services to be engaging, useful, user-friendly, and appropriate to their needs. The survey identified a few significant differences in users' perceptions of each platform: empathy toward users and the quality and safety of the care received were rated significantly higher on the CareSimple-Covid platform than on the Telecare-Covid platform. Users appreciated four aspects of these telehealth approaches: (1) the ease of access to services and the availability of care team members; (2) the user-friendliness of the platforms; (3) the continuity of care provided, and (4) the wide range of services delivered. Users identified some technical limitations and raised certain issues, such as the importance of maintaining human contact, data security, and confidentiality. Improvement suggestions include promoting access to connected devices; enhancing communications between institutions, healthcare users, and the public on confidentiality and personal data protection standards; and integrating a participatory approach to telehealth platform development and deployment efforts. Conclusion: This study provides preliminary evidence that the two remote monitoring platforms are well-received by users, with very few significant differences between them concerning users' experiences and views. This type of program could be considered for use in a post-pandemic era and for other post-hospitalization clienteles. To maximize efficiency, the areas for improvement and the issues identified should be addressed with a patient-centered approach.

Learning From a Massive Open Online COVID-19 Vaccination Training Experience: Survey Study

Background

To prepare key stakeholders for the global COVID-19 vaccination rollout, the World Health Organization and partners developed online vaccination training packages. The online course was launched in December 2020 on the OpenWHO learning platform. This paper presents the findings of an evaluation of this course.

Objective

The aim of this evaluation was to provide insights into user experiences and challenges, measure the impact of the course in terms of knowledge gained, and anticipate potential interest in future online vaccination courses.

Methods

The primary source of data was the anonymized information on course participants, enrollment, completion, and scores from the OpenWHO platform’s statistical data and metric reporting system. Data from the OpenWHO platform were analyzed from the opening of the courses in mid-December 2020 to mid-April 2021. In addition, a learner feedback survey was sent by email to all course participants to complete within a 3-week period (March 19 to April 9, 2021). The survey was designed to determine the perceived strengths and weaknesses of the training packages and to understand barriers to access.

Results

During the study period, 53,593 learners enrolled in the course. Of them, 30,034 (56.0%) completed the course, which is substantially higher than the industry benchmark of 5%-10% for a massive open online course (MOOC). Overall, learners averaged 76.5% on the prequiz compared to 85% on the postquiz, resulting in an increase in average score of 9%. A total of 2019 learners from the course participated in the survey. Nearly 98% (n=1647 fully agree, n=308 somewhat agree; N=1986 survey respondents excluding missing values) of respondents fully or somewhat agreed that they had more confidence in their ability to support COVID-19 vaccination following completion of this course.

Conclusions

The online vaccine training was well received by the target audience, with a measurable impact on knowledge gained. The key benefits of online training were the convenience, self-paced nature, access to downloadable material, and ability to replay material, as well as an increased ability to concentrate. Online training was identified as a timely, cost-effective way of delivering essential training to a large number of people to prepare for the COVID-19 vaccination rollout.

Establishing a Working Definition of User Experience for eHealth Interventions of Self-reported User Experience Measures With eHealth Researchers and Adolescents: Scoping Review

BACKGROUND

Across eHealth intervention studies involving children, adolescents, and their parents, researchers have measured user experience to assist with intervention development, refinement, and evaluation. To date, no widely accepted definitions or measures of user experience exist to support a standardized approach for evaluation and comparison within or across interventions.

OBJECTIVE

We conduct a scoping review with subsequent Delphi consultation to identify how user experience is defined and measured in eHealth research studies, characterize the measurement tools used, and establish working definitions for domains of user experience that could be used in future eHealth evaluations.

METHODS

We systematically searched electronic databases for published and gray literature available from January 1, 2005, to April 11, 2019. We included studies assessing an eHealth intervention that targeted any health condition and was designed for use by children, adolescents, and their parents. eHealth interventions needed to be web-, computer-, or mobile-based, mediated by the internet with some degree of interactivity. We required studies to report the measurement of user experience as first-person experiences, involving cognitive and behavioral factors reported by intervention users. We appraised the quality of user experience measures in included studies using published criteria: well-established, approaching well-established, promising, or not yet established. We conducted a descriptive analysis of how user experience was defined and measured in each study. Review findings subsequently informed the survey questions used in the Delphi consultations with eHealth researchers and adolescent users for how user experience should be defined and measured.

RESULTS

Of the 8634 articles screened for eligibility, 129 articles and 1 erratum were included in the review. A total of 30 eHealth researchers and 27 adolescents participated in the Delphi consultations. On the basis of the literature and consultations, we proposed working definitions for 6 main user experience domains: acceptability, satisfaction, credibility, usability, user-reported adherence, and perceived impact. Although most studies incorporated a study-specific measure, we identified 10 well-established measures to quantify 5 of the 6 domains of user experience (all except for self-reported adherence). Our adolescent and researcher participants ranked perceived impact as one of the most important domains of user experience and usability as one of the least important domains. Rankings between adolescents and researchers diverged for other domains.

CONCLUSIONS

Findings highlight the various ways in which user experience has been defined and measured across studies and what aspects are most valued by researchers and adolescent users. We propose incorporating the working definitions and available measures of user experience to support consistent evaluation and reporting of outcomes across studies. Future studies can refine the definitions and measurement of user experience, explore how user experience relates to other eHealth outcomes, and inform the design and use of human-centered eHealth interventions.

User Experience in Remote Surgical Consultation: Survey Study of User Acceptance and Satisfaction in Real-Time Use of a Telemedicine Service

Background

Teleguidance, a promising telemedicine service for intraoperative surgical consultation, was planned to scale up at a major academic hospital in partnership with 5 other hospitals. If the service was adopted and used over time, it was expected to provide educational benefits and improve clinical outcomes during endoscopic retrograde cholangiopancreatography (ERCP), which is a technically advanced procedure for biliary and pancreatic disease. However, it is known that seemingly successful innovations can play out differently in new settings, which might cause variability in clinical outcomes. In addition, few telemedicine services survive long enough to deliver system-level outcomes, the causes of which are not well understood.

Objective

We were interested in factors related to usability and user experience of the telemedicine service, which might affect adoption. Therefore, we investigated perceptions and responses to the use and anticipated use of a system. Technology acceptance, a construct referring to how users perceive a technology’s usefulness, is commonly considered to indicate whether a new technology will actually be used in a real-life setting. Satisfaction measures were used to investigate whether user expectations and needs have been met through the use of technology. In this study, we asked surgeons to rate the perceived usefulness of teleguidance, and their satisfaction with the telemedicine service in direct conjunction with real-time use during clinical procedures.

Methods

We designed domain-specific measures for perceived usefulness and satisfaction, based on performance and outcome measures for the clinical procedure. Surgeons were asked to rate their user experience with the telemedicine service in direct conjunction with real-time use during clinical procedures.

Results

In total, 142 remote intraoperative consultations were conducted during ERCP procedures at 5 hospitals. The demand for teleguidance was more pronounced in cases with higher complexity. Operating surgeons rated teleguidance to have contributed to performance and outcomes to a moderate or large extent in 111 of 140 (79.3%) cases. Specific examples were that teleguidance was rated as having contributed to intervention success and avoiding a repeated ERCP in 23 cases, avoiding 3 PTC, and 11 referrals, and in 11 cases, combinations of these outcomes. Preprocedure beliefs about the usefulness of teleguidance were generally lower than postprocedure satisfaction ratings. The usefulness of teleguidance was mainly experienced through practical advice from the consulting specialist (119/140, 85%) and support with assessment and decision-making (122/140, 87%).

Conclusions

Users’ satisfaction with teleguidance surpassed their initial expectations, mainly through contribution to nontechnical aspects of performance, and through help with general assessment. Teleguidance shows the potential to improve performance and outcomes during ERCP. However, it takes hands-on experience for practitioners to understand how the new telemedicine service contributes to performance and outcomes.

A Smartphone-Based Self-Management Intervention for Individuals with Bipolar Disorder (LiveWell): Qualitative Study on User Experiences of the Behavior Change Process

BACKGROUND

Bipolar disorder is a severe mental illness characterized by recurrent episodes of depressed, elevated, and mixed mood states. The addition of psychotherapy to pharmacological management can decrease symptoms, lower relapse rates, and improve quality of life; however, access to psychotherapy is limited. Mental health technologies such as smartphone apps are being studied as a means to increase access to and enhance the effectiveness of adjunctive psychotherapies for bipolar disorder. Individuals with bipolar disorder find this intervention format acceptable, but our understanding of how people utilize and integrate these tools into their behavior change and maintenance processes remains limited.

OBJECTIVE

The objective of this study was to explore how individuals with bipolar disorder perceive and utilize a smartphone intervention for health behavior change and maintenance.

METHODS

Individuals with bipolar disorder were recruited via flyers placed at university-affiliated and private outpatient mental health practices to participate in a pilot study of LiveWell, a smartphone-based self-management intervention. At the end of the study, all participants completed in-depth qualitative exit interviews. The behavior change framework developed to organize the intervention design was used to deductively code behavioral targets and determinants involved in target engagement. Inductive coding was used to identify themes not captured by this framework.

RESULTS

In terms of behavioral targets, participants emphasized the importance of managing mood episode-related signs and symptoms. They also discussed the importance of maintaining regular routines, sleep duration, and medication adherence. Participants emphasized that receiving support from a coach as well as seeking and receiving assistance from family, friends, and providers were important for managing behavioral targets and staying well. In terms of determinants, participants stressed the important role of monitoring for their behavior change and maintenance efforts. Monitoring facilitated self-awareness and reflection, which was considered valuable for staying well. Some participants also felt that the intervention facilitated learning information necessary for managing bipolar disorder but others felt that the information provided was too basic.

CONCLUSIONS

In addition to addressing acceptability, satisfaction, and engagement, a person-based design of mental health technologies can be used to understand how people experience the impact of these technologies on their behavior change and maintenance efforts. This understanding may then be used to guide ongoing intervention development. The participants' perceptions aligned with the intervention's primary behavioral targets and use of a monitoring tool as a core intervention feature. Participant feedback further indicates that developing additional content and tools to address building and engaging social support may be an important avenue for improving LiveWell. A comprehensive behavior change framework to understand participant perceptions of their behavior change and maintenance efforts may help facilitate ongoing intervention development.

Co-production in coping with care dependency in Germany: How can integrated local care centres contribute?

In Germany, most care dependent people are looked after by family members at home. Professional support can help ease the burden of caring relatives and stabilise home care. Ideally, care then is provided through the co-production of formal and informal caregivers. This article analyses how care dependent people and their family caregivers integrate professional support into their care arrangements. An analysis was conducted using data collected for a qualitative study evaluating integrated local care centres in North-Rhine-Westphalia, Germany. The study is based on episodic interviews with users of these care centres and their family caregivers (N = 26). During the analysis, three interpretive and practice patterns relating to co-production of care were identified. These patterns reveal how the interviewees deal with (increasing) needs for assistance and care while incorporating professional care into their lives. The patterns help differentiate whether the interviewees (a) use developed care skills to contribute actively to the co-production with their layman knowledge, or (b) seek relief of their care responsibilities and withdraw temporarily from the direct sphere of care applying freed capacities to organise family daily life, or (c) use the services of the care centres to meet with other older people and to develop spaces for mutual help and co-production. The interpretive and practice patterns thus differ in the extent to which care users and family caregivers continue to play an 'active role' in the care process and contribute their own knowledge, ideas, expectations and particular care activities. In order to achieve a functioning co-production, professionals face the challenge of understanding these patterns that have been established over many years and of taking them into account appropriately.

A Smartphone Application Designed to Engage the Elderly in Home-Based Rehabilitation

As life expectancy increases, it is imperative that the elderly take advantage of the benefits of technology to remain active and independent. Mobile health applications are widely used nowadays as they promote a healthy lifestyle and self-management of diseases, opening new horizons in the interactive health service delivery. However, adapting these applications to the needs and requirements of the elderly is still a challenge. This article presents a smartphone application that is part of a multifactorial intervention to support older people with balance disorders. The application aims to enable users to self-evaluate their activity and progress, to communicate with each other and, through strategically selected motivational features, to engage with the system with undiminished interest for a long period of time. Mock-up interfaces were evaluated in semi-structured focus groups and interviews that were performed across three European countries. Further evaluation in the form of four pilot studies with 160 participants will be performed and qualitative and quantitative measures will be used to process the feedback about the use of the application.

The More, the Better? Improving VR Firefighting Training System with Realistic Firefighter Tools as Controllers

A virtual reality (VR) controller plays a key role in supporting interactions between users and the virtual environment. This paper investigates the relationship between the user experience and VR control device modality. We developed a VR firefighting training system integrated with four control devices adapted from real firefighting tools. We iteratively improved the controllers and VR system through a pilot study with six participants and conducted a user study with 30 participants to assess two salient human factor constructs—perceived presence and cognitive load—with three device modality conditions (two standard VR controllers, four real tools, and a hybrid of one real tool and one standard VR controller). We found that having more realistic devices that simulate real tools does not necessarily guarantee a higher level of user experience, highlighting a strategic approach to the development and utilization of VR control devices. Our study gives empirical insights on establishing appropriate combinations of VR control device modality in the context of field-based VR simulation and training.

Understanding UX Better: A New Technique to Go beyond Emotion Assessment

User experience (UX) is a quality aspect that considers the emotions evoked by the system, extending the usability concept beyond effectiveness, efficiency, and satisfaction. Practitioners and researchers are aware of the importance of evaluating UX. Thus, UX evaluation is a growing field with diverse approaches. Despite various approaches, most of them produce a general indication of the experience as a result and do not seek to capture the problem that gave rise to the bad UX. This information makes it difficult to obtain relevant results to improve the application, making it challenging to identify what caused a negative user experience. To address this gap, we developed a UX evaluation technique called UX-Tips. This paper presents UX-Tips and reports two empirical studies performed in an academic and an industrial setting to evaluate it. Our results show that UX-Tips had good performance in terms of efficiency and effectiveness, making it possible to identify the causes that led to a negative user experience, and it was easy to use. In this sense, we present a new technique suitable for use in both academic and industrial settings, allowing UX evaluation and finding the problems that may lead to a negative experience.

Automatic Mobile Health Arrhythmia Monitoring for the Detection of Atrial Fibrillation: Prospective Feasibility, Accuracy, and User Experience Study

Background

Atrial fibrillation (AF) is the most common tachyarrhythmia and associated with a risk of stroke. The detection and diagnosis of AF represent a major clinical challenge due to AF’s asymptomatic and intermittent nature. Novel consumer-grade mobile health (mHealth) products with automatic arrhythmia detection could be an option for long-term electrocardiogram (ECG)-based rhythm monitoring and AF detection.

Objective

We evaluated the feasibility and accuracy of a wearable automated mHealth arrhythmia monitoring system, including a consumer-grade, single-lead heart rate belt ECG device (heart belt), a mobile phone application, and a cloud service with an artificial intelligence (AI) arrhythmia detection algorithm for AF detection. The specific aim of this proof-of-concept study was to test the feasibility of the entire sequence of operations from ECG recording to AI arrhythmia analysis and ultimately to final AF detection.

Methods

Patients (n=159) with an AF (n=73) or sinus rhythm (n=86) were recruited from the emergency department. A single-lead heart belt ECG was recorded for 24 hours. Simultaneously registered 3-lead ECGs (Holter) served as the gold standard for the final rhythm diagnostics and as a reference device in a user experience survey with patients over 65 years of age (high-risk group).

Results

The heart belt provided a high-quality ECG recording for visual interpretation resulting in 100% accuracy, sensitivity, and specificity of AF detection. The accuracy of AF detection with the automatic AI arrhythmia detection from the heart belt ECG recording was also high (97.5%), and the sensitivity and specificity were 100% and 95.4%, respectively. The correlation between the automatic estimated AF burden and the true AF burden from Holter recording was >0.99 with a mean burden error of 0.05 (SD 0.26) hours. The heart belt demonstrated good user experience and did not significantly interfere with the patient’s daily activities. The patients preferred the heart belt over Holter ECG for rhythm monitoring (85/110, 77% heart belt vs 77/109, 71% Holter, P=.049).

Conclusions

A consumer-grade, single-lead ECG heart belt provided good-quality ECG for rhythm diagnosis. The mHealth arrhythmia monitoring system, consisting of heart-belt ECG, a mobile phone application, and an automated AF detection achieved AF detection with high accuracy, sensitivity, and specificity. In addition, the mHealth arrhythmia monitoring system showed good user experience.

Trial Registration

ClinicalTrials.gov NCT03507335; https://clinicaltrials.gov/ct2/show/NCT03507335

User Perception of Automated Dose Dispensed Medicine in Home Care: A Scoping Review

(1) Background: Automated dose dispensing (ADD) systems are today used around the world. The ADD robots are placed in patients' homes to increase medication safety as well as medication adherence; however, little is known about how ADD robots affect the patient's day-to-day lives, receiving the daily doses of medicine from a machine rather than from a human healthcare professional. The aim of this study is to review the available literature on users' perceptions of having an ADD robot and collect evidence on how they perceive having less human contact after implementing this technology in their homes. (2) Methods: References were searched for in Embase and PubMed. Literature investigating ADD robots in primary healthcare was included in this study and literature in a hospital setting was excluded. After screening processes, eleven publications were included in this review. (3) Results: The literature reported high medication adherence when using ADD robots and general satisfaction in terms of user experiences with the acceptability and functionality of ADD. (4) Conclusion: The review is the first focusing on user experience and perceptions regarding ADD robots. General satisfaction was shown towards ADD robots as an intervention, but the review indicates that research is missing on healthcare professionals and patient perceptions on how ADD affects their routines, both in relation to work and daily life.

Patients Prefer a Virtual Reality Approach Over a Similarly Performing Screen-Based Approach for Continuous Oculomotor-Based Screening of Glaucomatous and Neuro-Ophthalmological Visual Field Defects

Standard automated perimetry (SAP) is the gold standard for evaluating the presence of visual field defects (VFDs). Nevertheless, it has requirements such as prolonged attention, stable fixation, and a need for a motor response that limit application in various patient groups. Therefore, a novel approach using eye movements (EMs) - as a complementary technique to SAP - was developed and tested in clinical settings by our group. However, the original method uses a screen-based eye-tracker which still requires participants to keep their chin and head stable. Virtual reality (VR) has shown much promise in ophthalmic diagnostics - especially in terms of freedom of head movement and precise control over experimental settings, besides being portable. In this study, we set out to see if patients can be screened for VFDs based on their EM in a VR-based framework and if they are comparable to the screen-based eyetracker. Moreover, we wanted to know if this framework can provide an effective and enjoyable user experience (UX) compared to our previous approach and the conventional SAP. Therefore, we first modified our method and implemented it on a VR head-mounted device with built-in eye tracking. Subsequently, 15 controls naïve to SAP, 15 patients with a neuro-ophthalmological disorder, and 15 glaucoma patients performed three tasks in a counterbalanced manner: (1) a visual tracking task on the VR headset while their EM was recorded, (2) the preceding tracking task but on a conventional screen-based eye tracker, and (3) SAP. We then quantified the spatio-temporal properties (STP) of the EM of each group using a cross-correlogram analysis. Finally, we evaluated the human-computer interaction (HCI) aspects of the participants in the three methods using a user-experience questionnaire. We find that: (1) the VR framework can distinguish the participants according to their oculomotor characteristics; (2) the STP of the VR framework are similar to those from the screen-based eye tracker; and (3) participants from all the groups found the VR-screening test to be the most attractive. Thus, we conclude that the EM-based approach implemented in VR can be a user-friendly and portable companion to complement existing perimetric techniques in ophthalmic clinics.

User Experience in Health & Medicine: Building Methods for Patient Experience Design in Multidisciplinary Collaborations

Health and medical contexts have emerged as an important area of inquiry for researchers at the intersection of user experience and technical communication. In addressing this intersection, this article advocates and extends patient experience design or PXD (Melonçon, 2017) as an important framework for user experience research within health and medicine. Specifically, this article presents several PXD insights from a task-based usability study that examined an online intervention program for people with voice problems. We respond to Melonçon's call (2017) to build PXD as a framework for user experience and technical communication research by describing ways traditional usability methods can provide PXD insights and asking the following question: What insights can emerge from combining traditional usability methods and PXD research? In addressing this question, we outline two primary methodological and practical considerations we found central to conducting PXD research: 1) engaging patients as participants, and 2) leveraging multidisciplinary collaboration.

Usability, User Experience, and Acceptance Evaluation of CAPACITY: A Technological Ecosystem for Remote Follow-Up of Frailty

Frailty predisposes older persons to adverse events, and information and communication technologies can play a crucial role to prevent them. CAPACITY provides a means to remotely monitor variables with high predictive power for adverse events, enabling preventative personalized early interventions. This study aims at evaluating the usability, user experience, and acceptance of a novel mobile system to prevent disability. Usability was assessed using the system usability scale (SUS); user experience using the user experience questionnaire (UEQ); and acceptance with the technology acceptance model (TAM) and a customized quantitative questionnaire. Data were collected at baseline (recruitment), and after three and six months of use. Forty-six participants used CAPACITY for six months; nine dropped out, leaving a final sample of 37 subjects. SUS reached a maximum averaged value of 83.68 after six months of use; no statistically significant values have been found to demonstrate that usability improves with use, probably because of a ceiling effect. UEQ, obtained averages scores higher or very close to 2 in all categories. TAM reached a maximum of 51.54 points, showing an improvement trend. Results indicate the success of the participatory methodology, and support user centered design as a key methodology to design technologies for frail older persons. Involving potential end users and giving them voice during the design stage maximizes usability and acceptance.

Technology-Mediated Experiences and Social Context: Relevant Needs in Private Vs. Public Interaction and the Importance of Others for Positive Affect

Technologies, such as smartphones or wearables, take a central role in our daily lives. Making their use meaningful and enjoyable requires a better understanding of the prerequisites and underpinnings of positive experiences with such technologies. So far, a focus had been on the users themselves, that is, their individual goals, desires, feelings, and acceptance. However, technology is often used in a social context, observed by others or even used in interaction with others, and thus shapes social dynamics considerably. In the present paper, we start from the notion that meaningful and/or enjoyable experiences (i.e., wellbeing) are a major outcome of technology use. We investigate how these experiences are further shaped by social context, such as potential spectators. More specifically, we gathered private (while being alone) and public (while other people are present) positive experiences with technology and compared need fulfillment and affective experience. In addition, we asked participants to imagine a change in context (from private to public or public to private) and to report the impact of this change on experience. Results support the idea of particular social needs, such as relatedness and popularity, which are especially relevant and better fulfilled in public than in private contexts. Moreover, our findings show that participants experience less positive affect when imaginatively removing the present others from a formerly public interaction, i.e., when they imagine performing the same interaction but without the other people present. Overall, this underlines the importance of social context for Human-Computer Interaction practice and research. Practical implications relate to product development, e.g., designing interactive technologies that can adapt to context (changes) or allow for context-sensitive interaction sets. We discuss limitations related to the experimental exploration of social context, such as the method of data collection, as well as potential alternatives to address those limitations, such as diary studies.

Receiving healthcare for drug-resistant TB: a cross-sectional survey from Pakistan

OBJECTIVE

To describe and quantify patients' self-reported experiences of receiving healthcare from Pakistan's Programmatic Management of Drug-Resistant Tuberculosis (PMDT) model of care, and to understand these experiences within the broader context of Pakistan's health system.

METHOD

This was a cross-sectional survey of patients attending three PMDT clinics in Khyber-Pakhtunkhwa Province in Pakistan.

RESULTS

The median consultation time at the PMDT clinics was 10 minutes. In their most recent visit to the PMDT clinic, 34.9% of patients spent >40% of their monthly income to access treatment. To specify, 71% of patients reported spending out-of-pocket for ancillary medicines and 44.7% for laboratory tests. In 10.5% of cases, medicines for drug-resistant TB (DR-TB) were dispensed without the patient attending the clinic. Only 43.7% of treatment supporters regularly accompanied patients to the clinic, and 6% supervised the patient's intake of medicines. Disbursement of financial support was irregular in 98.6% of cases. Only 6.2% of patients received their daily injections from a public facility, the rest went elsewhere.

CONCLUSION

Several shortcomings in PMDT services, including hurried consultations, irregularities in financial support, and gaps in Pakistan's broader health system undermined healthcare experience of patients with DR-TB. To improve health outcomes and patients' care experience these service gaps need to be addressed.

Role of Interface Design: A Comparison of Different Online Learning System Designs

Mobile devices are becoming an indispensable part of the daily lives and learning habits of older adults with the easy access of the Internet. It enhances the connection between old users and online education, which supplies an approach to cultivate them with innovative concepts and entrepreneurship of education. However, the complicated navigation of information systems (IS) confuses older adults, and gets them disoriented in searches of information, in addition, to influencing online activities for older adults. This study aims to investigate what kind of navigation of IS is suitable for older adults. A 2 × 2 (2 factors, 2 levels) mixed experimental design was employed. The two factors were, respectively, cognitive load (CL) and navigation structure (NS). A sample of 40 older participants (mean age = 64.37, SD = 4.03) performed online learning tasks in terms of innovative concept using linear hierarchical or mixed NSs under different time pressures. The results showed that linear hierarchical navigation is more appropriate for the elderly when learning and generating innovative concepts on smartphones, as the interaction between CL and NS exists. Overall, the findings combined suggest that the linear hierarchical NS, compared to mixed hierarchical navigation, obtained better usability in terms of task efficiency, CL, and subjective ratings. The findings can provide theoretical support for designers to design and develop mobile websites for older adults.

Security, Privacy, and Usability in Continuous Authentication: A Survey

Continuous authentication has been proposed as a possible approach for passive and seamless user authentication, using sensor data comprising biometric, behavioral, and context-oriented characteristics. Since these are personal data being transmitted and are outside the control of the user, this approach causes privacy issues. Continuous authentication has security challenges concerning poor matching rates and susceptibility of replay attacks. The security issues are mainly poor matching rates and the problems of replay attacks. In this survey, we present an overview of continuous authentication and comprehensively discusses its different modes, and issues that these modes have related to security, privacy, and usability. A comparison of privacy-preserving approaches dealing with the privacy issues is provided, and lastly recommendations for secure, privacy-preserving, and user-friendly continuous authentication.

Smart Co-housing for People With Disabilities: A Preliminary Assessment of Caregivers' Interaction With the DOMHO System

Millions of people with motor and cognitive disabilities face hardships in daily life due to the limited accessibility and inclusiveness of living spaces which limit their autonomy and independence. The DOMHO project deals with these fundamental issues by leveraging an innovative solution: a smart co-housing apartment. Besides, the project aims at exploiting the well know effects of co-housing on individuals' health and well-being in combination with ambient assisted living technologies. The present study focused on the interaction of caregivers with the control application of an integrated smart system. Participants performed different tasks, fill out a questionnaire, and were interviewed. Performance and usability of the user interface, trust in technology, privacy, and attitudes towards home automation were explored. A series of guidelines for domotic technology control interfaces design was identified, and a high level of trust in these advanced tools was shown. Caregivers considered smart technologies as a work aid and a means for enhancing autonomy and life quality for users with disabilities.

EXecutive-Functions Innovative Tool (EXIT 360°): A Usability and User Experience Study of an Original 360°-Based Assessment Instrument

Over the last few decades, several studies have shown the feasibility, acceptability, and efficacy of VR-based instruments in the early evaluation of executive dysfunction (ED) in psychiatric and neurologic conditions. Due to the negative impact of ED in everyday functioning, identifying innovative strategies for evaluating ED allows clinicians to detect executive impairment early and minimize its effects. This work aimed to test the usability and user experience (UX) of EXecutive-functions Innovative Tool 360° (EXIT 360°), a 360°-based tool for assessing ED. Seventy-six healthy subjects underwent an evaluation that involved (1) usability assessment using System Usability Scale and (2) evaluation of UX using the ICT-Sense of Presence and UX Questionnaire. Results showed a satisfactory level of usability (mean = 75.9 ± 12.8), with good scores for usability and learnability. As regards UX, EXIT 360° showed an absence of negative effects (mean = 1.79 ± 0.95) and high scores in ecological validity (mean = 4.32 ± 0.54) and engagement (mean = 3.76 ± 0.56). Moreover, it obtained good scores in efficiency (mean = 1.84 ± 0.84), originality (mean = 2.49 ± 0.71), and attractiveness (mean = 1.93 ± 0.98). Interestingly, demographic characteristics and technological expertise had no impact on the performance (p > 0.05). Overall, EXIT 360° appeared to be a usable, learn-to-use, engaging, and creative tool with irrelevant negative effects. Further studies will be conducted to evaluate these aspects in the clinical population.

User Perceptions and Experiences of a Handheld 12-Lead Electrocardiographic Device in a Clinical Setting: Usability Evaluation

Background

Cardiac arrhythmias are a leading cause of death. The mainstay method for diagnosing arrhythmias (eg, atrial fibrillation) and cardiac conduction disorders (eg, prolonged corrected QT interval [QTc]) is by using 12-lead electrocardiography (ECG). Handheld 12-lead ECG devices are emerging in the market. In tandem with emerging technology options, evaluations of device usability should go beyond validation of the device in a controlled laboratory setting and assess user perceptions and experiences, which are crucial for successful implementation in clinical practice.

Objective

This study aimed to evaluate clinician and patient perceptions and experiences, regarding the usability of a handheld 12-lead ECG device compared to a conventional 12-lead ECG machine, and generalizability of this user-centered approach.

Methods

International Organization for Standardization Guidelines on Usability and the Technology Acceptance Model were integrated to form the framework for this study, which was conducted in outpatient clinics and cardiology wards at Westmead Hospital, New South Wales, Australia. Each patient underwent 2 ECGs (1 by each device) in 2 postures (supine and standing) acquired in random sequence. The times taken by clinicians to acquire the first ECG (efficiency) using the devices were analyzed using linear regression. Electrocardiographic parameters (QT interval, QTc interval, heart rate, PR interval, QRS interval) and participant satisfaction surveys were collected. Device reliability was assessed by evaluating the mean difference of QTc measurements within ±15 ms, intraclass correlation coefficient, and level of agreement of the devices in detecting atrial fibrillation and prolonged QTc. Clinicians’ perceptions and feedback were assessed with semistructured interviews based on the Technology Acceptance Model.

Results

A total of 100 patients (age: mean 57.9 years, SD 15.2; sex: male: n=64, female n=36) and 11 clinicians (experience acquiring ECGs daily or weekly 10/11, 91%) participated, and 783 ECGs were acquired. Mean differences in QTc measurements of both handheld and conventional devices were within ±15 ms with high intraclass correlation coefficients (range 0.90-0.96), and the devices had a good level of agreement in diagnosing atrial fibrillation and prolonged QTc (κ=0.68-0.93). Regardless of device, QTc measurements when patients were standing were longer duration than QTc measurements when patients were supine. Clinicians’ ECG acquisition times improved with usage (P<.001). Clinicians reported that device characteristics (small size, light weight, portability, and wireless ECG transmission) were highly desired features. Most clinicians agreed that the handheld device could be used for clinician-led mass screening with enhancement in efficiency by increasing user training. Regardless of device, patients reported that they felt comfortable when they were connected to the ECG devices.

Conclusions

Reliability and usability of the handheld 12-lead ECG device were comparable to those of a conventional ECG machine. The user-centered evaluation approach helped us identify remediable action to improve the efficiency in using the device and identified highly desirable device features that could potentially help mass screening and remote assessment of patients. The approach could be applied to evaluate and better understand the acceptability and usability of new medical devices.

Mental Health Service User and Worker Experiences of Psychosocial Support Via Telehealth Through the COVID-19 Pandemic: Qualitative Study

BACKGROUND

During the COVID-19 pandemic, we saw telehealth rapidly become the primary way to receive mental health care. International research has validated many of the benefits and challenges of telehealth known beforehand for specific population groups. However, if telehealth is to assume prominence in future mental health service delivery, greater understanding of its capacity to be used to provide psychosocial support to people with complex and enduring mental health conditions is needed.

OBJECTIVE

We focused on an Australian community-managed provider of psychosocial intervention and support. We aimed to understand service user and worker experiences of psychosocial support via telehealth throughout the COVID-19 pandemic.

METHODS

This study was jointly developed and conducted by people with lived experience of mental ill health or distress, mental health service providers, and university-based researchers. Semistructured interviews were conducted between August and November 2020 and explored participant experiences of receiving or providing psychosocial support via telehealth, including telephone, text, and videoconferencing. Qualitative data were analyzed thematically; quantitative data were collated and analyzed using descriptive statistics.

RESULTS

Service users (n=20) and workers (n=8) completed individual interviews via telephone or videoconferencing platform. Service users received psychosocial support services by telephone (12/20, 60%), by videoconferencing (6/20, 30%), and by both telephone and videoconferencing (2/20, 10%). Of note, 55% (11/20) of service user participants stated a future preference for in-person psychosocial support services, 30% (6/20) preferred to receive a mixture of in-person and telehealth, and 15% (3/20) elected telehealth only. Two meta-themes emerged as integral to worker and service user experience of telehealth during the pandemic: (1) creating safety and comfort and (2) a whole new way of working. The first meta-theme comprises subthemes relating to a sense of safety and comfort while using telehealth; including trusting in the relationship and having and exercising choice and control. The second meta-theme contains subthemes reflecting key challenges and opportunities associated with the shift from in-person psychosocial support to telehealth.

CONCLUSIONS

Overall, our findings highlighted that most service users experienced telehealth positively, but this was dependent on them continuing to get the support they needed in a way that was safe and comfortable. While access difficulties of a subgroup of service users should not be ignored, most service users and workers were able to adapt to telehealth by focusing on maintaining the relationship and using choice and flexibility to maintain service delivery. Although most research participants expressed a preference for a return to in-person psychosocial support or hybrid in-person and telehealth models, there was a general recognition that intentional use of telehealth could contribute to flexible and responsive service delivery. Challenges to telehealth provision of psychosocial support identified in this study are yet to be fully understood.

Usability and Effectiveness of Immersive Virtual Grocery Shopping for Assessing Cognitive Fatigue in Healthy Controls: Protocol for a Randomized Controlled Trial

BACKGROUND

Cognitive fatigue (CF) is a human response to stimulation and stress and is a common comorbidity in many medical conditions that can result in serious consequences; however, studying CF under controlled conditions is difficult. Immersive virtual reality provides an experimental environment that enables the precise measurement of the response of an individual to complex stimuli in a controlled environment.

OBJECTIVE

We aim to examine the development of an immersive virtual shopping experience to measure subjective and objective indicators of CF induced by instrumental activities of daily living.

METHODS

We will recruit 84 healthy participants (aged 18-75 years) for a 2-phase study. Phase 1 is a user experience study for testing the software functionality, user interface, and realism of the virtual shopping environment. Phase 2 uses a 3-arm randomized controlled trial to determine the effect that the immersive environment has on fatigue. Participants will be randomized into 1 of 3 conditions exploring fatigue response during a typical human activity (grocery shopping). The level of cognitive and emotional challenges will change during each activity. The primary outcome of phase 1 is the experience of user interface difficulties. The primary outcome of phase 2 is self-reported CF. The core secondary phase 2 outcomes include subjective cognitive load, change in task performance behavior, and eye tracking. Phase 2 uses within-subject repeated measures analysis of variance to compare pre- and postfatigue measures under 3 conditions (control, cognitive challenge, and emotional challenge).

RESULTS

This study was approved by the scientific review committee of the National Institute of Nursing Research and was identified as an exempt study by the institutional review board of the National Institutes of Health. Data collection will begin in spring 2021.

CONCLUSIONS

Immersive virtual reality may be a useful research platform for simulating the induction of CF associated with the cognitive and emotional challenges of instrumental activities of daily living.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04883359; http://clinicaltrials.gov/ct2/show/NCT04883359.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/28073.

Why and How to Approach User Experience in Safety-Critical Domains: The Example of Health Care

OBJECTIVE

To highlight the importance of the personal experience of users who interact with technology in safety-critical domains and summarize three interaction concepts and the associated theories that provide the means for addressing user experience.

BACKGROUND

In health care, the dominant concepts of interaction are based on theories arising from classic cognitive psychology. These concepts focus mainly on safety and efficiency, with too little consideration being given to user experience.

METHOD

Users in complex socio-technical and safety-critical domains such as health care interact with many technological devices. Enhancing the user experience could improve the design of technology, enhance the well-being of staff, and contribute to modern safety management. We summarize concepts of "interaction" based on modern theories of human-computer interaction, which include the personal experience of users as an important construct.

RESULTS AND CONCLUSION

Activity theory, embodiment, and interaction as experience provide a theoretical foundation for considering user experience in safety-critical domains. Using an example from anesthesiology, we demonstrate how each theory provides a unique but complementary view on experience. Finally, the methodological possibilities for considering personal experience in design and evaluations vary among the theories.

APPLICATION

Considering user experience in health care and potentially other safety-critical domains can provide an additional means of optimizing interaction with technology, contributing to the well-being of staff, and improving safety.

Using electrophysiological measures to evaluate the sense of presence in immersive virtual environments: An event-related potential study

Sense of presence has been often explored in the context of virtual reality (VR) and immersive visual technologies; however, standardized and objective measures of the sense of presence have been difficult to find. Studies attempting to find physiological correlates of sense presence using electroencephalography (EEG) have reported mixed results. In the present study, we used brain event-related potentials (ERPs) elicited by auditory stimuli to identify an objective physiological index of sense of presence during VR, attempting to replicate the findings of previous studies and explain the heterogeneity of results reported in the literature. Participants in our experiment were asked to experience an immersive virtual environment using a modern head-mounted display while passively hearing task-irrelevant frequent standard and infrequent deviant tones as in a classic auditory oddball paradigm. Subsequently, they were asked to complete a battery of questionnaires aimed to estimate their sense of presence during the VR. EEG and questionnaire data from three-seventh participants were analyzed. ERP components evoked by the auditory stimuli were then analyzed. Late ERP components (after 450 ms from stimulus onset) registered over central brain areas were associated with the sense of presence as measured with questionnaires, while earlier components were not associated with presence. The use of different questionnaires and the content of the VR environment may both be a plausible explanation for heterogeneous results as reported in previous studies. The present study showed that late ERP components recorded over the central brain may represent good electrophysiological correlates of the subjective sense of presence.

Development of a Video-Observed Therapy System to Improve Monitoring of Tuberculosis Treatment in Thailand: Mixed-Methods Study

Background

Directly observed therapy programs for monitoring tuberculosis (TB) treatment in Thailand are unsustainable, especially during the COVID-19 pandemic. The current video-observed therapy (VOT) system, the Thai VOT (TH VOT), was developed to replace the directly observed therapy program.

Objective

This study aimed to describe the VOT system design and identify the potential for system improvements.

Methods

This pilot study was conducted in Na Yong district, a small district in Trang province, south of Thailand. The TH VOT system consists of a smartphone app for patients, a secured web-based platform for staff, items used, and standard operating procedures. There were three groups of users: observers who were TB staff, healthy volunteers as simulated patients, and patients with active TB. All participants were trained to follow the standard operating procedures. After 2-week usage, VOT session records were analyzed to measure the compliance of the patients and observers. The User Experience Questionnaire was used to lead the participant users to focus on 6 standard dimensions of usability, and was supplemented with an in-depth interview to identify potential system improvements from users’ experience.

Results

Only 2 of 16 patients with currently active TB had a usable smartphone. Sixty of 70 drug-taking sessions among 2 patients and 3 simulated patients in 2 weeks were recorded and uploaded. Only 37 sessions were inspected by the observers within 24 hours. All participants needed a proper notification system. An audit system was also requested.

Conclusions

Before upscaling, the cost of smartphone lending, audit management, and notification systems should be elucidated.

Examining Challenges to the Incorporation of End Users in the Design of Digital Health Interventions: Protocol for a Systematic Review

Background

The process of designing a digital health intervention (DHI)—also referred to as mobile health or eHealth—spans needs assessments, technical functionality and feasibility, user satisfaction, effectiveness, impact, and value. These interventions are causing a rapid evolution in the landscape of health care. Multiple studies have shown their propensity to extend both the quality and reach of interventions. However, failure to improve DHI design is linked to failed uptake and health outcomes. This dilemma is further conflicted by the colliding backdrops of the digital and health industries, both of which approach, understand, and involve end users differently in the framing of a DHI.

Objective

The objective of this systematic review is to assess the challenges to incorporating end users in the design stage of digital health interventions, to identify key pain points, and to identify limitations and gaps for areas of future investigation.

Methods

The PRISMA-P (Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocols) checklist will be used to structure this protocol. A systematic search of the PsycINFO, PubMed (MEDLINE), Web of Science, CINAHL, Scopus, and IEEE Xplore databases will be conducted. Additionally, the PerSPEcTiF guidelines for complex interventions will be consulted. Two reviewers will independently screen the titles and abstracts of the identified references and select studies according to the eligibility criteria. Any discrepancies will then be discussed and resolved. Two reviewers will independently extract and validate data from the included studies into a standardized form and conduct quality appraisal.

Results

As of February 2021, we have completed a preliminary literature search examining challenges to the incorporation of end users in the design stage of DHIs. Systematic searches, data extraction and analysis, and writing of the systematic review are expected to be completed by December 2021.

Conclusions

This systematic review aims to provide an effective summary of key pain points toward incorporating end users in DHIs. Results from this review will provide an evidence base for a better approach to end user involvement in the interest of improving efficacy and uptake of DHIs.

Trial Registration

PROSPERO International Prospective Register of Systematic Reviews CRD42021238164; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=238164

International Registered Report Identifier (IRRID)

PRR1-10.2196/28083

A Qualitative Approach to Help Adjust the Design of Management Subjects in ICT Engineering Undergraduate Programs through User Experience in a Smart Classroom Context

Qualitative research activities, including first-day of class surveys and user experience interviews on completion of a subject were carried out to obtain students' feedback in order to improve the design of the subject 'Information Systems' as a part of a general initiative to enhance ICT (Information and Communication Technologies) engineering programs. Due to the COVID-19 (corona virus disease 2019) pandemic, La Salle URL adopted an Emergency Remote Teaching tactical solution in the second semester of the 2019-2020 academic year, just before implementing a strategic learning approach based on a new Smart Classroom (SC) system deployed in the campus facilities. The latter solution was developed to ensure that both on-campus and off-campus students could effectively follow the course syllabus through the use of new technological devices introduced in classrooms and laboratories, reducing the inherent difficulties of online learning. The results of our findings show: (1) No major concerns about the subject were identified by students; (2) Interaction and class dynamics were the main issues identified by students, while saving time on commuting when learning from home and access to recorded class sessions were the aspects that students considered the most advantageous about the SC.