A Scale to Assess the Methodological Quality of Studies Assessing Usability of Electronic Health Products and Services: Delphi Study Followed by Validity and Reliability Testing

Acceptance and Privacy Perceptions Toward Video-based Active and Assisted Living technologies: Scoping Review (Preprint)

BACKGROUND

The aging society posits new socioeconomic challenges to which a potential solution is active and assisted living (AAL) technologies. Visual-based sensing systems are technologically among the most advantageous forms of AAL technologies in providing health and social care; however, they come at the risk of violating rights to privacy. With the immersion of video-based technologies, privacy-preserving smart solutions are being developed; however, the user acceptance research about these developments is not yet being systematized.

OBJECTIVE

With this scoping review, we aimed to gain an overview of existing studies examining the viewpoints of older adults and/or their caregivers on technology acceptance and privacy perceptions, specifically toward video-based AAL technology.

METHODS

A total of 22 studies were identified with a primary focus on user acceptance and privacy attitudes during a literature search of major databases. Methodological quality assessment and thematic analysis of the selected studies were executed and principal findings are summarized. The PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) guidelines were followed at every step of this scoping review.

RESULTS

Acceptance attitudes toward video-based AAL technologies are rather conditional, and are summarized into five main themes seen from the two end-user perspectives: caregiver and care receiver. With privacy being a major barrier to video-based AAL technologies, security and medical safety were identified as the major benefits across the studies.

CONCLUSIONS

This review reveals a very low methodological quality of the empirical studies assessing user acceptance of video-based AAL technologies. We propose that more specific and more end user- and real life-targeting research is needed to assess the acceptance of proposed solutions.

A systematic review and meta-analysis on cognitive frailty in community-dwelling older adults: risk and associated factors

OBJECTIVES

To identify which factors are associated with cognitive frailty (CF), as well as the impact of CF on the incidence of dementia and mortality.

METHODS

A systematic review with meta-analysis was carried out using papers that enrolled a total of 75,379 participants and were published up to January 2020.

RESULTS

Of the 558 identified records, 28 studies met the inclusion criteria and were included in the review. The meta-analysis of cross-sectional studies showed that CF has a significant association of having an older age and a history of falls. In longitudinal studies, the analysis showed a significant increase in risk of mortality and dementia for those with CF.

DISCUSSION

This is the first systematic review and meta-analysis on CF, which addressed a wide variety of factors associated with the theme and which pointed out some as a potential target for prevention or management with different interventions or treatments, showing the clinical importance of its identification in the most vulnerable and susceptible groups.

A roadmap for the development and evaluation of the eHealthResp online course

Background

Inappropriate antibiotic use constitutes one of the most concerning public health issues, being one of the main causes of antibiotic resistance. Hence, to tackle this issue, it is important to encourage the development of educational interventions for health practitioners, namely by using digital health tools. This study focuses on the description of the development and validation process of the eHealthResp online course, a web platform directed to physicians and pharmacists, with the overall goal of improving antibiotic use for respiratory tract infections, along with the assessment of its usability.

Methods

The eHealthResp platform and the courses, developed with a user-centered design and based on Wordpress and MySQL, were based on a previously developed online course. A questionnaire to assess the usability was distributed among physicians (n = 6) and pharmacists (n = 6). Based on the obtained results, statistical analyses were conducted to calculate the usability score and appraise the design of the online course, as well as to compare the overall scores attributed by both groups. Further qualitative comments provided by the participants have also been analyzed.

Results

The eHealthResp contains two online courses directed to physicians and pharmacists aiming to aid in the management of respiratory tract infections. The average usability score of the eHealthResp online courses for physicians and pharmacists was of 78.33 (±11.57, 95%CI), and 83.75 (±15.90, 95%CI), respectively. Qualitative feedback emphasized the usefulness of the course, including overall positive reviews regarding user-friendliness and consistency.

Conclusions

This study led us to conclude that the eHealthResp online course is not recognized as a complex web platform, as both qualitative and quantitative feedback obtained were globally positive.

Methodological Quality of User-Centered Usability Evaluation of Ambient Assisted Living Solutions: A Systematic Literature Review

This study aimed to determine the methodological quality of user-centered usability evaluation of Ambient Assisted Living (AAL) solutions by (i) identifying the characteristics of the AAL studies reporting on user-centered usability evaluation, (ii) systematizing the methods, procedures and instruments being used, and (iii) verifying if there is evidence of a common understanding on methods, procedures, and instruments for user-centered usability evaluation. An electronic search was conducted on Web of Science, Scopus, and IEEE Xplore databases, combining relevant keywords. Then, titles and abstracts were screened against inclusion and exclusion criteria, and the full texts of the eligible studies were retrieved and screened for inclusion. A total of 44 studies were included. The results show a great heterogeneity of methods, procedures, and instruments to evaluate the usability of AAL solutions and, in general, the researchers fail to consider and report relevant methodological aspects. Guidelines and instruments to assess the quality of the studies might help improving the experimental design and reporting of studies on user-centered usability evaluation of AAL solutions.

Characterisation and assessment of the most popular mobile apps designed for neck pain self-management: A systematic search in app stores

PURPOSE

This study aimed to assess commonly downloaded apps targeting neck pain (NP) using the Mobile Apps Rating Scale (MARS), to characterise the interventions described in the apps and to correlate the ratings of quality retrieved from app stores against the ratings of quality assessed using MARS.

MATERIALS AND METHODS

A systematic search was conducted at the Apple App Store, Google Play, Windows Phone Store and Huawei App Gallery. Mobile apps with 100 or more ratings, a user rating of 4 or higher and which were available for free were retrieved for assessment.

RESULTS

A total of 17 apps met the inclusion criteria. The mean user rating was 4.4 ± 0.28, the mean MARS score was 4.0 ± 0.58 and the correlation between user ratings and MARS total scores and subdomains varied between a maximum of 0.41 (p > 0.05) and a minimum of -0.11 (p > 0.05). All apps included exercise but only 5 (29%) included an educational component.

CONCLUSIONS

The most downloaded apps on NP available for free in app stores are, generally, of acceptable quality as assessed by the MARS total score, target exercise, but fail to include an appropriate educational component. The user rating available in app stores is not an indicator of the quality of the apps. Therefore, health professionals should carefully assess the mobile apps before recommending them to the patients.

A Provider-Facing eHealth Tool for Transitioning Youth With Special Health Care Needs From Pediatric to Adult Care: Mixed Methods, User-Engaged Usability Study

BACKGROUND

There is a need for medical education on health care transitions for youth with special health care needs. The Texas Transition Toolkit (the tool) supports providers through a one-stop shop for researching literature on care transitions, a catalog of care transition tools, and guides for developing care transition programs.

OBJECTIVE

This study aims to assess the functionality and usability of the tool with providers working with transition-aged children and youth with special health care needs (representative users).

METHODS

The tool was evaluated using a triangulated mixed methods case study approach consisting of a concurrent think-aloud phase, a satisfaction survey, and a survey of problem relevance and task performance to operationalize and capture functionality and usability. Our mixed methods deep dive into the functionality and usability of the tool focused on 10 representative users from one medical home in Texas and 5 website design experts.

RESULTS

Representative users found the tool to be highly relevant, as demonstrated by the satisfaction score for relevance (138/150, 92%). According to the users, the tool provided comprehensive information related to health care transitions for youth with special health care needs, with a satisfaction score of 87.3% (131/150) for comprehensive. Overall satisfaction with the tool was high at 81.92% (1065/1300) with a cutoff score of 73.33% (953.4/1300) indicating high satisfaction, but users reported relatively lower satisfaction with search (114/150, 76%) and navigation (ease of use: 114/150, 76%; hyperlinks: 163/200, 81.5%; structure: 159/200, 79.5%). They experienced search- and navigation-related problems (total problems detected: 21/31, 68%) and, based on quality checks, had a relatively low task completion rate for tasks involving finding information (60/80, 75%), which required searching and navigation. The problems identified around search and navigation functionality were relevant (relevance scores ranging from 14.5 to 22, with a cutoff score of 11.7 indicating relevance).

CONCLUSIONS

The tool may help bridge the gaps in training on health care transitions for youth with special health care needs in US medical education. The tool can be used to create structured protocols to help improve provider knowledge, collaboration across pediatric and adult care providers, and the continuity of care as youth with special health care needs transition from pediatric to adult care. The results provided a road map for optimizing the tool and highlighted the importance of evaluating eHealth technologies with representative users.

Validation of the eHealthResp online course for pharmacists and physicians: A Delphi method approach

FRAMEWORK

The inappropriate use of antibiotics for respiratory tract infections is dispersed worldwide, thus being a strong contributor to antibiotic resistances. As the use of educational interventions among health practitioners is shown to have an impact on judicious antibiotic use, an online course (eHealthResp) has been developed, especially targeted to pharmacists and physicians. Thus, the main goal of this study is to validate the contents of the online course eHealthResp.

METHODS

This two-round Delphi study involved the recruitment of a multidisciplinary panel (n = 19), to which the questionnaires of the first round were sent. After the first round, a report summing up the results has been forwarded to the panel, along with a new, reformulated version of the questionnaire.

RESULTS

After the two rounds of the Delphi process, consensus was evaluated. Six clinical cases and fifty-one treatments obtained minor consensus [60-75%] or full consensus (≥75%). The question on antibiotic practice has obtained a consensus >90% on both rounds.

CONCLUSIONS

The validation of the contents based on experts' consensus has been an essential approach to improve eHealthResp's online course, as valuable feedback has been provided by the panel on both rounds.

User-Centered Development of a Mobile App for Biopsychosocial Pain Assessment in Adults: Usability, Reliability, and Validity Study

Background

Pain-related mobile apps targeting pain assessment commonly limit pain assessment to pain behaviors and physiological aspects. However, current guidelines state that pain assessment should follow the biopsychosocial model, clearly addressing biological, psychological, and social aspects of the pain experience. Existing reviews also highlight that pain specialists and end users are not commonly involved in the development process of mobile apps for pain assessment, negatively affecting the quality of the available apps.

Objective

This study aimed to develop a mobile app for pain assessment (AvaliaDor) and assess its usability, validity, reliability, and measurement error in a sample of real patients with chronic pain recruited from a physiotherapy clinic.

Methods

This study was divided into 2 phases: phase 1—development of the AvaliaDor app; and phase 2—assessment of the apps’ usability, reliability, measurement error, and validity. AvaliaDor was developed (phase 1) based on the literature and the recommendations of physiotherapists and patients with pain in cycles of evaluation, inclusion of recommendations, and reevaluation until no further changes were required. The final version of the app was then tested in patients with musculoskeletal pain attending a private physiotherapy practice (phase 2) who were asked to use the app twice on 2 consecutive days for reliability purposes. In addition, participants had to complete a set of paper-based scales (Brief Pain Inventory, painDETECT, Pain Catastrophizing Scale, and Tampa Scale for Kinesiophobia), which were used to assess the validity (criterion validity and hypothesis testing) of the app, and the Post-Study System Usability Questionnaire was used to assess its usability.

Results

The development process (phase 1) included 5 physiotherapists external to the research team and 5 patients with musculoskeletal pain, and it resulted in the creation of an app named AvaliaDor, which includes an assessment of pain intensity, location, and phenotype; associated disability; and the issues of pain catastrophizing and fear of movement. A total of 52 patients with pain (mean age 50.12 years, SD 11.71 years; 39 females) participated in phase 2 and used the app twice. The Pearson correlation coefficient between the scores on the paper-based scales and the app ranged between 0.81 and 0.93 for criterion validity and between 0.41 and 0.59 for hypothesis testing. Test-retest reliability was moderate to good (intraclass correlation coefficient between 0.67 and 0.90) and the score for usability was 1.16 (SD 0.27), indicating good usability.

Conclusions

A mobile app named AvaliaDor was developed to assess the intensity, location, and phenotype of pain; associated disability; and the issues of pain catastrophizing and fear of movement in a user-centered design process. The app was shown to be usable, valid, and reliable for assessing pain from a biopsychosocial perspective in a heterogeneous group of patients with pain. Future work can explore the long-term use of AvaliaDor in clinical contexts and its advantages for the assessment and management of patients with pain.

E-Health Tools to Improve Antibiotic Use and Resistances: A Systematic Review

(1) Background: e-Health tools, especially in the form of clinical decision support systems (CDSSs), have been emerging more quickly than ever before. The main objective of this systematic review is to assess the influence of these tools on antibiotic use for respiratory tract infections. (2) Methods: The scientific databases, MEDLINE-PubMed and EMBASE, were searched. The search was conducted by two independent researchers. The search strategy was mainly designed to identify relevant studies on the effectiveness of CDSSs in improving antibiotic use, as a primary outcome, and on the acceptability and usability of CDSSs, as a secondary outcome. (3) Results: After the selection, 22 articles were included. The outcomes were grouped either into antibiotics prescription practices or adherence to guidelines concerning antibiotics prescription. Overall, 15 out of the 22 studies had statistically significant outcomes related to the interventions. (4) Conclusions: Overall, the results show a positive impact on the prescription and conscientious use of antibiotics for respiratory tract infections, both with respect to patients and prescribing healthcare professionals. CDSSs have been shown to have great potential as powerful tools for improving both clinical care and patient outcomes.

Methodological Quality of Manuscripts Reporting on the Usability of Mobile Applications for Pain Assessment and Management: A Systematic Review

BACKGROUND

There has been increasing use of mobile mHealth applications, including pain assessment and pain self-management apps. The usability of mHealth applications has vital importance as it affects the quality of apps. Thus, usability assessment with methodological rigor is essential to minimize errors and undesirable consequences, as well as to increase user acceptance.

OBJECTIVE

this study aimed to synthesize and evaluate existing studies on the assessment of the usability of pain-related apps using a newly developed scale.

METHODS

an electronic search was conducted in several databases, combining relevant keywords. Then titles and abstracts were screened against inclusion and exclusion criteria. The eligible studies were retrieved and independently screened for inclusion by two authors. Disagreements were resolved by discussion until consensus was reached.

RESULTS

a total of 31 articles were eligible for inclusion. Quality assessment revealed that most manuscripts did not assess usability using valid instruments or triangulation of methods of usability assessment. Most manuscripts also failed to assess the three domains of usability (effectiveness, efficiency and satisfaction).

CONCLUSIONS

future studies should consider existing guidelines on usability assessment design, development and assessment of pain-related apps.