A Wearable Sensor-Based Exercise Biofeedback System: Mixed Methods Evaluation of Formulift

Clinical Utility and Usability of the Digital Box and Block Test: Mixed Methods Study

Background

The Box and Block Test (BBT) is a clinical tool used to measure hand dexterity, which is often used for tracking disease progression or the effectiveness of therapy, particularly benefiting older adults and those with neurological conditions. Digitizing the measurement of hand function may enhance the quality of data collection. We have developed and validated a prototype that digitizes this test, known as the digital BBT (dBBT), which automatically measures time and determines and displays the test result.

Objective

This study aimed to investigate the clinical utility and usability of the newly developed dBBT and to collect suggestions for future improvements.

Methods

A total of 4 occupational therapists participated in our study. To evaluate the clinical utility, we compared the dBBT to the BBT across dimensions such as acceptance, portability, energy and effort, time, and costs. We observed therapists using the dBBT as a dexterity measurement tool and conducted a quantitative usability questionnaire using the System Usability Scale (SUS), along with a focus group. Evaluative, structured, and qualitative content analysis was used for the qualitative data, whereas quantitative analysis was applied to questionnaire data. The qualitative and quantitative data were merged and analyzed using a convergent mixed methods approach.

Results

Overall, the results of the evaluative content analysis suggested that the dBBT had a better clinical utility than the original BBT, with ratings of all collected participant statements for the dBBT being 45% (45/99) equal to, 48% (48/99) better than, and 6% (6/99) lesser than the BBT. Particularly in the subcategories "acceptance," "time required for evaluation," and "purchase costs," the dBBT was rated as being better than the original BBT. The dBBT achieved a mean SUS score of 83 (95% CI 76-96). Additionally, several suggested changes to the system were identified.

Conclusions

The study demonstrated an overall positive evaluation of the clinical utility and usability of the dBBT. Valuable insights were gathered for future system iterations. These pioneering results highlight the potential of digitizing hand dexterity assessments.

Attributes, Methods, and Frameworks Used to Evaluate Wearables and Their Companion mHealth Apps: Scoping Review

BACKGROUND

Wearable devices, mobile technologies, and their combination have been accepted into clinical use to better assess the physical fitness and quality of life of patients and as preventive measures. Usability is pivotal for overcoming constraints and gaining users' acceptance of technology such as wearables and their companion mobile health (mHealth) apps. However, owing to limitations in design and evaluation, interactive wearables and mHealth apps have often been restricted from their full potential.

OBJECTIVE

This study aims to identify studies that have incorporated wearable devices and determine their frequency of use in conjunction with mHealth apps or their combination. Specifically, this study aims to understand the attributes and evaluation techniques used to evaluate usability in the health care domain for these technologies and their combinations.

METHODS

We conducted an extensive search across 4 electronic databases, spanning the last 30 years up to December 2021. Studies including the keywords "wearable devices," "mobile apps," "mHealth apps," "physiological data," "usability," "user experience," and "user evaluation" were considered for inclusion. A team of 5 reviewers screened the collected publications and charted the features based on the research questions. Subsequently, we categorized these characteristics following existing usability and wearable taxonomies. We applied a methodological framework for scoping reviews and the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) checklist.

RESULTS

A total of 382 reports were identified from the search strategy, and 68 articles were included. Most of the studies (57/68, 84%) involved the simultaneous use of wearables and connected mobile apps. Wrist-worn commercial consumer devices such as wristbands were the most prevalent, accounting for 66% (45/68) of the wearables identified in our review. Approximately half of the data from the medical domain (32/68, 47%) focused on studies involving participants with chronic illnesses or disorders. Overall, 29 usability attributes were identified, and 5 attributes were frequently used for evaluation: satisfaction (34/68, 50%), ease of use (27/68, 40%), user experience (16/68, 24%), perceived usefulness (18/68, 26%), and effectiveness (15/68, 22%). Only 10% (7/68) of the studies used a user- or human-centered design paradigm for usability evaluation.

CONCLUSIONS

Our scoping review identified the types and categories of wearable devices and mHealth apps, their frequency of use in studies, and their implementation in the medical context. In addition, we examined the usability evaluation of these technologies: methods, attributes, and frameworks. Within the array of available wearables and mHealth apps, health care providers encounter the challenge of selecting devices and companion apps that are effective, user-friendly, and compatible with user interactions. The current gap in usability and user experience in health care research limits our understanding of the strengths and limitations of wearable technologies and their companion apps. Additional research is necessary to overcome these limitations.

The Role and Importance of Using Sensor-Based Devices in Medical Rehabilitation: A Literature Review on the New Therapeutic Approaches

Due to the growth of sensor technology, more affordable integrated circuits, and connectivity technologies, the usage of wearable equipment and sensing devices for monitoring physical activities, whether for wellness, sports monitoring, or medical rehabilitation, has exploded. The current literature review was performed between October 2022 and February 2023 using PubMed, Web of Science, and Scopus in accordance with P.R.I.S.M.A. criteria. The screening phase resulted in the exclusion of 69 articles that did not fit the themes developed in all subchapters of the study, 41 articles that dealt exclusively with rehabilitation and orthopaedics, 28 articles whose abstracts were not visible, and 10 articles that dealt exclusively with other sensor-based devices and not medical ones; the inclusion phase resulted in the inclusion of 111 articles. Patients who utilise sensor-based devices have several advantages due to rehabilitating a missing component, which marks the accomplishment of a fundamental goal within the rehabilitation program. As technology moves faster and faster forward, the field of medical rehabilitation has to adapt to the time we live in by using technology and intelligent devices. This means changing every part of rehabilitation and finding the most valuable and helpful gadgets that can be used to regain lost functions, keep people healthy, or prevent diseases.

Alberta Rating Index for Apps: Study of Reliability and Validity

Background. The number of mobile health applications is rapidly increasing, yet no reliable tool exists for occupational therapists and their clients to rate the quality of these apps. Purpose. To develop the Alberta Rating Index for Apps (ARIA). Methods. Through a sequential design in three phases, we developed a rating index for mobile health applications and examined its reliability and validity with 10 participants. Findings. The coefficients of reliability were 0.95 for occupational therapists, 0.60 for older adults, and 0.88 for adults with a mental health condition. ARIA's correlation with another scale used in app review studies, U-MARS, was low to moderate. Implications. ARIA showed a high inter-rater reliability in two of the three user groups. ARIA is comprehensive and includes criteria not captured by U-MARS, such as privacy and security. Further studies are warranted with diverse raters and health apps.

Spanish adaptation and validation of the User Version of the Mobile Application Rating Scale (uMARS)

OBJECTIVE

While the professional version of the Mobile App Rating Scale (MARS) has already been translated, and validated into the Spanish language, its user-centered counterpart has not yet been adapted. Furthermore, no other similar tools exist in the Spanish language. The aim of this paper is to adapt and validate User Version of the MARS (uMARS) into the Spanish language.

MATERIALS AND METHODS

Cross-cultural adaptation, translation, and metric evaluation. The internal consistency and test-retest reliability of the Spanish version of the uMARS were evaluated using the RadarCovid app. Two hundred and sixteen participants rated the app using the translated scale. The app was then rated again 2 weeks later by 21 of these participants to measure test-retest reliability.

RESULTS

No major differences were observed between the uMARS original and the Spanish version. Discrimination indices (item-scale correlation) obtained appropriate results for both raters. The Spanish uMARS presented with excellent internal consistency, α = .89 and .67 for objective and subjective quality, respectively, and temporal stability (r > 0.82 for all items and subscales).

DISCUSSION

The Spanish uMARS is a useful tool for health professionals to recommend high-quality mobile apps to their patients based on the user's perspective and for researchers and app developers to use end-user feedback and evaluation, to help them identify highly appraised and valued components, as well as areas for further development, to continue ensuring the increasing quality and prominence of the area of mHealth.

CONCLUSION

uMARS Spanish version is an instrument with adequate metric properties to assess the quality of health apps from the user perspective.

A Biofeedback App for Migraine: Development and Usability Study

Background

Biofeedback is effective in treating migraines. It is believed to have a beneficial effect on autonomous nervous system activity and render individuals resilient to stressors that may trigger a migraine. However, widespread use of biofeedback is hampered by the need for a trained therapist and specialized equipment. Emerging digital health technology, including smartphones and wearables (mHealth), enables new ways of administering biofeedback. Currently, mHealth interventions for migraine appear feasible, but development processes and usability testing remain insufficient.

Objective

The objective of this study was to evaluate and improve the feasibility and usability of an mHealth biofeedback treatment app for adults with migraine.

Methods

In a prospective development and usability study, 18 adults with migraine completed a 4-week testing period of self-administered therapist-independent biofeedback treatment consisting of a smartphone app connected to wearable sensors (Cerebri, Nordic Brain Tech AS). The app included biofeedback training, instructions for self-delivery, and a headache diary. Two wearable sensors were used to measure surface electromyographic voltage at the trapezius muscle and peripheral skin temperature and heart rate at the right second fingertip. Participants were instructed to complete a daily headache diary entry and biofeedback session of 10 minutes duration. The testing period was preceded by a preusability expectation interview and succeeded by a postusability experience interview. In addition, an evaluation questionnaire was completed at weeks 2 and 4. Adherence was calculated as the proportion of 10-minute sessions completed within the first 28 days of treatment. Usability and feasibility were analyzed and summarized quantitatively and qualitatively.

Results

A total of 391 biofeedback sessions were completed with a median of 25 (IQR 17-28) per participant. The mean adherence rate was 0.76 (SD 0.26). The evaluation questionnaire revealed that functionality and design had the highest scores, whereas engagement and biofeedback were lower. Qualitative preexpectation analysis revealed that participants expected to become better familiar with physical signals and gain more understanding of their migraine attacks and noted that the app should be simple and understandable. Postusability analysis indicated that participants had an overall positive user experience with some suggestions for improvement regarding the design of the wearables and app content. The intervention was safe and tolerable. One case of prespecified adverse events was recorded in which a patient developed a skin rash from the sticky surface electromyography electrodes.

Conclusions

The app underwent a rigorous development process that indicated an overall positive user experience, good usability, and high adherence rate. This study highlights the value of usability testing in the development of mHealth apps.

The WOMEN-UP Solution, a Patient-Centered Innovative e-Health Tool for Pelvic Floor Muscle Training: Qualitative and Usability Study during Early-Stage Development

e-Health may enhance self-management of pelvic floor muscle training (PFMT) to treat stress urinary incontinence (SUI). It is crucial to involve patients in planning, developing and monitoring the optimal e-Health solution. This research aims to describe patient-centered innovation in an early developmental stage of the WOMEN-UP solution. We conducted a qualitative study through a self-developed questionnaire in 22 women with SUI, to define system requirements from a patient’s perspective. The first prototype of the WOMEN-UP solution was developed. It was tested by 9 patients in a usability study (think-aloud protocol and retrospective interviews). Patient preferences regarding the possible use of an e-Health solution with serious games for PFMT were: (1) to receive feedback about PFMT; (2) convenient home-use; (3) increasing motivation; (4) available in medical centers. Identified usability aids (31) reassured our design-development plan, which considered the biofeedback and serious games as key factors. Patient’s perspective detected some unexpected issues related to the calibration and serious games, involving a change in the ongoing development to get an improved WOMEN-UP solution; the value of patient-centered innovation during the development of an e-Health solution for PFMT (WOMEN-UP solution). To identify patients’ unmet needs, we proposed a longitudinal approach for the future eHealth-related patient-centered innovations.

ExerSense: Physical Exercise Recognition and Counting Algorithm from Wearables Robust to Positioning

Wearable devices are currently popular for fitness tracking. However, these general usage devices only can track limited and prespecified exercises. In our previous work, we introduced ExerSense that segments, classifies, and counts multiple physical exercises in real-time based on a correlation method. It also can track user-specified exercises collected only one motion in advance. This paper is the extension of that work. We collected acceleration data for five types of regular exercises by four different wearable devices. To find the best accurate device and its position for multiple exercise recognition, we conducted 50 times random validations. Our result shows the robustness of ExerSense, working well with various devices. Among the four general usage devices, the chest-mounted sensor is the best for our target exercises, and the upper-arm-mounted smartphone is a close second. The wrist-mounted smartwatch is third, and the worst one is the ear-mounted sensor.

Design of a Low-Cost, Wearable Device for Kinematic Analysis in Physical Therapy Settings

BACKGROUND

Unsupervised home exercise is a major component of physical therapy (PT). This study proposes an inexpensive, inertial measurement unit-based wearable device to capture kinematic data to facilitate exercise. However, conveying and interpreting kinematic data to non-experts poses a challenge due to the complexity and background knowledge required that most patients lack.

OBJECTIVES

The objectives of this study were to identify key user interface and user experience features that would likely improve device adoption and assess participant receptiveness toward the device.

METHODS

Fifty participants were recruited to perform nine upper extremity exercises while wearing the device. Prior to exercise, participants completed an orientation of the device, which included examples of software graphics with exercise data. Surveys that measured receptiveness toward the device, software graphics, and ergonomics were given before and after exercise.

RESULTS

Participants were highly receptive to the device with 90% of the participants likely to use the device during PT. Participants understood how the simple kinematic data could be used to aid exercise, but the data could be difficult to comprehend with more complex movements. Devices should incorporate wireless sensors and emphasize ease of wear.

CONCLUSION

Device-guided home physical rehabilitation can allow for individualized treatment protocols and improve exercise self-efficacy through kinematic analysis. Future studies should implement clinical testing to evaluate the impact a wearable device can have on rehabilitation outcomes.

Comparing the Usability and Acceptability of Wearable Sensors Among Older Irish Adults in a Real-World Context: Observational Study

Background

Wearable devices are valuable assessment tools for patient outcomes in contexts such as clinical trials. To be successfully deployed, however, participants must be willing to wear them. Another concern is that usability studies are rarely published, often fail to test devices beyond 24 hours, and need to be repeated frequently to ensure that contemporary devices are assessed.

Objective

This study aimed to compare multiple wearable sensors in a real-world context to establish their usability within an older adult (>50 years) population.

Methods

Eight older adults wore seven devices for a minimum of 1 week each: Actigraph GT9x, Actibelt, Actiwatch, Biovotion, Hexoskin, Mc10 Biostamp_RC, and Wavelet. Usability was established through mixed methods using semistructured interviews and three questionnaires, namely, the Intrinsic Motivation Inventory (IMI), the System Usability Scale (SUS), and an acceptability questionnaire. Quantitative data were reported descriptively and qualitative data were analyzed using deductive content analysis. Data were then integrated using triangulation.

Results

Results demonstrated that no device was considered optimal as all scored below average in the SUS (median, IQR; min-max=57.5, 12.5; 47.5-63.8). Hexoskin was the lowest scored device based on the IMI (3.6; 3.4-4.5), while Biovotion, Actibelt, and Mc10 Biostamp_RC achieved the highest median results on the acceptability questionnaire (3.6 on a 6-point Likert scale). Qualitatively, participants were willing to accept less comfort, less device discretion, and high charging burdens if the devices were perceived as useful, namely through the provision of feedback for the user. Participants agreed that the purpose of use is a key enabler for long-term compliance. These views were particularly noted by those not currently wearing an activity-tracking device. Participants believed that wrist-worn sensors were the most versatile and easy to use, and therefore, the most suitable for long-term use. In particular, Actiwatch and Wavelet stood out for their comfort. The convergence of quantitative and qualitative data was demonstrated in the study.

Conclusions

Based on the results, the following context-specific recommendations can be made: (1) researchers should consider their device selection in relation to both individual and environmental factors, and not simply the primary outcome of the research study; (2) if researchers do not wish their participants to have access to feedback from the devices, then a simple, wrist-worn device that acts as a watch is preferable; (3) if feedback is allowed, then it should be made available to help participants remain engaged; this is likely to apply only to people without cognitive impairments; (4) battery life of 1 week should be considered as a necessary feature to enhance data capture; (5) researchers should consider providing additional information about the purpose of devices to participants to support their continued use.

Measuring Mental Effort for Creating Mobile Data Collection Applications

To deal with drawbacks of paper-based data collection procedures, the QuestionSys approach empowers researchers with none or little programming knowledge to flexibly configure mobile data collection applications on demand. The mobile application approach of QuestionSys mainly pursues the goal to mitigate existing drawbacks of paper-based collection procedures in mHealth scenarios. Importantly, researchers shall be enabled to gather data in an efficient way. To evaluate the applicability of QuestionSys, several studies have been carried out to measure the efforts when using the framework in practice. In this work, the results of a study that investigated psychological insights on the required mental effort to configure the mobile applications are presented. Specifically, the mental effort for creating data collection instruments is validated in a study with N = 80 participants across two sessions. Thereby, participants were categorized into novices and experts based on prior knowledge on process modeling, which is a fundamental pillar of the developed approach. Each participant modeled 10 instruments during the course of the study, while concurrently several performance measures are assessed (e.g., time needed or errors). The results of these measures are then compared to the self-reported mental effort with respect to the tasks that had to be modeled. On one hand, the obtained results reveal a strong correlation between mental effort and performance measures. On the other, the self-reported mental effort decreased significantly over the course of the study, and therefore had a positive impact on measured performance metrics. Altogether, this study indicates that novices with no prior knowledge gain enough experience over the short amount of time to successfully model data collection instruments on their own. Therefore, QuestionSys is a helpful instrument to properly deal with large-scale data collection scenarios like clinical trials.

Wearable Sensor-Based Exercise Biofeedback for Orthopaedic Rehabilitation: A Mixed Methods User Evaluation of a Prototype System

The majority of wearable sensor-based biofeedback systems used in exercise rehabilitation lack end-user evaluation as part of the development process. This study sought to evaluate an exemplar sensor-based biofeedback system, investigating the feasibility, usability, perceived impact and user experience of using the platform. Fifteen patients participated in the study having recently undergone knee replacement surgery. Participants were provided with the system for two weeks at home, completing a semi-structured interview alongside the System Usability Scale (SUS) and user version of the Mobile Application Rating Scale (uMARS). The analysis from the SUS (mean = 90.8 [SD = 7.8]) suggests a high degree of usability, supported by qualitative findings. The mean adherence rate was 79% with participants reporting a largely positive user experience, suggesting it offers additional support with the rehabilitation regime. Overall quality from the mean uMARS score was 4.1 out of 5 (SD = 0.39), however a number of bugs and inaccuracies were highlighted along with suggestions for additional features to enhance engagement. This study has shown that patients perceive value in the use of wearable sensor-based biofeedback systems and has highlighted the benefit of user-evaluation during the design process, illustrated the need for real-world accuracy validation, and supports the ongoing development of such systems.

Wearables, Biomechanical Feedback, and Human Motor-Skills’ Learning & Optimization

Biomechanical feedback is a relevant key to improving sports and arts performance. Yet, the bibliometric keyword analysis on Web of Science publications reveals that, when comparing to other biofeedback applications, the real-time biomechanical feedback application lags far behind in sports and arts practice. While real-time physiological and biochemical biofeedback have seen routine applications, the use of real-time biomechanical feedback in motor learning and training is still rare. On that account, the paper aims to extract the specific research areas, such as three-dimensional (3D) motion capture, anthropometry, biomechanical modeling, sensing technology, and artificial intelligent (AI)/deep learning, which could contribute to the development of the real-time biomechanical feedback system. The review summarizes the past and current state of biomechanical feedback studies in sports and arts performance; and, by integrating the results of the studies with the contemporary wearable technology, proposes a two-chain body model monitoring using six IMUs (inertial measurement unit) with deep learning technology. The framework can serve as a basis for a breakthrough in the development. The review indicates that the vital step in the development is to establish a massive data, which could be obtained by using the synchronized measurement of 3D motion capture and IMUs, and that should cover diverse sports and arts skills. As such, wearables powered by deep learning models trained by the massive and diverse datasets can supply a feasible, reliable, and practical biomechanical feedback for athletic and artistic training.