Acceptance of New Technology: A Usability Test of a Computerized Adaptive Test for Fatigue in Rheumatoid Arthritis

Examining Usability, Acceptability, and Adoption of a Self-Directed, Technology-Based Intervention for Upper Limb Rehabilitation After Stroke: Cohort Study

BACKGROUND

Upper limb (UL) recovery after stroke is strongly dependent upon rehabilitation dose. Rehabilitation technologies present pragmatic solutions to dose enhancement, complementing therapeutic activity within conventional rehabilitation, connecting clinicians with patients remotely, and empowering patients to drive their own recovery. To date, rehabilitation technologies have been poorly adopted. Understanding the barriers to adoption may shape strategies to enhance technology use and therefore increase rehabilitation dose, thus optimizing recovery potential.

OBJECTIVE

We examined the usability, acceptability, and adoption of a self-directed, exercise-gaming technology within a heterogeneous stroke survivor cohort and investigated how stroke survivor characteristics, technology usability, and attitudes toward technology influenced adoption.

METHODS

A feasibility study of a novel exercise-gaming technology for self-directed UL rehabilitation in early subacute stroke survivors (N=30) was conducted in an inpatient, acute hospital setting. Demographic and clinical characteristics were recorded; participants' performance in using the system (usability) was assessed using a 4-point performance rating scale (adapted from the Barthel index), and adherence with the system was electronically logged throughout the trial. The technology acceptance model was used to formulate a survey examining the acceptability of the system. Spearman rank correlations were used to examine associations between participant characteristics, user performance (usability), end-point technology acceptance, and intervention adherence (adoption).

RESULTS

The technology was usable for 87% (n=26) of participants, and the overall technology acceptance rating was 68% (95% CI 56%-79%). Participants trained with the device for a median of 26 (IQR 16-31) minutes daily over an enrollment period of 8 (IQR 5-14) days. Technology adoption positively correlated with user performance (usability) (ρ=0.55; 95% CI 0.23-0.75; P=.007) and acceptability as well as domains of perceived usefulness (ρ=0.42; 95% CI 0.09-0.68; P=.03) and perceived ease of use (ρ=0.46; 95% CI 0.10-0.74; P=.02). Technology acceptance decreased with increased global stroke severity (ρ=-0.56; 95% CI -0.79 to -0.22; P=.007).

CONCLUSIONS

This technology was usable and acceptable for the majority of the cohort, who achieved an intervention dose with technology-facilitated, self-directed UL training that exceeded conventional care norms. Technology usability and acceptability were determinants of adoption and appear to be mediated by stroke severity. The results demonstrate the importance of selecting technologies for stroke survivors on the basis of individual needs and abilities, as well as optimizing the accessibility of technologies for the target user group. Facilitating changes in stroke survivors' beliefs and attitudes toward rehabilitation technologies may enhance adoption. Further work is needed to understand how technology can be optimized to benefit those with more severe stroke.

A Smartphone App to Increase Immunizations in the Pediatric Solid Organ Transplant Population: Development and Initial Usability Study

Background

Vaccine-preventable infections result in significant morbidity, mortality, and costs in pediatric transplant recipients. However, at the time of transplant, less than 20% of children are up-to-date for age-appropriate immunizations that could prevent these diseases. Smartphone apps have the potential to increase immunization rates through their ability to provide vaccine education, send vaccine reminders, and facilitate communication between parents and a multidisciplinary medical group.

Objective

The aim of this study was to describe the development of a smartphone app, Immunize PediatricTransplant, to promote pretransplant immunization and to report on app functionality and usability when applied to the target population.

Methods

We used a mixed methods study design guided by the Mobile Health Agile Development and Evaluation Lifecycle. We first completed a formative research including semistructured interviews with transplant stakeholders (12 primary care physicians, 40 parents or guardians of transplant recipients, 11 transplant nurse coordinators, and 19 transplant subspecialists) to explore the acceptability of an immunization app to be used in the pretransplant period. Based on these findings, CANImmunize Inc developed the Immunize PediatricTransplant app. We next held 2 focus group discussions with 5-6 transplant stakeholders/group (n=11; 5 parents of transplant recipients, 2 primary care physicians, 2 transplant nurse coordinators, and 2 transplant subspecialists) to receive feedback on the app. After the app modifications were made, alpha testing was conducted on the functional prototype. We then implemented beta testing with 12 stakeholders (6 parents of transplant recipients, 2 primary care doctors, 2 transplant nurse coordinators, and 2 transplant subspecialists) to refine the app through an iterative process. Finally, the stakeholders completed the user version of the Mobile Application Rating Scale (uMARS) to assess the functionality and quality of the app.

Results

A new Android- and Apple-compatible app, Immunize PediatricTransplant, was developed to improve immunization delivery in the pretransplant period. The app contains information about vaccine use in the pretransplant period, houses a complete immunization record for each child, includes a communication tool for parents and care providers, and sends automated reminders to parents and care providers when immunizations are due. During usability testing, the stakeholders were able to enter a mock vaccine record containing 16 vaccines in an average of 8.1 minutes (SD 1.8) with 87% accuracy. The stakeholders rated engagement, functionality, aesthetics, and information quality of the app as 4.2/5, 4.5/5, 4.6/5, and 4.8/5, respectively. All participants reported that they would recommend this app to families and care teams with a child awaiting solid organ transplant.

Conclusions

Through a systematic, user-centered, agile, iterative approach, the Immunize PediatricTransplant app was developed to improve immunization delivery in the pretransplant period. The app tested well with end users. Further testing and agile development among patients awaiting transplant are needed to understand real-world acceptability and effectiveness in improving immunization rates in children awaiting transplant.

Usability of eHealth and Mobile Health Interventions by Young People Living With Juvenile Idiopathic Arthritis: Systematic Review

BACKGROUND

Considering the changing landscape of internet use and rising ownership of digital technology by young people, new methods could be considered to improve the current model of juvenile idiopathic arthritis (JIA) management.

OBJECTIVE

This systematic review aims to evaluate the usability of eHealth and mobile health (mHealth) interventions currently available for young people living with JIA.

METHODS

The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines were used to oversee this review. We systematically searched 15 databases for 252 potential studies; 2 authors independently screened all quantitative studies reporting the use of eHealth and mHealth interventions for young people (aged 1-18 years) diagnosed with JIA. Studies were excluded if they did not report outcome measures or were reviews, commentaries, or qualitative studies. Study methodological quality was scored using the Down and Black (modified) checklist. A narrative descriptive methodology was used to quantify the data because of heterogeneity across the studies.

RESULTS

A total of 11 studies were included in this review, reporting 7 eHealth and mHealth interventions for young people (aged 4-18 years) living with JIA, targeting health issues such as pain, health-related quality of life, physical activity, and chronic disease self-management. The usability of the interventions was facilitated through training and ongoing support. The engagement was promoted by a combination of persuasive influences, and barriers preventing adherence were removed through personal reminders and flexible program schedules to cater to JIA and non-JIA illnesses or other commonly seen activities in childhood. The feedback obtained was that most young people and their parents liked the interventions.

CONCLUSIONS

The results of this review need to be considered cautiously because of the lack of rigorous testing and heterogeneity, which limits the detailed descriptions of data synthesis. Further research is needed to consider gender differences, associated costs, and the effectiveness of the interventions on health outcomes to better support young people living with JIA.

Measuring Patient-Reported Outcomes Adaptively: Multidimensionality Matters!

As there is currently a marked increase in the use of both unidimensional (UCAT) and multidimensional computerized adaptive testing (MCAT) in psychological and health measurement, the main aim of the present study is to assess the incremental value of using MCAT rather than separate UCATs for each dimension. Simulations are based on empirical data that could be considered typical for health measurement: a large number of dimensions (4), strong correlations among dimensions (.77-.87), and polytomously scored response data. Both variable- (SE < .316, SE < .387) and fixed-length conditions (total test length of 12, 20, or 32 items) are studied. The item parameters and variance-covariance matrix Φ are estimated with the multidimensional graded response model (GRM). Outcome variables include computerized adaptive test (CAT) length, root mean square error (RMSE), and bias. Both simulated and empirical latent trait distributions are used to sample vectors of true scores. MCATs were generally more efficient (in terms of test length) and more accurate (in terms of RMSE) than their UCAT counterparts. Absolute average bias was highest for variable-length UCATs with termination rule SE < .387. Test length of variable-length MCATs was on average 20% to 25% shorter than test length across separate UCATs. This study showed that there are clear advantages of using MCAT rather than UCAT in a setting typical for health measurement.

An Online Hand Exercise Intervention for Adults With Rheumatoid Arthritis (mySARAH): Design, Development, and Usability Testing

BACKGROUND

The Strengthening and Stretching for Rheumatoid Arthritis of the Hand (SARAH) program is a tailored, progressive 12-week exercise program for people with hand problems due to rheumatoid arthritis. The program was shown to be clinically and cost-effective in a large clinical trial and is recommended by the UK National Institute for Health and Care Excellence (NICE) guidelines for rheumatoid arthritis in adults.

OBJECTIVE

We have developed an online version of the SARAH program (mySARAH) to make the SARAH program widely accessible to people with rheumatoid arthritis. The purposes of this study were to develop mySARAH and to evaluate and address its usability issues.

METHODS

We developed mySARAH using a three-step process and gaining feedback from patient contributors. After initial development, mySARAH was tested in two iterative usability cycles in nine participants using a simplified think-aloud protocol and self-reported questionnaires. We also evaluated if participants executed the SARAH exercises correctly after watching the exercise videos included on the website.

RESULTS

A preliminary version of mySARAH consisting of six sessions over a 12-week period and delivered via text, exercise videos, images, exercise plan form, exercise calendar, and links to additional information on rheumatoid arthritis was developed. Five participants (1 male; 4 females; median age 64 years) and four participants (four females; median age 64.5 years) took part in the first and second usability testing cycles respectively. Usability issues identified from Cycle 1 such as having a navigation tutorial video and individualised feedback on pain levels were addressed prior to Cycle 2. The need for more instructions to complete the mySARAH patient forms was identified in Cycle 2 and was rectified. Self-reports from both cycles indicated that participants found the program useful and easy to use and were confident in performing the SARAH exercises themselves. Eight of the nine participants correctly demonstrated all the exercises.

CONCLUSIONS

mySARAH is the first online hand exercise intervention for people with rheumatoid arthritis. We actively involved target users in the development and usability evaluation and ensured mySARAH met their needs and preferences.

Older Cancer Patients' User Experiences With Web-Based Health Information Tools: A Think-Aloud Study

Background

Health information is increasingly presented on the Internet. Several Web design guidelines for older Web users have been proposed; however, these guidelines are often not applied in website development. Furthermore, although we know that older individuals use the Internet to search for health information, we lack knowledge on how they use and evaluate Web-based health information.

Objective

This study evaluates user experiences with existing Web-based health information tools among older (≥ 65 years) cancer patients and survivors and their partners. The aim was to gain insight into usability issues and the perceived usefulness of cancer-related Web-based health information tools.

Methods

We conducted video-recorded think-aloud observations for 7 Web-based health information tools, specifically 3 websites providing cancer-related information, 3 Web-based question prompt lists (QPLs), and 1 values clarification tool, with colorectal cancer patients or survivors (n=15) and their partners (n=8) (median age: 73; interquartile range 70-79). Participants were asked to think aloud while performing search, evaluation, and application tasks using the Web-based health information tools.

Results

Overall, participants perceived Web-based health information tools as highly useful and indicated a willingness to use such tools. However, they experienced problems in terms of usability and perceived usefulness due to difficulties in using navigational elements, shortcomings in the layout, a lack of instructions on how to use the tools, difficulties with comprehensibility, and a large amount of variety in terms of the preferred amount of information. Although participants frequently commented that it was easy for them to find requested information, we observed that the large majority of the participants were not able to find it.

Conclusions

Overall, older cancer patients appreciate and are able to use cancer information websites. However, this study shows the importance of maintaining awareness of age-related problems such as cognitive and functional decline and navigation difficulties with this target group in mind. The results of this study can be used to design usable and useful Web-based health information tools for older (cancer) patients.

Theory-Based Design and Development of a Socially Connected, Gamified Mobile App for Men About Breastfeeding (Milk Man)

Background

Despite evidence of the benefits of breastfeeding, <15% of Australian babies are exclusively breastfed to the recommended 6 months. The support of the father is one of the most important factors in breastfeeding success, and targeting breastfeeding interventions to the father has been a successful strategy in previous research. Mobile technology offers unique opportunities to engage and reach populations to enhance health literacy and healthy behavior.

Objective

The objective of our study was to use previous research, formative evaluation, and behavior change theory to develop the first evidence-based breastfeeding app targeted at men. We designed the app to provide men with social support and information aiming to increase the support men can offer their breastfeeding partners.

Methods

We used social cognitive theory to design and develop the Milk Man app through stages of formative research, testing, and iteration. We held focus groups with new and expectant fathers (n=18), as well as health professionals (n=16), and used qualitative data to inform the design and development of the app. We tested a prototype with fathers (n=4) via a think-aloud study and the completion of the Mobile Application Rating Scale (MARS).

Results

Fathers and health professionals provided input through the focus groups that informed the app development. The think-aloud walkthroughs identified 6 areas of functionality and usability to be addressed, including the addition of a tutorial, increased size of text and icons, and greater personalization. Testers rated the app highly, and the average MARS score for the app was 4.3 out of 5.

Conclusions

To our knowledge, Milk Man is the first breastfeeding app targeted specifically at men. The development of Milk Man followed a best practice approach, including the involvement of a multidisciplinary team and grounding in behavior change theory. It tested well with end users during development. Milk Man is currently being trialed as part of the Parent Infant Feeding Initiative (ACTRN12614000605695).

Construct Validation of a Multidimensional Computerized Adaptive Test for Fatigue in Rheumatoid Arthritis

OBJECTIVE

Multidimensional computerized adaptive testing enables precise measurements of patient-reported outcomes at an individual level across different dimensions. This study examined the construct validity of a multidimensional computerized adaptive test (CAT) for fatigue in rheumatoid arthritis (RA).

METHODS

The 'CAT Fatigue RA' was constructed based on a previously calibrated item bank. It contains 196 items and three dimensions: 'severity', 'impact' and 'variability' of fatigue. The CAT was administered to 166 patients with RA. They also completed a traditional, multidimensional fatigue questionnaire (BRAF-MDQ) and the SF-36 in order to examine the CAT's construct validity. A priori criterion for construct validity was that 75% of the correlations between the CAT dimensions and the subscales of the other questionnaires were as expected. Furthermore, comprehensive use of the item bank, measurement precision and score distribution were investigated.

RESULTS

The a priori criterion for construct validity was supported for two of the three CAT dimensions (severity and impact but not for variability). For severity and impact, 87% of the correlations with the subscales of the well-established questionnaires were as expected but for variability, 53% of the hypothesised relations were found. Eighty-nine percent of the items were selected between one and 137 times for CAT administrations. Measurement precision was excellent for the severity and impact dimensions, with more than 90% of the CAT administrations reaching a standard error below 0.32. The variability dimension showed good measurement precision with 90% of the CAT administrations reaching a standard error below 0.44. No floor- or ceiling-effects were found for the three dimensions.

CONCLUSION

The CAT Fatigue RA showed good construct validity and excellent measurement precision on the dimensions severity and impact. The dimension variability had less ideal measurement characteristics, pointing to the need to recalibrate the CAT item bank with a two-dimensional model, solely consisting of severity and impact.

Adaptive Measurement of Well-Being: Maximizing Efficiency and Optimizing User Experience during Individual Assessment

Well-being is linked to important societal factors such as health care costs and productivity and has experienced a surge in development activity of both theories and measurement. This study builds on validation of the Well-Being 5 survey and for the first time applies Item Response Theory, a modern and flexible measurement paradigm, to form the basis of adaptive population well-being measurement. Adaptive testing allows survey questions to be administered selectively, thereby reducing the number of questions required of the participant. After the graded response model was fit to a sample of size N = 12,035, theta scores were estimated based on both the full-item bank and a simulation of Computerized Adaptive Testing (CAT). Comparisons of these 2 sets of score estimates with each other and of their correlations with external outcomes of job performance, absenteeism, and hospital admissions demonstrate that the CAT well-being scores maintain accuracy and validity. The simulation indicates that the average survey taker can expect a reduction in number of items administered during the CAT process of almost 50%. An increase in efficiency of this extent is of considerable value because of the time savings during the administration of the survey and the potential improvement of user experience, which in turn can help secure the success of a total population-based well-being improvement program. (Population Health Management 2016;19:284–290)

On the use of symmetrized dot patterns for the visual characterization of speech waveforms and other sampled data

While the spectrogram (and related graphic analyses) have been invaluable in showing the general frequency content of an input signal, sometimes it is difficult for trained and untrained users to see on the spectrogram differences which are perceptible to the ear. In this paper, several demonstrations of a novel representation are presented which, in some cases, can make subtle differences in input signals obvious to the human analyst. The representation, a "symmetrized dot pattern" (SDP), provides a stimulus in which local visual correlations are integrated to form a global percept and can potentially be applied to the detection and characterization of significant features of any sampled data.