A Mobile Patient-Reported Outcome Measure App With Talking Touchscreen: Usability Assessment

Evaluating the usability of a cancer registry system using Cognitive Walkthrough, and assessing user agreement with its problems

OBJECTIVE/AIM

Good design of cancer registry systems makes them easy to use, while poor design of their user interfaces leads to user dissatisfaction and resistance. The objective of this study was to evaluate the usability of a cancer registry system using Cognitive Walkthrough (CW) and to assess users' agreement with its usability problems.

METHODS

CW was used to evaluate the registry system. We developed a checklist to help evaluators speed up the evaluation process, a problems form to collect the usability issues identified by the evaluators, and a problems severity form to determine the severity of problems by the evaluators. The problems were classified into two categories according to the CW questions and the system tasks. The agreement of the users with the system problems was examined by an online questionnaire. Users' agreement with the problems was then analyzed using the Interclass Correlation Coefficient in the SPSS 22 (Statistical Package for Social Science).

RESULTS

In this study, 114 problems were identified. In the categorization of problems based on the CW questions, 41% (n = 47) of the problems concerned the issue of "users do not know what to do at each stage of working with the system", 24% (n = 27) were classified as "users cannot link what they intend to do with system controls", and 22% (n = 25) were related to "user's lack of understanding of the system processes". Based on user tasks, about 36% (n = 41) of the problems were related to "removing patient duplication" and 33% (n = 38) were related to "registration of patient identification information". User agreement with the problems was high (CI 95% = 0.9 (0.96, 0.98)).

CONCLUSION

System problems often originate from user ignorance about what to do at each stage of using the system. Also, half of the system problems concern a mismatch between what users want to do and the system controls, or a lack of understanding about what the system does at different stages. Therefore, to avoid user confusion, designers should use clues and guides on the screen for users, design controls consistent with the user model of thinking, and provide appropriate feedback after each user action to help users understand what the system is doing. The high agreement of users with the problems showed that in the absence of users system designers can use CW to identify the problems that users face in the real environment.

User-Centered Development and Testing of the Online Patient-Reported Outcomes, Burdens, and Experiences (PROBE) Survey and the myPROBE App and Integration With the Canadian Bleeding Disorder Registry: Mixed Methods Study

Background

The Patient-Reported Outcomes, Burdens, and Experiences (PROBE) questionnaire is a tool for assessing the quality of life and disease burden in people living with hemophilia.

Objective

The objectives of our study were (1) to assess the needs of relevant stakeholders involved in the use of PROBE, (2) to develop the software infrastructure needed to meet these needs, and (3) to test the usability of the final product.

Methods

We conducted a series of semistructured interviews of relevant stakeholders, including PROBE investigators, people with hemophilia, and representatives of the sponsor. Based on these, we developed an online survey and a mobile app for iOS and Android. A user group evaluated the final product using the System Usability Scale (SUS) and an open feedback framework.

Results

The online survey was updated, and the myPROBE app for mobile devices and a new application programming interface were developed. The app was tested and modified according to user feedback over multiple cycles. The final version of the app was released in July 2019. Seventeen users aged 23 to 67 years evaluated the final version of the app using the SUS. The median (first, third quartile) SUS score for the app was 85 (68, 88) out of 100. The newly introduced functionalities were as follows: (1) capability to longitudinally track repeated fillings of the questionnaire at different time points by the same participant (as opposed to anonymous completion); (2) linking of the questionnaire with hemophilia registries, starting with the Canadian Bleeding Disorders Registry as a proof of concept; (3) removing or adding questions as needed; and (4) sending notifications to the users (eg, reminders). A new secure database was built for securely storing personal information separately from the questionnaire data. The PROBE online survey is currently available in 96 countries and 34 languages.

Conclusions

The online survey was updated successfully, and the myPROBE app was developed, with a SUS score of 85 (out of 100). The app has been released in 81 countries and 34 languages. This will facilitate data collection for research and advocacy purposes, and the use of this tool in everyday clinical practice.

Mobile app requirements for patients with rare liver diseases: A single center survey for the ERN RARE-LIVER‬‬‬

BACKGROUND

More patient data are needed to improve research on rare liver diseases. Mobile health apps enable an exhaustive data collection. Therefore, the European Reference Network on Hepatological diseases (ERN RARE-LIVER) intends to implement an app for patients with rare liver diseases communicating with a patient registry, but little is known about which features patients and their healthcare providers regard as being useful.

AIMS

This study aimed to investigate how an app for rare liver diseases would be accepted, and to find out which features are considered useful.

METHODS

An anonymous survey was conducted on adult patients with rare liver diseases at a single academic, tertiary care outpatient-service. Additionally, medical experts of the ERN working group on autoimmune hepatitis were invited to participate in an online survey.

RESULTS

In total, the responses from 100 patients with autoimmune (n = 90) or other rare (n = 10) liver diseases and 32 experts were analyzed. Patients were convinced to use a disease specific app (80%) and expected some benefit to their health (78%) but responses differed significantly between younger and older patients (93% vs. 62%, p < 0.001; 88% vs. 64%, p < 0.01). Comparing patients' and experts' feedback, patients more often expected a simplified healthcare pathway (e.g. 89% vs. 59% (p < 0.001) wanted access to one's own medical records), while healthcare providers saw the benefit mainly in improving compliance and treatment outcome (e.g. 93% vs. 31% (p < 0.001) and 70% vs. 21% (p < 0.001) expected the app to reduce mistakes in taking medication and improve quality of life, respectively).

CONCLUSION

Our results underline the great desire for disease-specific apps but also the need to involve patients and healthcare providers in the development of such apps in order to achieve long-term use and, thereby, improvements of patient care and research. The results of this study will help tremendously to implement the first cross-country app that communicates with an ERN patient registry.

Usability of the Turkish Translation of the Dutch Talking Touch Screen Questionnaire for Physical Therapy Patients With a Turkish Background: Qualitative Study

Background

The Turkish translation of the Dutch Talking Touch Screen Questionnaire (TTSQ) has been developed to help physical therapy patients with a Turkish background in the Netherlands to autonomously elucidate their health problems and impairments and set treatment goals, regardless of their level of health literacy.

Objective

The aim of this study was to evaluate the usability of the Turkish TTSQ for physical therapy patients with a Turkish background with diverse levels of health literacy and experience in using mobile technology.

Methods

The qualitative Three-Step Test-Interview method was carried out to gain insight into the usability of the Turkish TTSQ. A total of 10 physical therapy patients participated. The interview data were analyzed using a thematic content analysis approach aimed at determining the accuracy and completeness with which participants completed the questionnaire (effectiveness), the time it took participants to complete the questionnaire (efficiency), and the extent to which the participants were satisfied with the ease of use of the questionnaire (satisfaction). The problems encountered by the participants in this study were given a severity rating, which was used to provide a rough estimate of the need for additional usability improvements.

Results

No participant in this study was able to complete the questionnaire without encountering at least one usability problem. A total of 17 different kinds of problems were found. On the basis of their severity score, 3 problems that should be addressed during future development of the tool were “Not using the navigation function of the photo gallery in Question 4 causing the participant to not see all presented response items;” “Touching the text underneath a photo in Question 4 to select an activity instead of touching the photo itself, causing the activity not to be selected;” and “Pushing too hard or tapping too softly on the touch screen causing the touch screen to not respond.” The data on efficiency within this study were not valid and are, therefore, not reported in this study. No participant was completely satisfied or dissatisfied with the overall ease of use of the Turkish TTSQ. Two participants with no prior experience of using tablet computers felt that, regardless of what kinds of improvement might be made, it would just be too difficult for them to learn to work with the device.

Conclusions

As with the Dutch TTSQ, the Turkish TTSQ needs improvement before it can be released. The results of this study confirm the conclusion of the Dutch TTSQ study that participants with low levels of education and little experience in using mobile technology are less able to operate the TTSQ effectively. Using a Dutch speaking interviewer and Turkish interpreter has had a negative effect on data collection in this study.