Evaluation of a telerehabilitation system for community-based rehabilitation

Evaluation of Psychometric Properties of Thai Version Telehealth Usability Questionnaire (T-TUQ)

This cross-sectional validation study aimed to translate, cross-culturally adapt, and investigate the psychometric properties of a Thai version of the Telehealth Usability Questionnaire (T-TUQ). Two hundred and ten Thai participants, mean age of 61.2±15.2 years, were recruited from three specialty clinics: 50 (23.8%) hematology, 70 (33.3%) movement disorders, and 90 (42.9%) general neurology. The T-TUQ was translated from the original English version to produce a Thai language version. Back translation and pilot cognitive interviews were completed. All five subscales (usefulness, ease of use, effectiveness, reliability, and satisfaction) showed excellent internal consistency (alpha >0.80), displayed by Cronbach's alpha coefficient of 0.83, 0.94, 0.86, 0.83, and 0.92, respectively. For construct validity, exploratory factor analysis revealed two dimensions from eigenvalues and scree plot, defined as utility and accessibility subscales. In conclusion, the T-TUQ could be a reliable and valid instrument to evaluate the usability of telehealth with a Thai population.

A Scoping Review for Usage of Telerehabilitation among Older Adults with Mild Cognitive Impairment or Cognitive Frailty

Older adults are vulnerable towards cognitive frailty that can lead to adverse health outcomes and telerehabilitation appears to be a potential platform to reverse cognitive frailty among older adults. The aim of this coping review is to identify the usage of telerehabilitation and its common platform of delivery among older adults with mild cognitive impairment (MCI) or cognitive frailty (CF). Articles published from January 2015 until October 2020 were selected. Out of the 1738 articles retrieved, six studies were identified. Two articles were randomized controlled trials, one was a pilot study and three were qualitative studies. The outcome suggests that telerehabilitation may improve the quality of life among participants as well as it can be a useful and supportive digital platform for health care. Some types of technologies commonly used were smartphones or telephones with internet, television-based assistive integrated technology, mobile application and videoconference. Telerehabilitation utilization in managing cognitive frailty among older adults is still limited and more research is required to evaluate its feasibility and acceptability. Although telerehabilitation appears to be implemented among older adults with MCI and CF, some social support is still required to improve the adherence and effectiveness of telerehabilitation. Future research should focus on the evaluation of acceptance and participants’ existing knowledge towards telerehabilitation to achieve its target.

The most used questionnaires for evaluating telemedicine services

Background

Questionnaires are commonly used tools in telemedicine services that can help to evaluate different aspects. Selecting the ideal questionnaire for this purpose may be challenging for researchers. This study aims to review which questionnaires are used to evaluate telemedicine services in the studies, which are most common, and what aspects of telemedicine evaluation do they capture.

Methods

The PubMed database was searched in August 2020 to retrieve articles. Data extracted from the final list of articles included author/year of publication, journal of publication, type of evaluation, and evaluation questionnaire. Data were analyzed using descriptive statistics.

Results

Fifty-three articles were included in this study. The questionnaire was used for evaluating the satisfaction (49%), usability (34%), acceptance (11.5%), and implementation (2%) of telemedicine services. Among telemedicine specific questionnaires, Telehealth Usability Questionnaire (TUQ) (19%), Telemedicine Satisfaction Questionnaire (TSQ) (13%), and Service User Technology Acceptability Questionnaire (SUTAQ) (5.5%), were respectively most frequently used in the collected articles. Other most used questionnaires generally used for evaluating the users’ satisfaction, usability, and acceptance of technology were Client Satisfaction Questionnaire (CSQ) (5.5%), Questionnaire for User Interaction Satisfaction (QUIS) (5.5%), System Usability Scale (SUS) (5.5%), Patient Satisfaction Questionnaire (PSQ) (5.5%), and Technology Acceptance Model (TAM) (3.5%) respectively.

Conclusion

Employing specifically designed questionnaires or designing a new questionnaire with fewer questions and more comprehensiveness in terms of the issues studied provides a better evaluation. Attention to user needs, end-user acceptance, and implementation processes, along with users' satisfaction and usability evaluation, may optimize telemedicine efforts in the future.

Novel use of existing technology: A preliminary study of patient portal use for telerehabilitation

CLINICALTRIALS.GOV IDENTIFIER

NCT02665052. Registered 27 January 2016. https://clinicaltrials.gov/ct2/show/NCT02665052.

Measuring patient satisfaction with video consultation: a systematic review of assessment tools and their measurement properties

BACKGROUND

Video consultation (VC) is considered promising in delivering healthcare closer to the patient and improving patient satisfaction. Indeed, providing care-at-distance via VC is believed to be promising for some situations and patients, serving their needs without associated concomitant costs. In order to assess implementation and perceived benefits, patient satisfaction is frequently measured. Measuring patient satisfaction with VC in healthcare is often performed using quantitative and qualitative outcome analysis. As studies employ different surveys, pooling of data on the topic is troublesome. This systematic review critically appraises, summarizes, and compares available questionnaires in order to identify the most suitable questionnaire for qualitative outcome research using VC in clinical outpatient care.

METHODS

PubMed, Embase, and Cochrane were searched for relevant articles using predefined inclusion criteria. Methodological quality appraisal of yielded questionnaires to assess VC was performed using the validated COSMIN guideline.

RESULTS

This systematic search identified twelve studies that used ten different patient satisfaction questionnaires. The overall quality of nine questionnaires was rated as "inadequate" to "doubtful" according to the COSMIN criteria. None of the questionnaires retrieved completed a robust validation process for the purpose of use.

CONCLUSION AND RECOMMENDATIONS

Although high-quality studies on measurement properties of these questionnaires are scarce, the questionnaire developed by Mekhjian has the highest methodological quality achieving validity on internal consistency and the use of a large sample size. Moreover, this questionnaire can be used across healthcare settings. This finding may be instrumental in further studies measuring patient satisfaction with VC.

Usability of an Immersive Augmented Reality Based Telerehabilitation System with Haptics (ARTESH) for Synchronous Remote Musculoskeletal Examination

This study describes the features and utility of a novel augmented reality based telemedicine system with haptics that allows the sense of touch and direct physical examination during a synchronous immersive telemedicine consultation and physical examination. The system employs novel engineering features: (a) a new force enhancement algorithm to improve force rendering and overcoming the “just-noticeable-difference” limitation; (b) an improved force compensation method to reduce the delay in force rendering; (c) use of the “haptic interface point” to reduce disparity between the visual and haptic data; and (d) implementation of efficient algorithms to process, compress, decompress, transmit and render 3-D tele-immersion data. A qualitative pilot study (n=20) evaluated the usability of the system. Users rated the system on a 26-question survey using a seven-point Likert scale, with percent agreement calculated from the total users who agreed with a given statement. Survey questions fell into three main categories: (1) ease and simplicity of use, (2) quality of experience, and (3) comparison to in-person evaluation. Average percent agreements between the telemedicine and in-person evaluation were highest for ease and simplicity of use (86%) and quality of experience (85%), followed by comparison to in-person evaluation (58%). Eighty-nine percent (89%) of respondents expressed satisfaction with the overall quality of experience. Results suggest that the system was effective at conveying audio-visual and touch data in real-time across 20.3 miles, and warrants further development.

A Usability Study of 3 Radiotherapy Systems: A Comparative Evaluation Based on Expert Evaluation and User Experience

BACKGROUND The complex user interface design of radiotherapy treatment delivery systems can lead to use error and patient harm. In this study, we present the results of a comparison of 3 radiotherapy treatment delivery systems now used in China. MATERIAL AND METHODS We conducted a comprehensive usability study of 3 radiotherapy treatment delivery systems. Expert evaluation was performed through heuristic evaluation with 3 human-factors experts and 1 experienced radiation therapist for each system. User experience was assessed through perceived system usability and workload, using the National Aeronautics and Space Administration Task Load Index and the Post-Study System Usability Questionnaire. RESULTS For the expert evaluation, 47 usability problems were identified for Varian Trilogy, 75 for Elekta Precise, and 37 for Shinva XHA600E. Most problems were classified as major and minor usability problems, and were found in the process of patient setup and setup verification. For the user experience, radiation therapists presented a lower workload for Varian Trilogy compared to Elekta Precise (P<0.01) and Shinva XHA600E (P<0.01), and a lower workload for Elekta Precise compared to Shinva XHA600E (P=0.020). Radiation therapists perceived a higher system usability for Varian Trilogy compared to Shinva XHA600E (P<0.01), and a higher system usability for Elekta Precise compared to Shinva XHA600E (P<0.01). CONCLUSIONS This research provides valuable data on how 3 radiotherapy treatment delivery systems compare. The results of this study may be useful for hospital equipment procurement decisions, and designing next-generation products to improve patient safety.

An mHealth App for Users with Dexterity Impairments: Accessibility Study

BACKGROUND

A mobile health (mHealth) system called iMHere (interactive mobile health and rehabilitation) was developed to support individuals with chronic conditions and disability in their self-management regimens. The initial design of iMHere, however, lacked sufficient accessibility for users with a myriad of dexterity impairments. The accessibility of self-management apps is essential in ensuring usability.

OBJECTIVE

This study aims to increase the usability of the iMHere system for users with dexterity impairments by increasing the app's accessibility.

METHODS

We targeted the accessibility redesign by focusing on the physical presentation and the navigability of the iMHere apps. Six participants presenting with dexterity impairments were included in the usability study of the original and redesigned apps.

RESULTS

We observed a lower number of touches needed to complete tasks (P=.09) and time to complete individual tasks (P=.06) with the redesigned app than with the original app; a significantly lower time for users to complete all tasks (P=.006); and a significantly lower error rate (P=.01) with the redesigned app than with the original app. In fact, no errors occurred with use of the redesigned app. Participant-reported overall average usability of the redesigned app (P=.007) and usability of individual modules (P<.001) were significantly higher than that of the original app due mostly to better ease of use and learnability, interface quality, and reliability.

CONCLUSIONS

Improved usability was achieved using a redesigned app. This study offers insight into the importance of personalization in enhancing the accessibility and also identifies strategies for improving usability in app development.

Development of a Just-in-Time Adaptive mHealth Intervention for Insomnia: Usability Study

Background

Healthy sleep is a fundamental component of physical and brain health. Insomnia, however, is a prevalent sleep disorder that compromises functioning, productivity, and health. Therefore, developing efficient treatment delivery methods for insomnia can have significant societal and personal health impacts. Cognitive behavioral therapy for insomnia (CBTI) is the recommended first-line treatment of insomnia but access is currently limited for patients, since treatment must occur in specialty sleep clinics, which suffer from an insufficient number of trained clinicians. Smartphone-based interventions offer a promising means for improving the delivery of CBTI. Furthermore, novel features such as real-time monitoring and assessment, personalization, dynamic adaptations of the intervention, and context awareness can enhance treatment personalization and effectiveness, and reduce associated costs. Ultimately, this “Just in Time Adaptive Intervention” for insomnia—an intervention approach that is acceptable to patients and clinicians, and is based on mobile health (mHealth) platform and tools—can significantly improve patient access and clinician delivery of evidence-based insomnia treatments.

Objective

This study aims to develop and assess the usability of a Just in Time Adaptive Intervention application platform called iREST (“interactive Resilience Enhancing Sleep Tactics”) for use in behavioral insomnia interventions. iREST can be used by both patients and clinicians.

Methods

The development of iREST was based on the Iterative and Incremental Development software development model. Requirement analysis was based on the case study’s description, workflow and needs, clinician inputs, and a previously conducted BBTI military study/implementation of the Just in Time Adaptive Intervention architecture. To evaluate the usability of the iREST mHealth tool, a pilot usability study was conducted. Additionally, this study explores the feasibility of using an off-the-shelf wearable device to supplement the subjective assessment of patient sleep patterns.

Results

The iREST app was developed from the mobile logical architecture of Just in Time Adaptive Intervention. It consists of a cross-platform smartphone app, a clinician portal, and secure 2-way communications platform between the app and the portal. The usability study comprised 19 Active Duty Service Members and Veterans between the ages of 18 and 60. Descriptive statistics based on in-app questionnaires indicate that on average, 12 (mean 12.23, SD 8.96) unique devices accessed the clinician portal per day for more than two years, while the app was rated as “highly usable”, achieving a mean System Usability Score score of 85.74 (SD 12.37), which translates to an adjective rating of “Excellent”. The participants also gave high scores on “ease of use and learnability” with an average score of 4.33 (SD 0.65) on a scale of 1 to 5.

Conclusions

iREST provides a feasible platform for the implementation of Just in Time Adaptive Intervention in mHealth-based and remote intervention settings. The system was rated highly usable and its cross-platformness made it readily implemented within the heavily segregated smartphone market. The use of wearables to track sleep is promising; yet the accuracy of this technology needs further improvement. Ultimately, iREST demonstrates that mHealth-based Just in Time Adaptive Intervention is not only feasible, but also works effectively.

What Is the Proportion of Studies Reporting Patient and Practitioner Satisfaction with Software Support Tools Used in the Management of Knee Pain and Is This Related to Sample Size, Effect Size, and Journal Impact Factor?

INTRODUCTION

E-health software tools have been deployed in managing knee conditions. Reporting of patient and practitioner satisfaction in studies regarding e-health usage is not widely explored. The objective of this review was to identify studies describing patient and practitioner satisfaction with software use concerning knee pain.

MATERIALS AND METHODS

A computerized search was undertaken: four electronic databases were searched from January 2007 until January 2017. Keywords were decision dashboard, clinical decision, Web-based resource, evidence support, and knee. Full texts were scanned for effect of size reporting and satisfaction scales from participants and practitioners. Binary regression was run; impact factor and sample size were predictors with indicators for satisfaction and effect size reporting as dependent variables.

RESULTS

Seventy-seven articles were retrieved; 37 studies were included in final analysis. Ten studies reported patient satisfaction ratings (27.8%): a single study reported both patient and practitioner satisfaction (2.8%). Randomized control trials were the most common design (35%) and knee osteoarthritis the most prevalent condition (38%). Electronic patient-reported outcome measures and Web-based training were the most common interventions. No significant dependency was found within the regression models (p > 0.05).

DISCUSSION AND CONCLUSIONS

The proportion of reporting of patient satisfaction was low; practitioner satisfaction was poorly represented. There may be implications for the suitability of administering e-health, a medium for capturing further meta-evidence needs to be established and used as best practice for implicated studies in future. This is the first review of its kind to address patient and practitioner satisfaction with knee e-health.

A Commentary on Telerehabilitation Services in Pakistan: Current Trends and Future Possibilities

A 2014 World Health Organization (WHO) study reported that almost 27 million people with disability live in Pakistan with fewer than one allied rehabilitation professional per 10,000 people. The current study sought to determine the attitudes toward telerehabilitation via a survey administered to 329 Pakistani rehabilitation professionals. Study results indicate that rehabilitation professionals in Pakistan are knowledgeable about telerehabilitation and Information and Communication Technology (ICT), and are receptive to employing telerehabilitation programs and applications. Therefore, we can infer that the future of telerehabilitation can be bright in Pakistan but requires the attention of policy makers and non-government organizations to launch an appropriate program nationwide. The authors suggest that a range of telerehabilitation services (e.g., consultation, assessment, and therapy) could alleviate the shortage of rehabilitation personnel in Pakistan.

Towards Usable E-Health. A Systematic Review of Usability Questionnaires

BACKGROUND

The use of e-health can lead to several positive outcomes. However, the potential for e-health to improve healthcare is partially dependent on its ease of use. In order to determine the usability for any technology, rigorously developed and appropriate measures must be chosen.

OBJECTIVES

To identify psychometrically tested questionnaires that measure usability of e-health tools, and to appraise their generalizability, attributes coverage, and quality.

METHODS

We conducted a systematic review of studies that measured usability of e-health tools using four databases (Scopus, PubMed, CINAHL, and HAPI). Non-primary research, studies that did not report measures, studies with children or people with cognitive limitations, and studies about assistive devices or medical equipment were systematically excluded. Two authors independently extracted information including: questionnaire name, number of questions, scoring method, item generation, and psychometrics using a data extraction tool with pre-established categories and a quality appraisal scoring table.

RESULTS

Using a broad search strategy, 5,558 potentially relevant papers were identified. After removing duplicates and applying exclusion criteria, 35 articles remained that used 15 unique questionnaires. From the 15 questionnaires, only 5 were general enough to be used across studies. Usability attributes covered by the questionnaires were: learnability (15), efficiency (12), and satisfaction (11). Memorability (1) was the least covered attribute. Quality appraisal showed that face/content (14) and construct (7) validity were the most frequent types of validity assessed. All questionnaires reported reliability measurement. Some questionnaires scored low in the quality appraisal for the following reasons: limited validity testing (7), small sample size (3), no reporting of user centeredness (9) or feasibility estimates of time, effort, and expense (7).

CONCLUSIONS

Existing questionnaires provide a foundation for research on e-health usability. However, future research is needed to broaden the coverage of the usability attributes and psychometric properties of the available questionnaires.

Consumer Satisfaction with Telerehabilitation Service Provision of Alternative Computer Access and Augmentative and Alternative Communication

Telerehabilitation (TR) services for assistive technology evaluation and training have the potential to reduce travel demands for consumers and assistive technology professionals while allowing evaluation in more familiar, salient environments for the consumer. Sixty-five consumers received TR services for augmentative and alternative communication or alternative computer access, and consumer satisfaction was compared with twenty-eight consumers who received exclusively in-person services. TR recipients rated their TR services at a median of 6 on a 6-point Likert scale TR satisfaction questionnaire, although individual responses did indicate room for improvement in the technology. Overall satisfaction with AT services was rated highly by both in-person (100% satisfaction) and TR (99% satisfaction) service recipients.

The usability of ventilators: a comparative evaluation of use safety and user experience

Background

The design complexity of critical care ventilators (CCVs) can lead to use errors and patient harm. In this study, we present the results of a comparison of four CCVs from market leaders, using a rigorous methodology for the evaluation of use safety and user experience of medical devices.

Methods

We carried out a comparative usability study of four CCVs: Hamilton G5, Puritan Bennett 980, Maquet SERVO-U, and Dräger Evita V500. Forty-eight critical care respiratory therapists participated in this fully counterbalanced, repeated measures study. Participants completed seven clinical scenarios composed of 16 tasks on each ventilator.

Use safety was measured by percentage of tasks with use errors or close calls (UE/CCs). User experience was measured by system usability and workload metrics, using the Post-Study System Usability Questionnaire (PSSUQ) and the National Aeronautics and Space Administration Task Load Index (NASA-TLX).

Results

Nine of 18 post hoc contrasts between pairs of ventilators were significant after Bonferroni correction, with effect sizes between 0.4 and 1.09 (Cohen’s d). There were significantly fewer UE/CCs with SERVO-U when compared to G5 (p = 0.044) and V500 (p = 0.020). Participants reported higher system usability for G5 when compared to PB980 (p = 0.035) and higher system usability for SERVO-U when compared to G5 (p < 0.001), PB980 (p < 0.001), and V500 (p < 0.001). Participants reported lower workload for G5 when compared to PB980 (p < 0.001) and lower workload for SERVO-U when compared to PB980 (p < 0.001) and V500 (p < 0.001). G5 scored better on two of nine possible comparisons; SERVO-U scored better on seven of nine possible comparisons. Aspects influencing participants’ performance and perception include the low sensitivity of G5’s touchscreen and the positive effect from the quality of SERVO-U’s user interface design.

Conclusions

This study provides empirical evidence of how four ventilators from market leaders compare and highlights the importance of medical technology design. Within the boundaries of this study, we can infer that SERVO-U demonstrated the highest levels of use safety and user experience, followed by G5. Based on qualitative data, differences in outcomes could be explained by interaction design, quality of hardware components used in manufacturing, and influence of consumer product technology on users’ expectations.

Electronic supplementary material

The online version of this article (doi:10.1186/s13054-016-1431-1) contains supplementary material, which is available to authorized users.

The Use of Video Conferencing for Persons with Chronic Conditions: A Systematic Review

The purpose of this paper is to present a systematic review of studies that used Video Conferencing (VC) intervention for common chronic conditions. Chronic conditions account for the majority of poor health, disability, and death, and for a major portion of health-care expenditures in the United States. Innovative methods and interventions are needed to enhance care and management, improve access to care, improve patient outcomes, narrow health disparities and reduce health-care costs. Video Conferencing could be particularly relevant in improving health, care management, access and cost in the care of chronic illnesses. A comprehensive literature search process guided by the PRISMA statement led to the inclusion of 27 articles measuring video conferencing, at least one chronic illness, and patient outcomes for adults living in a community setting. While VC has been found to be feasible and effective, a low number of randomized controlled trials limit evidence. In addition, studies in this review were not designed to address the question of whether access to care in rural areas is improved through VC. Hence, more research is needed.

Feasibility of Telerehabilitation Implementation as a Novel Experience in Rehabilitation Academic Centers and Affiliated Clinics in Tehran: Assessment of Rehabilitation Professionals' Attitudes

Introduction. This study aimed to assess rehabilitation professionals' attitude toward implementation and application of telerehabilitation technology as a novel study in rehabilitation academic centers and affiliated clinics in Tehran. Methods. It was a descriptive cross-sectional study. To collect data, a researcher-designed questionnaire was developed. 141 rehabilitation experts participated in the study. Results. A majority of faculty members (78%) and clinicians (89.7%) either were in “definite agreement” or “somewhat agreed” with implementation and application of this technology, which demonstrates an overall positive attitude. Discussion. Based on the positive attitudes of the majority of participants toward implementation and application of this technology and their preferences in offering different telerehabilitation services, it seems that there is an appropriate and desirable acceptance and administrative culture to implement this technology among rehabilitation experts in Tehran. It is thus expected that implementation and application of this technology will be a promising experience in rehabilitation academic centers and affiliate clinics in Tehran.

Accessibility needs and challenges of a mHealth system for patients with dexterity impairments

Purpose State: An innovative mobile health system called iMHere (interactive Mobile Health and Rehabilitation) has been developed at the University of Pittsburgh to support self-care and adherence to self-care regimens for patients with chronic conditions. The goal of this study is to explore and to identify the accessibility needs and preferences of individuals with dexterity impairments when they use the iMHere system.

METHOD

Participants were asked to perform tasks after a one-week field trial. The time for a participant to complete each task, the number of possible errors a participant made and the number of errors a participant was able to self-correct were recorded and analyzed. The Telehealth Usability Questionnaire was collected to rate participants' experiences.

RESULTS

Nine participants with various levels of dexterity abilities were included in the study. A statistically significant negative correlation was identified between participants' dexterity levels and their error ratios by using a Pearson product-moment correlation (r = -0.434, n = 36, p = 0.004). Approximately 51% of errors detected was self-corrected without any help, but other errors called for resolution from a researcher.

CONCLUSIONS

Due to the diversity of participants' dexterity impairments, their needs and preferences differ one from another. Personalized design may be the key to approaching these challenges in improving accessibility. Implications for Rehabilitation An innovative mobile health system called iMHere has been developed to support self-care and adherence to self-care regimens as part of rehabilitation for patients with chronic conditions. Before populations with these limitations can harness the potential of mHealth, accessibility of mHealth has to be addressed to ensure its quality and value. Dexterity limitations are commonly associated with chronic disease, accidents or aging. The study presented here identified the potential issues and barriers to accessibility related to user interface components for persons with dexterity impairments. The findings should be of interest for those working with mobile health, accessibility, smartphone apps, wellness and self-care.