Engaging community stakeholders to evaluate the design, usability, and acceptability of a chronic obstructive pulmonary disease social media resource center

Towards co-design of rehabilitation technologies: a collaborative approach to prioritize usability issues

Introduction

Early stakeholder engagement is critical to the successful development and translation of rehabilitation technologies, a pivotal step of which is usability testing with intended end-users. To this end, several methods employ end-user feedback to identify usability and implementation issues. However, the process of prioritizing identified issues seldom leverages the knowledge and expertise of the range of stakeholders who will ultimately affect the demand and supply of a device. This paper describes a novel method to prioritize end-user feedback using transdisciplinary stakeholder consultation and address it in subsequent product development. The proposed approach was demonstrated using a case study relating to the development of a novel technology for neural recovery after spinal cord injury.

Method

Feedback from five individuals with chronic spinal cord injury was collected during two-hour usability evaluation sessions with a fully functional high-fidelity system prototype. A think-aloud and semi-structured interview protocol was used with each participant to identify usability and acceptability issues relating to the system in a 3-phase approach. Phase 1 involved extracting usability issues from think-aloud and semi-structured interview data. Phase 2 involved rating the usability issues based on their significance, technical feasibility, and implementation priority by relevant internal and external stakeholders. Finally, Phase 3 involved aggregating the usability issues according to design and implementation elements to facilitate solution generation, and these solutions were then raised as action tasks for future design iterations.

Results

Sixty usability issues representing nine facets of usability were rated. Eighty percent of issues were rated to be of moderate to high significance, 83% were rated as being feasible to address, and 75% were rated as addressable using existing project resources. Fifty percent of the issues were rated to be a high priority for implementation. Evaluation of the grouped issues identified 21 tasks which were mapped to the product roadmap for integration into future design iterations.

Discussion

This paper presents a method for meaningful transdisciplinary stakeholder engagement in rehabilitation technology development that can extended to other projects. Alongside a worked example, we offer practical considerations for others seeking to co-develop rehabilitation technologies.

Co-Design of a website for women with pelvic organ prolapse: A study protocol

Background

Despite high reported prevalence of pelvic organ prolapse (POP), women report difficulties accessing evidence-based and reliable information about the condition. Many rely on social media and other popular and highly visible internet platforms which have been found to contain poor quality information that is difficult for the average patient to understand. The aim of the study is to co-design an information website for premenopausal women with POP. The website design will be based on the Website Developmental Model for the Healthcare Consumer (WDMHC) framework.

Methods

A four phase process will be utilised as per the WDMHC framework: 1) User, task and environmental analysis; 2) Functional and representational analysis; 3) Cognitive walkthrough, keystroke level model, heuristic testing; 4) Content based testing, expert testing and user-based testing.Ethics approval has been obtained (LS-23-19-Carroll-Ful). Two groups of stakeholders will be recruited (i) patient group (ii) healthcare professional (HCP) group. Patient participants will be recruited from an online pelvic floor dysfunction (PFD) support group (n=950 members). A website designer and HCP stakeholders involved in the multidisciplinary team caring for women with POP will be invited to participate.Both groups will participate in separate co-design online workshops. Focus group workshops will be video-recorded, transcribed and imported into NVivo. Themes and subthemes will be developed.The website will be designed and disseminated to all participants for feedback. Cognitive walkthrough and heuristic testing will be undertaken. Following this, necessary modifications will be made to the website. Participants will then complete a modified System Usability Scale (SUS) and the eHealth Impact Questionnaire, while five HCPs will complete the DISCERN instrument.

Conclusion

This study will inform the design and testing of an information website for women with POP. The website design and content will be informed by patient and HCP stakeholder voices and the health literacy literature.

Barriers and facilitators of electronic patient portal uptake for asthma management

OBJECTIVE

An active patient-practitioner partnership is a key aspect of asthma management and patient-reported data helps with shared decision making. Technological advances such as the electronic patient portal can facilitate partnership, with the goal of improved asthma outcomes. However, uptake of portals by end-users for asthma management has been low. We studied portal-based asthma interventions to understand barriers and facilitators to its use.

DATA SOURCES

We searched within the PubMed, Web of Science, Scopus, MEDLINE, and Google Scholar databases.

STUDY SELECTIONS

We used the PRISMA extension for scoping reviews to guide our analysis of studies related to asthma and patient portals. We summarized relevant studies in terms of barriers and facilitators as well as study characteristics.

RESULTS

Sixteen studies were included in our final analysis. Common barriers to patient portal use for asthma management were lack of perceived value by the end-user, low end-user technological literacy, and limited resources. Facilitators of portal use included ease of use, personalization, and adequate technical support. Patient portals in these studies were used for a variety of applications related to core asthma management concepts of assessment and monitoring, education for a partnership in asthma care, environmental factors, co-morbidities, and medications.

CONCLUSIONS

Patient portal use for asthma management can be encouraged by ensuring the portal is easy to access and navigate, demonstrates values, as well as has readily available technical support. Involving end-users closely in the design process and implementation may help address barriers. Special attention is needed for groups with technological resource limitations.

#CRCFREE: Using Social Media to Reduce Colorectal Cancer Risk in Rural Adults

Objectives: In this study, we pilot-tested #CRCFree, a Facebook-based intervention aimed at reducing colorectal cancer (CRC) risk in rural Appalachian adults at risk for CRC. Methods: Participants were 56 rural Appalachian adults aged > 50 years. Daily #CRCFree Facebook posts addressed diet, physical activity, and CRC screening. Participants' sociodemographics, diet, body mass index, physical activity, and CRC screening status were measured pre- and post-intervention. The Healthy Eating Index (HEI) and the Dietary Inflammatory Index (DII) assessed dietary patterns. Facebook engagement was measured throughout the intervention. A post-intervention focus group evaluated intervention acceptability. Results: Participants were Caucasian, aged 58 ± 6 years, and predominantly female (66%). Post-intervention, HEI scores increased (49.9 ± 9.9 vs 58.6 ± 12.1, p = <.001), and DII scores decreased from baseline (2.8 ± 1.1 vs 1.6 ± 1.7, p = .002). There was no change in physical activity, BMI, or CRC screening status. Focus group participants found the intervention to be educational and motivating. Conclusions: These results provide preliminary evidence to support using Facebook to address CRC risk in this population. Participants were responsive to this intervention, and Facebook is a novel and accessible modality for health promotion.

Facebook Groups on Chronic Obstructive Pulmonary Disease: Social Media Content Analysis

Facebook Groups facilitate information exchange and engagement for patients with chronic conditions, including those living with Chronic Obstructive Pulmonary Disease (COPD); however, little is known about how knowledge is diffused throughout these communities. This study aimed to evaluate the content that is available on COPD-related Facebook Groups, as well as the communication (self-disclosures, social support) and engagement (agreement, emotional reaction) strategies used by members to facilitate these resources. Two researchers independently searched the "Groups" category using the terms "COPD", "emphysema", and "chronic bronchitis". Twenty-six closed (n = 23) and public (n = 3) COPD Facebook Groups were identified with 87,082 total members. The vast majority of Group members belonged to closed (n = 84,684; 97.25%) as compared to open (n = 2398; 2.75%) groups. Medications were the most commonly addressed self-management topic (n = 48; 26.7%). While overall engagement with wall posts was low, the number of "likes" (an indicator of agreement) was significantly greater for wall posts that demonstrated social support as compared to posts that did not (p < 0.001). Findings from this study showed that COPD Facebook group members share specific disease-related experiences and request information about select self-management topics. This information can be used to improve the quality of self-management support provided to members of popular COPD Facebook groups.

Automated analysis of free-text comments and dashboard representations in patient experience surveys: a multimethod co-design study

Background

Patient experience surveys (PESs) often include informative free-text comments, but with no way of systematically, efficiently and usefully analysing and reporting these. The National Cancer Patient Experience Survey (CPES), used to model the approach reported here, generates > 70,000 free-text comments annually.

Main aim

To improve the use and usefulness of PES free-text comments in driving health service changes that improve the patient experience.

Secondary aims

(1) To structure CPES free-text comments using rule-based information retrieval (IR) (‘text engineering’), drawing on health-care domain-specific gazetteers of terms, with in-built transferability to other surveys and conditions; (2) to display the results usefully for health-care professionals, in a digital toolkit dashboard display that drills down to the original free text; (3) to explore the usefulness of interdisciplinary mixed stakeholder co-design and consensus-forming approaches in technology development, ensuring that outputs have meaning for all; and (4) to explore the usefulness of Normalisation Process Theory (NPT) in structuring outputs for implementation and sustainability.

Design

A scoping review, rapid review and surveys with stakeholders in health care (patients, carers, health-care providers, commissioners, policy-makers and charities) explored clinical dashboard design/patient experience themes. The findings informed the rules for the draft rule-based IR [developed using half of the 2013 Wales CPES (WCPES) data set] and prototype toolkit dashboards summarising PES data. These were refined following mixed stakeholder, concept-mapping workshops and interviews, which were structured to enable consensus-forming ‘co-design’ work. IR validation used the second half of the WCPES, with comparison against its manual analysis; transferability was tested using further health-care data sets. A discrete choice experiment (DCE) explored which toolkit features were preferred by health-care professionals, with a simple cost–benefit analysis. Structured walk-throughs with NHS managers in Wessex, London and Leeds explored usability and general implementation into practice.

Key outcomes

A taxonomy of ranked PES themes, a checklist of key features recommended for digital clinical toolkits, rule-based IR validation and transferability scores, usability, and goal-oriented, cost–benefit and marketability results. The secondary outputs were a survey, scoping and rapid review findings, and concordance and discordance between stakeholders and methods.

Results

(1) The surveys, rapid review and workshops showed that stakeholders differed in their understandings of the patient experience and priorities for change, but that they reached consensus on a shortlist of 19 themes; six were considered to be core; (2) the scoping review and one survey explored the clinical toolkit design, emphasising that such toolkits should be quick and easy to use, and embedded in workflows; the workshop discussions, the DCE and the walk-throughs confirmed this and foregrounded other features to form the toolkit design checklist; and (3) the rule-based IR, developed using noun and verb phrases and lookup gazetteers, was 86% accurate on the WCPES, but needs modification to improve this and to be accurate with other data sets. The DCE and the walk-through suggest that the toolkit would be well accepted, with a favourable cost–benefit ratio, if implemented into practice with appropriate infrastructure support.

Limitations

Small participant numbers and sampling bias across component studies. The scoping review studies mostly used top-down approaches and focused on professional dashboards. The rapid review of themes had limited scope, with no second reviewer. The IR needs further refinement, especially for transferability. New governance restrictions further limit immediate use.

Conclusions

Using a multidisciplinary, mixed stakeholder, use of co-design, proof of concept was shown for an automated display of patient experience free-text comments in a way that could drive health-care improvements in real time. The approach is easily modified for transferable application.

Future work

Further exploration is needed of implementation into practice, transferable uses and technology development co-design approaches.

Funding

The National Institute for Health Research Health Services and Delivery Research programme.

Web-based Health Information Seeking and eHealth Literacy among Patients Living with Chronic Obstructive Pulmonary Disease (COPD)

Many people living with Chronic Obstructive Pulmonary Disease (COPD) have low general health literacy; however, there is little information available on these patients' eHealth literacy, or their ability to seek, find, understand, and appraise online health information and apply this knowledge to address or solve disease-related health concerns. A nationally representative sample of patients registered in the COPD Foundation's National Research Registry (N = 1,270) was invited to complete a web-based survey to assess socio-demographic (age, gender, marital status, education), health status (generic and lung-specific health-related quality of life), and socio-cognitive (social support, self-efficacy, COPD knowledge) predictors of eHealth literacy, measured using the 8-item eHealth literacy scale (eHEALS). Over 50% of the respondents (n = 176) were female (n = 89), with a mean age of 66.19 (SD = 9.47). Overall, participants reported moderate levels of eHealth literacy, with more than 70% feeling confident in their ability to find helpful health resources on the Internet. However, respondents were much less confident in their ability to distinguish between high- and low-quality sources of web-based health information. Very severe versus less severe COPD (β = 4.15), lower lung-specific health-related quality of life (β = -0.19), and greater COPD knowledge (β = 0.62) were significantly associated with higher eHealth literacy. Higher COPD knowledge was also significantly associated with greater knowledge (ρ = 0.24, p = .001) and use (ρ = 0.24, p = .001) of web-based health resources. Findings emphasize the importance of integrating skill-building activities into comprehensive patient education programs that enable patients with severe cases of COPD to identify high-quality sources of web-based health information. Additional research is needed to understand how new social technologies can be used to help medically underserved COPD patients benefit from web-based self-management support resources.

Speaking Up: How Patient and Physician Voices Shaped a Trial to Improve Goals-of-Care Discussions

BACKGROUND

Patients with advanced cancer benefit from early goals-of-care (GoC) conversations, but few facilitators are known.

OBJECTIVE

We describe the process and outcomes of involving patient and physician stakeholders in the design and development of a trial, funded by the Patient-Centered Outcomes Research Institute (PCORI), to enhance oncologists' communication skills and their propensity to facilitate productive, meaningful GoC discussions with patients with advanced cancer.

METHODS

We recruited oncologists, palliative care physicians, and patient stakeholders to participate in proposal development, intervention design and modification, identification of outcome measures, and refinement of study tools. Formats for exchange included 1:1 structured interviews, workshops, and stakeholder meetings.

RESULTS

Patient and physician voices helped craft and implement a study of an intervention to enhance oncologists' ability to facilitate GoC discussions with patients with advanced cancer. Physician inputs guided the creation of an oncologist and palliative care physician "joint visit" intervention at a turning point in disease management. Patient inputs impacted on the language used, outcome measures assessed, and approaches used to introduce patients to the intervention visit.

CONCLUSIONS

Stakeholder input informed the development of a novel intervention that physicians seemed to find both valuable and in sync with their needs and their practice schedules. Where communication about difficult subjects and shared decision making are involved, including multiple stakeholder groups in study design, implementation, and outcomes measurement may have far-reaching effects.

eHealth Literacy and Health Behaviors Affecting Modern College Students: A Pilot Study of Issues Identified by the American College Health Association

Background

The eHealth Literacy Scale (eHEALS) has been widely adopted by researchers to understand how eHealth literacy can be put into context. eHealth researchers need to know how to promote positive health behavior changes across college students, given the importance of the Internet to acquire and use health information. The American College Health Association identified a set of key health issues that affect college students today. By understanding how eHEALS might be related to college students’ maintenance of their health and their use of online health resources, researchers will be provided with a better understanding of eHealth literacy and its pragmatic implications for health campaigns and future interventions.

Objective

The goal of the study was to examine what eHEALS reveals about college student health behaviors identified by the American College Health Association. To understand college student current health maintenance and their intentions to maintain their health and use online resources, the theory of planned behavior was used as the theoretical framework for the study.

Methods

Data were collected via a survey of 422 college students that included the eHEALS measure and questions about health issues based on the recommendations of the American College Health Association. These questions asked about college student current health, subsequent use of online health resources, and their intention to maintain their health and make use of such resources in the future.

Results

eHEALS was positively and significantly associated with all 8 areas of health issues identified by the American College Health Association for college student current maintenance of health and use of online health resources and for future intention of health maintenance and use of online resources. Key issues that emerged with eHealth literacy were maintaining safe sex practices and seeking out related information, seeking out information on an exercise regime, information on vaccinations, and maintaining a balanced diet.

Conclusions

These results suggest several areas that may be targeted for future health campaigns toward college students. In addition, eHEALS was found to be a useful instrument for college students in the United States. Lastly, these results point to a need to deliver targeted information to college students, particularly since eHEALS captures literacy based on positively phrased items.

Living with asthma and chronic obstructive airways disease: Using technology to support self-management - An overview

Long-term respiratory conditions such as asthma and chronic obstructive pulmonary disease (COPD) are common, and cause high levels of morbidity and mortality. Supporting self-management is advocated for both asthma and increasingly so for COPD, and there is growing interest in the potential role of a range of new technologies, such as smartphone apps, the web or telehealth to facilitate and promote self-management in these conditions. Treatment goals for both asthma and COPD include aiming to control symptoms, maintain activities, achieve the best possible quality of life and minimize risks of exacerbation. To do this, health professionals should be (a) helping patients to recognize deteriorating symptoms and act appropriately; (b) promoting adherence to maintenance therapy; (c) promoting a regular review where triggers can be established, and strategies for managing such triggers discussed; and (d) promoting healthy lifestyles and positive self-management of symptoms. In particular, low uptake of asthma action plans is a modifiable contributor to morbidity and possibly also to mortality in those with asthma and should be addressed as a priority. Using technology to support self-management is an evolving strategy that shows promise. This review provides an overview of self-management support and discusses how newer technologies may help patients and health professionals to meet key treatment goals.

Towards Usable E-Health. A Systematic Review of Usability Questionnaires

BACKGROUND

The use of e-health can lead to several positive outcomes. However, the potential for e-health to improve healthcare is partially dependent on its ease of use. In order to determine the usability for any technology, rigorously developed and appropriate measures must be chosen.

OBJECTIVES

To identify psychometrically tested questionnaires that measure usability of e-health tools, and to appraise their generalizability, attributes coverage, and quality.

METHODS

We conducted a systematic review of studies that measured usability of e-health tools using four databases (Scopus, PubMed, CINAHL, and HAPI). Non-primary research, studies that did not report measures, studies with children or people with cognitive limitations, and studies about assistive devices or medical equipment were systematically excluded. Two authors independently extracted information including: questionnaire name, number of questions, scoring method, item generation, and psychometrics using a data extraction tool with pre-established categories and a quality appraisal scoring table.

RESULTS

Using a broad search strategy, 5,558 potentially relevant papers were identified. After removing duplicates and applying exclusion criteria, 35 articles remained that used 15 unique questionnaires. From the 15 questionnaires, only 5 were general enough to be used across studies. Usability attributes covered by the questionnaires were: learnability (15), efficiency (12), and satisfaction (11). Memorability (1) was the least covered attribute. Quality appraisal showed that face/content (14) and construct (7) validity were the most frequent types of validity assessed. All questionnaires reported reliability measurement. Some questionnaires scored low in the quality appraisal for the following reasons: limited validity testing (7), small sample size (3), no reporting of user centeredness (9) or feasibility estimates of time, effort, and expense (7).

CONCLUSIONS

Existing questionnaires provide a foundation for research on e-health usability. However, future research is needed to broaden the coverage of the usability attributes and psychometric properties of the available questionnaires.

Rotterdam Prostate Cancer Risk Calculator: Development and Usability Testing of the Mobile Phone App

Background

The use of prostate cancer screening tools that take into account relevant prebiopsy information (ie, risk calculators) is recommended as a way of determining the risk of cancer and the subsequent need for a prostate biopsy. This has the potential to limit prostate cancer overdiagnosis and subsequent overtreatment. mHealth apps are gaining traction in urological practice and are used by both practitioners and patients for a variety of purposes.

Objective

The impetus of the study was to design, develop, and assess a smartphone app for prostate cancer screening, based on the Rotterdam Prostate Cancer Risk Calculator (RPCRC).

Methods

The results of the Rotterdam arm of the European Randomized Study of Screening for Prostate Cancer (ERSPC) study were used to elaborate several algorithms that allowed the risk of prostate cancer to be estimated. A step-by-step workflow was established to ensure that depending on the available clinical information the most complete risk model of the RPCRC was used. The user interface was designed and then the app was developed as a native app for iOS. The usability of the app was assessed using the Post-Study System Usability Questionnaire (PSSUQ) developed by IBM, in a group of 92 participants comprising urologists, general practitioners, and medical students.

Results

A total of 11 questions were built into the app, and, depending on the answers, one of the different algorithms of the RPCRC could be used to predict the risk of prostate cancer and of clinically significant prostate cancer (Gleason score ≥7 and clinical stage >T2b). The system usefulness, information quality, and interface quality scores were high—92% (27.7/30), 87% (26.2/30), and 89% (13.4/15), respectively. No usability problems were identified.

Conclusions

The RPCRC app is helpful in predicting the risk of prostate cancer and, even more importantly, clinically significant prostate cancer. Its algorithms have been externally validated before and the usability score shows the app’s interface is well designed. Further usability testing is required in different populations to verify these results and ensure that it is easy to use, to warrant a broad appeal, and to provide better patient care.

Evaluating Comparative Effectiveness Research Priorities for Care Coordination in Chronic Obstructive Pulmonary Disease: A Community-Based eDelphi Study

BACKGROUND

Despite research supporting the use of care coordination in chronic obstructive pulmonary disease (COPD), there is relatively little known about the comparative effectiveness of different strategies used to organize care for patients. To investigate the most important COPD care coordination strategies, community-based stakeholder input is needed, especially from medically underserved populations. Web-based platforms are electronic tools now being used to bring together individuals from underrepresented populations to share input and obtain clarification on comparative effectiveness research (CER) ideas, questions, and hypotheses.

OBJECTIVE

Use low computer-literate, collaborative survey technology to evaluate stakeholder priorities for CER in COPD care coordination.

METHODS

A mixed-method, concurrent triangulation design was used to collect survey data from a virtual advisory board of community-based stakeholders including medically underserved patients with COPD, informal caregivers, clinicians, and research scientists. The eDelphi method was used to conduct 3 iterative rounds of Web-based surveys. In the first 2 survey rounds, panelists viewed a series of "mini research prospectus" YouTube video presentations and rated their level of agreement with the importance of 10 COPD care coordination topics using 7-point Likert scales. In the final third-round survey, panelists ranked (1=most important, 8=least important) and commented on 8 remaining topics that panelists favored most throughout the first 2 survey rounds. Following the third-round survey, panelists were asked to provide feedback on the potential impact of a Web-based stakeholder engagement network dedicated to improving CER in COPD.

RESULTS

Thirty-seven panelists rated the following care coordination topics as most important (lower means indicate greater importance): (1) measurement of quality of care (mean 2.73, SD 1.95); (2) management of COPD with other chronic health issues (mean 2.92, SD 1.67); (3) pulmonary rehabilitation as a model for care (mean 3.72; SD 1.93); (4) quality of care coordination (mean 4.12, SD 2.41); and (5) comprehensive COPD patient education (mean 4.27, SD 2.38). Stakeholder comments on the relative importance of these care coordination topics primarily addressed the importance of comparing strategies for COPD symptom management and evaluating new methods for patient-provider communication. Approximately one half of the virtual panel assembled indicated that a Web-based stakeholder engagement network could enable more online community meetings (n=19/37, 51%) and facilitate more opportunities to suggest, comment on, and vote for new CER ideas in COPD (n=18/37, 49%).

CONCLUSIONS

Members of this unique virtual advisory board engaged in a structured Web-based communication process that identified the most important community-specific COPD care coordination research topics and questions. Findings from this study support the need for more CER that evaluates quality of care measures used to assess the delivery of treatments and interventions among medically underserved patients with COPD.