Evaluating the Usability of the Wellness School Assessment Tool Whole School, Whole Community, Whole Child (WellSAT WSCC): A School Wellness Policy Assessment Tool

BACKGROUND

Adoption of the Whole School, Whole Community, Whole Child (WSCC) model has been slowed by a lack of available tools to support implementation. The Wellness School Assessment Tool (WellSAT) WSCC is an online assessment tool that allows schools to evaluate the alignment of their policies with the WSCC model. This study assesses the usability of the WellSAT WSCC.

METHODS

Using a convergent mixed methods design, we collected qualitative and quantitative data from 5 school-based participants with roles in development and evaluation of policy. Participants explored the platform while engaging in a think-aloud procedure and scored a sample policy using the platform. They also completed the System Usability Scale and responded to open-ended questions about the usability of the platform.

RESULTS

Participants rated the WellSAT WSCC as an above-average user experience, but data suggested several areas for improvement, including improved instructions, enhanced visual design of the platform, and guidance for subsequent policy changes.

CONCLUSION

The WellSAT WSCC provides an above-average user experience but can be improved to increase user experience. These improvements increase the potential for greater use to facilitate integration of the WSCC model into school policy.

Usability of an eHealth sleep education intervention for university students

Background

It has been estimated that more than one-third of university students suffer from insomnia. Few accessible eHealth sleep education programmes exist for university students and of the ones that do exist, fewer were developed using a user-centred approach, which allows for student input to be systematically collected and utilized to provide students with a programme that they consider to be easy to use and implement and to be effective. Better Nights, Better Days-Youth (BNBD-Youth) is a four-session eHealth sleep education programme designed for youth but previously only evaluated in younger adolescents (ages 14-18 years).

Aims

The purpose of this study is to evaluate the usability of the BNBD-Youth programme with university students using Morville's User Experience Honeycomb framework to determine if this programme would meet the needs of university students and if so what modifications would be needed.

Methods

Canadian undergraduate students (n = 46) completed the BNBD-Youth programme. Students completed online usability questionnaires based on the seven dimensions of Morville's User Experience Honeycomb (i.e. useful, usable, valuable, credible, desirable, accessible and findable) after each session and after completion of the programme. Open- and closed-ended questions were used to obtain both quantitative and qualitative responses.

Results

Average quantitative ratings were positive across user experience dimensions, ranging from 3.43 to 4.46 (out of 5). Qualitative responses indicated overall positive experiences with the programme. The only constructive feedback that met the criteria for revising the programme was to include more interactive features in Session 4.

Conclusions

This study demonstrates that university students found BNBD-Youth to be a usable programme for older youth. Demonstrating usability is an essential step in developing a programme with a user-centred design that university students will want to use in the future. Once the BNBD-Youth programme is revised to create the BNBD-University (BNBD-Uni) programme, additional usability and effectiveness testing will be conducted.

Methods for Human-Centered eHealth Development: Narrative Scoping Review

Background

Thorough holistic development of eHealth can contribute to a good fit among the technology, its users, and the context. However, despite the availability of frameworks, not much is known about specific research activities for different aims, phases, and settings. This results in researchers having to reinvent the wheel. Consequently, there is a need to synthesize existing knowledge on research activities for participatory eHealth development processes.

Objective

The 3 main goals of this review are to create an overview of the development strategies used in studies based on the CeHRes (Center for eHealth Research) Roadmap, create an overview of the goals for which these methods can be used, and provide insight into the lessons learned about these methods.

Methods

We included eHealth development studies that were based on the phases and/or principles of the CeHRes Roadmap. This framework was selected because of its focus on participatory, iterative eHealth design in context and to limit the scope of this review. Data were extracted about the type of strategy used, rationale for using the strategy, research questions, and reported information on lessons learned. The most frequently mentioned lessons learned were summarized using a narrative, inductive approach.

Results

In the included 160 papers, a distinction was made between overarching development methods (n=10) and products (n=7). Methods are used to gather new data, whereas products can be used to synthesize previously collected data and support the collection of new data. The identified methods were focus groups, interviews, questionnaires, usability tests, literature studies, desk research, log data analyses, card sorting, Delphi studies, and experience sampling. The identified products were prototypes, requirements, stakeholder maps, values, behavior change strategies, personas, and business models. Examples of how these methods and products were applied in the development process and information about lessons learned were provided.

Conclusions

This study shows that there is a plethora of methods and products that can be used at different points in the development process and in different settings. To do justice to the complexity of eHealth development, it seems that multiple strategies should be combined. In addition, we found no evidence for an optimal single step-by-step approach to develop eHealth. Rather, researchers need to select the most suitable research methods for their research objectives, the context in which data are collected, and the characteristics of the participants. This study serves as a first step toward creating a toolkit to support researchers in applying the CeHRes Roadmap to practice. In this way, they can shape the most suitable and efficient eHealth development process.

Chronic pain self-management in middle-aged and older adults: A collective intelligence approach to identifying barriers and user needs in eHealth interventions

Objectives

eHealth refers to health services and health information delivered or enhanced through the internet and related technologies. The number of eHealth interventions for chronic pain self-management is increasing. However, little evidence has been found for the overall efficacy of these interventions for older adults. The aim of the current study was to use a Collective Intelligence approach to identify the barriers and specific user needs of middle-aged and older adults using eHealth for chronic pain self-management.

Methods

A Collective Intelligence workshop was conducted with middle-aged and older adults to generate, clarify, select, and structure ideas in relation to barriers to eHealth use and specific design requirements for the purposes of chronic pain self-management. Prior to attending the workshop, participants received a trigger question requesting the identification of five barriers to eHealth use for chronic pain self-management. These barriers were categorised and presented to the group along with barrier-related scenarios and user need prompts, resulting in the generation of a set of ranked barriers and a set of user needs.

Results

A total of 78 barriers were identified, from which six categories emerged: Content, Support, Technological, Personal, Computer Literacy and Accessibility. Additional idea-writing and group reflection in response to these barriers revealed 97 user needs.

Conclusion

This is the first study to use Collective Intelligence methods to investigate barriers to eHealth technology use and the specific user needs of middle-aged and older adults in the context of chronic pain self-management. The results of the current study provide a platform for the design and development of enhanced eHealth interventions for this population.

Mapping the evidence and gaps of interventions for pediatric chronic pain to inform policy, research, and practice: A systematic review and quality assessment of systematic reviews

Background: Reviews in pediatric chronic pain often focus on only one intervention or population, making it difficult for policymakers and decision makers to quickly synthesize knowledge to inform larger-scale policy and funding priorities. Aims: The aims of this study were to (1) create an evidence and gap map of interventions for pediatric chronic pain and (2) identify gaps between existing evidence and recently identified patient-oriented research priorities. Methods: We performed a systematic review of English-language peer-reviewed systematic reviews or clinical practice guidelines of pediatric chronic pain intervention published in the past 20 years. Database searches of Medline, Embase, PsycINFO, Web of Science, CINAHL, and SCOPUS were conducted inclusive to June 3, 2019. Review quality was assessed using the AMSTAR-2. Results: Of 4168 unique abstracts, 50 systematic reviews (including 2 clinical practice guidelines) crossing diverse pediatric chronic pain populations and intervention settings were included. One third were rated high quality, with half rated low to critically low quality. The largest proportion of reviews addressed psychological and pharmacological interventions, followed by interdisciplinary, other (e.g., dietary), and physical interventions. Most common outcomes included pain, physical, emotional, and role functioning and quality of life. Treatment satisfaction and adverse events were less common, with minimal report of sleep or economic factors. Most patient-oriented research priorities had not been investigated. Conclusions: Sufficient quality evidence is available to guide evidence-informed policies in pediatric chronic pain, most notably regarding psychological and pharmacological interventions. Numerous evidence gaps in patient-oriented research priorities and treatment outcomes should guide prioritization of research funds, as well as study aims and design.