Assessing Digital Health Implementation for a Pediatric Chronic Pain Intervention: Comparing the RE-AIM and BIT Frameworks Against Real-World Trial Data and Recommendations for Future Studies

A WHO digital intervention to address depression among young Chinese adults: a type 1 effectiveness-implementation randomized controlled trial

Common mental disorders among young people are rising globally. Current university-based interventions are inadequate to address the need for evidence-based interventions. We investigated the effectiveness and implementation of Step-by-Step (SbS), a WHO digital intervention to address depression, among Chinese university students with depressive symptoms. In this paper, we report a type 1 hybrid effectiveness-implementation randomized controlled trial conducted between September 2021 and September 2022. The control condition was enhanced treatment as usual (ETAU, psychoeducation). The primary outcome was improvement in depression symptoms. Secondary outcomes were improvements in psychological well-being, anxiety symptoms, and self-identified psychosocial problems. Effectiveness of the intervention was evaluated using generalized linear mixed models. Implementation outcomes were evaluated by thematic analysis of participant interviews. A total of 371 participants were enrolled to two treatment conditions in a 1:1 ratio. SbS resulted in a greater reduction in depressive symptoms at posttreatment (p = 0.004, Hedges' g = 0.35), but no significant difference between SbS and ETAU was observed at three-month follow-up (p = 0.179, Hedges' g = 0.16). The treatment effect was larger among those who adhered to the treatment (Hedges' gs = 0.59 and 0.30). Subjective well-being also improved for SbS at both time points (Hedges' gs = 0.31 and 0.30). In addition, SbS resulted in more improvement in anxiety symptoms at posttreatment (p = 0.029, Hedges' g = 0.26), but not at three-month follow-up (p = 0.265, Hedges' g = 0.13). The qualitative results demonstrated that the intervention was well-implemented as a self-help mental health service, with minimal support from peer supporters. In conclusion, Step-by-Step, a digital intervention developed by WHO, was effective in reducing depressive symptoms in the short term and improving psychological well-being in a longer term. The sustained effect on depression needs further investigation. Improving uptake and engagement in the program is needed for its scale-up implementation as a university-based mental health service for Chinese young adults. Trial registration: ChiCTR2100050214.

A sociotechnical framework to assess patient-facing eHealth tools: results of a modified Delphi process

Among the thousands of eHealth tools available, the vast majority do not get past pilot phases because they cannot prove value, and only a few have been systematically assessed. Although multiple eHealth assessment frameworks have been developed, these efforts face multiple challenges. This study aimed to address some of these challenges by validating and refining an initial list of 55 assessment criteria based on previous frameworks through a two-round modified Delphi process with in-between rounds of interviews. The expert panel (n = 57) included participants from 18 countries and 9 concerned parties. A consensus was reached on 46 criteria that were classified into foundational and contextual criteria. The 36 foundational criteria focus on evaluating the eHealth tool itself and were grouped into nine clusters: technical aspects, clinical utility and safety, usability and human centricity, functionality, content, data management, endorsement, maintenance, and developer. The 10 contextual criteria focus on evaluating the factors that vary depending on the context the tool is being evaluated for and were grouped into seven clusters: data-protection compliance, safety regulatory compliance, interoperability and data integration, cultural requirements, affordability, cost-benefit, and implementability. The classification of criteria into foundational and contextual helps us assess not only the quality of an isolated tool, but also its potential fit in a specific setting. Criteria subscales may be particularly relevant when determining the strengths and weaknesses of the tool being evaluated. This granularity enables different concerned parties to make informed decisions about which tools to consider according to their specific needs and priorities.

Assessing the Quality and Impact of eHealth Tools: Systematic Literature Review and Narrative Synthesis

BACKGROUND

Technological advancements have opened the path for many technology providers to easily develop and introduce eHealth tools to the public. The use of these tools is increasingly recognized as a critical quality driver in health care; however, choosing a quality tool from the myriad of tools available for a specific health need does not come without challenges.

OBJECTIVE

This review aimed to systematically investigate the literature to understand the different approaches and criteria used to assess the quality and impact of eHealth tools by considering sociotechnical factors (from technical, social, and organizational perspectives).

METHODS

A structured search was completed following the participants, intervention, comparators, and outcomes framework. We searched the PubMed, Cochrane, Web of Science, Scopus, and ProQuest databases for studies published between January 2012 and January 2022 in English, which yielded 675 results, of which 40 (5.9%) studies met the inclusion criteria. The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines and the Cochrane Handbook for Systematic Reviews of Interventions were followed to ensure a systematic process. Extracted data were analyzed using NVivo (QSR International), with a thematic analysis and narrative synthesis of emergent themes.

RESULTS

Similar measures from the different papers, frameworks, and initiatives were aggregated into 36 unique criteria grouped into 13 clusters. Using the sociotechnical approach, we classified the relevant criteria into technical, social, and organizational assessment criteria. Technical assessment criteria were grouped into 5 clusters: technical aspects, functionality, content, data management, and design. Social assessment criteria were grouped into 4 clusters: human centricity, health outcomes, visible popularity metrics, and social aspects. Organizational assessment criteria were grouped into 4 clusters: sustainability and scalability, health care organization, health care context, and developer.

CONCLUSIONS

This review builds on the growing body of research that investigates the criteria used to assess the quality and impact of eHealth tools and highlights the complexity and challenges facing these initiatives. It demonstrates that there is no single framework that is used uniformly to assess the quality and impact of eHealth tools. It also highlights the need for a more comprehensive approach that balances the social, organizational, and technical assessment criteria in a way that reflects the complexity and interdependence of the health care ecosystem and is aligned with the factors affecting users' adoption to ensure uptake and adherence in the long term.

Protocol for a multi-phase, multi-center, real-world, hybrid effectiveness-implementation study of a digital intervention for pediatric chronic pain co-designed with patients (Digital SPA)

Background

Children and adolescents with chronic pain are a vulnerable population who often lack the resources to manage their condition. Due to high personal, social, and economic consequences, proper management in its early stages is key to reducing disability. The aim of this project is to co-develop a digital intervention for pediatric chronic pain (Digital SPA) with end-users and to evaluate its effectiveness and implementation outcomes in Spain.

Methods

(Phase 1) Focus groups with patients, parents, and clinicians (n = 5-6 each) will inform about unmet pain care needs and provide a starting point for co-designing the intervention. (Phase 2) Content creation and usability testing will be based on the results of Phase 1, and the theory-driven development will follow the latest available evidence. The intervention will use validated psychological techniques focused on improving functioning by teaching pain coping skills. (Phase 3) Hybrid effectiveness-implementation trial. Participants (n = 195) will be adolescents aged 12-17 years old with chronic pain and one of their parents. Assessments include physical function, pain, sleep, anxiety, mood, satisfaction and adherence to the treatment, and number of visits to the emergency room. A qualitative framework analysis will be conducted with data from Phase 1. Effects of the intervention will be evaluated using linear multilevel modeling. The Consolidated Framework for Implementation Research (CFIR) and Behavioral Interventions Using Technology (BIT) frameworks will be used to evaluate implementation.

Discussion

This study is expected to produce a co-created evidence-based digital intervention for pediatric chronic pain and a roadmap for successful implementation.

Trial registration number TRN and date of registration

ClinicalTrials.gov (registered on 26 June 2023: https://clinicaltrials.gov/study/NCT05917626). Contributions to the literature The implementation of digital health interventions has two major gaps: (1) adherence to treatment is suboptimal, and (2) the process of making the interventions available to the end-user in a sustainable way is often unsuccessful.In this study, we expect that assessing users' needs and co-designing an intervention with them will improve adherence.Documenting the implementation process from the project inception and integrating the results into an implementation framework will allow for replication and extension in different contexts.This study will increase the knowledge about implementation in a vulnerable population: adolescents with chronic pain without access to in-person multidisciplinary pain care.

Impact of coronavirus disease 2019 on chronic pain structures: data from French national survey

Aims: The authors evaluated the impact of the first coronavirus disease 2019 pandemic wave on French chronic pain structures (CPSs). Methods: An online survey assessed CPS resource allocation, workflow and perceived impact on patient care. Results: All CPS workflow was severely impacted by the reallocation of 42% of specialists. In-person appointments were cancelled by 72% of participants. Follow-up was maintained in 91% of participants (telemedicine). Skills in end-of-life decision-making/counseling were rarely solicited. The perceived impact of the crisis on the experience of patients was high (eight out of ten), with a significant increase in access-to-care delay. Conclusion: CPSs maintained patient follow-up. Special features of CPS specialists were rarely solicited by coronavirus disease 2019 teams experiencing a high workload. Recommendations on optimal CPS resource reallocations have to be standardized in crisis conditions.

Rapid Evidence and Gap Map of virtual care solutions across a stepped care continuum for youth with chronic pain and their families in response to the COVID-19 pandemic

ABSTRACT

Poor access to pediatric chronic pain care is a longstanding concern. The COVID-19 pandemic has necessitated virtual care delivery at an unprecedented pace and scale. We conducted a scoping review to create an interactive Evidence and Gap Map of virtual care solutions across a stepped care continuum (ie, from self-directed to specialist care) for youth with chronic pain and their families. Review methodology was codesigned with 8 youth with chronic pain and 7 parents/caregivers. Data sources included peer-reviewed scientific literature, gray literature (app stores and web sites), and a call for innovations. Records were independently coded and assessed for quality. Overall, 185 records were included (105 scientific records, 56 apps, 16 web sites, and 8 innovations). Most virtual care solutions were applicable across pediatric chronic pain diagnoses, with the greatest proportion at lower levels of stepped care (ie, >100 self-guided apps and web sites). Virtual delivery of psychological strategies was common. Evidence gaps were noted at higher levels of stepped care (ie, requiring more resource and health professional involvement), integration with health records, communication with health professionals, web accessibility, and content addressing social/family support, medications, school, substance use, sleep, diet, and acute pain flares or crises. Evidence and Gap Maps are a novel visual knowledge synthesis tool, which enable rapid evidence-informed decision-making by patients and families, health professionals, and policymakers. This evidence and gap map identified high-quality virtual care solutions for immediate scale and spread and areas with no evidence in need of prioritization. Virtual care should address priorities identified by youth with chronic pain and their families.

A Cost-effectiveness Analysis of an Internet-delivered Pain Management Program Delivered With Different Levels of Clinician Support: Results From a Randomised Controlled Trial

There is growing interest in the potential of internet-delivered pain management programs (PMPs) to increase access to care for people with chronic pain. However, very few economic evaluations of these interventions have been reported. Using existing data, the current study examined the cost-effectiveness of an internet-delivered PMP for a mixed group chronic pain patients (n = 490) provided with different levels of clinician support. The findings indicated that each additional clinical outcome (defined as a ≥ 30% reduction in disability, depression, anxiety, and pain) was associated with cost-savings when the intervention was provided in a self-guided format (ICER range: -$404--$808 AUD) or an optional-guided format (ICER range: -$314--$541 AUD), and a relatively small fixed cost when provided in the clinician-guided format (ICER range: $88-$225 AUD). The results were driven by a reduction in service use costs among the treatment groups, which offset the costs of providing the internet-delivered PMP in the self-guided and optional-guided formats. The same general pattern of results was found when more stringent clinical outcomes (defined as a ≥ 50% reduction) were employed. These findings suggest that carefully developed and administered internet-delivered PMPs, provided with different levels of clinician support, can be highly cost effective for patients with a broad range of pain conditions. PERSPECTIVE: This study examines the cost-effectiveness of an internet-delivered PMP provided to adults with a broad range of chronic pain conditions. Evidence of cost-effectiveness was found across a broad range of clinical outcomes and with different levels of clinician support.