Identifying and Categorizing Adverse Events in Trials of Digital Mental Health Interventions: Narrative Scoping Review of Trials in the International Standard Randomized Controlled Trial Number Registry

Bridging the gap from medical to psychological safety assessment: consensus study in a digital mental health context

BACKGROUND

Digital Mental Health Interventions (DMHIs) that meet the definition of a medical device are regulated by the Medicines and Healthcare products Regulatory Agency (MHRA) in the UK. The MHRA uses procedures that were originally developed for pharmaceuticals to assess the safety of DMHIs. There is recognition that this may not be ideal, as is evident by an ongoing consultation for reform led by the MHRA and the National Institute for Health and Care Excellence.

AIMS

The aim of this study was to generate an experts' consensus on how the medical regulatory method used for assessing safety could best be adapted for DMHIs.

METHOD

An online Delphi study containing three rounds was conducted with an international panel of 20 experts with experience/knowledge in the field of UK digital mental health.

RESULTS

Sixty-four items were generated, of which 41 achieved consensus (64%). Consensus emerged around ten recommendations, falling into five main themes: Enhancing the quality of adverse events data in DMHIs; Re-defining serious adverse events for DMHIs; Reassessing short-term symptom deterioration in psychological interventions as a therapeutic risk; Maximising the benefit of the Yellow Card Scheme; and Developing a harmonised approach for assessing the safety of psychological interventions in general.

CONCLUSION

The implementation of the recommendations provided by this consensus could improve the assessment of safety of DMHIs, making them more effective in detecting and mitigating risk.

Measurement of Adverse Events in Studies of Digital Health Interventions for Psychosis: Guidance and Recommendations Based on a Literature Search and Framework Analysis of Standard Operating Procedures

BACKGROUND

Given the rapid expansion of research into digital health interventions (DHIs) for severe mental illness (SMI; eg, schizophrenia and other psychosis diagnoses), there is an emergent need for clear safety measures. Currently, measurement and reporting of adverse events (AEs) are inconsistent across studies. Therefore, an international network, iCharts, was assembled to systematically identify and refine a set of standard operating procedures (SOPs) for AE reporting in DHI studies for SMI.

DESIGN

The iCharts network comprised experts on DHIs for SMI from seven countries (United Kingdom, Belgium, Germany, Pakistan, Australia, United States, and China) and various professional backgrounds. Following a literature search, SOPs of AEs were obtained from authors of relevant studies, and from grey literature.

RESULTS

A thorough framework analysis of SOPs (n = 32) identified commonalities for best practice for certain domains, along with significant gaps in others; particularly around the classification of AEs during trials, and the provision of training/supervision for research staff in measuring and reporting AEs. Several areas which could lead to the observed inconsistencies in AE reporting and handling were also identified.

CONCLUSIONS

The iCharts network developed best-practice guidelines and a practical resource for AE monitoring in DHI studies for psychosis, based on a systematic process which identified common features and evidence gaps. This work contributes to international efforts to standardize AE measurement and reporting in this emerging field, ensuring that safety aspects of DHIs for SMI are well-studied across the translational pathway, with monitoring systems set-up from the outset to support safe implementation in healthcare systems.

Adverse Events Reporting in Digital Interventions Evaluations for Psychosis: A Systematic Literature Search and Individual Level Content Analysis of Adverse Event Reports

BACKGROUND

Digital health interventions (DHIs) have significant potential to upscale treatment access to people experiencing psychosis but raise questions around patient safety. Adverse event (AE) monitoring is used to identify, record, and manage safety issues in clinical trials, but little is known about the specific content and context contained within extant AE reports. This study aimed to assess current AE reporting in DHIs.

STUDY DESIGN

A systematic literature search was conducted by the iCharts network (representing academic, clinical, and experts by experience) to identify trials of DHIs in psychosis. Authors were invited to share AE reports recorded in their trials. A content analysis was conducted on the shared reports.

STUDY RESULTS

We identified 593 AE reports from 18 DHI evaluations, yielding 19 codes. Only 29 AEs (4.9% of total) were preidentified by those who shared AEs as being related to the intervention or trial procedures. While overall results support the safety of DHIs, DHIs were linked to mood problems and psychosis exacerbation in a few cases. Additionally, 27% of studies did not report information on relatedness for all or at least some AEs; 9.6% of AE reports were coded as unclear because it could not be determined what had happened to participants.

CONCLUSIONS

The results support the safety of DHIs, but AEs must be routinely monitored and evaluated according to best practice. Individual-level analyses of AEs have merit to understand safety in this emerging field. Recommendations for best practice reporting in future studies are provided.

A digital health intervention to support patients with chronic pain during prescription opioid tapering: a pilot randomised controlled trial

Introduction

Recent changes in opioid prescribing guidelines have led to an increasing number of patients with chronic pain being recommended to taper. However, opioid tapering can be challenging, and many patients require support.

Objectives

We evaluated the feasibility, acceptability, and potential efficacy of a codesigned digital health intervention to support patients with chronic pain during voluntary prescription opioid tapering.

Methods

In a pilot randomised controlled trial, participants received a psychoeducational video and 28 days of text messages (2 SMS/day) in addition to their usual care (intervention) or usual care alone (control). The feasibility, acceptability, and potential efficacy of the intervention were evaluated. The primary outcome was opioid tapering self-efficacy. Secondary outcomes were pain intensity and interference, anxiety and depression symptom severity, pain catastrophising, and pain self-efficacy.

Results

Of 28 randomised participants, 26 completed the study (13 per group). Text message delivery was high (99.2%), but fidelity of video delivery was low (57.1%). Most participants rated the messages as useful, supportive, encouraging, and engaging; 78.5% would recommend the intervention to others; and 64.2% desired a longer intervention period. Tapering self-efficacy (Cohen d = 0.74) and pain self-efficacy (d = 0.41) were higher, and pain intensity (d = 0.65) and affective interference (d = 0.45) were lower in the intervention group at week 4.

Conclusion

First evidence supports the feasibility, acceptability, and potentially efficacy of a psychoeducational video and SMS text messaging intervention to support patients with chronic pain during voluntary prescription opioid tapering. Definitive trials with longer intervention duration are warranted.

Assessing the Potential Risks of Digital Therapeutics (DTX): The DTX Risk Assessment Canvas

MOTIVATION

Digital therapeutics (DTX), i.e., health interventions that are provided through digital means, are increasingly available for use; in some countries, physicians can even prescribe selected DTX following a reimbursement by health insurances. This results in an increasing need for methodologies to consider and monitor DTX's negative consequences, their risks to patient safety, and possible adverse events. However, it is completely unknown which aspects should be subject to surveillance given the missing experiences with the tools and their negative impacts.

OBJECTIVE

Our aim is to develop a tool-the DTX Risk Assessment Canvas-that enables researchers, developers, and practitioners to reflect on the negative consequences of DTX in a participatory process.

METHOD

Taking the well-established business model canvas as a starting point, we identified relevant aspects to be considered in a risk assessment of a DTX. The aspects or building blocks of the canvas were constructed in a two-way process: first, we defined the aspects relevant for discussing and reflecting on how a DTX might bring negative consequences and risks for its users by considering ISO/TS 82304-2, the scientific literature, and by reviewing existing DTX and their listed adverse effects. The resulting aspects were grouped into thematic blocks and the canvas was created. Second, six experts in health informatics and mental health provided feedback and tested the understandability of the initial canvas by individually applying it to a DTX of their choice. Based on their feedback, the canvas was modified.

RESULTS

The DTX Risk Assessment Canvas is organized into 15 thematic blocks which are in turn grouped into three thematic groups considering the DTX itself, the users of the DTX, and the effects of the DTX. For each thematic block, questions have been formulated to guide the user of the canvas in reflecting on the single aspects. Conclusions: The DTX Risk Assessment Canvas is a tool to reflect the negative consequences and risks of a DTX by discussing different thematic blocks that together constitute a comprehensive interpretation of a DTX regarding possible risks. Applied during the DTX design and development phase, it can help in implementing countermeasures for mitigation or means for their monitoring.

The Safety of Digital Mental Health Interventions: Systematic Review and Recommendations

BACKGROUND

Evidence suggests that digital mental health interventions (DMHIs) for common mental health conditions are effective. However, digital interventions, such as face-to-face therapies, pose risks to patients. A safe intervention is considered one in which the measured benefits outweigh the identified and mitigated risks.

OBJECTIVE

This study aims to review the literature to assess how DMHIs assess safety, what risks are reported, and how they are mitigated in both the research and postmarket phases and building on existing recommendations for assessing, reporting, and mitigating safety in the DMHI and standardizing practice.

METHODS

PsycINFO, Embase, and MEDLINE databases were searched for studies that addressed the safety of DMHIs. The inclusion criteria were any study that addressed the safety of a clinical DMHI, even if not as a main outcome, in an adult population, and in English. As the outcome data were mainly qualitative in nature, a meta-analysis was not possible, and qualitative analysis was used to collate the results. Quantitative results were synthesized in the form of tables and percentages. To illustrate the use of a single common safety metric across studies, we calculated odds ratios and CIs, wherever possible.

RESULTS

Overall, 23 studies were included in this review. Although many of the included studies assessed safety by actively collecting adverse event (AE) data, over one-third (8/23, 35%) did not assess or collect any safety data. The methods and frequency of safety data collection varied widely, and very few studies have performed formal statistical analyses. The main treatment-related reported AE was symptom deterioration. The main method used to mitigate risk was exclusion of high-risk groups. A secondary web-based search found that 6 DMHIs were available for users or patients to use (postmarket phase), all of which used indications and contraindications to mitigate risk, although there was no evidence of ongoing safety review.

CONCLUSIONS

The findings of this review show the need for a standardized classification of AEs, a standardized method for assessing AEs to statically analyze AE data, and evidence-based practices for mitigating risk in DMHIs, both in the research and postmarket phases. This review produced 7 specific, measurable, and achievable recommendations with the potential to have an immediate impact on the field, which were implemented across ongoing and future research. Improving the quality of DMHI safety data will allow meaningful assessment of the safety of DMHIs and confidence in whether the benefits of a new DMHI outweigh its risks.

TRIAL REGISTRATION

PROSPERO CRD42022333181; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=333181.