1
|
Hall CL, Gómez Bergin AD, Rennick-Egglestone S. Research Into Digital Health Intervention for Mental Health: 25-Year Retrospective on the Ethical and Legal Challenges. J Med Internet Res 2024; 26:e58939. [PMID: 39250796 PMCID: PMC11420603 DOI: 10.2196/58939] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2024] [Revised: 05/15/2024] [Accepted: 07/16/2024] [Indexed: 09/11/2024] Open
Abstract
Digital mental health interventions are routinely integrated into mental health services internationally and can contribute to reducing the global mental health treatment gap identified by the World Health Organization. Research teams designing and delivering evaluations frequently invest substantial effort in deliberating on ethical and legal challenges around digital mental health interventions. In this article, we reflect on our own research experience with digital mental health intervention design and evaluation to identify 8 of the most critical challenges that we or others have faced, and that have ethical or legal consequences. These include: (1) harm caused by online recruitment work; (2) monitoring of intervention safety; (3) exclusion of specific demographic or clinical groups; (4) inadequate robustness of effectiveness and cost-effectiveness findings; (5) adequately conceptualizing and supporting engagement and adherence; (6) structural barriers to implementation; (7) data protection and intellectual property; and (8) regulatory ambiguity relating to digital mental health interventions that are medical devices. As we describe these challenges, we have highlighted serious consequences that can or have occurred, such as substantial delays to studies if regulations around Software as a Medical Device (SaMD) are not fully understood, or if regulations change substantially during the study lifecycle. Collectively, the challenges we have identified highlight a substantial body of required knowledge and expertise, either within the team or through access to external experts. Ensuring access to knowledge requires careful planning and adequate financial resources (for example, paying public contributors to engage in debate on critical ethical issues or paying for legal opinions on regulatory issues). Access to such resources can be planned for on a per-study basis and enabled through funding proposals. However, organizations regularly engaged in the development and evaluation of digital mental health interventions should consider creating or supporting structures such as advisory groups that can retain necessary competencies, such as in medical device regulation.
Collapse
Affiliation(s)
- Charlotte L Hall
- Institute of Mental Health, Mental Health and Clinical Neurosciences, School of Medicine, University of Nottingham, Nottingham, United Kingdom
- National Institute for Health and Care Research (NIHR) Nottingham Biomedical Research Centre, Institute of Mental Health, University of Nottingham, Nottingham, United Kingdom
- National Institute for Health and Care Research (NIHR) MindTech HealthTech Research Centre, Institute of Mental Health, University of Nottingham, Nottingham, United Kingdom
| | - Aislinn D Gómez Bergin
- National Institute for Health and Care Research (NIHR) MindTech HealthTech Research Centre, Institute of Mental Health, University of Nottingham, Nottingham, United Kingdom
- Responsible AI UK, School of Computer Science, University of Nottingham, Nottingham, United Kingdom
| | - Stefan Rennick-Egglestone
- National Institute for Health and Care Research (NIHR) Nottingham Biomedical Research Centre, Institute of Mental Health, University of Nottingham, Nottingham, United Kingdom
- School of Health Sciences, Institute of Mental Health, University of Nottingham, Nottingham, United Kingdom
| |
Collapse
|
2
|
Slade E, Rennick-Egglestone S, Ng F, Kotera Y, Llewellyn-Beardsley J, Newby C, Glover T, Keppens J, Slade M. The Implementation of Recommender Systems for Mental Health Recovery Narratives: Evaluation of Use and Performance. JMIR Ment Health 2024; 11:e45754. [PMID: 38551630 PMCID: PMC11015364 DOI: 10.2196/45754] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/16/2023] [Revised: 09/11/2023] [Accepted: 02/15/2024] [Indexed: 04/02/2024] Open
Abstract
BACKGROUND Recommender systems help narrow down a large range of items to a smaller, personalized set. NarraGive is a first-in-field hybrid recommender system for mental health recovery narratives, recommending narratives based on their content and narrator characteristics (using content-based filtering) and on narratives beneficially impacting other similar users (using collaborative filtering). NarraGive is integrated into the Narrative Experiences Online (NEON) intervention, a web application providing access to the NEON Collection of recovery narratives. OBJECTIVE This study aims to analyze the 3 recommender system algorithms used in NarraGive to inform future interventions using recommender systems for lived experience narratives. METHODS Using a recently published framework for evaluating recommender systems to structure the analysis, we compared the content-based filtering algorithm and collaborative filtering algorithms by evaluating the accuracy (how close the predicted ratings are to the true ratings), precision (the proportion of the recommended narratives that are relevant), diversity (how diverse the recommended narratives are), coverage (the proportion of all available narratives that can be recommended), and unfairness (whether the algorithms produce less accurate predictions for disadvantaged participants) across gender and ethnicity. We used data from all participants in 2 parallel-group, waitlist control clinical trials of the NEON intervention (NEON trial: N=739; NEON for other [eg, nonpsychosis] mental health problems [NEON-O] trial: N=1023). Both trials included people with self-reported mental health problems who had and had not used statutory mental health services. In addition, NEON trial participants had experienced self-reported psychosis in the previous 5 years. Our evaluation used a database of Likert-scale narrative ratings provided by trial participants in response to validated narrative feedback questions. RESULTS Participants from the NEON and NEON-O trials provided 2288 and 1896 narrative ratings, respectively. Each rated narrative had a median of 3 ratings and 2 ratings, respectively. For the NEON trial, the content-based filtering algorithm performed better for coverage; the collaborative filtering algorithms performed better for accuracy, diversity, and unfairness across both gender and ethnicity; and neither algorithm performed better for precision. For the NEON-O trial, the content-based filtering algorithm did not perform better on any metric; the collaborative filtering algorithms performed better on accuracy and unfairness across both gender and ethnicity; and neither algorithm performed better for precision, diversity, or coverage. CONCLUSIONS Clinical population may be associated with recommender system performance. Recommender systems are susceptible to a wide range of undesirable biases. Approaches to mitigating these include providing enough initial data for the recommender system (to prevent overfitting), ensuring that items can be accessed outside the recommender system (to prevent a feedback loop between accessed items and recommended items), and encouraging participants to provide feedback on every narrative they interact with (to prevent participants from only providing feedback when they have strong opinions).
Collapse
Affiliation(s)
- Emily Slade
- School of Health Sciences, Institute of Mental Health, University of Nottingham, Nottingham, United Kingdom
- Department of Computer Science, University of Oxford, Oxford, United Kingdom
| | - Stefan Rennick-Egglestone
- School of Health Sciences, Institute of Mental Health, University of Nottingham, Nottingham, United Kingdom
| | - Fiona Ng
- School of Health Sciences, Institute of Mental Health, University of Nottingham, Nottingham, United Kingdom
| | - Yasuhiro Kotera
- School of Health Sciences, Institute of Mental Health, University of Nottingham, Nottingham, United Kingdom
- Center for Infectious Disease Education and Research, Osaka University, Osaka, Japan
| | - Joy Llewellyn-Beardsley
- School of Health Sciences, Institute of Mental Health, University of Nottingham, Nottingham, United Kingdom
| | - Chris Newby
- School of Medicine, University of Nottingham, Nottingham, United Kingdom
| | - Tony Glover
- DRT Software Ltd., Nottingham, United Kingdom
| | - Jeroen Keppens
- Department of Informatics, King's College London, London, United Kingdom
| | - Mike Slade
- School of Health Sciences, Institute of Mental Health, University of Nottingham, Nottingham, United Kingdom
- Nord University, Faculty of Nursing and Health Sciences, Namsos, Norway
| |
Collapse
|
3
|
Slade M, Rennick-Egglestone S, Elliott RA, Newby C, Robinson C, Gavan SP, Paterson L, Ali Y, Yeo C, Glover T, Pollock K, Callard F, Priebe S, Thornicroft G, Repper J, Keppens J, Smuk M, Franklin D, Walcott R, Harrison J, Smith R, Robotham D, Bradstreet S, Gillard S, Cuijpers P, Farkas M, Zeev DB, Davidson L, Kotera Y, Roe J, Ng F, Llewellyn-Beardsley J. Effectiveness and cost-effectiveness of online recorded recovery narratives in improving quality of life for people with non-psychotic mental health problems: a pragmatic randomized controlled trial. World Psychiatry 2024; 23:101-112. [PMID: 38214639 PMCID: PMC10785987 DOI: 10.1002/wps.21176] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/13/2024] Open
Abstract
Narratives describing first-hand experiences of recovery from mental health problems are widely available. Emerging evidence suggests that engaging with mental health recovery narratives can benefit people experiencing mental health problems, but no randomized controlled trial has been conducted as yet. We developed the Narrative Experiences Online (NEON) Intervention, a web application providing self-guided and recommender systems access to a collection of recorded mental health recovery narratives (n=659). We investigated whether NEON Intervention access benefited adults experiencing non-psychotic mental health problems by conducting a pragmatic parallel-group randomized trial, with usual care as control condition. The primary endpoint was quality of life at week 52 assessed by the Manchester Short Assessment (MANSA). Secondary outcomes were psychological distress, hope, self-efficacy, and meaning in life at week 52. Between March 9, 2020 and March 26, 2021, we recruited 1,023 participants from across England (the target based on power analysis was 994), of whom 827 (80.8%) identified as White British, 811 (79.3%) were female, 586 (57.3%) were employed, and 272 (26.6%) were unemployed. Their mean age was 38.4±13.6 years. Mood and/or anxiety disorders (N=626, 61.2%) and stress-related disorders (N=152, 14.9%) were the most common mental health problems. At week 52, our intention-to-treat analysis found a significant baseline-adjusted difference of 0.13 (95% CI: 0.01-0.26, p=0.041) in the MANSA score between the intervention and control groups, corresponding to a mean change of 1.56 scale points per participant, which indicates that the intervention increased quality of life. We also detected a significant baseline-adjusted difference of 0.22 (95% CI: 0.05-0.40, p=0.014) between the groups in the score on the "presence of meaning" subscale of the Meaning in Life Questionnaire, corresponding to a mean change of 1.1 scale points per participant. We found an incremental gain of 0.0142 quality-adjusted life years (QALYs) (95% credible interval: 0.0059 to 0.0226) and a £178 incremental increase in cost (95% credible interval: -£154 to £455) per participant, generating an incremental cost-effectiveness ratio of £12,526 per QALY compared with usual care. This was lower than the £20,000 per QALY threshold used by the National Health Service in England, indicating that the intervention would be a cost-effective use of health service resources. In the subgroup analysis including participants who had used specialist mental health services at baseline, the intervention both reduced cost (-£98, 95% credible interval: -£606 to £309) and improved QALYs (0.0165, 95% credible interval: 0.0057 to 0.0273) per participant as compared to usual care. We conclude that the NEON Intervention is an effective and cost-effective new intervention for people experiencing non-psychotic mental health problems.
Collapse
Affiliation(s)
- Mike Slade
- School of Health Sciences, Institute of Mental Health, University of Nottingham, Nottingham, UK
- Faculty of Nursing and Health Sciences, Health and Community Participation Division, Nord University, Namsos, Norway
| | | | - Rachel A Elliott
- Manchester Centre for Health Economics, Division of Population Health, Health Services Research & Primary Care, University of Manchester, Manchester, UK
| | - Chris Newby
- School of Medicine, University of Nottingham, Nottingham, UK
| | - Clare Robinson
- Centre for Evaluation and Methods, Wolfson Institute of Population Health, Pragmatic Clinical Trials Unit, Queen Mary University of London, London, UK
| | - Sean P Gavan
- Manchester Centre for Health Economics, Division of Population Health, Health Services Research & Primary Care, University of Manchester, Manchester, UK
| | - Luke Paterson
- Manchester Centre for Health Economics, Division of Population Health, Health Services Research & Primary Care, University of Manchester, Manchester, UK
| | - Yasmin Ali
- School of Health Sciences, Institute of Mental Health, University of Nottingham, Nottingham, UK
| | - Caroline Yeo
- School of Health Sciences, Institute of Mental Health, University of Nottingham, Nottingham, UK
- Department of Architecture and Built Environment, Faculty of Engineering, University of Nottingham, Nottingham, UK
| | | | - Kristian Pollock
- School of Health Sciences, University of Nottingham, Nottingham, UK
| | - Felicity Callard
- School of Geographical & Earth Sciences, University of Glasgow, Glasgow, UK
| | - Stefan Priebe
- Unit for Social and Community Psychiatry, East London NHS Foundation Trust, London, UK
| | - Graham Thornicroft
- Centre for Implementation Science and Centre for Global Mental Health, Health Service and Population Research Department, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, UK
| | | | - Jeroen Keppens
- Department of Informatics, King's College London, London, UK
| | - Melanie Smuk
- Centre for Genomics and Child Health, Blizard Institute, Queen Mary University of London, London, UK
| | | | - Rianna Walcott
- Black Communication and Technology Lab, Department of Communication, University of Maryland, College Park, MD, USA
| | | | - Roger Smith
- NEON Lived Experience Advisory Panel, Nottingham, UK
| | | | - Simon Bradstreet
- Institute of Health and Wellbeing, University of Glasgow, Glasgow, UK
| | - Steve Gillard
- School of Health Sciences, City, University of London, London, UK
| | - Pim Cuijpers
- Department of Clinical, Neuro and Developmental Psychology, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
- International Institute for Psychotherapy, Babes¸-Bolyai University, Cluj-Napoca, Romania
| | - Marianne Farkas
- Center for Psychiatric Rehabilitation, College of Health and Rehabilitation Sciences, Boston University, Boston, MA, USA
| | - Dror Ben Zeev
- School of Medicine, University of Washington, Seattle, WA, USA
| | - Larry Davidson
- Yale School of Medicine, Yale University, New Haven, CT, USA
| | - Yasuhiro Kotera
- School of Health Sciences, Institute of Mental Health, University of Nottingham, Nottingham, UK
- Center for Infectious Disease Education and Research, Osaka University, Osaka, Japan
| | - James Roe
- National Institute for Health and Care Research (NIHR) Applied Research Collaboration East Midlands, University of Nottingham, Nottingham, UK
| | - Fiona Ng
- School of Health Sciences, Institute of Mental Health, University of Nottingham, Nottingham, UK
| | - Joy Llewellyn-Beardsley
- School of Health Sciences, Institute of Mental Health, University of Nottingham, Nottingham, UK
| |
Collapse
|
4
|
Ng F, Rennick-Egglestone S, Onwumere J, Newby C, Llewellyn-Beardsley J, Yeo C, Ali Y, Pollock K, Kotera Y, Pomberth S, Gavan SP, van der Krieke L, Robotham D, Gillard S, Thornicroft G, Slade M. Pragmatic, feasibility randomized controlled trial of a recorded mental health recovery narrative intervention: narrative experiences online intervention for informal carers (NEON-C). Front Psychiatry 2024; 14:1272396. [PMID: 38323025 PMCID: PMC10845336 DOI: 10.3389/fpsyt.2023.1272396] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/04/2023] [Accepted: 12/14/2023] [Indexed: 02/08/2024] Open
Abstract
Introduction Informal carers of people with mental health problems often have unmet support needs. Mental health recovery narratives are increasingly accessible, but their relevance to and effect on informal carers have been minimally investigated. The Narrative Experiences Online (NEON) Intervention is a first-in-field intervention that provides informal carers with access to a diverse collection of recorded mental health recovery narratives. This trial aimed to examine the feasibility and acceptability of the NEON Intervention for informal carers. Methods This study involved a two-arm feasibility randomized controlled trial. Carers were randomly assigned to receiving versus not receiving the NEON Intervention. The feasibility aspects investigated included the acceptability of the intervention and of randomization, trial processes, engagement rates, recruitment procedures, attrition, sample size estimation, identification of candidate primary and secondary outcomes, and the feasibility of conducting a definitive trial. A qualitative process evaluation was conducted. Findings A total of 121 carers were eligible, of whom 54 were randomized (intervention: 27, control: 27). Twelve-month follow-up data were available for 36 carers. Carers accessed a mean of 25 narratives over a 12-month period, and the intervention group, compared with the control group, reported a small effect on hope and a moderate effect on the presence of meaning in life. Five modifications were recommended to improve the user experience, applicability, and trial processes. Discussion The NEON Intervention is feasible and acceptable. Significant refinement of the NEON Intervention and trial processes is required to personalize and ensure applicability to carers. Further feasibility testing is recommended prior to a definitive trial.
Collapse
Affiliation(s)
- Fiona Ng
- School of Health Sciences, Institute of Mental Health, University of Nottingham, Nottingham, United Kingdom
| | - Stefan Rennick-Egglestone
- School of Health Sciences, Institute of Mental Health, University of Nottingham, Nottingham, United Kingdom
| | - Juliana Onwumere
- Department of Psychology, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, United Kingdom
- South London and Maudsley NHS Foundation Trust, Bethlem Royal Hospital, Kent, United Kingdom
| | - Christopher Newby
- School of Health Sciences, Institute of Mental Health, University of Nottingham, Nottingham, United Kingdom
| | - Joy Llewellyn-Beardsley
- School of Health Sciences, Institute of Mental Health, University of Nottingham, Nottingham, United Kingdom
| | - Caroline Yeo
- School of Health Sciences, Institute of Mental Health, University of Nottingham, Nottingham, United Kingdom
| | - Yasmin Ali
- School of Health Sciences, Institute of Mental Health, University of Nottingham, Nottingham, United Kingdom
| | - Kristian Pollock
- School of Health Sciences, University of Nottingham, Nottingham, United Kingdom
| | - Yasuhiro Kotera
- School of Health Sciences, Institute of Mental Health, University of Nottingham, Nottingham, United Kingdom
| | - Scott Pomberth
- NEON Lived Experience Advisory Panel, Nottingham, United Kingdom
| | - Sean P. Gavan
- Manchester Centre for Health Economics, Division of Population Health, Health Services Research and Primary Care, The University of Manchester, Manchester, United Kingdom
| | - Lian van der Krieke
- University Medical Center Groningen, University Center of Psychiatry, University of Groningen, Groningen, Netherlands
| | | | - Steve Gillard
- School of Health & Psychological Sciences, City, University of London, London, United Kingdom
| | - Graham Thornicroft
- Centre for Implementation Science, Health Service and Population Research Department, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, United Kingdom
| | - Mike Slade
- School of Health Sciences, Institute of Mental Health, University of Nottingham, Nottingham, United Kingdom
- Health and Community Participation Division, Faculty of Nursing and Health Sciences, Nord University, Namsos, Norway
| | | |
Collapse
|
5
|
Rennick-Egglestone S, Newby C, Robinson C, Yeo C, Ng F, Elliott R, Ali Y, Llewellyn-Beardsley J, Pomberth S, Harrison J, Gavan SP, Cuijpers P, Priebe S, Hall C, Slade M. Differences between online trial participants who have used statutory mental health services and who have not: analysis of baseline data from two pragmatic trials of a digital health intervention (Preprint). J Med Internet Res 2022. [DOI: 10.2196/44687] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/01/2023] Open
|
6
|
Rennick-Egglestone S, Elliott R, Newby C, Robinson C, Slade M. Impact of receiving recorded mental health recovery narratives on quality of life in people experiencing non-psychosis mental health problems (NEON-O Trial): updated randomised controlled trial protocol. Trials 2022; 23:90. [PMID: 35093141 PMCID: PMC8800358 DOI: 10.1186/s13063-022-06027-z] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/27/2020] [Accepted: 01/13/2022] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Mental health recovery narratives are first-person lived experience accounts of recovery from mental health problems, which refer to events or actions over a period of time, and which include elements of adversity or struggle, and also self-defined or observable strengths, successes, or survival. Recorded recovery narratives are those presented in invariant form, including text, audio, or video. In a previous publication, we presented a protocol for three pragmatic trials of the Narrative Experiences Online (NEON) Intervention, a web application recommending recorded recovery narratives to participants. The aim of the definitive NEON Trial was to understand whether the NEON Intervention benefitted people with experience of psychosis. The aim of the smaller NEON-O and NEON-C trials was to evaluate the feasibility of conducting definitive trials of the NEON Intervention with people (1) experiencing non-psychosis mental health problems and (2) who informally care for others experiencing mental health problems. An open recruitment strategy with a 60-week recruitment period was developed. Recruitment for the NEON Trial and NEON-O Trial targeted mental health service users and people not using mental health services. The NEON Trial recruited to time and target. The NEON-O Trial achieved its target in 10 weeks. Analysis considered by a Programme Steering Committee after the target was achieved demonstrated a definitive result could be obtained if the trial was adapted for recruitment to continue. The UK Health Research Authority approved all needed amendments following ethical review. PURPOSE OF THIS ARTICLE To describe the decision-making process for amending the NEON-O Trial and to describe amendments made to the NEON-O Trial to enable a definitive result. The article describes amendments to the aims, objectives, design, power calculation, recruitment rate, process evaluation design, and informed consent documents. The extended NEON-O Trial adopts analysis principles previously specified for the NEON Trial. The article provides a model for other studies adapting feasibility trials into definitive trials. TRIAL REGISTRATION All trials prospectively registered. NEON Trial: ISRCTN11152837 . Registered on 13th August 2018. NEON-C Trial: ISRCTN76355273 . Registered on 9th January 2020. NEON-O Trial: ISRCTN63197153 . Registered on 9th January 2020. The NEON-O Trial ISRCTN was updated when amendments were approved. Amendment details: NOSA2, 30th October 2020.
Collapse
Affiliation(s)
- Stefan Rennick-Egglestone
- School of Health Sciences, Institute of Mental Health, University of Nottingham Innovation Park, Triumph Road, Nottingham, NG7 2TU UK
| | - Rachel Elliott
- Division of Population Health, Health Services Research & Primary Care, University of Manchester, Oxford Road, Manchester, M13 9PL UK
| | - Chris Newby
- School of Medicine, Institute of Mental Health, University of Nottingham Innovation Park, Triumph Road, Nottingham, NG7 2TU UK
| | - Clare Robinson
- Centre for Primary Care & Public Health, Pragmatic Clinical Trials Unit, Queen Mary University of London, 58 Turner St, London, E1 2AB UK
| | - Mike Slade
- School of Health Sciences, Institute of Mental Health, University of Nottingham Innovation Park, Triumph Road, Nottingham, NG7 2TU UK
| |
Collapse
|
7
|
Paterson L, Rennick-Egglestone S, Gavan SP, Slade M, Ng F, Llewellyn-Beardsley J, Bond C, Grundy A, Nicholson J, Quadri D, Bailey S, Elliott RA. Development and delivery cost of digital health technologies for mental health: Application to the Narrative Experiences Online Intervention. Front Psychiatry 2022; 13:1028156. [PMID: 36419974 PMCID: PMC9676659 DOI: 10.3389/fpsyt.2022.1028156] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/25/2022] [Accepted: 10/17/2022] [Indexed: 11/09/2022] Open
Abstract
BACKGROUND The increasing development and use of digital health interventions requires good quality costing information to inform development and commissioning choices about resource allocation decisions. The Narrative Experiences Online (NEON) Intervention is a web-application that delivers recorded mental health recovery narratives to its users. Two randomized controlled trials are testing the NEON Intervention in people with experience of psychosis (NEON) and people experiencing non-psychosis mental health problems (NEON-O). AIM This study describes and estimates the cost components and total cost of developing and delivering the NEON Intervention. MATERIALS AND METHODS Total costs for the NEON Trial (739 participants) and NEON-O Trial (1,024 participants) were estimated by: identifying resource use categories involved in intervention development and delivery; accurate measurement or estimation of resource use; and a valuation of resource use to generate overall costs, using relevant unit costs. Resource use categories were identified through consultation with literature, costing reporting standards and iterative consultation with health researchers involved in NEON Intervention development and delivery. Sensitivity analysis was used to test assumptions made. RESULTS The total cost of developing the NEON Intervention was £182,851. The largest cost components were software development (27%); Lived Experience Advisory Panel workshops (23%); coding the narratives (9%); and researchers' time to source narratives (9%). The total cost of NEON Intervention delivery during the NEON Trial was £118,663 (£349 per NEON Intervention user). In the NEON-O Trial, the total delivery cost of the NEON Intervention was £123,444 (£241 per NEON Intervention user). The largest cost components include updating the narrative collection (50%); advertising (19%); administration (14%); and software maintenance (11%). Uncertainty in the cost of administration had the largest effect on delivery cost estimates. CONCLUSION Our work shows that developing and delivering a digital health intervention requires expertise and time commitment from a range of personnel. Teams developing digital narrative interventions need to allocate substantial resources to curating narrative collections. IMPLICATIONS FOR PRACTICE This study identifies the development and delivery resource use categories of a digital health intervention to promote the consistent reporting of costs and informs future decision-making about the costs of delivering the NEON Intervention at scale. TRIAL REGISTRATION NEON Trial: ISRCTN11152837, registered 13 August 2018, http://www.isrctn.com/ISRCTN11152837. NEON-O Trial: ISRCTN63197153, registered 9 January 2020, http://www.isrctn.com/ISRCTN63197153.
Collapse
Affiliation(s)
- Luke Paterson
- Manchester Centre for Health Economics, Division of Population Health, Health Services Research and Primary Care, University of Manchester, Manchester, United Kingdom
| | - Stefan Rennick-Egglestone
- School of Health Sciences, Institute of Mental Health, University of Nottingham, Nottingham, United Kingdom
| | - Sean P Gavan
- Manchester Centre for Health Economics, Division of Population Health, Health Services Research and Primary Care, University of Manchester, Manchester, United Kingdom
| | - Mike Slade
- School of Health Sciences, Institute of Mental Health, University of Nottingham, Nottingham, United Kingdom.,Health and Community Participation Division, Faculty of Nursing and Health Sciences, Nord University, Namsos, Norway
| | - Fiona Ng
- School of Health Sciences, Institute of Mental Health, University of Nottingham, Nottingham, United Kingdom
| | - Joy Llewellyn-Beardsley
- School of Health Sciences, Institute of Mental Health, University of Nottingham, Nottingham, United Kingdom
| | - Carmel Bond
- Nottingham University Business School, The University of Nottingham, Nottingham, United Kingdom
| | - Andrew Grundy
- School of Health Sciences, University of Manchester, Manchester, United Kingdom
| | - Joe Nicholson
- School of Humanities, The University of Nottingham, Nottingham, United Kingdom
| | - Dania Quadri
- GKT School of Medical Education, King's College London, London, United Kingdom
| | - Sylvia Bailey
- Narrative Experiences Online Intervention (NEON) Lived Experience Advisory Panel, Nottingham, United Kingdom
| | - Rachel A Elliott
- Manchester Centre for Health Economics, Division of Population Health, Health Services Research and Primary Care, University of Manchester, Manchester, United Kingdom
| |
Collapse
|