Adaptation of the technology readiness levels for impact assessment in implementation sciences: The TRL-IS checklist

Background

Intervention development is a critical process in implementation research. There are key stages involved in the process to design, pilot, demonstrate and release a technology or an intervention. The Technology Readiness Level (TRL) is a globally accepted instrument for assessing the maturity of research development. However, the original levels do not fit all, and some adjustments are required for its applicability in implementation sciences.

Aims

This study aimed to gather the prior knowledge base on TRL in public and population health research; to develop a standard definition of readiness, and to adapt and validate the TRL to an implementation science context (TRL-IS).

Materials and methods

A Mixed methods approach has been followed in this study. A scoping review using the PRISMA extension (PRISMA-ScR) informed a nominal expert panel for developing a standard definition of readiness and to modify the TRL following an ontoterminology approach. Then the maturity of six practical case study examples were rated by ten researchers using the modified TRL to estimate inter-rater reliability, and a group of experts provided final content and face validity and feasibility.This mixed methods study included 1) a scoping review to examine the current literature and develop a knowledge base, identify knowledge gaps and to clarify concepts; 2) the development of a standard definition of 'Readiness' and related terms; and 3) adaptation of the TRL to implementation science and development of a checklist to rate the maturity of applications.A standard definition of readiness and related terms was produced by the core team, and an international nominal group (n = 30) was conducted to discuss and validate the definition and terms, and the location of 'Readiness' in the initiation and early development phases of implementation.Following feedback from the nominal group, the development of the TRL-IS was finalised and a TRL-IS rating checklist was developed to rate the maturity of applications. The TRL-IS checklist was tested using six cases based on real world studies on implementation research.The inter-rater reliability of the TRL-IS was evaluated by ten raters and finally six raters evaluated the content and face validity, and feasibility, of the TRL-IS checklist using the System Usability Scale (SUS).

Results

Few papers (n = 11) utilised the TRL to evaluate the readiness of readiness of health and social science implementation research. The main changes in the adaption of the TRL-IS included the removal of laboratory testing, limiting the use of "operational" environment and a clearer distinction between level 6 (pilot in a relevant environment) and 7 (demonstration in the real world prior to release). The adapted version was considered relevant by the expert panel. The TRL-IS checklist showed evidence of good inter-rater reliability (ICC = 0.90 with 95 % confident interval = 0.74-0.98, p < .001) and provides a consistent metric.

Conclusions

In spite of recommendations made by national and international research funding agencies, few health and social science implementation studies include the TRL as part of their evaluation protocol. The TRL-IS offers a high degree of conceptual clarity between scientific maturity phases or readiness levels, and good reliability among raters of varying experience. This study highlights that adoption of the TRL-IS framework in implementation sciences will bolster the scientific robustness and comparability of research maturity in this domain.

Emotion-induced frontal α asymmetry as a candidate predictor of relapse after discontinuation of antidepressant medication

BACKGROUND

One in three patients relapse after antidepressant discontinuation. Thus, the prevention of relapse after achieving remission is an important component in the long-term management of Major Depressive Disorder (MDD). However, no clinical or other predictors are established. Frontal reactivity to sad mood as measured by fMRI has been reported to relate to relapse independently of antidepressant discontinuation and is an interesting candidate predictor.

METHODS

Patients (n=56) who had remitted from a depressive episode while taking antidepressants underwent EEG recording during a sad mood induction procedure prior to gradually discontinuing their medication. Relapse was assessed over a six-months follow-up period. 35 healthy controls were also tested. Current source density of the EEG power in the α band (8-13Hz) was extracted and alpha-asymmetry was computed by comparing the power across two hemispheres at frontal electrodes (F5 and F6).

OUTCOMES

Sad mood induction was robust across all groups. Reactivity of α-asymmetry to sad mood did not distinguish healthy controls from patients with remitted MDD on medication. However, the 14 (25%) patients who relapsed during the follow-up period after discontinuing medication showed significantly reduced reactivity in α- asymmetry compared to patients who remained well. This EEG signal provided predictive power (69% out-of-sample balanced accuracy and a positive predictive value of 0.75).

INTERPRETATION

A simple EEG-based measure of emotional reactivity may have potential to contribute to clinical prediction models of antidepressant discontinuation. Given the very small sample size, this finding must be interpreted with caution and requires replication in a larger study.

Physical Activity Participation Decreases the Risk of Depression in Older Adults: The ATHLOS Population-Based Cohort Study

BACKGROUND

To which extent physical activity is associated with depression independent of older adults' physical and cognitive functioning is largely unknown. This cohort study using harmonised data by the EU Ageing Trajectories of Health: Longitudinal Opportunities and Synergies consortium, including over 20 countries, to evaluate the longitudinal association of physical activity (light-to-moderate or vigorous intensity) with depression in older adults (aged ≥ 50 years).

RESULTS

We evaluated 56,818 participants (light-to-moderate models; 52.7% females, age 50-102 years) and 62,656 participants (vigorous models; 52.7% females, age 50-105 years). Compared to never, light-to-moderate or vigorous physical activity was associated with a lower incidence rate ratio (IRR) of depression (light-to-moderate model: once/week: 0.632, 95% CI 0.602-0.663; twice or more/week: 0.488, 95% CI 0.468-0.510; vigorous model: once/week: 0.652, 95% CI 0.623-0.683; twice or more/week: 0.591, 95% CI 0.566-0.616). Physical activity remained associated with depression after adjustment for the healthy ageing scale, which is a scale that incorporated 41 items of physical and cognitive functioning (light-to-moderate model: once/week: 0.787, 95% CI 0.752-0.824; twice or more/week: 0.711, 95% CI 0.682-0.742; vigorous model: once/week: 0.828, 95% CI 0.792-0.866; twice or more/week: 0.820, 95% CI 0.786-0.856).

CONCLUSIONS

Physical activity, of any intensity and weekly frequency, was a strong protective factor against depression, independent of physical and mental functioning. Health policies could stimulate the incorporation of lower physical activity intensity to protect against depression, which might be more feasible at the population level.

Design and development of a digital intervention for workplace stress and mental health (EMPOWER)

Purpose

We describe the design and development of the European Platform to Promote health and wellbeing in the workplace (EMPOWER) digital intervention that provides an integrative user programme meeting the needs of employees and employers in addressing work stress.

Results

A user-centred design process was followed from January 2020 until November 2021. A tailored algorithm was developed to provide support at the individual employee level and the company level. Each element of the digital intervention was developed in English and then translated in Spanish, English, Polish and Finnish. The digital intervention consists of a website and a mobile application (app) that provides algorithm-based personalised content after assessing a user's somatic and psychological symptoms, work functioning, and psychosocial risk factors for work stress. It has a public section and an employer portal that provides recommendations to reduce psychosocial risks in their company based upon clustered input from employees. Usability testing was conducted and showed high ease of use and completion of tasks by participants.

Conclusion

The EMPOWER digital intervention is a tailored multimodal intervention addressing wellbeing, work stress, mental and physical health problems, and work productivity. This will be used in a planned RCT in four countries to evaluate its effectiveness.

Study protocol of EMPOWER: A cluster randomized trial of a multimodal eHealth intervention for promoting mental health in the workplace following a stepped wedge trial design

Objective: This article describes the EMPOWER study, a controlled trial aiming to determine the effectiveness and cost-effectiveness of an eHealth intervention to prevent common health problems and reduce presenteeism and absenteeism in the workplace. Intervention: The EMPOWER intervention spans universal, secondary and tertiary prevention and consists of an eHealth platform delivered via a website and a smartphone app designed to guide employees throughout different modules according to their specific profiles. Design: A stepped-wedge cluster randomized trial will be implemented in four countries (Finland, Poland, Spain and UK) with employees from small and medium enterprises (SMEs) and public agencies. Companies will be randomly allocated in one of three groups with different times at which the intervention is implemented. The intervention will last 7 weeks. Employees will answer several questionnaires at baseline, pre- and post-intervention and follow-up. Outcome measures: The main outcome is presenteeism. Secondary outcomes include depression, anxiety, insomnia, stress levels, wellbeing and absenteeism. Analyses will be conducted at the individual level using the intention-to-treat approach and mixed models. Additional analyses will evaluate the intervention effects according to gender, country or type of company. Cost-effectiveness and cost-utility analyses [based on the use of quality-adjusted life-years (QALYS)] will consider a societal, employers' and employees' perspective.

The mediating role of general and cognitive stress on the effect of an app-based intervention on productivity measures in workers: A randomized controlled trial (Preprint)

BACKGROUND

Loss of productivity is a result of absence from work (absenteeism) or of working with limitations due to illness (presenteeism). Recently, occupational mental health interventions have increasingly been delivered in digital format, as this is thought to be more convenient, flexible, easily accessible, and anonymous. However, the effectiveness of electronic mental health (e-mental health) interventions in the workplace to improve presenteeism and absenteeism remains unknown, and could be potentially mediated by psychological variables such as stress levels.

OBJECTIVE

The aim of this study was to determine the effectiveness of an e-mental health intervention to reduce absenteeism and presenteeism in employees, as well as to investigate the mediating role of stress in this effect.

METHODS

Employees of six companies in two countries participated in a randomized controlled trial (n=210 in the intervention group and n=322 in the waitlist control group). Participants in the intervention group could use the Kelaa Mental Resilience app for 4 weeks. All participants were asked to complete assessments at baseline, during the intervention, postintervention, and at a 2-week follow-up. Absenteeism and presenteeism were assessed by means of the Work Productivity and Activity Impairment Questionnaire: General Health, while general and cognitive stress were assessed through the Copenhagen Psychosocial Questionnaire-Revised Version. Regression and mediation analyses were performed to evaluate the effect of the Kelaa Mental Resilience app on presenteeism and absenteeism.

RESULTS

The intervention did not have a direct effect on presenteeism or absenteeism, neither at postintervention nor at follow-up. Nevertheless, general stress significantly mediated the intervention effect on presenteeism (P=.005) but not on absenteeism (P=.92), and cognitive stress mediated the effect of the intervention on both presenteeism (P<.001) and absenteeism (P=.02) right after the intervention. At the 2-week follow-up, the mediating effect of cognitive stress on presenteeism was significant (P=.04), although this was not the case for its mediating effect on absenteeism (P=.36). Additionally, at the 2-week follow-up, general stress did not mediate the intervention effect on presenteeism (P=.25) or on absenteeism (P=.72).

CONCLUSIONS

While no direct effect of the e-mental health intervention on productivity was found in this study, our findings suggest that stress reduction could mediate the effect of the intervention on presenteeism and absenteeism. As such, e-mental health interventions that address stress in employees might also indirectly reduce presenteeism and absenteeism in these employees. However, due to study limitations such as an overrepresentation of female participants in the sample and a high proportion of attrition, these results should be interpreted with caution. Future research is needed to better understand the mechanisms of interventions on productivity in the workplace.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05924542; https://clinicaltrials.gov/study/NCT05924542.

Strategies for Implementing Occupational eMental Health Interventions: Scoping Review

BACKGROUND

The implementation of eMental health interventions, especially in the workplace, is a complex process. Therefore, learning from existing implementation strategies is imperative to ensure improvements in the adoption, development, and scalability of occupational eMental health (OeMH) interventions. However, the implementation strategies used for these interventions are often undocumented or inadequately reported and have not been systematically gathered across implementations in a way that can serve as a much-needed guide for researchers.

OBJECTIVE

The objective of this scoping review was to identify implementation strategies relevant to the uptake of OeMH interventions that target employees and detail the associated barriers and facilitation measures.

METHODS

A scoping review was conducted. The descriptive synthesis was guided by the RE-AIM (reach, effectiveness, adoption, implementation, and maintenance) framework and the Consolidated Framework for Implementation Research.

RESULTS

A total of 31 of 32,916 (0.09%) publications reporting the use of the web-, smartphone-, telephone-, and email-based OeMH interventions were included. In all, 98 implementation strategies, 114 barriers, and 131 facilitators were identified. The synthesis of barriers and facilitators produced 19 facilitation measures that provide initial recommendations for improving the implementation of OeMH interventions.

CONCLUSIONS

This scoping review represents one of the first steps in a research agenda aimed at improving the implementation of OeMH interventions by systematically selecting, shaping, evaluating, and reporting implementation strategies. There is a dire need for improved reporting of implementation strategies and combining common implementation frameworks with more technology-centric implementation frameworks to fully capture the complexities of eHealth implementation. Future research should investigate a wider range of common implementation outcomes for OeMH interventions that also focus on a wider set of common mental health problems in the workplace. This scoping review's findings can be critically leveraged by discerning decision-makers to improve the reach, effectiveness, adoption, implementation, and maintenance of OeMH interventions.

The Mental Health of Employees with Job Loss and Income Loss during the COVID-19 Pandemic: The Mediating Role of Perceived Financial Stress

The COVID-19 outbreak, which was followed by home confinement, is expected to have had profound negative impact on the mental health of people. Associated factors, such as losing jobs and income, can be expected to lead to an increased risk of suffering from psychopathological problems. Therefore, this study was aimed at researching the associations of job and income loss with mental health, as well as the possible mediating role of perceived financial stress during the COVID-19 outbreak. The sample included 2381 Spanish workers who were interviewed right after the first COVID-19 lockdown. Measures were taken for generalized anxiety disorder, panic attacks, depression, post-traumatic stress disorder, substance abuse, suicidal thoughts and behaviors, working conditions, sociodemographic variables, and perceived financial stress. Logistic regression models were calculated with psychological variables as outcomes, and with job loss and income loss as predictors. Mediation analyses were performed by adding the financial threat as a mediator. Nineteen point six percent and 33.9% of participants reported having lost their jobs and incomes due to the pandemic, respectively. Only income loss was related to a higher risk of suffering from depression and panic attacks. When adding financial stress as a mediator, the indirect effects of job and income loss on the mental health measures were found to be significant, therefore indicating mediation. These findings pinpoint the vulnerability of this population, and highlight the need for interventional and preventive programs targeting mental health in economic crisis scenarios, such as the current one. They also highlight the importance of implementing social and income policies during the COVID-19 pandemic to prevent mental health problems.

Duration of untreated illness and bipolar disorder: time for a new definition? Results from a cross-sectional study

BACKGROUND

We primarily aimed to explore the associations between duration of untreated illness (DUI), treatment response, and functioning in a cohort of patients with bipolar disorder (BD).

METHODS

261 participants with BD were recruited. DUI was defined as months from the first affective episode to the start of a mood-stabilizer. The functioning assessment short test (FAST) scores and treatment response scores for lithium, valproate, or lamotrigine according to the Alda Scale Total Score (TS) were compared between patients with short (<24 months) or long DUI. Differences in FAST scores among good (GR; TS≥7), poor (PR; TS=2-6), or non-responders (NR; TS<2) to each mood-stabilizer were analyzed. Linear regression was computed using the FAST global score as the dependent variable.

RESULTS

DUI and FAST scores showed no statistically significant correlation. Patients with a longer DUI showed poorer response to lithium (Z=-3.196; p<0.001), but not to valproate or lamotrigine. Response to lithium (β=-1.814; p<0.001), number of hospitalizations (β=0.237; p<0.001), and illness duration (β=0.160; p=0.028) were associated with FAST total scores. GR to lithium was associated with better global functioning compared to PR or NR [H=27.631; p<0.001].

LIMITATIONS

The retrospective design could expose our data to a recall bias. Also, only few patients were on valproate or lamotrigine treatment.

CONCLUSIONS

Poor functioning in BD could be the result of multiple affective relapses, rather than a direct effect of DUI. A timely diagnosis with subsequent effective prophylactic treatment, such as lithium, may prevent poor functional outcomes in real-world patients with BD.

The two-way relationship between nicotine and cortical activity: a systematic review of neurobiological and treatment aspects

Nicotine intake and cortical activity are closely related, as they can influence each other. Nicotine is implicated in the induction and modification of cortical plasticity and excitability, whereas a change on cortical plasticity and excitability can also lead to a modification of the smoking behaviour of an individual. The aim of this systematic review was, on the one hand, to evaluate the effects of nicotinergic modulation on cortical excitability and plasticity, and, on the other hand, to assess if modifying the brain's excitability and plasticity could influence one's smoking behaviour. Two systematic literature searches in the PubMed/MEDLINE and PsycINFO databases were conducted. Studies focusing either on the impact of nicotinergic modulation on cortical activity or the treatment effect of non-invasive brain stimulation techniques (NIBS) on smoking behaviour were included. A total of 22 studies for the first systematic search and 35 studies for the second one were included after full-text screening. Nicotine's effect on cortical activity appeared to depend on smoking status of the individual. While deprived smokers seem to generally profit from nicotine consumption in terms of cortical excitability and plasticity, the contrary was true for non-smokers. Regarding the questions of how changes in cortical excitability can influence smoking behaviour, a trend points towards NIBS being a potential intervention technique for smoking cessation.