"Did the trial kill the intervention?" experiences from the development, implementation and evaluation of a complex intervention

'Smart' BLE wearables for digital contact tracing in care homes during the COVID-19 pandemic-a process evaluation of the CONTACT feasibility study

BACKGROUND

Rapid and mass transmission of the SARS-CoV-2 virus amongst vulnerable people led to devastating effects from COVID-19 in care homes. The CONTACT intervention introduced Bluetooth Low Energy 'smart' wearable devices (BLE wearables) as a basis for automated contact tracing in, and feedback on infection risks and patterns to, care homes to try and improve infection prevention and control (IPC). We planned a cluster randomised controlled trial (RCT) of CONTACT. To be feasible, homes had to adopt CONTACT's technology and new ways of working. This paper reports on the process evaluation conducted alongside CONTACT's feasibility study and explains why it lacked the feasibility and acceptability for a definitive RCT.

METHODS

This mixed method process evaluation used Normalisation Process Theory (NPT) qualitative (interviews, field notes, study case report forms and documents, and observation) and quantitative (survey instruments, counts of activity) data to plan, implement, and analyse the mechanisms, effects, and contextual factors that shaped the feasibility and acceptability of the CONTACT intervention.

RESULTS

Thirteen themes within four core NPT constructs explained CONTACT's lack of feasibility. Coherence: the home's varied in the scale and extent of commitment and understanding of the technology and study procedures. Leadership credibility was important but compromised by competing priorities. Management and direct care staff saw CONTACT differently. Work to promote (cognitive participation) and enact (collective action) CONTACT was burdensome and failed to be prioritised over competing COVID-19-related demands on time and scarce human and cognitive resources. Ultimately, staff appraisal of the value of CONTACT-generated information and study procedures (reflexivity) was that any utility for IPC was insufficient to outweigh the perceived burden and complexity involved.

CONCLUSIONS

Despite implementation failure, dismissing BLE wearables' potential for contact tracing is premature. In non-pandemic conditions, with more time, better co-design and integration of theory-driven implementation strategies tailored to care homes' unique contexts, researchers could enhance normalisation in readiness for future pandemic challenges.

TRIAL REGISTRATION

ISRCTN registration: 11,204,126 registered 17/02/2021.

Oral Health Coaches at Well-Baby Clinics to Promote Oral Health in Preschool Children From the First Erupted Tooth: Protocol for a Multisite, Pragmatic Randomized Controlled Trial

Background

Early childhood caries is considered one of the most prevalent diseases in childhood, affecting almost half of preschool-age children globally. In the Netherlands, approximately one-third of children aged 5 years already have dental caries, and dental care providers experience problems reaching out to these children.

Objective

Within the proposed trial, we aim to test the hypothesis that, compared to children who receive usual care, children who receive the Toddler Oral Health Intervention as add-on care will have a reduced cumulative caries incidence and caries incidence density at the age of 48 months.

Methods

This pragmatic, 2-arm, individually randomized controlled trial is being conducted in the Netherlands and has been approved by the Medical Ethics Research Board of University Medical Center Utrecht. Parents with children aged 6 to 12 months attending 1 of the 9 selected well-baby clinics are invited to participate. Only healthy children (ie, not requiring any form of specialized health care) with parents that have sufficient command of the Dutch language and have no plans to move outside the well-baby clinic region are eligible. Both groups receive conventional oral health education in well-baby clinics during regular well-baby clinic visits between the ages of 6 to 48 months. After concealed random allocation of interventions, the intervention group also receives the Toddler Oral Health Intervention from an oral health coach. The Toddler Oral Health Intervention combines behavioral interventions of proven effectiveness in caries prevention. Data are collected at baseline, at 24 months, and at 48 months. The primary study endpoint is cumulative caries incidence for children aged 48 months, and will be analyzed according to the intention-to-treat principle. For children aged 48 months, the balance between costs and effects of the Toddler Oral Health Intervention will be evaluated, and for children aged 24 months, the effects of the Toddler Oral Health Intervention on behavioral determinants, alongside cumulative caries incidence, will be compared.

Results

The first parent-child dyads were enrolled in June 2017, and recruitment was finished in June 2019. We enrolled 402 parent-child dyads.

Conclusions

All follow-up interventions and data collection will be completed by the end of 2022, and the trial results are expected soon thereafter. Results will be shared at international conferences and via peer-reviewed publication.

Trial Registration

Netherlands Trial Register NL8737; https://trialsearch.who.int/Trial2.aspx?TrialID=NL8737

International Registered Report Identifier (IRRID)

DERR1-10.2196/39683

Feasibility of a peer-led, after-school physical activity intervention for disadvantaged adolescent females during the COVID-19 pandemic: results from the Girls Active Project (GAP)

Introduction

There is a critical need for interventions that can be feasibly implemented and are effective in successfully engaging adolescent females in physical activity (PA). A theory-based, peer-led, after-school PA intervention, the Girls Active Project (GAP), was codesigned with adolescent females. This study aimed to assess the feasibility of implementing and evaluating the GAP programme.

Setting

One single-sex, female-only, designated disadvantaged postprimary school (students aged 12–18) in Dublin, Ireland.

Methods

Mixed methods were applied with multiple stakeholders over a 12-week trial (March to May 2021). A single-arm study design was used to examine intervention: reach, dose, fidelity, acceptability, compatibility and context. Feasibility of using proposed self-reported outcome measures (moderate-to-vigorous PA levels, self-rated health, life satisfaction, PA self-efficacy and PA enjoyment) was also explored. Due to school closure resulting from the COVID-19 pandemic, the intervention was delivered both online and in person in the school setting.

Results

Eight exercise classes were peer delivered by project leaders (n = 6, students aged 15–17) to intervention recipients (students aged 13-14). Recruitment was low (n = 8, 10% of eligible students, mean age: 13.3 SD: 0.46), yet retention was high (n = 7/8, 88%). Attendance rates were satisfactory (68%), and the intervention was implemented with high fidelity (87%). Data completion rates suggested proposed self-reported outcome measures were deemed appropriate (≥ 95%), except for weight (50%) and height data (80%). Despite COVID-19 hindering intervention implementation, both quantitative and qualitative data suggested that stakeholders were satisfied and perceived the in-person delivered intervention to be compatible with the school setting. Recommended refinements included extending class duration, introducing different rewards, and boosting programme awareness.

Conclusions

Further thought must be given on how to increase recruitment. Overall, the in-person delivered after-school PA programme was well-received by stakeholders and shows promise as an intervention that can be feasibly implemented and evaluated. Suggested improvements to the GAP intervention programme are recommended, before continuing to a more robust evaluation.

Trial registration

10.17605/OSF.IO/75HWJ (prospectively registered, date of registration: 9th December 2020)

Supplementary Information

The online version contains supplementary material available at 10.1186/s40814-022-01149-2.

Contextual factors influencing complex intervention research processes in care homes: a systematic review and framework synthesis

Background

Care homes are complex settings to undertake intervention research. Barriers to research implementation processes can threaten studies’ validity, reducing the value to residents, staff, researchers and funders. We aimed to (i) identify and categorise contextual factors that may mediate outcomes of complex intervention studies in care homes and (ii) provide recommendations to minimise the risk of expensive research implementation failures.

Methods

We conducted a systematic review using a framework synthesis approach viewed through a complex adaptive systems lens. We searched: MEDLINE, Embase, CINAHL, ASSIA databases and grey literature. We sought process evaluations of care home complex interventions published in English. Narrative data were indexed under 28 context domains. We performed an inductive thematic analysis across the context domains.

Results

We included 33 process evaluations conducted in high-income countries, published between 2005 and 2019. Framework synthesis identified barriers to implementation that were more common at the task and organisational level. Inductive thematic analysis identified (i) avoiding procedural drift and (ii) participatory action and learning as key priorities for research teams. Research team recommendations include advice for protocol design and care home engagement. Care home team recommendations focus on internal resources and team dynamics. Collaborative recommendations apply to care homes’ individual context and the importance of maintaining positive working relationships.

Discussion

Researchers planning and undertaking research with care homes need a sensitive appreciation of the complex care home context. Study implementation is most effective where an intervention is co-produced, with agreed purpose and adequate resources to incorporate within existing routines and care practices.

Optimising recruitment to a late-phase tuberculosis clinical trial: a qualitative study exploring patient and practitioner experiences in Uzbekistan

BACKGROUND

Addressing the global burden of multidrug-resistant tuberculosis (MDR-TB) requires identification of shorter, less toxic treatment regimens. Médecins Sans Frontières (MSF) is currently conducting a phase II/III randomised controlled clinical trial, to find more effective, shorter and tolerable treatments for people with MDR-TB. Recruitment to the trial in Uzbekistan has been slower than expected; we aimed to study patient and health worker experiences of the trial, examining potential factors perceived to impede and facilitate trial recruitment, as well as general perceptions of clinical research in this context.

METHODS

We conducted a qualitative study using maximum variation, purposive sampling of participants. We carried out in-depth interviews (IDIs) and focus group discussions (FGDs) guided by semi-structured topic guides. In December 2019 and January 2020, 26 interviews were conducted with patients, Ministry of Health (MoH) and MSF staff and trial health workers, to explore challenges and barriers to patient recruitment as well as perceptions of the trial and research in general. Preliminary findings from the interviews informed three subsequent focus group discussions held with patients, nurses and counsellors. Focus groups adopted a person-centred design, brainstorming potential solutions to problems and barriers. Interviews and FGDs were audio recorded, translated and transcribed verbatim. Thematic analysis, drawing on constant comparison, was used to analyse the data.

RESULTS

Health system contexts may compete with new approaches especially when legislative health regulations or policy around treatment is ingrained in staff beliefs, perceptions and practice, which can undermine clinical trial recruitment. Trust plays a significant role in how patients engage with the trial. Decision-making processes are dynamic and associated with relationship to diagnosis, assimilation of information, previous knowledge or experience and influence of peers and close relations.

CONCLUSIONS

This qualitative analysis highlights ways in which insights developed together with patients and healthcare workers might inform approaches towards improved recruitment into trials, with the overall objective of delivering evidence for better treatments.

Implementing a telehealth prehabilitation education session for patients preparing for major cancer surgery

Background

Prehabilitation services assist patients in preparing for surgery, yet access to these services are often limited by geographical factors. Enabling rural and regional patients to access specialist surgical prehabilitation support with the use of telehealth technology has the potential to overcome health inequities and improve post-operative outcomes.

Aim

To evaluate the current and likely future impact of a telehealth preoperative education package for patients preparing for major abdominal cancer surgery.

Methods

A telehealth alternative to a hospital based pre-operative education session was developed and implemented at a dedicated cancer hospital. Adult patients (≥18 years) scheduled for elective major cancer surgery were offered this telehealth alternative. Impact evaluation was conducted using the RE-AIM framework.

Results

To date, 35 participants have consented to participate in the study. Thirty-one participants attended the intervention; 24 (69%) residing in rural or regional areas. Twenty-four (77%) reported that if given a choice they would prefer the online session as opposed to attending the hospital in person. The majority (97%) reported they would recommend the intervention to others preparing for surgery. Session information was recalled by all 26 participants and 77% of participants reported acting on recommendations 2 weeks after the session. Lessons learnt and recommendations for providers implementing similar programs are reported.

Conclusion

Telehealth alternatives to hospital based pre-operative education are well received by patients preparing for major cancer surgery. We make seven recommendations to improve implementation. Further evaluation of implementation strategies alongside clinical effectiveness in future studies is essential.

Trial registration

ACTRN12620000096954, 04/02/2020.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-021-06437-w.

Sample size, study length, and inadequate controls were the most common self-acknowledged limitations in manual therapy trials: A methodological review

OBJECTIVES

The aim of this study was to quantify and analyze the presence and type of self-acknowledged limitations (SALs) in a sample of manual therapy (MT) randomized controlled trials.

STUDY DESIGN AND SETTING

We randomly selected 120 MT trials. We extracted data related to SALs from the original reports and classified them into 12 categories. After data extraction, specific limitations within each category were identified. A descriptive analysis was performed using frequencies and percentages for qualitative variables.

RESULTS

The number of SALs per trial article ranged from 0 to 8, and more than two-thirds of trials acknowledged at least two different limitations. Despite its small proportion, 9% of trials did not report SALs. The most common limitation declared, in almost half of our sample, related to sample size (47.5%) followed by limitations related to study length and follow-up (33.3%) and inadequate controls (32.5%).

CONCLUSION

Our results indicate that at least two different limitations are consistently acknowledged in MT trial reports, the most common being those related to sample size, study length, follow-up, and inadequate controls. Analysis of the reasons behind the SALs gives some insights about the main difficulties in conducting research in this field and may help develop strategies to improve future research.

The value of real-world testing: a qualitative feasibility study to explore staff and organisational barriers and strategies to support implementation of a clinical pathway for the management of anxiety and depression in adult cancer patients

Background

Effective translation of evidence-based research into clinical practice requires assessment of the many factors that can impact implementation success. Research methods that draw on recognised implementation frameworks, such as the Promoting Action Research in Health Services (PARiHS) framework, and that test feasibility to gain information prior to full-scale roll-out, can support a more structured approach to implementation.

Objective

This paper presents qualitative findings from a feasibility study in one cancer service of an online portal to operationalise a clinical pathway for the screening, assessment and management of anxiety and depression in adult cancer patients. The aim of this study was to explore staff perspectives on the feasibility and acceptance of a range of strategies to support implementation in order to inform the full-scale roll-out.

Methods

Semi-structured interviews were conducted with fifteen hospital staff holding a range of clinical, administrative and managerial roles, and with differing levels of exposure to the pathway. Qualitative data were analysed thematically, and themes were subsequently organised within the constructs of the PARiHS framework.

Results

Barriers and facilitators that affected the feasibility of the online portal and implementation strategies were organised across eight key themes: staff perceptions, culture, external influences, attitudes to psychosocial care, intervention fit, familiarity, burden and engagement. These themes mapped to the PARiHS framework's three domains of evidence, context and facilitation.

Conclusions

Implementation success may be threatened by a range of factors related to the real-world context, perceptions of the intervention (evidence) and the process by which it is introduced (facilitation). Feasibility testing of implementation strategies can provide unique insights into issues likely to influence full-scale implementation, allowing for early tailoring and more effective facilitation which may save time, money and effort in the long-term. Use of a determinant implementation framework can assist researchers to synthesise and effectively respond to barriers as they arise. While the current feasibility study related to a specific implementation, strategies such as regular engagement with local stakeholders, and discussion of barriers arising in real-time during early testing is likely to be of benefit to all researchers and clinicians seeking to maximise the likelihood of long-term implementation success.

Decision-based models of the implementation of interventions in systems of healthcare: Implementation outcomes and intervention effectiveness in complex service environments

Implementation is a crucial component for the success of interventions in health service systems, as failure to implement well can have detrimental impacts on the effectiveness of evidence-based practices. Therefore, evaluations conducted in real-world contexts should consider how interventions are implemented and sustained. However, the complexity of healthcare environments poses considerable challenges to the evaluation of interventions and the impact of implementation efforts on the effectiveness of evidence-based practices. In consequence, implementation and intervention effectiveness are often assessed separately in health services research, which prevents the direct investigation of the relationships of implementation components and effectiveness of the intervention. This article describes multilevel decision juncture models based on advances in implementation research and causal inference to study implementation in health service systems. The multilevel decision juncture model is a theory-driven systems approach that integrates structural causal models with frameworks for implementation. This integration enables investigation of interventions and their implementation within a single model that considers the causal links between levels of the system. Using a hypothetical youth mental health intervention inspired by published studies from the health service research and implementation literature, we demonstrate that such theory-based systems models enable investigations of the causal pathways between the implementation outcomes as well as their links to patient outcomes. Results from Monte Carlo simulations also highlight the benefits of structural causal models for covariate selection as consistent estimation requires only the inclusion of a minimal set of covariates. Such models are applicable to real-world context using different study designs, including longitudinal analyses which facilitates the investigation of sustainment of interventions.

Implementation of a Stepwise, Multidisciplinary Intervention for Pain and Challenging Behaviour in Dementia (STA OP!): A Process Evaluation

Background

A stepwise, multidisciplinary and multicomponent intervention (called STA OP!) was implemented in Dutch nursing home units, which included a comprehensive multidisciplinary team training. A cluster-randomised controlled trial showed that the intervention reduced symptoms of pain and challenging behaviour.

Objectives

To describe the experiences around the implementation of the intervention; to examine the extent to which the STA OP! intervention was delivered and implemented as intended (at the level of the team, and the individual resident/professional); and to understand factors influencing the implementation process.

Methods

A process evaluation was performed using a mixed-methods design encompassing several data sources. Quantitative data (i.e. from the written evaluations by healthcare professionals, management, and the research database) were analysed using descriptive statistics. Qualitative data (i.e. semi-structured interviews, notes, completed intervention forms, and written evaluations) were analysed according to the principles of thematic analysis. The implementation process and the influencing factors were categorised according to the i) organisational level, ii) the team level, and iii) the level of the individual resident/professional.

Results

In total, 39.2% of the residents with pain and/or challenging behaviour were treated following the stepwise approach of the STA OP! intervention. The training manual and forms used were found to be relevant and feasible. Factors inhibiting the implementation process at the i) organisational level concerned instability of the organisation and the team (e.g. involvement in multiple projects/new innovations, staff turnover/absence of essential disciplines, and/or high workload). At the team level (ii), we found that presence of a person with a motivational leadership style facilitated the implementation. Also, interdisciplinary cooperation through the design/setting of the multidisciplinary training, securing the intervention by use of clear agreements, and written reporting or transfers facilitated implementation. At the individual level (iii), perceived value of the stepwise working method, and enhanced awareness facilitated the implementation.

Conclusion

Although the intervention was not implemented as planned, the intervention empowered healthcare professionals and increased their awareness of the signals of pain and challenging behaviour. Future implementation of the intervention should start on units with a motivational leader, and specific features of the organisation and the team should be considered to facilitate implementation, e.g. stability, support, and shared focus to change.

An evaluation of a referral management and triage system for oral surgery referrals from primary care dentists: a mixed-methods study

Background

Oral surgery referrals from dentists are rising and putting increased pressure on finite hospital resources. It has been suggested that primary care specialist services can provide care for selected patients at reduced costs and similar levels of quality and patient satisfaction.

Research questions

Can an electronic referral system with consultant- or peer-led triage effectively divert patients requiring oral surgery into primary care specialist settings safely, and at a reduced cost, without destabilising existing services?

Design

A mixed-methods, interrupted time study (ITS) with adjunct diagnostic test accuracy assessment and health economic evaluation.

Setting

The ITS was conducted in a geographically defined health economy with appropriate hospital services and no pre-existing referral management or primary care oral surgery service. Hospital services included a district general, a foundation trust and a dental hospital.

Participants

Patients, carers, general and specialist dentists, consultants (both surgical and Dental Public Health), hospital managers, commissioners and dental educators contributed to the qualitative component of the work. Referrals from primary care dental practices for oral surgery procedures over a 3-year period were utilised for the quantitative and health economic evaluation.

Interventions

A consultant- then practitioner-led triage system for oral surgery referrals embedded within an electronic referral system for oral surgery with an adjunct primary care service.

Main outcome measures

Diagnostic test accuracy metrics for sensitivity and specificity were calculated. Total referrals, numbers of referrals sent to primary care and the cost per referral are reported for the main intervention. Qualitative findings in relation to patient experience and whole-system impact are described.

Results

In the diagnostic test accuracy study, remote triage was found to be highly specific (mean 88.4, confidence intervals 82.6 and 92.8) but with lower values for sensitivity. The implementation of the referral system and primary care service was uneventful. During consultant triage in the active phases of the study, 45% of referrals were diverted to primary care, and when general practitioner triage was used this dropped to 43%. Only 4% of referrals were sent from specialist primary care to hospital, suggesting highly efficient triage of referrals. A significant per-referral saving of £108.23 [standard error (SE) £11.59] was seen with consultant triage, and £84.13 (SE £11.56) with practitioner triage. Cost savings varied according the differing methods of applying the national tariff. Patients reported similar levels of satisfaction for both settings, and speed of treatment was their over-riding concern.

Conclusions

Implementation of electronic referral management in primary care can lead, when combined with triage, to diversions of appropriate cases to primary care. Cost savings can be realised but are dependent on tariff application by hospitals, with a risk of overestimating where hospitals are using day case tariffs extensively.

Study limitations

The geographical footprint of the study was relatively small and, hence, the impact on services was minimal and could not be fully assessed across all three hospitals.

Future work

The findings suggest that the intervention should be tested in other localities and disciplines, especially those, such as dermatology, that present the opportunity to use imaging to triage.

Funding

The National Institute for Health Research Health Services and Delivery Research programme.

Whole Medical Systems versus the System of Conventional Biomedicine: A Critical, Narrative Review of Similarities, Differences, and Factors That Promote the Integration Process

BACKGROUND

There is an increasing need for a worldwide professional integration of conventional medicine and traditional/complementary whole medical systems (WMSs). However, the integration is perceived by conventional medicine as problematic or unacceptable, because of a supposed lack of evidence for specific effects of WMSs therapies and supposed prescientific or unscientific paradigms of WMSs.

OBJECTIVES

To review the literature on the features of WMSs, similarities and differences between conventional medicine and WMSs, and scientific and clinical practice issues that should be dealt with in order to promote the integration process.

METHODS

A critical, narrative review of the literature on six WMSs.

RESULTS AND CONCLUSIONS

Key factors for the integration of WMSs and conventional medicine are as follows: legal frameworks, quality standards, high-quality research on safety and efficacy of WMS interventions, infrastructure, and financial resources. For scientific assessment of WMSs, there are unresolved ontological, epistemological, and methodological issues and issues of diagnostics, therapy delivery, and outcome assessment in clinical practice. Future research not only should be directed at quality assurance and generating the necessary data on safety and efficacy/effectiveness but also should address more fundamental (ontological, epistemological, and methodological) issues, in order to overcome the differences between WMSs and conventional medicine.

Patient and carer experience of hospital-based rehabilitation from intensive care to hospital discharge: mixed methods process evaluation of the RECOVER randomised clinical trial

OBJECTIVES

To explore and compare patient/carer experiences of rehabilitation in the intervention and usual care arms of the RECOVER trial (ISRCTN09412438); a randomised controlled trial of a complex intervention of post-intensive care unit (ICU) acute hospital-based rehabilitation following critical illness.

DESIGN

Mixed methods process evaluation including comparison of patients' and carers' experience of usual care versus the complex intervention. We integrated and compared quantitative data from a patient experience questionnaire (PEQ) with qualitative data from focus groups with patients and carers.

SETTING

Two university-affiliated hospitals in Scotland.

PARTICIPANTS

240 patients discharged from ICU who required ≥48 hours of mechanical ventilation were randomised into the trial (120 per trial arm). Exclusion criteria comprised: primary neurologic diagnosis, palliative care, current/planned home ventilation and age <18 years. 182 patients completed the PEQ at 3 months postrandomisation. 22 participants (14 patients and 8 carers) took part in focus groups (2 per trial group) at >3 months postrandomisation.

INTERVENTIONS

A complex intervention of post-ICU acute hospital rehabilitation, comprising enhanced physiotherapy, nutritional care and information provision, case-managed by dedicated rehabilitation assistants (RAs) working within existing ward-based clinical teams, delivered between ICU discharge and hospital discharge. Comparator was usual care.

OUTCOME MEASURES

A novel PEQ capturing patient-reported aspects of quality care.

RESULTS

The PEQ revealed statistically significant between-group differences across 4 key intervention components: physiotherapy (p=0.039), nutritional care (p=0.038), case management (p=0.045) and information provision (p<0.001), suggesting greater patient satisfaction in the intervention group. Focus group data strongly supported and helped explain these findings. Specifically, case management by dedicated RAs facilitated greater access to physiotherapy, nutritional care and information that cut across disciplinary boundaries and staffing constraints. Patients highly valued its individualisation according to their needs, abilities and preferences.

CONCLUSIONS

Case management by dedicated RAs improves patients' experiences of post-ICU hospital-based rehabilitation and increases perceived quality of care.

TRIAL REGISTRATION NUMBER

ISRCTN09412438.

Execution of a participatory supportive return to work program within the Dutch social security sector: a qualitative evaluation of stakeholders' perceptions

Background

A process evaluation of a participatory supportive return to work program, aimed at workers without a (permanent) employment contract who are sick-listed due to a common mental disorder, revealed that this program was executed less successfully than similar programs evaluated in earlier studies. The program consisted of a participatory approach, integrated care and direct placement in competitive employment. Aim of this study was to get a better understanding of the execution of the program by evaluating stakeholders’ perceptions. In the absence of an employer, the program was applied by the Dutch Social Security Agency, in collaboration with vocational rehabilitation agencies. Together with the sick-listed workers, these were the main stakeholders. Our research questions involved stakeholders’ perceptions of the function(s) of the program, and their perceptions of barriers and facilitators for a successful execution of the program within the Dutch social security sector.

Methods

Semi-structured interviews were held with five sick-listed workers, eight professionals of the Social Security Agency, and two case managers of vocational rehabilitation agencies. Interview topics were related to experiences with different components of the program. Selection of respondents was based on purposive sampling and continued until data saturation was reached. Content analysis was applied to identify patterns in the data. Two researchers developed a coding system, based on predefined topics and themes emerging from the data.

Results

Although perceived functions of some components of the program were as intended, all stakeholders stressed that the program often had not resulted in return to work. Perceived barriers for a successful execution were related to a poor collaboration between the Dutch Social Security Agency, vocational rehabilitation agencies and healthcare providers, the type of experienced (health) problems, time constraints, and limited job opportunities.

Conclusions

For future implementation of the program, it will be important to consider how a better integration of services by the Dutch Social Security Agency, vocational rehabilitation agencies and the mental healthcare sector can be improved in order to address treatment and vocational needs simultaneously, and to better match the sick-listed worker with the limited opportunities in the Dutch labor market.

Trial registration

NTR3563

Electronic supplementary material

The online version of this article (doi:10.1186/s12889-016-2997-x) contains supplementary material, which is available to authorized users.

Behind the scenes of the PRIME intervention: designing a complex intervention to improve malaria care at public health centres in Uganda

Background

In Uganda, health system challenges limit access to good quality healthcare and contribute to slow progress on malaria control. We developed a complex intervention (PRIME), which was designed to improve quality of care for malaria at public health centres.

Objective

Responding to calls for increased transparency, we describe the PRIME intervention's design process, rationale, and final content and reflect on the choices and challenges encountered during the design of this complex intervention.

Design

To develop the intervention, we followed a multistep approach, including the following: 1) formative research to identify intervention target areas and objectives; 2) prioritization of intervention components; 3) review of relevant evidence; 4) development of intervention components; 5) piloting and refinement of workshop modules; and 6) consolidation of the PRIME intervention theories of change to articulate why and how the intervention was hypothesized to produce desired outcomes. We aimed to develop an intervention that was evidence-based, grounded in theory, and appropriate for the study context; could be evaluated within a randomized controlled trial; and had the potential to be scaled up sustainably.

Results

The process of developing the PRIME intervention package was lengthy and dynamic. The final intervention package consisted of four components: 1) training in fever case management and use of rapid diagnostic tests for malaria (mRDTs); 2) workshops in health centre management; 3) workshops in patient-centred services; and 4) provision of mRDTs and antimalarials when stocks ran low.

Conclusions

The slow and iterative process of intervention design contrasted with the continually shifting study context. We highlight the considerations and choices made at each design stage, discussing elements we included and why, as well as those that were ultimately excluded. Reflection on and reporting of ‘behind the scenes’ accounts of intervention design may improve the design, assessment, and generalizability of complex interventions and their evaluations.

Methodological Challenges of Mixed Methods Intervention Evaluations

Abstract. This paper addresses the methodological challenges that accompany the use of a combination of research methods to evaluate complex interventions. In evaluating complex interventions, the question about effectiveness is not the only question that needs to be answered. Of equal interest are questions about acceptability, feasibility, and implementation of the intervention and the evaluation study itself. Using qualitative research in conjunction with trials enables us to address this diversity of questions. The combination of methods results in a mixed methods intervention evaluation (MMIE). In this article we demonstrate the relevance of mixed methods evaluation studies and provide case studies from health care. Methodological challenges that need our attention are, among others, choosing appropriate designs for MMIEs, determining realistic expectations of both components, and assigning adequate resources to both components. Solving these methodological issues will improve our research designs and provide further insights into complex interventions.

COSMOS--improving the quality of life in nursing home patients: protocol for an effectiveness-implementation cluster randomized clinical hybrid trial

BACKGROUND

Nursing home patients have complex mental and physical health problems, disabilities and social needs, combined with widespread prescription of psychotropic drugs. Preservation of their quality of life is an important goal. This can only be achieved within nursing homes that offer competent clinical conditions of treatment and care. COmmunication, Systematic assessment and treatment of pain, Medication review, Occupational therapy, Safety (COSMOS) is an effectiveness-implementation hybrid trial that combines and implements organization of activities evidence-based interventions to improve staff competence and thereby the patients' quality of life, mental health and safety. The aim of this paper is to describe the development, content and implementation process of the COSMOS trial.

METHODS/DESIGN

COSMOS includes a 2-month pilot study with 128 participants distributed among nine Norwegian nursing homes, and a 4-month multicenter, cluster randomized effectiveness-implementation clinical hybrid trial with follow-up at month 9, including 571 patients from 67 nursing home units (one unit defined as one cluster). Clusters are randomized to COSMOS intervention or current best practice (control group). The intervention group will receive a 2-day education program including written guidelines, repeated theoretical and practical training (credited education of caregivers, physicians and nursing home managers), case discussions and role play. The 1-day midway evaluation, information and interviews of nursing staff and a telephone hotline all support the implementation process. Outcome measures include quality of life in late-stage dementia, neuropsychiatric symptoms, activities of daily living, pain, depression, sleep, medication, cost-utility analysis, hospital admission and mortality.

DISCUSSION

Despite complex medical and psychosocial challenges, nursing home patients are often treated by staff possessing low level skills, lacking education and in facilities with a high staff turnover. Implementation of a research-based multicomponent intervention may improve staff's knowledge and competence and consequently the quality of life of nursing home patients in general and people with dementia in particular.

TRIAL REGISTRATION

ClinicalTrials.gov NCT02238652.

Mindfulness-based cognitive therapy for inflammatory bowel disease patients: findings from an exploratory pilot randomised controlled trial

Background

Inflammatory bowel disease (IBD) is a chronic gastrointestinal condition with a relapsing disease course. Managing the relapsing nature of the disease causes daily stress for IBD patients; thus, IBD patients report higher rates of depression and anxiety than the general population.

Mindfulness-based Cognitive Therapy (MBCT) is an evidence-based psychological program designed to help manage depressive and stress symptoms. There has been no randomized controlled trial (RCT) testing the use of MBCT in IBD patients.

The purpose of this pilot study is to test the trial methodology and assess the feasibility of conducting a large RCT testing the effectiveness of MBCT in IBD.

Methods

The IBD patients, who were recruited from gastroenterology outpatient clinics at two Scottish NHS Boards, were randomly allocated to an MBCT intervention group (n = 22) or a wait-list control group (n = 22). The MBCT intervention consisted of 16 hours of structured group training over 8 consecutive weeks plus guided home practice and follow-up sessions. The wait-list group received a leaflet entitled ‘Staying well with IBD’. All participants completed a baseline, post-intervention and 6-month follow up assessment. The key objectives were to assess patient eligibility and recruitment/dropout rate, to calculate initial estimates of parameters to the proposed outcome measures (depression, anxiety, disease activity, dispositional mindfulness and quality of life) and to estimate sample size for a future large RCT.

Results

In total, 350 patients were assessed for eligibility. Of these, 44 eligible patients consented to participate. The recruitment rate was 15 %, with main reasons for ineligibility indicated as follows: non-response to invitation, active disease symptoms, planned surgery or incompatibility with group schedule. There was a higher than expected dropout rate of 44 %. Initial estimates of parameters to the proposed outcomes at post-intervention and follow-up showed a significant improvement of scores in the MBCT group when compared to the control for depression, trait anxiety and dispositional mindfulness. The sample-size calculation was guided by estimates of clinically important effects in depression scores.

Conclusions

This pilot study suggests that a multicentre randomized clinical trial testing the effectiveness of MBCT for IBD patients is feasible with some changes to the protocol. Improvement in depression, trait anxiety and dispositional mindfulness scores are promising when coupled with patients reporting a perceived improvement of their quality of life.

Trial registration

ISRCTN27934462. 2 August 2013.

Electronic supplementary material

The online version of this article (doi:10.1186/s13063-015-0909-5) contains supplementary material, which is available to authorized users.

Searching for randomized controlled trials and systematic reviews on exercise. A descriptive study

CONTEXT AND OBJECTIVE

The current paradigm of science is to accumulate as much research data as possible, with less thought given to navigation or synthesis of the resulting mass, which hampers locating and using the research. The aim here was to describe the number of randomized controlled trials (RCTs) and systematic reviews (SRs) focusing on exercise, and their journal sources, that have been indexed in PubMed over time.

DESIGN AND SETTING

Descriptive study conducted at Bond University, Australia.

METHOD

To find RCTs, a search was conducted in PubMed Clinical Queries, using the category "Therapy" and the Medical Subject Headings (MeSH) term "Exercise". To find SRs, a search was conducted in PubMed Clinical Queries, using the category "Therapy", the MeSH term "Exercise" and various methodological filters.

RESULTS

Up until 2011, 9,354 RCTs about exercise were published in 1,250 journals and 1,262 SRs in 513 journals. Journals in the area of Sports Science published the greatest number of RCTs and journals categorized as belonging to "Other health professions" area (for example nursing or psychology) published the greatest number of SRs. The Cochrane Database of Systematic Reviews was the principal source for SRs, with 9.8% of the total, while the Journal of Strength and Conditioning Research and Medicine & Science in Sports & Exercise published 4.4% and 5.0% of the RCTs, respectively.

CONCLUSIONS

The rapid growth and resulting scatter of RCTs and SRs on exercise presents challenges for locating and using this research. Solutions for this issue need to be considered.

A pilot randomised controlled trial of a Telehealth intervention in patients with chronic obstructive pulmonary disease: challenges of clinician-led data collection

BACKGROUND

The increasing prevalence and associated cost of treating chronic obstructive pulmonary disease (COPD) is unsustainable, and focus is needed on self-management and prevention of hospital admissions. Telehealth monitoring of patients' vital signs allows clinicians to prioritise their workload and enables patients to take more responsibility for their health. This paper reports the results of a pilot randomised controlled trial (RCT) of Telehealth-supported care within a community-based COPD supported-discharge service.

METHODS

A two-arm pragmatic pilot RCT was conducted comparing the standard service with a Telehealth-supported service and assessed the potential for progressing into a full RCT. The co-primary outcome measures were the proportion of COPD patients readmitted to hospital and changes in patients' self-reported quality of life. The objectives were to assess the suitability of the methodology, produce a sample size calculation for a full RCT, and to give an indication of cost-effectiveness for both pathways.

RESULTS

Sixty three participants were recruited (n = 31 Standard; n = 32 Telehealth); 15 participants were excluded from analysis due to inadequate data completion or withdrawal from the Telehealth arm. Recruitment was slow with significant gaps in data collection, due predominantly to an unanticipated 60% reduction of staff capacity within the clinical team. The sample size calculation was guided by estimates of clinically important effects and COPD readmission rates derived from the literature. Descriptive analyses showed that the standard service group had a lower proportion of patients with hospital readmissions and a greater increase in self-reported quality of life compared to the Telehealth-supported group. Telehealth was cost-effective only if hospital admissions data were excluded.

CONCLUSIONS

Slow recruitment rates and service reconfigurations prevented progression to a full RCT. Although there are advantages to conducting an RCT with data collection conducted by a frontline clinical team, in this case, challenges arose when resources within the team were reduced by external events. Gaps in data collection were resolved by recruiting a research nurse. This study reinforces previous findings regarding the difficulty of undertaking evaluation of complex interventions, and provides recommendations for the introduction and evaluation of complex interventions within clinical settings, such as prioritisation of research within the clinical remit.

TRIAL REGISTRATION

Current Controlled Trials ISRCTN68856013, registered Nov 2010.

The impact of a self-administered coping intervention on emotional well-being in women awaiting the outcome of IVF treatment: a randomized controlled trial

STUDY QUESTION

What is the effect of the positive reappraisal coping intervention (PRCI) on anxiety in women awaiting the outcome of an IVF/ICSI cycle?

SUMMARY ANSWER

Women reported significantly more anxiety during the waiting period than before treatment, but the use of the PRCI did not significantly reduce anxiety during the waiting period.

WHAT IS KNOWN ALREADY

Waiting for the outcome of IVF/ICSI treatment after embryo transfer is one of the most stressful periods of fertility treatments. At present, no evidence-based coping interventions are available to assist women though this waiting period. The PRCI has been designed to address this unmet need by promoting positive reappraisal coping.

STUDY DESIGN, SIZE, DURATION

A three-armed randomized controlled trail (RCT) was designed to evaluate the PRCI in women undergoing IVF/ICSI. Data were collected between October 2010 and June 2012. A total of 377 participants were randomized to receive either the PRCI and emotional monitoring, emotional monitoring only, or routine care. Only the PRCI-monitoring group received the coping intervention, comprising an explanatory leaflet and ten statements to be read at least once in the morning and once in the evening.

PARTICIPANT, MATERIALS, SETTING, METHODS

To capture the general impact of the PRCI, all three groups completed questionnaires at three time points: just before the waiting period (time 1: stimulation phase), on Day 10 of the 14-day waiting period (time 2: waiting period) and 6 weeks after the start of the waiting period (time 3: 6-week follow-up). In addition, to capture the specific impacts of the PRCI on the days of the waiting period, the PRCI-monitoring group and the monitoring-control group also rated their emotions and reactions daily, for the 14-day waiting period.

MAIN RESULTS AND THE ROLE OF CHANCE

Of the women who agreed to participate and who met eligibility criteria, 377 were randomized. All study participants reported significantly more anxiety and depression during the waiting period than before treatment (P < 0.001). The mean difference in anxiety between time 1 versus time 2 was 1.465 (95% CI 1.098–1.832). The mean difference in depression between time 1 versus time 2 was 0.514 (95% CI 0.215–0.813). Use of the PRCI did not significantly reduce anxiety or depression, or daily negative emotions during the waiting period. However, patients randomized to the PRCI reported significantly more positive emotions during the waiting period (P < 0.001) than the monitoring-control group, and reported the intervention to be easy to use, and as having a positive psychological effect. No significant differences were found between the groups in treatment outcome.

LIMITATIONS, REASONS FOR CAUTION

The lack of difference observed in the present study for anxiety levels between the PRCI and the monitoring-control group could have been due to the effects of monitoring itself or its ability to attenuate or obscure the effects of the PRCI intervention in unknown ways. A randomized group of women who used only the PRCI without daily monitoring would provide more insight.

WIDER IMPLICATIONS OF THE FINDINGS

The PRCI was shown to help women reinterpret the demands of the waiting period in a more positive way. These results are consistent with previous studies showing that positive reappraisal coping is a useful strategy for unpredictable and uncontrollable situations represented by a medical waiting period. This simple low cost self-help coping intervention increases positive affect during the waiting period in an IVF/ICSI treatment. STUDY FUNDING/COMPETENT INTEREST(S): The Women and Baby Division of the University Medical Centre Utrecht funded the study. The authors have no conflicting interest(s).

TRIAL REGISTRATION NUMBER

The study is registered at the Clinical Tials.gov (NCT01701011).

Feasibility of a psychosocial rehabilitation intervention to enhance the involvement of relatives in cancer rehabilitation: pilot study for a randomized controlled trial

BACKGROUND

Cancer often affects the quality of life and well-being of patients as well as their relatives. Previous studies have suggested that relatives should be involved in psychosocial rehabilitation to address the needs for an interpersonal relationship with others in the disease trajectory. We developed an innovative rehabilitation program to be offered to the patient and a relative as a pair.

OBJECTIVE

The aim of the present pilot study was to examine the feasibility of the intervention in a randomized controlled trial (RCT) and to evaluate the impact on quality of life.

METHODS

The study was designed as an RCT comparing the new multimodal psychosocial rehabilitation with the usual services. The intervention comprised three 'supportive talks' and a residential rehabilitation course. From March 2010 to March 2011, participation was offered at the time of diagnosis to patients with lung or gynecological cancer from two departments at Odense University Hospital in Denmark. Questionnaires were used to estimate changes in quality of life (EORTC-QLQ-C30 on global health status) and well-being (WHO-Five Well-Being Index) at baseline and after 2 and 12 months. Information on the participants' views about the rehabilitation intervention was obtained from assessment charts and qualitative interviews.

RESULTS

A total of 209 patients were assessed for eligibility, but only 42 pairs were randomized to the study. The 2-month follow-up was completed by 34 patients and 32 relatives, and 19 patients and 21 relatives completed the 12-month follow-up. A higher dropout rate at the 12-month follow-up was reported in the intervention group compared with controls. Quality of life and well-being increased for patients and relatives in both the intervention and the control group, and no clinically significant difference was observed between the intervention and the control group. Pairs reported that the time of inclusion was inconvenient and that rehabilitation ought to meet their changing needs.

CONCLUSIONS

The pilot study showed that it may be difficult to conduct an RCT of a psychosocial rehabilitation intervention for pairs, and difficulties with inclusion and drop out have to be addressed. Interventions need to be carefully developed and tested before evaluating an effect in a large-scale study.

Challenges to evaluating complex interventions: a content analysis of published papers

BACKGROUND

There is continuing interest among practitioners, policymakers and researchers in the evaluation of complex interventions stemming from the need to further develop the evidence base on the effectiveness of healthcare and public health interventions, and an awareness that evaluation becomes more challenging if interventions are complex.We undertook an analysis of published journal articles in order to identify aspects of complexity described by writers, the fields in which complex interventions are being evaluated and the challenges experienced in design, implementation and evaluation. This paper outlines the findings of this documentary analysis.

METHODS

The PubMed electronic database was searched for the ten year period, January 2002 to December 2011, using the term "complex intervention*" in the title and/or abstract of a paper. We extracted text from papers to a table and carried out a thematic analysis to identify authors' descriptions of challenges faced in developing, implementing and evaluating complex interventions.

RESULTS

The search resulted in a sample of 221 papers of which full text of 216 was obtained and 207 were included in the analysis. The 207 papers broadly cover clinical, public health and methodological topics. Challenges described included the content and standardisation of interventions, the impact of the people involved (staff and patients), the organisational context of implementation, the development of outcome measures, and evaluation.

CONCLUSIONS

Our analysis of these papers suggests that more detailed reporting of information on outcomes, context and intervention is required for complex interventions. Future revisions to reporting guidelines for both primary and secondary research may need to take aspects of complexity into account to enhance their value to both researchers and users of research.

A new generation of pragmatic trials of psychosocial interventions is needed

This Editorial addresses the crucial issue of which research methodology is most suited for capturing the complexity of psychosocial interventions conducted in 'real world' mental health settings. It first examines conventional randomized controlled trial (RCT) methodology and critically appraises its strengths and weaknesses. It then considers the specificity of mental health care treatments and defines the term 'complex' intervention and its implications for RCT design. The salient features of pragmatic RCTs aimed at generating evidence of psychosocial intervention effectiveness are then described. Subsequently, the conceptualization of pragmatic RCTs, and of their further developments - which we propose to call 'new generation' pragmatic trials - in the broader routine mental health service context, is explored. Helpful tools for planning pragmatic RCTs, such as the CONSORT extension for pragmatic trials, and the PRECIS tool are also examined. We then discuss some practical challenges that are involved in the design and implementation of pragmatic trials based on our own experience in conducting the GET UP PIANO Trial. Lastly, we speculate on the ways in which current ideas on the purpose, scope and ethics of mental health care research may determine further challenges for clinical research and evidence-based practice.

Patient and phaRmacist telephonic encounters (PARTE) in an underserved rural patient population with asthma: results of a pilot study

OBJECTIVE

To assess the feasibility, acceptability, and preliminary impact of a telepharmacy intervention in an underserved, rural asthma patient population.

SUBJECTS AND METHODS

Patients with asthma were randomized to receive either standard care or telephone consultations from pharmacists regarding asthma self-management over a 3-month period. Qualitative interviews were conducted to identify participants' attitudes/opinions regarding the intervention. Baseline and follow-up surveys assessed asthma control, patient activation, and medication utilization.

RESULTS

Ninety-eight adults were recruited (78% accrual); 83 completed the study (15% dropout). Participants reported positive opinions and believed the intervention improved their asthma self-management. The intervention group had significantly higher patient activation compared with the control (p<0.05). There were no significant between-group differences regarding asthma control. However, within-group analyses of the intervention group showed an improvement in asthma control (p<0.01) and medication adherence (p<0.01). No within-group differences were found for the control group.

CONCLUSIONS

This telepharmacy intervention is feasible and showed indicators of effectiveness, suggesting the design is well suited for a robust study to evaluate its impact in uncontrolled asthma patients. Pharmacists helping patients manage asthma through telecommunications may resolve access barriers and improve care.

Prevention of sexually transmitted infections in urban communities (Peru PREVEN): a multicomponent community-randomised controlled trial

BACKGROUND

Previous community-randomised trials of interventions to control sexually transmitted infections (STIs) have involved rural settings, were rarely multicomponent, and had varying results. We aimed to assess the effect of a multicomponent intervention on curable STIs in urban young adults and female sex workers (FSWs).

METHODS

In this community-randomised trial, baseline STI screening was done between August, and November, 2002, in random household samples of young adults (aged 18-29 years) and in FSWs in Peruvian cities with more than 50,000 inhabitants. Geographically separate cities were selected, matched into pairs, and randomly allocated to intervention or control groups with an S-PLUS program. Follow-up surveys of random samples were done after 2 years and 3 years. The intervention comprised four modalities: strengthened STI syndromic management by pharmacy workers and clinicians; mobile-team outreach to FSWs for STI screening and pathogen-specific treatment; periodic presumptive treatment of FSWs for trichomoniasis; and condom promotion for FSWs and the general population. Individuals in control cities received standard care. The composite primary endpoint was infection of young adults with Chlamydia trachomatis, Trichomonas vaginalis, or Neisseria gonorrhoeae, or syphilis seroreactivity. Laboratory workers and the data analyst were masked, but fieldworkers, the Peruvian study team, and participants in the outcome surveys were not. All analyses were done by intention to treat. This trial is registered, ISRCTN43722548.

FINDINGS

We did baseline surveys of 15,261 young adults in 24 Peruvian cities. Of those, 20 geographically separate cities were matched into pairs, in each of which one city was assigned to intervention and the other to standard of care. In the 2006 follow-up survey, data for the composite primary outcome were available for 12,930 young adults. We report a non-significant reduction in prevalence of STIs in young adults, adjusted for baseline prevalence, in intervention cities compared with control cities (relative risk 0·84, 95% CI 0·69-1·02; p=0·096). In subgroup analyses, significant reductions were noted in intervention cities in young adult women and FSWs.

INTERPRETATION

Syndromic management of STIs, mobile-team outreach to FSWs, presumptive treatment for trichomoniasis in FSWs, and condom promotion might reduce the composite prevalence of any of the four curable STIs investigated in this trial.

FUNDING

Wellcome Trust and Burroughs Wellcome Fund, National Institutes of Health, Center for AIDS Research, CIPRA, and USAID-Peru.