Does establishing fidelity of treatment help in understanding treatment efficacy? Comment on Bellg et al. (2004)

Fidelity of primary care nurses' delivery of a behavioural change intervention enhancing physical activity in patients at risk of cardiovascular disease: an observational study

OBJECTIVE

To evaluate the fidelity of delivery of a nurse-led intervention to enhance physical activity in patients at risk for cardiovascular diseases, the Activate intervention, by assessing: (1) self-reported fidelity of delivery; (2) observed fidelity of delivery; (3) quality of delivery of the Activate intervention and (4) nurses' beliefs about their capability, motivation, confidence and effectiveness towards delivering the Activate intervention, including behavioural change techniques.

DESIGN

An observational study.

SETTING

General practices in the Netherlands.

PARTICIPANTS

Primary care nurses (n=20) from 16 general practices.

PRIMARY AND SECONDARY OUTCOME MEASURES

Nurses' self-reported fidelity was evaluated using checklists (n=282), and the observed fidelity and quality of delivery were examined using audiorecordings of consultations of the delivery of the Activate intervention (n=42). Nurses' beliefs towards delivering the intervention were assessed using questionnaires (n=72).

RESULTS

The self-reported fidelity was 88.1% and observed fidelity was 85.4%, representing high fidelity. The observed fidelity of applied behavioural change techniques was moderate (75.0%). The observed quality of delivery was sufficient and varied among nurses (mean 2.9; SD 4.4; range 0-4). Nurses' beliefs about their capability, motivation, confidence and effectiveness towards delivering the intervention increased over time.

CONCLUSIONS

Nurses delivered most intervention components as intended with sufficient quality. Nurses believed they were capable, motivated and confident to deliver the intervention. They believed the intervention was effective to increase patients' physical activity level. Despite the high fidelity and moderate fidelity of applied behavioural change techniques, the varying quality of delivery within and across nurses might have diluted the effectiveness of the Activate intervention.

TRIAL REGISTRATION NUMBER

NCT02725203.

Intervention Fidelity Within Trials of Infant Feeding Behavioral Interventions to Prevent Childhood Obesity: A Systematic Review

Background

Intervention fidelity refers to whether an intervention has been implemented as intended. Trials of infant feeding behavioral interventions to prevent childhood obesity show inconsistent evidence of effectiveness. However, intervention fidelity has not been previously explored within these trials, limiting interpretation of findings.

Purpose

To review the use and/or reporting of strategies to enhance and assess intervention fidelity within trials of infant feeding interventions to prevent childhood obesity, and their association with study quality, effectiveness, and publication year.

Methods

Seven electronic databases were searched, with articles screened for inclusion by two reviewers. The National Institutes of Health Behaviour Change Consortium fidelity checklist was used to assess use and/or reporting of fidelity strategies across five domains (design, provider training, delivery, receipt, and enactment).

Results

Ten trials (16 papers) were identified. Average use/reporting of fidelity strategies was moderate (54%), ranging from 28.9% to 76.7%. Levels of use/reporting ranged from 15.9% in the domain of provider training to 95% for enactment. No association was found between these levels and study quality, effectiveness, or publication year.

Conclusions

The moderate use/reporting of fidelity strategies within trials of infant feeding interventions suggests that previous findings of inconsistent effectiveness may not fully reflect the intended interventions. The review highlights key considerations for improving future research, both in the area of behavioral infant feeding and wider behavior change literature. This includes improving reporting across all fidelity domains and ensuring an enhanced focus on provider training and control group content to optimize the translation of research into practice.

PROSPERO Registration number

CRD42016033492.

STructured lifestyle education for people WIth SchizophrEnia (STEPWISE): mixed methods process evaluation of a group-based lifestyle education programme to support weight loss in people with schizophrenia

BACKGROUND

STEPWISE is a theory-informed self-management education programme that was co-produced with service users, healthcare professionals and interventionists to support weight loss for people with schizophrenia. We report the process evaluation to inform understanding about the intervention and its effectiveness in a randomised controlled trial (RCT) that evaluated its efficacy.

METHODS

Following the UK Medical Research Council (MRC) Guidelines for developing and evaluating complex interventions, we explored implementation quality. We considered causal mechanisms, unanticipated consequences and contextual factors associated with variation in actual and intended outcomes, and integrated treatment fidelity, using the programme theory and a pipeline logic model. We followed a modified version of Linnan and Steckler's framework and single case design. Qualitative data from semi-structured telephone interviews with service-users (n = 24), healthcare professionals delivering the intervention (n = 20) and interventionists (n = 7) were triangulated with quantitative process and RCT outcome data and with observations by interventionists, to examine convergence within logic model components.

RESULTS

Training and course materials were available although lacked co-ordination in some trusts. Healthcare professionals gained knowledge and some contemplated changing their practice to reflect the (facilitative) 'style' of delivery. They were often responsible for administrative activities increasing the burden of delivery. Healthcare professionals recognised the need to address antipsychotic-induced weight gain and reported potential value from the intervention (subject to the RCT results). However, some doubted senior management commitment and sustainability post-trial. Service-users found the intervention highly acceptable, especially being in a group of people with similar experiences. Service-users perceived weight loss and lifestyle benefits; however, session attendance varied with 23% (n = 47) attending all group-sessions and 17% (n = 36) attending none. Service-users who lost weight wanted closer monitoring and many healthcare professionals wanted to monitor outcomes (e.g. weight) but it was outside the intervention design. No clinical or cost benefit was demonstrated from the intermediate outcomes (RCT) and any changes in RCT outcomes were not due to the intervention.

CONCLUSIONS

This process evaluation provides a greater understanding of why STEPWISE was unsuccessful in promoting weight loss during the clinical trial. Further research is required to evaluate whether different levels of contact and objective monitoring can support people with schizophrenia to lose weight.

TRIAL REGISTRATION

ISRCTN, ISRCTN19447796. Registered 20 March 2014.

Structured lifestyle education to support weight loss for people with schizophrenia, schizoaffective disorder and first episode psychosis: the STEPWISE RCT

BACKGROUND

Obesity is twice as common in people with schizophrenia as in the general population. The National Institute for Health and Care Excellence guidance recommends that people with psychosis or schizophrenia, especially those taking antipsychotics, be offered a healthy eating and physical activity programme by their mental health care provider. There is insufficient evidence to inform how these lifestyle services should be commissioned.

OBJECTIVES

To develop a lifestyle intervention for people with first episode psychosis or schizophrenia and to evaluate its clinical effectiveness, cost-effectiveness, delivery and acceptability.

DESIGN

A two-arm, analyst-blind, parallel-group, randomised controlled trial, with a 1 : 1 allocation ratio, using web-based randomisation; a mixed-methods process evaluation, including qualitative case study methods and logic modelling; and a cost-utility analysis.

SETTING

Ten community mental health trusts in England.

PARTICIPANTS

People with first episode psychosis, schizophrenia or schizoaffective disorder.

INTERVENTIONS

Intervention group: (1) four 2.5-hour group-based structured lifestyle self-management education sessions, 1 week apart; (2) multimodal fortnightly support contacts; (3) three 2.5-hour group booster sessions at 3-monthly intervals, post core sessions. Control group: usual care assessed through a longitudinal survey. All participants received standard written lifestyle information.

MAIN OUTCOME MEASURES

The primary outcome was change in weight (kg) at 12 months post randomisation. The key secondary outcomes measured at 3 and 12 months included self-reported nutrition (measured with the Dietary Instrument for Nutrition Education questionnaire), objectively measured physical activity measured by accelerometry [GENEActiv (Activinsights, Kimbolton, UK)], biomedical measures, adverse events, patient-reported outcome measures and a health economic assessment.

RESULTS

The trial recruited 414 participants (intervention arm: 208 participants; usual care: 206 participants) between 10 March 2015 and 31 March 2016. A total of 341 participants (81.6%) completed the trial. A total of 412 participants were analysed. After 12 months, weight change did not differ between the groups (mean difference 0.0 kg, 95% confidence interval -1.59 to 1.67 kg; p = 0.964); physical activity, dietary intake and biochemical measures were unchanged. Glycated haemoglobin, fasting glucose and lipid profile were unchanged by the intervention. Quality of life, psychiatric symptoms and illness perception did not change during the trial. There were three deaths, but none was related to the intervention. Most adverse events were expected and related to the psychiatric illness. The process evaluation showed that the intervention was acceptable, with participants valuing the opportunity to interact with others facing similar challenges. Session feedback indicated that 87.2% of participants agreed that the sessions had met their needs. Some indicated the desire for more ongoing support. Professionals felt that the intervention was under-resourced and questioned the long-term sustainability within current NHS settings. Professionals would have preferred greater access to participants' behaviour data to tailor the intervention better. The incremental cost-effectiveness ratio from the health-care perspective is £246,921 per quality-adjusted life-year (QALY) gained and the incremental cost-effectiveness ratio from the societal perspective is £367,543 per QALY gained.

CONCLUSIONS

Despite the challenges of undertaking clinical research in this population, the trial successfully recruited and retained participants, indicating a high level of interest in weight management interventions; however, the STEPWISE intervention was neither clinically effective nor cost-effective. Further research will be required to define how overweight and obesity in people with schizophrenia should be managed. The trial results suggest that lifestyle programmes for people with schizophrenia may need greater resourcing than for other populations, and interventions that have been shown to be effective in other populations, such as people with diabetes mellitus, are not necessarily effective in people with schizophrenia.

TRIAL REGISTRATION

Current Controlled Trials ISRCTN19447796.

FUNDING

This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 22, No. 65. See the NIHR Journals Library website for further project information.

Use of dynamic systems methods to characterize dyadic interactions in smoking cessation behavioural support sessions: A feasibility study

BACKGROUND

Understanding how behaviour change techniques (BCTs) operate in practice requires a method for characterizing the reciprocal, dynamic, and real-time nature of behavioural support interactions between practitioners and clients. State space grids (SSGs) are an observational, dynamic systems methodology used to map the trajectory of dyadic interactions in real time. By mapping the flow of events in terms of practitioner and client actions, SSGs are potentially well suited to characterize behavioural support sessions.

PURPOSE

To develop reliable methods and examine the feasibility of using the SSG methodology for characterizing practitioners' delivery of and clients' response to BCTs in smoking cessation behavioural support sessions.

METHODS

Smoking cessation behavioural support sessions were video-recorded and transcribed verbatim (n = 6 recordings; 2,916 statements). All speech was coded independently by two researchers for content and duration using published frameworks for specifying practitioner-delivered and client-received BCTs in smoking cessation interactions. Inter-rater reliability was assessed. Indices of practitioner-client interaction dynamics were derived: (1) reciprocity (i.e., attractor states, content congruence, conditional pairing) and (2) temporal patterning (i.e., variability, inter-grid distance, combinatory micro-patterning, sessional macropatterning). The extent to which indices can describe differences between sessions involving different practitioners and clients was examined.

RESULTS

Inter-rater reliability was moderate at 72% agreement. Indices of reciprocity and temporal patterning characterized differences between sessions involving different practitioners and clients.

CONCLUSIONS

State space grids provide a method for characterizing the complexity and variability of practitioner-delivered and client-received BCTs in behavioural support sessions. This method has potential to add explanatory value to smoking cessation intervention outcomes. Statement of Contribution What is already known on this subject? Frameworks exist for characterizing practitioner-delivered and client-received behaviour change techniques (BCTs). Methods are still needed to investigate which BCTs are effective under what conditions. State space grids (SSGs) are a dynamic systems method that may better characterize behavioural support interactions. What does this study add? First reliable, dynamic systems, SSG coding procedures, methods, and measures to characterize behavioural support. A method for examining reciprocality and temporal patterning of BCT delivery and receipt. Establishes a dynamic systems method that adds explanatory value to the outcomes of interventions.

An integrative literature review on nursing interventions aimed at increasing self-care among heart failure patients

OBJECTIVE

to analyze and summarize knowledge concerning critical components of interventions that have been proposed and implemented by nurses with the aim of optimizing self-care by heart failure patients.

METHODS

PubMed and CINAHL were the electronic databases used to search full peer-reviewed papers, presenting descriptions of nursing interventions directed to patients or to patients and their families and designed to optimize self-care. Forty-two studies were included in the final sample (n=4,799 patients).

RESULTS

this review pointed to a variety and complexity of nursing interventions. As self-care encompasses several behaviors, interventions targeted an average of 3.6 behaviors. Educational/counselling activities were combined or not with cognitive behavioral strategies, but only about half of the studies used a theoretical background to guide interventions. Clinical assessment and management were frequently associated with self-care interventions, which varied in number of sessions (1 to 30); length of follow-up (2 weeks to 12 months) and endpoints.

CONCLUSIONS

these findings may be useful to inform nurses about further research in self-care interventions in order to propose the comparison of different modalities of intervention, the use of theoretical background and the establishment of endpoints to evaluate their effectiveness.

Using mixed methods to assess fidelity of delivery and its influencing factors in a complex self-management intervention for people with osteoarthritis and low back pain

OBJECTIVES AND DESIGN

Despite an increasing awareness of the importance of fidelity of delivery within complex behaviour change interventions, it is often poorly assessed. This mixed methods study aimed to establish the fidelity of delivery of a complex self-management intervention and explore the reasons for these findings using a convergent/triangulation design.

SETTING

Feasibility trial of the Self-management of Osteoarthritis and Low back pain through Activity and Skills (SOLAS) intervention (ISRCTN49875385), delivered in primary care physiotherapy.

METHODS AND OUTCOMES

60 SOLAS sessions were delivered across seven sites by nine physiotherapists. Fidelity of delivery of prespecified intervention components was evaluated using (1) audio-recordings (n=60), direct observations (n=24) and self-report checklists (n=60) and (2) individual interviews with physiotherapists (n=9). Quantitatively, fidelity scores were calculated using percentage means and SD of components delivered. Associations between fidelity scores and physiotherapist variables were analysed using Spearman's correlations. Interviews were analysed using thematic analysis to explore potential reasons for fidelity scores. Integration of quantitative and qualitative data occurred at an interpretation level using triangulation.

RESULTS

Quantitatively, fidelity scores were high for all assessment methods; with self-report (92.7%) consistently higher than direct observations (82.7%) or audio-recordings (81.7%). There was significant variation between physiotherapists' individual scores (69.8% - 100%). Both qualitative and quantitative data (from physiotherapist variables) found that physiotherapists' knowledge (Spearman's association at p=0.003) and previous experience (p=0.008) were factors that influenced their fidelity. The qualitative data also postulated participant-level (eg, individual needs) and programme-level factors (eg, resources) as additional elements that influenced fidelity.

CONCLUSION

The intervention was delivered with high fidelity. This study contributes to the limited evidence regarding fidelity assessment methods within complex behaviour change interventions. The findings suggest a combination of quantitative methods is suitable for the assessment of fidelity of delivery. A mixed methods approach provided a more insightful understanding of fidelity and its influencing factors.

TRIAL REGISTRATION NUMBER

ISRCTN49875385; Pre-results.

A multidimensional approach to assessing intervention fidelity in a process evaluation of audit and feedback interventions to reduce unnecessary blood transfusions: a study protocol

Background

In England, NHS Blood and Transplant conducts national audits of transfusion and provides feedback to hospitals to promote evidence-based practice. Audits demonstrate 20% of transfusions fall outside guidelines. The AFFINITIE programme (Development & Evaluation of Audit and Feedback INterventions to Increase evidence-based Transfusion practIcE) involves two linked, 2×2 factorial, cluster-randomised trials, each evaluating two theoretically-enhanced audit and feedback interventions to reduce unnecessary blood transfusions in UK hospitals. The first intervention concerns the content/format of feedback reports. The second aims to support hospital transfusion staff to plan their response to feedback and includes a web-based toolkit and telephone support. Interpretation of trials is enhanced by comprehensively assessing intervention fidelity. However, reviews demonstrate fidelity evaluations are often limited, typically only assessing whether interventions were delivered as intended. This protocol presents methods for assessing fidelity across five dimensions proposed by the Behaviour Change Consortium fidelity framework, including intervention designer-, provider- and recipient-levels.

Methods

(1) Design: Intervention content will be specified in intervention manuals in terms of component behaviour change techniques (BCTs). Treatment differentiation will be examined by comparing BCTs across intervention/standard practice, noting the proportion of unique/convergent BCTs. (2) Training: draft feedback reports and audio-recorded role-play telephone support scenarios will be content analysed to assess intervention providers’ competence to deliver manual-specified BCTs. (3) Delivery: intervention materials (feedback reports, toolkit) and audio-recorded telephone support session transcripts will be content analysed to assess actual delivery of manual-specified BCTs during the intervention period. (4) Receipt and (5) enactment: questionnaires, semi-structured interviews based on the Theoretical Domains Framework, and objective web-analytics data (report downloads, toolkit usage patterns) will be analysed to assess hospital transfusion staff exposure to, understanding and enactment of the interventions, and to identify contextual barriers/enablers to implementation. Associations between observed fidelity and trial outcomes (% unnecessary transfusions) will be examined using mediation analyses.

Discussion

If the interventions have acceptable fidelity, then results of the AFFINITIE trials can be attributed to effectiveness, or lack of effectiveness, of the interventions. Hence, this comprehensive assessment of fidelity will be used to interpret trial findings. These methods may inform fidelity assessments in future trials.

Trial registration

ISRCTN 15490813. Registered 11/03/2015

Electronic supplementary material

The online version of this article (doi:10.1186/s13012-016-0528-x) contains supplementary material, which is available to authorized users.

Fidelity monitoring across the seven studies in the Consortium of Hospitals Advancing Research on Tobacco (CHART)

BACKGROUND

This paper describes fidelity monitoring (treatment differentiation, training, delivery, receipt and enactment) across the seven National Institutes of Health-supported Consortium of Hospitals Advancing Research on Tobacco (CHART) studies. The objectives of the study were to describe approaches to monitoring fidelity including treatment differentiation (lack of crossover), provider training, provider delivery of treatment, patient receipt of treatment, and patient enactment (behavior) and provide examples of application of these principles.

METHODS

Conducted between 2010 and 2014 and collectively enrolling over 9500 inpatient cigarette smokers, the CHART studies tested different smoking cessation interventions (counseling, medications, and follow-up calls) shown to be efficacious in Cochrane Collaborative Reviews. The CHART studies compared their unique treatment arm(s) to usual care, used common core measures at baseline and 6-month follow-up, but varied in their approaches to monitoring the fidelity with which the interventions were implemented.

RESULTS

Treatment differentiation strategies included the use of a quasi-experimental design and monitoring of both the intervention and control group. Almost all of the studies had extensive training for personnel and used a checklist to monitor the intervention components, but the items on these checklists varied widely and were based on unique aspects of the interventions, US Public Health Service and Joint Commission smoking cessation standards, or counselor rapport. Delivery of medications ranged from 31 to 100 % across the studies, with higher levels from studies that gave away free medications and lower levels from studies that sought to obtain prescriptions for the patient in real world systems. Treatment delivery was highest among those studies that used automated (interactive voice response and website) systems, but this did not automatically translate into treatment receipt and enactment. Some studies measured treatment enactment in two ways (e.g., counselor or automated system report versus patient report) showing concurrence or discordance between the two measures.

CONCLUSION

While fidelity monitoring can be challenging especially in dissemination trials, the seven CHART studies used a variety of methods to enhance fidelity with consideration for feasibility and sustainability.

TRIAL REGISTRATION

Dissemination of Tobacco Tactics for hospitalized smokers. Clinical Trials Registration No. NCT01309217.Smoking cessation in hospitalized smokers. Clinical Trials Registration No. NCT01289275.Using "warm handoffs" to link hospitalized smokers with tobacco treatment after discharge: study protocol of a randomized controlled trial. Clinical Trials Registration No. NCT01305928.Web-based smoking cessation intervention that transitions from inpatient to outpatient. Clinical Trials Registration No. NCT01277250.Effectiveness of smoking-cessation interventions for urban hospital patients. Clinical Trials Registration No. NCT01363245.Comparative effectiveness of post-discharge interventions for hospitalized smokers. Clinical Trials Registration No. NCT01177176.Health and economic effects from linking bedside and outpatient tobacco cessation services for hospitalized smokers in two large hospitals. Clinical Trials Registration No. NCT01236079.

Treatment manuals, training and successful provision of stop smoking behavioural support

OBJECTIVE

Translating evidence-based behaviour change interventions into practice is aided by use of treatment manuals specifying the recommended content and format of interventions, and evidence-based training. This study examined whether outcomes of stop smoking behavioural support differed with practitioner's use and evaluation of treatment manuals, or practitioner's training.

METHODS

English stop smoking practitioners were invited to complete an online survey including questions on: practitioners' training, availability, use and perceived utility of manuals, and annual biochemically-validated success rates of quit attempts supported (practitioner-reported). Mean success rates were compared between practitioners with/without access to manuals, those using/not using manuals, perceived utility ratings of manuals, and consecutive levels of training completed.

RESULTS

Success rates were higher if practitioners had a manual (Mean (SD) = 54.0 (24.0) versus 48.0 (25.3), t(838) = 2.48, p = 0.013; n = 840), used a manual (F(2,8237) = 4.78, p = 0.009, n = 840), perceived manuals as more useful (F(3,834) = 2.90, p = 0.034, n = 840), and had completed training (F(3,709) = 4.81, p = 0.002, n = 713). Differences were diminished when adjusting for professional and demographic characteristics and no longer reached statistical significance using a conventional alpha for perceived utility of manuals and training status (both p = 0.1).

CONCLUSIONS

Practitioners' performance in supporting smokers to quit varied with availability and use of treatment manuals. Evidence was weaker for perceived utility of manuals and practitioners' evidence-based training. Ensuring practitioners have access to treatment manuals within their service, promoting manual use, and training practitioners to competently apply manuals is likely to contribute to higher success rates in clinical practice.

Implementation fidelity of physiotherapist-delivered group education and exercise interventions to promote self-management in people with osteoarthritis and chronic low back pain: a rapid review part II

BACKGROUND

Implementation fidelity is the extent to which an intervention is delivered as intended by intervention developers, and is extremely important as it increases confidence that changes in study outcomes are due to the effect of the intervention itself and not due to variability in implementation. A paucity of literature exists concerning implementation fidelity in physiotherapy research.

DESIGN AND OBJECTIVES

This rapid review aimed to evaluate the implementation fidelity of group-based self-management interventions for people with osteoarthritis (OA) and/or chronic low back pain (CLBP).

METHOD

Group-based self-management interventions delivered by health-care professionals (including at least one physiotherapist) involving adults with OA and/or CLBP were eligible for inclusion. The National Institutes of Health Behaviour Change Consortium Treatment Fidelity checklist was used to assess fidelity and applied independently by two reviewers.

RESULTS

In total, 22 studies were found. Fidelity was found to be very low (mean score 36%) within the included studies with only one study achieving >80% on the framework. The domain of Training of Providers achieved the lowest fidelity rating (10%) across all studies.

CONCLUSIONS

Overall levels of implementation fidelity are low in self-management interventions for CLBP and/or OA; however it is unclear whether fidelity is poor within the trials included in this review, or just poorly reported. There is a need for the development of fidelity reporting guidelines and for the refinement of fidelity frameworks upon which to base these guidelines.

Assessing fidelity of delivery of smoking cessation behavioural support in practice

BACKGROUND

Effectiveness of evidence-based behaviour change interventions is likely to be undermined by failure to deliver interventions as planned. Behavioural support for smoking cessation can be a highly cost-effective, life-saving intervention. However, in practice, outcomes are highly variable. Part of this may be due to variability in fidelity of intervention implementation. To date, there have been no published studies on this. The present study aimed to: evaluate a method for assessing fidelity of behavioural support; assess fidelity of delivery in two English Stop-Smoking Services; and compare the extent of fidelity according to session types, duration, individual practitioners, and component behaviour change techniques (BCTs).

METHODS

Treatment manuals and transcripts of 34 audio-recorded behavioural support sessions were obtained from two Stop-Smoking Services and coded into component BCTs using a taxonomy of 43 BCTs. Inter-rater reliability was assessed using percentage agreement. Fidelity was assessed by examining the proportion of BCTs specified in the manuals that were delivered in individual sessions. This was assessed by session type (i.e., pre-quit, quit, post-quit), duration, individual practitioner, and BCT.

RESULTS

Inter-coder reliability was high (87.1%). On average, 66% of manual-specified BCTs were delivered per session (SD 15.3, range: 35% to 90%). In Service 1, average fidelity was highest for post-quit sessions (69%) and lowest for pre-quit (58%). In Service 2, fidelity was highest for quit-day (81%) and lowest for post-quit sessions (56%). Session duration was not significantly correlated with fidelity. Individual practitioner fidelity ranged from 55% to 78%. Individual manual-specified BCTs were delivered on average 63% of the time (SD 28.5, range: 0 to 100%).

CONCLUSIONS

The extent to which smoking cessation behavioural support is delivered as specified in treatment manuals can be reliably assessed using transcripts of audiotaped sessions. This allows the investigation of the implementation of evidence-based practice in relation to smoking cessation, a first step in designing interventions to improve it. There are grounds for believing that fidelity in the English Stop-Smoking Services may be low and that routine monitoring is warranted.

Strategies used and data obtained during treatment fidelity monitoring

BACKGROUND

Treatment fidelity, also called intervention fidelity, is an important component of testing treatment efficacy. Although examples of strategies needed to address treatment fidelity have been provided in several published reports, data describing variations that might compromise efficacy testing have been omitted.

OBJECTIVES

The aim of this study is to describe treatment fidelity monitoring strategies and data within the context of a nursing clinical trial.

METHODS

A three-group, randomized, controlled trial compared intervention (paced respiration) to attention control (fast, shallow breathing) to usual care for management of hot flashes and other menopausal symptoms. Data from both staff and participants were collected to assess treatment fidelity.

RESULTS

Staff measures for treatment delivery indicated good adherence to protocols. Participant ratings of expectancy and credibility were not statistically different between intervention and attention control; however, the attention control was significantly more acceptable (p < .05). Intervention participant data indicated good treatment receipt and enactment with mean breath rates at each time point falling within the target range. Practice log data for both intervention and attention control indicated lower adherence of once-daily rather than twice-daily practice.

DISCUSSION

Despite strengths in fidelity monitoring, some challenges were identified that have implications for other similar intervention studies.

Intervention fidelity: ensuring application to practice for youth and families

PURPOSE

Limited information on intervention fidelity is available in published studies with youth and families. The components of intervention fidelity, the complexity of measurement in these studies, and strategies for measuring intervention fidelity are described.

CONCLUSIONS

Strategies for ensuring intervention fidelity according to the Treatment Fidelity Workgroup of the Behavior Change Consortium in the areas of study design, provider training, treatment delivery, treatment receipt, and treatment enactment provide guidance for evaluating or developing intervention fidelity plans.

PRACTICE IMPLICATIONS

Ensuring the quality of intervention fidelity in evidence-based reviews or when developing new interventions is essential for translating findings into practice.

Intervention fidelity in psychosocial oncology

Intervention fidelity refers to strategies that practitioners and researchers use to monitor, enhance, or evaluate the accuracy and consistency of the delivery of an intervention to ensure that it is implemented according to how it was planned. The purpose of the authors in this article was to evaluate intervention fidelity in the psychosocial oncology intervention effectiveness research. Twenty-eight studies located in a previous systematic review on psychosocial oncology intervention effectiveness comprised the sample for this research. A treatment fidelity checklist was applied to each study independently by each author (MP & PB). Percent agreement between raters ranged from 68% to 100% (M = 89%). Overall, the mean proportion of adherence was 0.57 (SD 0.12), which may be considered to be moderate fidelity. Critical examination and applicability of the checklist in examining and assessing intervention fidelity were highlighted and discussed. Overall, intervention fidelity was adequately addressed in the psychosocial oncology intervention effectiveness research, and integrity was confirmed in the majority of studies reviewed. Suggestions for future psychosocial oncology effectiveness research were made.

The Assessment, Monitoring, and Enhancement of Treatment Fidelity In Public Health Clinical Trials

OBJECTIVES: To discuss methods of preservation of treatment fidelity in health behavior change trials conducted in public health contexts. METHODS: The treatment fidelity framework provided by the NIH's Behavioral Change Consortium (BCC) (1) includes five domains of treatment fidelity (Study Design, Training, Delivery, Receipt, and Enactment). A measure of treatment fidelity was previously developed and validated using these categories. RESULTS: Strategies for assessment, monitoring, and enhancing treatment fidelity within each of the five treatment fidelity domains are discussed. The previously created measure of treatment fidelity is updated to include additional items on selecting providers, additional confounders, theory testing, and multicultural considerations. CONCLUSIONS: Implementation of a treatment fidelity plan may require extra staff time and costs. However, the economic and scientific costs of lack of attention to treatment fidelity are far greater than the costs of treatment fidelity implementation. Maintaining high levels of treatment fidelity with flexible adaptation according to setting, provider, and patient is the goal for public health trials.

Therapeutic evaluation on complex interventions of integrative medicine and the potential role of data mining

It is a common view that the integration of Chinese medicine (CM) and modern Western medicine is an efficient way to facilitate the development of CM. Integrative medicine is a kind of complex interventions. Scientific therapeutic evaluation plays a crucial role in making integrative medicine universally acknowledged. However, the modern method of clinical study, which is based on the concept of evidence-based medicine, mostly focuses on the population characteristics and single interventional factor. As a result, it is difficult for this method to totally adapt to the clinical features of CM and integrative medicine as complex interventions. One possible way to solve this issue is to improve and integrate with the existing method and to utilize the evaluation model on complex interventions from abroad. As an interdisciplinary technique, data mining involves database technology, artificial intelligence, machine learning, statistics, neural network and some other latest technologies, and has been widely used in the field of CM. Therefore, the application of data mining in the therapeutic evaluation of integrative medicine has broad prospects.

Deconstruction of nurse-delivered patient self-management interventions for symptom management: factors related to delivery enactment and response

This research examines the processes by which patient self-management interventions are related to symptom responses among cancer patients. A total of 333 patients from two randomized clinical trials were combined. Each patient received a six-contact 8-week patient self-management intervention delivered by a nurse to address symptoms. Nurses' decisions to deliver the strategies, patient enactment of strategies, and their success were investigated using patient- and symptom-level characteristics. Generalized estimating equation modeling accounted for clustering of symptoms and strategies delivered for each symptom within patient. Patient self-management intervention strategies were classified into four classes. Strategies were delivered by nurses for symptoms with higher interference and longer duration. Patient and symptom factors were related to enactment strategies. Symptom responses were related to number of strategies tried by patients. Delivery and enactment of strategies were related to both patient and symptom characteristics.

Development of a tool to assess fidelity to a psycho-educational intervention

AIM

This paper is a description of a method to develop and conduct a customized psycho-educational intervention fidelity assessment as part of pilot work for an efficacy study. A tool designed to assess treatment fidelity to a psycho-education intervention for patients with end-stage renal disease and their surrogate decision makers, Sharing the Patient's Illness Representations to Increase Trust, is presented as an illustration.

BACKGROUND

Despite the specificity and idiosyncrasy of individual interventions and the call to systematically evaluate treatment fidelity, how to accomplish this goal has not been clarified. Tools to adequately measure treatment fidelity are lacking.

METHODS

We developed the Sharing the Patient's Illness Representations to Increase Trust Treatment Fidelity Assessment tool by identifying elements that were idiosyncratic to the intervention and those that could be adapted from existing tools. The tool has four components: overall adherence to the intervention content elements; pacing of the intervention delivery; overall dyad responsiveness; and, overall quality index of intervention delivery. The study was undertaken between 2006 and 2008.

RESULTS

Inter-rater reliability ranged from 0.80 to 0.87 for the four components. The tool showed utility in training and monitoring, such as detecting unplanned content elements delivered and the use of proscribed communication behaviours.

CONCLUSION

Psycho-educational interventions are one of the most common types of nursing interventions worldwide. Use of fidelity assessment tools customized to the individual interventions may enhance systematic evaluation of training and monitoring treatment fidelity.

Monitoring treatment fidelity in a randomized controlled trial of a complex intervention

AIM

This paper is a report of a study to describe how treatment fidelity is being enhanced and monitored, using a model from the National Institutes of Health Behavior Change Consortium.

BACKGROUND

The objective of treatment fidelity is to minimize errors in interpreting research trial outcomes, and to ascribe those outcomes directly to the intervention at hand. Treatment fidelity procedures are included in trials of complex interventions to account for inferences made from study outcomes. Monitoring treatment fidelity can help improve study design, maximize reliability of results, increase statistical power, determine whether theory-based interventions are responsible for observed changes, and inform the research dissemination process.

METHODS

Treatment fidelity recommendations from the Behavior Change Consortium were applied to the SPHERE study (Secondary Prevention of Heart DiseasE in GeneRal PracticE), a randomized controlled trial of a complex intervention. Procedures to enhance and monitor intervention implementation included standardizing training sessions, observing intervention consultations, structuring patient recall systems, and using written practice and patient care plans. The research nurse plays an important role in monitoring intervention implementation.

FINDINGS

Several methods of applying treatment fidelity procedures to monitoring interventions are possible. The procedure used may be determined by availability of appropriate personnel, fiscal constraints, or time limits. Complex interventions are not straightforward and necessitate a monitoring process at trial stage.

CONCLUSION

The Behavior Change Consortium's model of treatment fidelity is useful for structuring a system to monitor the implementation of a complex intervention, and helps to increase the reliability and validity of evaluation findings.

Factors contributing to intervention fidelity in a multi-site chronic disease self-management program

BACKGROUND AND OBJECTIVES

Disease self-management programs have been a popular approach to reducing morbidity and mortality from chronic disease. Replicating an evidence-based disease management program successfully requires practitioners to ensure fidelity to the original program design.

METHODS

The Florida Health Literacy Study (FHLS) was conducted to investigate the implementation impact of the Pfizer, Inc. Diabetes Mellitus and Hypertension Disease Self-Management Program based on health literacy principles in 14 community health centers in Florida. The intervention components discussed include health educator recruitment and training, patient recruitment, class sessions, utilization of program materials, translation of program manuals, patient retention and follow-up, and technical assistance.

RESULTS

This report describes challenges associated with achieving a balance between adaptation for cultural relevance and fidelity when implementing the health education program across clinic sites. This balance was necessary to achieve effectiveness of the disease self-management program. The FHLS program was implemented with a high degree of fidelity to the original design and used original program materials. Adaptations identified as advantageous to program participation are discussed, such as implementing alternate methods for recruiting patients and developing staff incentives for participation.

CONCLUSION

Effective program implementation depends on the talent, skill and willing participation of clinic staff. Program adaptations that conserve staff time and resources and recognize their contribution can increase program effectiveness without jeopardizing its fidelity.

Implications of translating research into practice: a medication management intervention

Through programs such as the Administration on Aging's Evidence-Based Prevention Initiative, researchers and practitioners are developing translational research studies seeking to implement rigorously tested, evidence-based interventions in new practice settings and evaluate the continuing effectiveness of these interventions. One such translational study is the Community-Based Medications Management Intervention (CBM Intervention), a collaborative effort to implement a medication management screening and intervention protocol in community-based waiver care management programs. The overall goals of the CBM Intervention are to implement an evidence-based medication management intervention in a California Medicaid waiver care management program, and to evaluate the effect of client-, intervention-, and organizational-level characteristics on resolving identified medication problems. This article presents the need for improved medication management in a frail, community-dwelling, older adult population and describes the CBM Intervention as an example of translating an evidence-based practice beyond its original efficacy trial in a home healthcare program into a care management program. It discusses critical factors involved in translating research into practice using a translational research framework, Promoting Action on Research Implementation in Health Services (PARIHS). Our experience suggests that although implementing research into practice can positively impact client care, professional skill enhancement and organizational effectiveness, this is very challenging work requiring signification facilitation for successful outcomes.

Intervention fidelity: Lessons learned from an unsuccessful pilot study

Although the design of the study we undertook was randomized, the groups were statistically similar, the intervention was based on evidence of previous efficacy, and the treatment was delivered exactly as planned, this pilot study on an intervention to promote motivation for weight loss in new mothers failed to produce an effect. A closer examination using criteria for intervention fidelity revealed a number of weaknesses. Both treatment integrity and differentiation merit careful consideration in intervention design.

Adapting efficacious interventions: advancing translational research in HIV prevention

The Human Immunodeficiency Virus (HIV) has infected approximately 1.5 million people in the United States. Type 1 translation research (basic research, methods development, and efficacy trials) has yielded multiple efficacious behavioral HIV prevention programs. Type 2 translation research (dissemination and effectiveness studies) has been less prevalent or successful. Adaptation of efficacious interventions for culturally diverse populations has received increasing researcher attention, and empirical validation of adaptation procedures promises to help bridge the gap between Type 1 and Type 2 studies. In this article, the authors briefly discuss the development, testing, and dissemination of efficacious HIV prevention programs and then focus on research-based principles and processes that can guide researchers'adaptation efforts and steps that researchers can take to help empower practitioners to conduct science-based adaptation. Greater collaboration between researchers and service providers to test adaptation frameworks promises to benefit both research and practice.

Enhancing intervention fidelity: a means of strengthening study impact

PURPOSE

Intervention studies that address the goals of the Healthy People initiative are increasing, and many focus on health concerns of children and adolescents. The purpose of this paper is to present strategies and discuss the rationale for including intervention fidelity in study design.

CONCLUSION

Researchers need to verify that their interventions were delivered as designed (intervention fidelity), and that variations from the design can be assessed. Infidelity can result in non-significant findings that are not due to the study design but rather to elements that affected the intervention delivery.

IMPLICATIONS

Addressing intervention fidelity in study design supports the researcher's conclusion about associations between intervention and outcomes and helps reduce premature abandonment of potentially useful interventions.