Trials in primary care: statistical issues in the design, conduct and evaluation of complex interventions

Mapping review of 'proof-of-concept' in mental health implementation research using the TRL framework: a need for a better focus and conceptual clarification

BACKGROUND

Proof-of-concept (PoC) development is a key step in implementation sciences. However, there is a dearth of studies in this area and the use of this term in health and social sciences is ambiguous.

OBJECTIVE

The objective was to remove the ambiguity surrounding the PoC and pilot study stage in the research development process using a standard system to rate the development of projects and applications provided by the Technology Readiness Levels (TRL) framework.

DESIGN

Mapping review and critical analysis using TRL as the standard measure.

SEARCH STRATEGY AND CHARTING METHOD

PubMed and PsycInfo databases were searched for papers that reported PoC studies of mental health interventions up to August 2023. Data were extracted, described and tabulated.

ELIGIBILITY CRITERIA

Included were PoC studies in mental health implementation research. Exclusion criteria were research relating to biomedical (drugs) development, neurocognitive tools, neuropsychology, medical devices, literature reviews or discussion papers or that did not include the term 'proof-of-concept' in the title, abstract or text.

RESULTS

From the 83 citations generated from the database search, 22 studies were included in this mapping review. Based on the study title, abstract and text, studies were categorised by research development stage according to the TRL framework. This review showed 95% of the studies used PoC incorrectly to describe the development stage of their research but which were not at this specific level of project development.

CONCLUSIONS

The TRL was a useful reference framework to improve terminological clarity around the term 'proof-of-concept' in implementation research. To extend the use of TRL in implementation sciences, this framework has now been adapted and validated to a health and social science-related research context accompanied by a health-related glossary of research process terms and definitions to promote a common vocabulary and shared understanding in implementation sciences.

START NOW: a cognitive behavioral skills training for adolescent girls with conduct or oppositional defiant disorder - a randomized clinical trial

BACKGROUND

Conduct disorder (CD) and oppositional defiant disorder (ODD) both convey a high risk for maladjustment later in life and are understudied in girls. Here, we aimed at confirming the efficacy of START NOW, a cognitive-behavioral, dialectical behavior therapy-oriented skills training program aiming to enhance emotion regulation skills, interpersonal and psychosocial adjustment, adapted for female adolescents with CD or ODD.

METHODS

A total of 127 girls were included in this prospective, cluster randomized, multi-center, parallel group, quasi-randomized, controlled phase III trial, which tested the efficacy of START NOW (n = 72) compared with standard care (treatment as usual, TAU, n = 55). All female adolescents had a clinical diagnosis of CD or ODD, were 15.6 (±1.5) years on average (range: 12-20 years), and were institutionalized in youth welfare institutions. The two primary endpoints were the change in number of CD/ODD symptoms between (1) baseline (T1) and post-treatment (T3), and (2) between T1 and 12-week follow-up (T4).

RESULTS

Both treatment groups showed reduced CD/ODD symptoms at T3 compared with T1 (95% CI: START NOW = -4.87, -2.49; TAU = -4.94, -2.30). There was no significant mean difference in CD/ODD symptom reduction from T1 to T3 between START NOW and TAU (-0.056; 95% CI = -1.860, 1.749; Hedge's g = -0.011). However, the START NOW group showed greater mean symptom reduction from T1 to T4 (-2.326; 95% CI = -4.274, -0.378; Hedge's g = -0.563). Additionally, secondary endpoint results revealed a reduction in staff reported aggression and parent-reported irritability at post assessment.

CONCLUSIONS

Although START NOW did not result in greater symptom reduction from baseline to post-treatment compared with TAU, the START NOW group showed greater symptom reduction from baseline to follow-up with a medium effect size, which indicates a clinically meaningful delayed treatment effect.

Sharing Health Care Wishes in Primary Care (SHARE) among older adults with possible cognitive impairment in primary care: Study protocol for a randomized controlled trial

OBJECTIVE

Little is known about effective strategies to improve advance care planning (ACP) for persons with cognitive impairment in primary care, the most common setting of care. We describe a randomized controlled trial to test the efficacy of a multicomponent communication intervention, "Sharing Healthcare Wishes in Primary Care" (SHARE).

PARTICIPANTS

Planned enrollment of 248 dyads of adults 80 years and older with possible cognitive impairment and their care partner, from primary care clinics at 2 Mid-Atlantic health systems.

METHODS

The treatment protocol encompasses an introductory letter from the clinic; access to a designated facilitator trained in ACP; person-family agenda-setting to align perspectives about the family's role; and print education. The control protocol encompasses minimally enhanced usual care, which includes print education and a blank advance directive. Randomization occurs at the individual dyad-level. Patient and care partner surveys are fielded at baseline, 6-, 12-, and 24- months. Fidelity of interventionist delivery of the protocol is measured through audio-recordings of ACP conversations and post-meeting reports, and by ongoing monitoring and support of interventionists.

OUTCOMES

The primary outcome is quality of end-of-life care communication at 6 months; secondary outcomes include ACP process measures. An exploratory aim examines end-of-life care quality and bereaved care partner experiences for patients who die by 24 months.

CONCLUSIONS

Caregiver burden, clinician barriers, and impaired decisional capacity amplify the difficulty and importance of ACP discussions in the context of cognitive impairment: this intervention will comprehensively examine communication processes for this special subpopulation in a key setting of primary care.

REGISTRATION

ClinicalTrials.gov: NCT04593472.

PreEMPT (Preterm infant Early intervention for Movement and Participation Trial): Feasibility outcomes of a randomised controlled trial

PURPOSE

Investigate feasibility of PreEMPT: a novel participation-focused, early physiotherapy intervention for preterm infants in regional Australia.

MATERIALS AND METHODS

Participants were infants born <35 weeks, residing in regional Australia. Sixteen infants were recruited then randomised to usual physiotherapy care (UPC: n = 8) or PreEMPT (n = 8). PreEMPT involved 14-weeks of alternating clinic- or telehealth-based, participation-focused intervention. Feasibility was evaluated by: demand, practicality, acceptability, implementation and limited efficacy testing for infants (motor, participation) and parents (mental well-being, self-efficacy).

RESULTS

Demand was lower than expected (45% recruitment rate). For practicality, attrition was high in the PreEMPT group (mean assessment attendance 3.8/5 sessions, range 2-5) compared to UPC (4.8/5 sessions, range 4-5). In addition, mean PreEMPT treatment dose received was approximately half intended (overall: 7.3/14 sessions, range 0-12; equivalent for face-to-face: 3.9/7, range 0-6, versus telehealth 3.4/7, range 0-6). The most common reason cited for treatment non-attendance was maternal mental health (22 sessions). Treatment acceptability for parents was high, with PreEMPT parents reporting they were offered choices in sessions (p = .02), and increased their knowledge (p = .01) and confidence (p = .009). There was a large effect size in favour of PreEMPT for increased parental self-efficacy (p = .021, ES = 1.34).

CONCLUSION

Early post-discharge physiotherapy for preterm infants in regional Australia is beneficial according to families but logistically challenging.

Multiple health behaviour change primary care intervention for smoking cessation, physical activity and healthy diet in adults 45 to 75 years old (EIRA study): a hybrid effectiveness-implementation cluster randomised trial

BACKGROUND

This study aimed to evaluate the effectiveness of a) a Multiple Health Behaviour Change (MHBC) intervention on reducing smoking, increasing physical activity and adherence to a Mediterranean dietary pattern in people aged 45-75 years compared to usual care; and b) an implementation strategy.

METHODS

A cluster randomised effectiveness-implementation hybrid trial-type 2 with two parallel groups was conducted in 25 Spanish Primary Health Care (PHC) centres (3062 participants): 12 centres (1481 participants) were randomised to the intervention and 13 (1581 participants) to the control group (usual care). The intervention was based on the Transtheoretical Model and focused on all target behaviours using individual, group and community approaches. PHC professionals made it during routine care. The implementation strategy was based on the Consolidated Framework for Implementation Research (CFIR). Data were analysed using generalised linear mixed models, accounting for clustering. A mixed-methods data analysis was used to evaluate implementation outcomes (adoption, acceptability, appropriateness, feasibility and fidelity) and determinants of implementation success.

RESULTS

14.5% of participants in the intervention group and 8.9% in the usual care group showed a positive change in two or all the target behaviours. Intervention was more effective in promoting dietary behaviour change (31.9% vs 21.4%). The overall adoption rate by professionals was 48.7%. Early and final appropriateness were perceived by professionals as moderate. Early acceptability was high, whereas final acceptability was only moderate. Initial and final acceptability as perceived by the participants was high, and appropriateness moderate. Consent and recruitment rates were 82.0% and 65.5%, respectively, intervention uptake was 89.5% and completion rate 74.7%. The global value of the percentage of approaches with fidelity ≥50% was 16.7%. Eight CFIR constructs distinguished between high and low implementation, five corresponding to the Inner Setting domain.

CONCLUSIONS

Compared to usual care, the EIRA intervention was more effective in promoting MHBC and dietary behaviour change. Implementation outcomes were satisfactory except for the fidelity to the planned intervention, which was low. The organisational and structural contexts of the centres proved to be significant determinants of implementation effectiveness.

TRIAL REGISTRATION

ClinicalTrials.gov , NCT03136211 . Registered 2 May 2017, "retrospectively registered".

Status, reporting completeness and methodological quality of pilot randomised controlled trials in acupuncture: protocol for a systematic review

INTRODUCTION

To date, there has been a lack of knowledge about the status, reporting completeness and methodological quality of pilot trials in the acupuncture field. Thus, this systematic review protocol aims to: (1) investigate publication trends and aspects of feasibility evaluated in acupuncture pilot trials; (2) identify the proportion of acupuncture pilot trials that lead to definitive trials and (3) assess the reporting completeness and methodological quality of pilot trials in acupuncture.

METHODS AND ANALYSIS

Studies of acupuncture pilot randomised controlled trials published from 2011 to 2021 will be retrieved in seven databases in January 2022, including PubMed, Web of Science, EMBASE, Cochrane Library, Chinese National Knowledge Infrastructure, Wanfang Database and Chinese Biomedical Literature Database. The methodological quality and reporting completeness of all included studies will be assessed using the risk of bias 2.0 tool (RoB 2) and the Consolidated Standards of Reporting Trials (CONSORT) extension to randomised pilot and feasibility trials, respectively. For the primary analysis, publication trends, aspects of feasibility and the proportion of pilot trials that lead to definitive trials will be analysed. A quantitative analysis of the methodological quality and reporting completeness of the included trials will be implemented by calculating the percentage of items reported in each domain of RoB 2 and CONSORT. The secondary analysis will adopt a regression analysis to identify factors associated with the reporting completeness.

ETHICS AND DISSEMINATION

Ethical approval is not required for this study. This study is planned to be submitted to a peer-reviewed academic journal.

Value of surgical pilot and feasibility study protocols

BACKGROUND

RCTs in surgery are challenging owing to well established methodological issues. Well designed pilot and feasibility studies (PFS) may help overcome such issues to inform successful main trial design and conduct. This study aimed to analyse protocols of UK-funded studies to explore current use of PFS in surgery and identify areas for practice improvement.

METHODS

PFS of surgical interventions funded by UK National Institute for Health Research programmes from 2005 to 2015 were identified, and original study protocols and associated publications sourced. Data extracted included study design characteristics, reasons for performing the work including perceived uncertainties around conducting a definitive main trial, and whether the studies had been published.

RESULTS

Thirty-five surgical studies were identified, of which 29 were randomized, and over half (15 of 29) included additional methodological components (such as qualitative work examining recruitment, and participant surveys studying current interventions). Most studies focused on uncertainties around recruitment (32 of 35), with far fewer tackling uncertainties specific to surgery, such as intervention stability, implementation or delivery (10 of 35). Only half (19 of 35) had made their results available publicly, to date.

CONCLUSION

The full potential of pretrial work to inform and optimize definitive surgical studies is not being realized.

Statistical design and analysis in trials of proportionate interventions: a systematic review

BACKGROUND

In proportionate or adaptive interventions, the dose or intensity can be adjusted based on individual need at predefined decision stages during the delivery of the intervention. The development of such interventions may require an evaluation of the effectiveness of the individual stages in addition to the whole intervention. However, evaluating individual stages of an intervention has various challenges, particularly the statistical design and analysis. This review aimed to identify the use of trials of proportionate interventions and how they are being designed and analysed in current practice.

METHODS

We searched MEDLINE, Web of Science and PsycINFO for articles published between 2010 and 2015 inclusive. We considered trials of proportionate interventions in all fields of research. For each trial, its aims, design and analysis were extracted. The data synthesis was conducted using summary statistics and a narrative format.

RESULTS

Our review identified 44 proportionate intervention trials, comprising 28 trial results, 13 protocols and three secondary analyses. These were mostly described as stepped care (n=37) and mainly focussed on mental health research (n=30). The other studies were aimed at finding an optimal adaptive treatment strategy (n=7) in a variety of therapeutic areas. Further terminology used included adaptive intervention, staged intervention, sequentially multiple assignment trial or a two-phase design. The median number of decision stages in the interventions was two and only one study explicitly evaluated the effect of the individual stages.

CONCLUSIONS

Trials of proportionate staged interventions are being used predominantly within the mental health field. However, few studies consider the different stages of the interventions, either at the design or the analysis phase, and how they may interact with one another. There is a need for further guidance on the design, analyses and reporting across trials of proportionate interventions.

TRIAL REGISTRATION

Prospero, CRD42016033781. Registered on 2 February 2016.

A biomarker feasibility study in the South East Asia Community Observatory health and demographic surveillance system

Background

Integration of biomarker data with information on health and lifestyle provides a powerful tool to enhance the scientific value of health research. Existing health and demographic surveillance systems (HDSSs) present an opportunity to create novel biodata resources for this purpose, but data and biological sample collection often presents challenges. We outline some of the challenges in developing these resources and present the outcomes of a biomarker feasibility study embedded within the South East Asia Community Observatory (SEACO) HDSS.

Methods

We assessed study-related records to determine the pace of data collection, response from potential participants, and feedback following data and sample collection. Overall and stratified measures of data and sample availability were summarised. Crude prevalence of key risk factors was examined.

Results

Approximately half (49.5%) of invited individuals consented to participate in this study, for a final sample size of 203 (161 adults and 42 children). Women were more likely to consent to participate compared with men, whereas children, young adults and individuals of Malay ethnicity were less likely to consent compared with older individuals or those of any other ethnicity. At least one biological sample (blood from all participants – finger-prick and venous [for serum, plasma and whole blood samples], hair or urine for adults only) was successfully collected from all participants, with blood test data available from over 90% of individuals. Among adults, urine samples were most commonly collected (97.5%), followed by any blood samples (91.9%) and hair samples (83.2%). Cardiometabolic risk factor burden was high (prevalence of elevated HbA1c among adults: 23.8%; of elevated triglycerides among adults: 38.1%; of elevated total cholesterol among children: 19.5%).

Conclusions

In this study, we show that it is feasible to create biodata resources using existing HDSS frameworks, and identify a potentially high burden of cardiometabolic risk factors that requires further evaluation in this population.

Smartphones in the secondary prevention of cardiovascular disease: a systematic review

Background

Cardiac Rehabilitation (CR) and secondary prevention are effective components of evidence-based management for cardiac patients, resulting in improved clinical and behavioural outcomes. Mobile health (mHealth) is a rapidly growing health delivery method that has the potential to enhance CR and heart failure management. We undertook a systematic review to assess the evidence around mHealth interventions for CR and heart failure management for service and patient outcomes, cost effectiveness with a view to how mHealth could be utilized for rural, remote and Indigenous cardiac patients.

Methods

A comprehensive search of databases using key terms was conducted for the years 2000 to August 2016 to identify randomised and non-randomised trials utilizing smartphone functionality and a model of care that included CR and heart failure management. Included studies were assessed for quality and risk of bias and data extraction was undertaken by two independent reviewers.

Results

Nine studies described a mix of mHealth interventions for CR (5 studies) and heart failure (4 studies) in the following categories: feasibility, utility and uptake studies; and randomised controlled trials. Studies showed that mHealth delivery for CR and heart failure management is feasible with high rates of participant engagement, acceptance, usage, and adherence. Moreover, mHealth delivery of CR was as effective as traditional centre-based CR (TCR) with significant improvement in quality of life. Hospital utilization for heart failure patients showed inconsistent reductions. There was limited inclusion of rural participants.

Conclusion

Mobile health delivery has the potential to improve access to CR and heart failure management for patients unable to attend TCR programs. Feasibility testing of culturally appropriate mHealth delivery for CR and heart failure management is required in rural and remote settings with subsequent implementation and evaluation into local health care services.

Electronic supplementary material

The online version of this article (10.1186/s12872-018-0764-x) contains supplementary material, which is available to authorized users.

Guidance for using pilot studies to inform the design of intervention trials with continuous outcomes

Background

A pilot study can be an important step in the assessment of an intervention by providing information to design the future definitive trial. Pilot studies can be used to estimate the recruitment and retention rates and population variance and to provide preliminary evidence of efficacy potential. However, estimation is poor because pilot studies are small, so sensitivity analyses for the main trial’s sample size calculations should be undertaken.

Methods

We demonstrate how to carry out easy-to-perform sensitivity analysis for designing trials based on pilot data using an example. Furthermore, we introduce rules of thumb for the size of the pilot study so that the overall sample size, for both pilot and main trials, is minimized.

Results

The example illustrates how sample size estimates for the main trial can alter dramatically by plausibly varying assumptions. Required sample size for 90% power varied from 392 to 692 depending on assumptions. Some scenarios were not feasible based on the pilot study recruitment and retention rates.

Conclusion

Pilot studies can be used to help design the main trial, but caution should be exercised. We recommend the use of sensitivity analyses to assess the robustness of the design assumptions for a main trial.

Where have all the pilot studies gone? A follow-up on 30 years of pilot studies in Clinical Rehabilitation

Introduction:

Pilot studies are meritorious for determining the feasibility of a definitive clinical trial in terms of conduct and potential for efficacy, but their possible applications for planning a future trial are not always fully realized. The purpose of this review was to estimate the extent to which pilot/feasibility studies: (i) addressed needed objectives; (ii) led to definitive trials; and (iii) whether the subsequent undertaking of a definitive trial was influenced by the strength of the evidence of outcome improvement.

Methods:

Trials published in the journal Clinical Rehabilitation, since its inception, were eligible if the word ‘pilot’ or ‘feasibility’ was specified somewhere in the article. A total of 191 studies were reviewed, results were summarized descriptively, and between-group effect sizes were computed.

Results:

The specific purposes of piloting were stated in only 58% (n = 110) of the studies. The most frequent purpose was to estimate the potential for efficacy (85%), followed by testing the feasibility of the intervention (60%). Only 12% of the studies were followed by a definitive trial; <4% of studies had a main study underway or a published study protocol. There was no relationship between observed effect size and follow-up of pilot studies, although the confidence intervals were very wide owing to small number of trials that followed on.

Discussion:

Labelling and reporting of pilot studies needs to be improved to be concordant with the recently issued CONSORT guidelines. Feasibility needs to be fully tested and demonstrated prior to committing considerable human and monetary resources.

Cognitive Analytic Therapy for Bipolar Disorder: A Pilot Randomized Controlled Trial

The evidence base for treatment of bipolar affective disorder (BD) demands the evaluation of new psychotherapies in order to broaden patient choice. This study reports on the feasibility, safety, helpfulness and effectiveness of cognitive analytic therapy (CAT). In a pilot randomized controlled trial, BD patients in remission were randomized to either receiving 24 sessions of CAT (n = 9) or treatment as usual (n = 9) and were assessed in terms of symptoms, functioning and service usage over time. In the CAT arm no adverse events occurred, 8/9 completed treatment, 5/8 attended all 24 sessions and 2/8 were categorized as recovered. The most common helpful event during CAT was recognition of patterns in mood variability, with helpfulness themes changing according to phase of therapy. No major differences were found when comparing the arms over time in terms of service usage or psychometric outcomes. The study suggests that conducting further research into the effectiveness of CAT in treating BD is warranted and guidance regarding future trials is provided. Copyright © 2016 John Wiley & Sons, Ltd.

KEY PRACTITIONER MESSAGE

Treating BD with CAT appears feasible and safe. Retaining fidelity to the reformulation, recognition and revision structure of CAT appears useful. Participants stated that across the phases of CAT, focussing on patterns of mood variability was consistently helpful.

START NOW - a comprehensive skills training programme for female adolescents with oppositional defiant and conduct disorders: study protocol for a cluster-randomised controlled trial

Background

In Europe, the number of females exhibiting oppositional defiant disorder (ODD) and conduct disorder (CD) is growing. Many of these females live in youth welfare institutions. Consequently, there is a great need for evidence-based interventions within youth welfare settings. A recently developed approach targeting the specific needs of girls with ODD and CD in residential care is START NOW. The aim of this group-based behavioural skills training programme is to specifically enhance emotional regulation capacities to enable females with CD or ODD to appropriately deal with daily-life demands. It is intended to enhance psychosocial adjustment and well-being as well as reduce oppositional and aggressive behaviour. We present the study protocol (version 4.1; 10 February 2016) of the FemNAT-CD intervention trial titled ‘Group-Based Treatment of Adolescent Female Conduct Disorders: The Central Role of Emotion Regulation’.

Methods/design

The study is a prospective, confirmatory, cluster-randomised, parallel-group, multi-centre, randomised controlled trial with 128 institutionalised female adolescents who fulfil the diagnostic criteria of ODD and/or CD. Institutions/wards will be randomised either to provide the 12-week skills training as an add-on intervention or to provide treatment as usual. Once the first cycle is completed, each institution will run a second cycle with the opposite condition. Primary endpoints are the pre-post change in number of CD/ODD symptoms as assessed by a standardised, semi-structured psychiatric interview (Kiddie Schedule for Affective Disorders and Schizophrenia for School-Age Children–Present and Lifetime, CD/ODD section) between baseline and the end of intervention, as well as between baseline and a 3-month follow-up point. Secondary objectives include pre-post change in CD/ODD-related outcome measures, most notably emotional regulation on a behavioural and neurobiological level after completion of START NOW compared with treatment as usual.

Discussion

To our knowledge, this study is the first to date to systematically investigate the effectiveness of an adapted integrative psychosocial intervention designed for female adolescents with ODD and CD in youth welfare settings.

Trial registration

German Clinical Trials Register (DRKS) identifier: DRKS00007524. Registered on 18 December 2015 and with the World Health Organisation International Clinical Trials Registry Platform.

Electronic supplementary material

The online version of this article (doi:10.1186/s13063-016-1705-6) contains supplementary material, which is available to authorized users.

Intra-cluster and inter-period correlation coefficients for cross-sectional cluster randomised controlled trials for type-2 diabetes in UK primary care

Background

Clustered randomised controlled trials (CRCTs) are increasingly common in primary care. Outcomes within the same cluster tend to be correlated with one another. In sample size calculations, estimates of the intra-cluster correlation coefficient (ICC) are needed to allow for this nonindependence. In studies with observations over more than one time period, estimates of the inter-period correlation (IPC) and the within-period correlation (WPC) are also needed.

Methods

This is a retrospective cross-sectional study of all patients aged 18 or over with a diagnosis of type-2 diabetes, from The Health Improvement Network (THIN) database, between 1 October 2007 and 31 March 2010. We report estimates of the ICC, IPC, and WPC for typical outcomes using unadjusted and adjusted generalised linear mixed models with cluster and cluster by period random effects. For binary outcomes we report on the proportions scale, which is the appropriate scale for trial design. Estimated ICCs were compared to those reported from a systematic search of CRCTs undertaken in primary care in the UK in type-2 diabetes.

Results

Data from 430 general practices, with a median [IQR] number of diabetics per practice of 241 [150–351], were analysed. The ICC for HbA1c was 0.032 (95 % CI 0.026–0.038). For a two-period (each of 12 months) design, the WPC for HbA1c was 0.035 (95 % CI 0.030–0.040) and the IPC was 0.019 (95 % CI 0.014–0.026). The difference between the WPC and the IPC indicates a decay of correlation over time. Following dichotomisation at 7.5 %, the ICC for HbA1c was 0.026 (95 % CI 0.022–0.030). ICCs for other clinical measurements and clinical outcomes are presented. A systematic search of ICCs used in the design of CRCTs involving type-2 diabetes with HbA1c (undichotomised) as the outcome found that published trials tended to use more conservative ICC values (median 0.047, IQR 0.047–0.050) than those reported here.

Conclusions

These estimates of ICCs, IPCs, and WPCs for a variety of outcomes commonly used in diabetes trials can be useful for the design of CRCTs. In studies with observations taken at different time-points, the correlation of observations may decay over time, as reflected in lower values for the IPC than for the ICC. The IPC and WPC estimates are the first reported for UK primary care data.

Electronic supplementary material

The online version of this article (doi:10.1186/s13063-016-1532-9) contains supplementary material, which is available to authorized users.

Improving patients’ experience and outcome of total joint replacement: the RESTORE programme

Background

Total hip replacements (THRs) and total knee replacements (TKRs) are common elective procedures. In the REsearch STudies into the ORthopaedic Experience (RESTORE) programme, we explored the care and experiences of patients with osteoarthritis after being listed for THR and TKR up to the time when an optimal outcome should be expected.

Objective

To undertake a programme of research studies to work towards improving patient outcomes after THR and TKR.

Methods

We used methodologies appropriate to research questions: systematic reviews, qualitative studies, randomised controlled trials (RCTs), feasibility studies, cohort studies and a survey. Research was supported by patient and public involvement.

Results

Systematic review of longitudinal studies showed that moderate to severe long-term pain affects about 7–23% of patients after THR and 10–34% after TKR. In our cohort study, 10% of patients with hip replacement and 30% with knee replacement showed no clinically or statistically significant functional improvement. In our review of pain assessment few research studies used measures to capture the incidence, character and impact of long-term pain. Qualitative studies highlighted the importance of support by health and social professionals for patients at different stages of the joint replacement pathway. Our review of longitudinal studies suggested that patients with poorer psychological health, physical function or pain before surgery had poorer long-term outcomes and may benefit from pre-surgical interventions. However, uptake of a pre-operative pain management intervention was low. Although evidence relating to patient outcomes was limited, comorbidities are common and may lead to an increased risk of adverse events, suggesting the possible value of optimising pre-operative management. The evidence base on clinical effectiveness of pre-surgical interventions, occupational therapy and physiotherapy-based rehabilitation relied on small RCTs but suggested short-term benefit. Our feasibility studies showed that definitive trials of occupational therapy before surgery and post-discharge group-based physiotherapy exercise are feasible and acceptable to patients. Randomised trial results and systematic review suggest that patients with THR should receive local anaesthetic infiltration for the management of long-term pain, but in patients receiving TKR it may not provide additional benefit to femoral nerve block. From a NHS and Personal Social Services perspective, local anaesthetic infiltration was a cost-effective treatment in primary THR. In qualitative interviews, patients and health-care professionals recognised the importance of participating in the RCTs. To support future interventions and their evaluation, we conducted a study comparing outcome measures and analysed the RCTs as cohort studies. Analyses highlighted the importance of different methods in treating and assessing hip and knee osteoarthritis. There was an inverse association between radiographic severity of osteoarthritis and pain and function in patients waiting for TKR but no association in THR. Different pain characteristics predicted long-term pain in THR and TKR. Outcomes after joint replacement should be assessed with a patient-reported outcome and a functional test.

Conclusions

The RESTORE programme provides important information to guide the development of interventions to improve long-term outcomes for patients with osteoarthritis receiving THR and TKR. Issues relating to their evaluation and the assessment of patient outcomes are highlighted. Potential interventions at key times in the patient pathway were identified and deserve further study, ultimately in the context of a complex intervention.

Study registration

Current Controlled Trials ISRCTN52305381.

Funding

This project was funded by the NIHR Programme Grants for Applied Research programme and will be published in full inProgramme Grants for Applied Research; Vol. 4, No. 12. See the NIHR Journals Library website for further project information.

Pilot study to test the feasibility of a trial design and complex intervention on PRIoritising MUltimedication in Multimorbidity in general practices (PRIMUMpilot)

OBJECTIVE

To improve medication appropriateness and adherence in elderly patients with multimorbidity, we developed a complex intervention involving general practitioners (GPs) and their healthcare assistants (HCA). In accordance with the Medical Research Council guidance on developing and evaluating complex interventions, we prepared for the main study by testing the feasibility of the intervention and study design in a cluster randomised pilot study.

SETTING

20 general practices in Hesse, Germany.

PARTICIPANTS

100 cognitively intact patients ≥65 years with ≥3 chronic conditions, ≥5 chronic prescriptions and capable of participating in telephone interviews; 94 patients completed the study.

INTERVENTION

The HCA conducted a checklist-based interview with patients on medication-related problems and reconciled their medications. Assisted by a computerised decision-support system (CDSS), the GPs discussed medication intake with patients and adjusted their medication regimens. The control group continued with usual care.

OUTCOME MEASURES

Feasibility of the intervention and required time were assessed for GPs, HCAs and patients using mixed methods (questionnaires, interviews and case vignettes after completion of the study). The feasibility of the study was assessed concerning success of achieving recruitment targets, balancing cluster sizes and minimising drop-out rates. Exploratory outcomes included the medication appropriateness index (MAI), quality of life, functional status and adherence-related measures. MAI was evaluated blinded to group assignment, and intra-rater/inter-rater reliability was assessed for a subsample of prescriptions.

RESULTS

10 practices were randomised and analysed per group. GPs/HCAs were satisfied with the interventions despite the time required (35/45 min/patient). In case vignettes, GPs/HCAs needed help using the CDSS. The study made no patients feel uneasy. Intra-rater/inter-rater reliability for MAI was excellent. Inclusion criteria were challenging and potentially inadequate, and should therefore be adjusted. Outcome measures on pain, functionality and self-reported adherence were unfeasible due to frequent missing values, an incorrect manual or potentially invalid results.

CONCLUSIONS

Intervention and trial design were feasible. The pilot study revealed important limitations that influenced the design and conduct of the main study, thus highlighting the value of piloting complex interventions.

TRIAL REGISTRATION NUMBER

ISRCTN99691973; Results.

Effect of a multifaceted educational program for care staff concerning fecal incontinence in nursing home patients: study protocol of a cluster randomized controlled trial

Background

Fecal incontinence has a high prevalence in the older population, which cannot be explained by comorbidity or the anatomical or psychological changes of aging alone. Fecal incontinence leads to a high economic burden to the healthcare system and is an important cause of institutionalization. In addition, fecal incontinence is associated with shame, social isolation and reduced quality of life. The importance of identifying treatable causes in the frail elderly is strongly emphasized. It is recommended that an assessment of fecal incontinence should be implemented as part of an evaluation of older patients. Although there is a substantial evidence base to guide choice of implementation activities targeting healthcare professionals, little implementation research has focused on the care of older people nor involved care processes or care personnel. This study is based on the assumption that fecal incontinence among nursing home patients can be prevented, cured or ameliorated by offering care staff knowledge of best practice through a multifaceted educational program. The primary objective is to test the hypothesis that a multifaceted educational program for nursing home care staff on assessment and treatment of fecal incontinence reduces patients’ frequency of fecal incontinence.

Methods/design

The study is a two-armed, parallel cluster-randomized controlled trial. Primary outcome is the frequency of fecal incontinence among patients. Sample size calculations resulted in a need for a total sample of 240 patients. Twenty nursing home units in one city in Norway will be recruited and allocated to intervention or control by an independent statistician using computer-generated tables. The intervention is a multifaceted educational program. Units in the control arm will provide care as usual. The intervention period is 3 months. Data will be collected at baseline, 3, and 6 months. Data will be analyzed using mixed effect models with the cluster treated as a random effect.

Discussion

This study is the first randomized controlled trial specifically focusing on this neglected area. The result of the study will give evidence for best practice for continence care in nursing homes, and organizational advice concerning implementation strategies.

Trial registration

ClinicalTrials.gov: NCT02183740, registered June 2014.

Pilot and feasibility studies come of age!

This editorial introduces the new, online, open-access journal Pilot and Feasibility Studies. The journal considers manuscripts on any aspect of the design and analysis of pilot and feasibility studies, as well as protocols for pilot and feasibility studies, and discussions and reviews of methodological issues around the planning and reporting of such studies. These studies are generally carried out in preparation for future large-scale definitive randomised controlled trials or observational studies and address key issues of uncertainty. Objectives for conducting pilot and feasibility studies therefore differ from those of the future large-scale study and should be clearly expressed. The journal provides a dedicated place for publication of this important work as well as a forum for discussion of methodological issues that will lead to increased scientific rigour in this area.

Effectiveness of an intervention designed to optimize statins use: a primary prevention randomized clinical trial

BACKGROUND

Although hypercholesterolemia is considered a cardiovascular risk factor, in isolation it is not necessarily sufficient cause for a cardiovascular event. To improve event prediction, cardiovascular risk calculators have been developed; the REGICOR calculator has been validated for use in our population. The objective of this project is to develop an intervention with general practitioners (GPs) and evaluate its impact on prescription adequacy of cholesterol-lowering drugs in primary prevention of cardiovascular disease and in controlling the costs associated with this disease.

METHODS

This nonblinded, cluster-randomized clinical trial analyzes data from primary care electronic medical records (ECAP) and other databases. Inclusion criteria are patients aged 35 to 74 years with no known cardiovascular disease and a new prescription for cholesterol-lowering drugs during the 2-year study period. Dependent variables include the following: RETIRA, defined as new cholesterol-lowering drugs initiated during the year preceding the intervention, considered inadequate, and withdrawn during the study period; EVITA, defined as new cholesterol-lowering drugs initiated during the study period and considered inadequate; COST, defined as the total cost of inadequate new treatments prescribed; and REGISTER, defined as the recording of cardiovascular risk factors. Independent variables include the GP's quality-of-care indicators and randomly assigned study group (intervention vs control), patient demographics, and clinical variables. Aggregated descriptive analysis will be done at the GP level and multilevel analysis will be performed to estimate the intervention effect, adjusted for individual and GP variables.

DISCUSSION

The study objective is to generate evidence about the effectiveness of implementing feedback information programs directed to GPs in the context of Primary Care. The goal is to improve the prescription adequacy of lipid-lowering therapies for primary prevention.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT01997671. November 28, 2013.

Effectiveness and cost-effectiveness of a group-based pain self-management intervention for patients undergoing total hip replacement: feasibility study for a randomized controlled trial

Background

Total hip replacement (THR) is a common elective surgical procedure and can be effective for reducing chronic pain. However, waiting times can be considerable. A pain self-management intervention may provide patients with skills to more effectively manage their pain and its impact during their wait for surgery. This study aimed to evaluate the feasibility of conducting a randomized controlled trial to assess the effectiveness and cost-effectiveness of a group-based pain self-management course for patients undergoing THR.

Methods

Patients listed for a THR at one orthopedic center were posted a study invitation pack. Participants were randomized to attend a pain self-management course plus standard care or standard care only. The lay-led course was delivered by Arthritis Care and consisted of two half-day sessions prior to surgery and one full-day session after surgery. Participants provided outcome and resource-use data using a diary and postal questionnaires prior to surgery and one month, three months and six months after surgery. Brief telephone interviews were conducted with non-participants to explore barriers to participation.

Results

Invitations were sent to 385 eligible patients and 88 patients (23%) consented to participate. Interviews with 57 non-participants revealed the most common reasons for non-participation were views about the course and transport difficulties. Of the 43 patients randomized to the intervention group, 28 attended the pre-operative pain self-management sessions and 11 attended the post-operative sessions. Participant satisfaction with the course was high, and feedback highlighted that patients enjoyed the group format. Retention of participants was acceptable (83% of recruited patients completed follow-up) and questionnaire return rates were high (72% to 93%), with the exception of the pre-operative resource-use diary (35% return rate). Resource-use completion rates allowed for an economic evaluation from the health and social care payer perspective.

Conclusions

This study highlights the importance of feasibility work prior to a randomized controlled trial to assess recruitment methods and rates, barriers to participation, logistics of scheduling group-based interventions, acceptability of the intervention and piloting resource use questionnaires to improve data available for economic evaluations. This information is of value to researchers and funders in the design and commissioning of future research.

Trial registration

Current Controlled Trials ISRCTN52305381.

A multicenter, phased, cluster-randomized controlled trial to reduce central line-associated bloodstream infections in intensive care units*

OBJECTIVES

To determine the causal effects of an intervention proven effective in pre-post studies in reducing central line-associated bloodstream infections in the intensive care unit.

DESIGN

We conducted a multicenter, phased, cluster-randomized controlled trial in which hospitals were randomized into two groups. The intervention group started in March 2007 and the control group started in October 2007; the study period ended September 2008. Baseline data for both groups are from 2006.

SETTING

Forty-five intensive care units from 35 hospitals in two Adventist healthcare systems.

INTERVENTIONS

A multifaceted intervention involving evidence-based practices to prevent central line-associated bloodstream infections and the Comprehensive Unit-based Safety Program to improve safety, teamwork, and communication.

MEASUREMENTS AND RESULTS

We measured central line-associated bloodstream infections per 1,000 central line days and reported quarterly rates. Baseline average central line-associated bloodstream infections per 1,000 central line days was 4.48 and 2.71, for the intervention and control groups (p = .28), respectively. By October to December 2007, the infection rate declined to 1.33 in the intervention group compared to 2.16 in the control group (adjusted incidence rate ratio 0.19; p = .003; 95% confidence interval 0.06-0.57). The intervention group sustained rates <1/1,000 central line days at 19 months (an 81% reduction). The control group also reduced infection rates to <1/1,000 central line days (a 69% reduction) at 12 months.

CONCLUSIONS

This study demonstrated a causal relationship between the multifaceted intervention and the reduced central line-associated bloodstream infections. Both groups decreased infection rates after implementation and sustained these results over time, replicating the results found in previous, pre-post studies of this multifaceted intervention and providing further evidence that most central line-associated bloodstream infections are preventable.

Effect on adherence to nicotine replacement therapy of informing smokers their dose is determined by their genotype: a randomised controlled trial

BACKGROUND

The behavioural impact of pharmacogenomics is untested. We tested two hypotheses concerning the behavioural impact of informing smokers their oral dose of NRT is tailored to analysis of DNA.

METHODS AND FINDINGS

We conducted an RCT with smokers in smoking cessation clinics (N = 633). In combination with NRT patch, participants were informed that their doses of oral NRT were based either on their mu-opioid receptor (OPRM1) genotype, or their nicotine dependence questionnaire score (phenotype). The proportion of prescribed NRT consumed in the first 28 days following quitting was not significantly different between groups: (68.5% of prescribed NRT consumed in genotype vs 63.6%, phenotype group, difference = 5.0%, 95% CI -0.9,10.8, p = 0.098). Motivation to make another quit attempt among those (n = 331) not abstinent at six months was not significantly different between groups (p = 0.23). Abstinence at 28 days was not different between groups (p = 0.67); at six months was greater in genotype than phenotype group (13.7% vs 7.9%, difference = 5.8%, 95% CI 1.0,10.7, p = 0.018).

CONCLUSIONS

Informing smokers their oral dose of NRT was tailored to genotype not phenotype had a small, statistically non-significant effect on 28-day adherence to NRT. Among those still smoking at six months, there was no evidence that saying NRT was tailored to genotype adversely affected motivation to make another quit attempt. Higher abstinence rate at six months in the genotype arm requires investigation.

TRIAL REGISTRATION

Controlled-Trials.com ISRCTN14352545.

[Development, appraisal and synthesis of complex interventions - a methodological challenge]

BACKGROUND

Many medical interventions are of a complex nature. They comprise interdependent components differently interacting within various complex settings, e.g., stroke units. Appraising the efficacy, benefit and harm of complex interventions is far more difficult than appraising single interventions like specific drug treatments. Detailed methodological procedures for the development, appraisal and synthesis of complex interventions are increasingly discussed internationally.

METHODS

Systematic inventory of methodological guidance for the development and evaluation of complex interventions through searching scientific literature databases and web resources of international institutes. Review of the current state of discussion and suggestions for the development, appraisal and synthesis of complex interventions.

RESULTS

Five methodological guidance papers have been identified. In particular, the UK Medical Research Council's guidance offers a structured introduction to the development and evaluation of complex interventions in health care and medicine. Synthesis of complex interventions using customary methods of systematic reviews is not satisfying. Other approaches to synthesis like realist evaluation as well as statistical procedures exploring the active components of complex interventions have been discussed. However, post hoc analytical procedures could never adequately replace careful prospective development and exploration of complex interventions and interdependencies with contextual factors.

CONCLUSION

Complex interventions require multi-stage development, use of different methods, reporting on all developing phases and new approaches for synthesis. Presentation of the complete evidence on a specific complex intervention might be more useful than synthesis of a variety of different complex interventions by customarily applied methods of (meta-analytical) systematic review.

Efficient design of cluster randomized and multicentre trials with unknown intraclass correlation

For cluster randomized and multicentre trials evaluating the effect of a treatment on persons nested within clusters, equations have been published to compute the optimal sample sizes at the cluster and person level as a function of sampling costs and intraclass correlation ( ICC). Here, optimal means maximum power and precision for a given sampling budget, or minimum sampling costs for a given power and precision. However, the ICC is usually unknown, and the optimal sample sizes depend strongly on this ICC. To overcome this local optimality problem, this study presents Maximin designs (MMDs) based on relative efficiency (RE) and efficiency. These designs perform well over a range of possible ICC values either in terms of RE compared with the locally optimal designs, or in terms of minimum efficiency (maximum variance) of the treatment effect estimator. The use of MMDs is illustrated using information from many cluster randomized trials in primary care. It is concluded that MMDs and the optimal design for an ICC halfway its assumed range are efficient for a range of ICC values and recommendable for practical use. This requires that trial reports mention the study cost per cluster and person.

What is a pilot or feasibility study? A review of current practice and editorial policy

BACKGROUND

In 2004, a review of pilot studies published in seven major medical journals during 2000-01 recommended that the statistical analysis of such studies should be either mainly descriptive or focus on sample size estimation, while results from hypothesis testing must be interpreted with caution. We revisited these journals to see whether the subsequent recommendations have changed the practice of reporting pilot studies. We also conducted a survey to identify the methodological components in registered research studies which are described as 'pilot' or 'feasibility' studies. We extended this survey to grant-awarding bodies and editors of medical journals to discover their policies regarding the function and reporting of pilot studies.

METHODS

Papers from 2007-08 in seven medical journals were screened to retrieve published pilot studies. Reports of registered and completed studies on the UK Clinical Research Network (UKCRN) Portfolio database were retrieved and scrutinized. Guidance on the conduct and reporting of pilot studies was retrieved from the websites of three grant giving bodies and seven journal editors were canvassed.

RESULTS

54 pilot or feasibility studies published in 2007-8 were found, of which 26 (48%) were pilot studies of interventions and the remainder feasibility studies. The majority incorporated hypothesis-testing (81%), a control arm (69%) and a randomization procedure (62%). Most (81%) pointed towards the need for further research. Only 8 out of 90 pilot studies identified by the earlier review led to subsequent main studies. Twelve studies which were interventional pilot/feasibility studies and which included testing of some component of the research process were identified through the UKCRN Portfolio database. There was no clear distinction in use of the terms 'pilot' and 'feasibility'. Five journal editors replied to our entreaty. In general they were loathe to publish studies described as 'pilot'.

CONCLUSION

Pilot studies are still poorly reported, with inappropriate emphasis on hypothesis-testing. Authors should be aware of the different requirements of pilot studies, feasibility studies and main studies and report them appropriately. Authors should be explicit as to the purpose of a pilot study. The definitions of feasibility and pilot studies vary and we make proposals here to clarify terminology.