Challenges in Creating a Good Randomized Controlled Trial in Hand Surgery

A guide to appropriately planning and conducting meta-analyses-Part 1: indications, assumptions and understanding risk of bias

A meta-analysis is the quantitative synthesis of data from two or more individual studies and are as a rule an important method of obtaining a more accurate estimate of the direction and magnitude of a treatment effect. However, it is imperative that the meta-analysis be performed with proper, rigorous methodology to ensure validity of the results and their interpretation. In this article the authors will review the most important questions researchers should consider when planning a meta-analysis to ensure proper indications and methodologies, minimize the risk of bias, and avoid misleading conclusions.

Optimising recruitment to the HAND-1 RCT feasibility study: integration of the QuinteT Recruitment Intervention (QRI)

Background

Recruitment to randomised controlled trials (RCTs) can be challenging, with most trials not reaching recruitment targets. Randomised feasibility studies can be set up prior to a main trial to identify and overcome recruitment obstacles. This paper reports on an intervention—the QuinteT Recruitment Intervention (QRI)—to optimise recruitment within a randomised feasibility study of surgical treatments for patients with Dupuytren’s contracture (the HAND-1 study).

Methods

The QRI was introduced in 2-phases: phase 1 sought to understand the recruitment challenges by interviewing trial staff, scrutinising screening logs and analysing audio-recorded patient consultations; in phase 2 a tailored plan of action consisting of recruiter feedback and training was delivered to address the identified challenges.

Results

Two key recruitment obstacles emerged: (1) issues with the recruitment pathway, in particular methods to identify potentially eligible patients and (2) equipoise of recruiters and patients. These were addressed by liaising with centres to share good practice and refine their pathway and by providing bespoke feedback and training on consent discussions to individual recruiters and centres whilst recruitment was ongoing. The HAND-1 study subsequently achieved its recruitment target.

Conclusions

Transferable lessons learnt from the QRI in the feasibility study will be implemented in the definitive RCT, enabling a “head start” in the tackling of wider issues around screening methods and consent discussions in the set up/early recruitment study phases, with ongoing QRI addressing specific issues with new centres and recruiters. Findings from this study are likely to be relevant to other surgical and similar trials that are anticipated to encounter issues around patient and recruiter equipoise of treatments and variation in recruitment pathways across centres. The study also highlights the value of feasibility studies in fine-tuning design and conduct issues for definitive RCTs. Embedding a QRI in an RCT, at feasibility or main stage, offers an opportunity for a detailed and nuanced understanding of key recruitment challenges and the chance to address them in “real-time” as recruitment proceeds.

The Fragility Index in Hand Surgery Randomized Controlled Trials

PURPOSE

Randomized controlled trials (RCTs) are the gold standard for comparing clinical interventions. Statistical significance as reported via a P value has been used to determine if a difference between clinical interventions exists in an RCT. However, P values do not clearly convey information about the robustness of a study's conclusions. An emerging metric, called the fragility index (the number of subjects who would need to change outcome category to raise the P value above the .05 threshold), is an indirect measure of how likely a repeat of the trial would reach the same conclusions. This study addressed the fragility of RCTs using dichotomous outcomes in hand surgery.

METHODS

Using systematic searching of the MEDLINE database, we identified hand surgery RCTs published in 11 high-impact journals published in the last decade (2007-2017). Studies were identified that involved 2 parallel arms, allocated patients to treatment and control in a 1:1 ratio, and reported statistical significance for a dichotomous variable. The fragility index was calculated using Fisher's exact test, using previously published methods.

RESULTS

Five hand surgery RCTs were identified for inclusion reporting a range of fragility indices from 0 to 26. Two of the trials (40%) had a fragility index of 2 or less. Two of the trials (40%) reported that the number of patients lost to follow-up exceeded the fragility index, meaning that results of the patients lost to follow-up could theoretically completely reverse the study conclusions.

CONCLUSIONS

The range of fragility indices reported in the recent hand surgery literature is consistent with previous reporting within orthopedic surgery.

CLINICAL RELEVANCE

The fragility index is a useful metric to analyze the robustness of the study conclusions that should complement other methods of critical evaluation including the P value or effect sizes. Our results emphasize the need for future efforts to strengthen the robustness of RCT conclusions.

Measuring and understanding treatment effectiveness in hand surgery

Incorporating evidence-based medicine into practice is now an expectation for hand surgeons. Hand surgeons need to be able to assess associated benefits, risks, cost, and applicability of a treatment option when providing care to their patients. Using a clinical example, this article takes the reader through the three-step approach when using a publication from the medical literature on therapy. The focus of this article is primarily the second and third steps, which involve measuring and understanding treatment effectiveness.

Application of the CONSORT statement to randomized controlled trials comparing endoscopic and open carpal tunnel release

BACKGROUND

The CONSORT (Consolidated Standards of Reporting Trials) statement was developed by a group of clinical trialists, biostatisticians, epidemiologists and biomedical editors as a means to improve the quality of reports of randomized controlled trials (RCTs). The purpose of the present study is to assess the reporting quality of published RCTs that compare endoscopic carpal tunnel release (ECTR) with open carpal tunnel release (OCTR) using the CONSORT statement.

METHODS

A computerized literature search was conducted to identify all RCTs published from January 1989 to November 2004 that compared ECTR with OCTR. Foreign language studies were also included, and translated versions of these studies were obtained. Two investigators independently reviewed each eligible article and determined whether the authors reported on each of the 22 items of the CONSORT statement. Disagreements were resolved by consensus. The mean scores for studies published before the introduction of the CONSORT statement and those published afterward were compared. Similarly, a comparison was made between foreign language studies and those published in English.

RESULTS

Eighteen RCTs comparing ECTR with OCTR met the inclusion criteria. The total scores on the CONSORT checklist ranged from 3 to 20, with a mean score of 9.83+/-3.79 (the maximum possible score was 22). The six studies published in foreign language journals had a statistically significantly lower mean score than the studies published in English language journals (7.00+/-2.76 versus 11.25+/-3.49, respectively; P<0.05). The mean score was higher for studies published after 1996 than for those published in 1996 or earlier (12.14+/-3.80 versus 8.36+/-3.11, respectively; P<0.05).

CONCLUSIONS

The quality of reporting improved over time, but no study met all 22 criteria of the CONSORT statement. The CONSORT scores were higher for studies published after 1996 and for studies published in English language journals. Despite the improvement after 1996, most of these RCTs only reported one-half of the items listed on the CONSORT statement. Future investigators of surgical RCTs should make an effort to comply with the CONSORT checklist.

Departures from the protocol during conduct of a clinical trial: a pattern from the data record consistent with a learning curve

OBJECTIVE

Recognition of learning curves in medical skill acquisition has enhanced patient safety through improved training techniques. Clinical trials research has not been similarly scrutinised. The VALsartan In Acute myocardial iNfarcTion, a large multinational, pragmatic, randomised, double-blind, multicentre trial, was retrospectively evaluated for evidence of research conduct consistent with a performance "learning curve".

DESIGN

Records provided protocol departure (deviations/violations) and documentation query data. For each site, analysis included patient order (eg, first, second), recruitment rate and first enrollment relative to study start date.

SETTING

Computerised data from a trial coordinated by an academic research organisation collaborating with 10 academic and 2 commercial research organisations and an industry sponsor. Interventions 931 sites enrolled 14,703 patients. Departures were restricted to the first year. Exclusions included patient's death or loss to follow-up within 12 months and subjects enrolled 80th or higher at a site. Departures were assessed for variance with higher patient rank, more frequent recruitment and later start date.

METHODS AND RESULTS

12,367 patients at 931 sites were analysed. Departures were more common for patients enrolled earlier at a site (p<0.0001). For example, compared with the 30th patient, the first had 47% more departures. Departures were also more common with slower enrollment and site start closer to the trial start date (p<0.0001). Similar patterns existed for queries.

CONCLUSIONS

Research performance improved during the VALsartan In Acute myocardial iNfarcTion consistent with a "learning curve". Although effects were not related to a change in outcome (mortality), learning curves in clinical research may have important safety, ethical, research quality and economic implications for trial conduct.

Outcomes in microsurgery

The reporting of microsurgical outcomes has been variable. Historically, emphasis has been placed on flap and digit survival or failure in the case of free-tissue transfer or digit replantation, respectively. Outcomes have also been measured with indices such as range of motion or grip strength for digital replantations, the ability to eat or talk for head and neck microsurgery, and the ability to walk or return to work for lower extremity microsurgery. Although relevant, this type of reporting of outcomes may fail to capture the effectiveness of microsurgical intervention from the patient's, the third-party payer's, or society's perspective. Significant events have arisen in the past two decades, including the emphasis on outcomes research, recent recommendations to adopt evidence-based microsurgery, and the inclusion in academic training programs of the competency "manager" to the health care system. This necessitates rethinking the way we report outcomes in microsurgery. This article explains the need to (1) use health-related quality-of-life scales to measure the benefits of microsurgical interventions, (2) measure outcomes with high-quality clinical research designs, and (3) incorporate proper cost-effectiveness studies in our clinical research before adopting new technologies such as new free flaps or techniques.