The design and assessment of prospective randomised, controlled trials in orthopaedic surgery

Examination of early treatment effects and related biases during the conduct of two UK-wide pragmatic orthopaedic surgical trials: ProFHER and UK FROST

Early large treatment effects can arise in small studies, which lessen as more data accumulate. This study aimed to retrospectively examine whether early treatment effects occurred for two multicentre orthopaedic randomized controlled trials (RCTs) and explore biases related to this. Included RCTs were ProFHER (PROximal Fracture of the Humerus: Evaluation by Randomisation), a two-arm study of surgery versus non-surgical treatment for proximal humerus fractures, and UK FROST (United Kingdom Frozen Shoulder Trial), a three-arm study of two surgical and one non-surgical treatment for frozen shoulder. To determine whether early treatment effects were present, the primary outcome of Oxford Shoulder Score (OSS) was compared on forest plots for: the chief investigator's (CI) site to the remaining sites, the first five sites opened to the other sites, and patients grouped in quintiles by randomization date. Potential for bias was assessed by comparing mean age and proportion of patients with indicators of poor outcome between included and excluded/non-consenting participants. No bias in treatment effect was observed overall for the CI site, or the first five sites, compared with the remaining sites in either trial. An early treatment effect on the OSS was observed for the first quintile of participants recruited to ProFHER only (clinically relevant difference of seven points). Selection bias for age was observed in the ProFHER trial only, with slightly younger patients being recruited into the study. Both trials showed some selection bias for markers of poor prognosis, although these did not appear to change over time. No bias in treatment effects overall were found at the CI or early sites set-up. An early treatment effect was found in one of the two trials, which was likely a chance effect as this did not continue during the study. Selection bias was observed in both RCTs, however this was minimal and did not impact on outcome.

Reporting and Analyzing Demographics in the Journal of Arthroplasty: Are We Making Progress?

BACKGROUND

Demographic factors, including age, sex, body mass index (BMI), race, and ethnicity have great effects on the outcomes of patients undergoing total joint arthroplasty. A portion of this data is included in nearly every study, but the completeness with which it is reported is variable. The purpose of this study is to investigate the frequency at which demographic information is reported and analyzed through formal statistical methods in randomized controlled trials (RCTs) published in the Journal of Arthroplasty (JOA).

METHODS

A systematic review was conducted of RCTs published in JOA between 2015 and 2019. For each study, we determined if age, sex, weight, height, BMI, race, and ethnicity were reported and/or analyzed. The overall frequency was assessed, along with the rates of reporting by individual year. Studies were evaluated using Cochrane risk-of-bias tool.

RESULTS

Age (96.7%), sex (96.7%), and BMI (80.4%) were reported by the majority of studies. There was very little information provided regarding race (6.2%) and ethnicity (3.8%); although both were reported at the highest frequency in 2019, the final year of articles reviewed. Sex was the most frequently analyzed variable at 11.5%. Only 1 study (0.5%) analyzed ethnicity and no studies analyzed race.

CONCLUSION

Although age, sex, and BMI are reported at a high rate, RCTs published in JOA rarely reported information on patient race and ethnicity. Demographics were infrequently included as part of statistical analysis. The importance of this information should be recognized and included in the analysis and interpretation of future studies.

Quality of reporting of systematic reviews and meta-analyses of surgical randomized clinical trials

Background

Well designed and conducted systematic reviews are essential to clinical practice. Surgical intervention is more complex than medical intervention when considering special items related to procedures. There has been no cross‐sectional study of the reporting quality of systematic reviews of surgical randomized trials focused on special items relating to surgical interventions.

Methods

A cross‐sectional survey of systematic reviews of surgical randomized trials published in 2007 and 2017 was undertaken via a PubMed search. Quality of reporting was assessed by the PRISMA checklist, with intervention details containing 27 items. Univariable and multivariable linear regression was used to explore factors in the checklist as indicators of reporting quality.

Results

A total of 204 systematic reviews were identified. The median score for the PRISMA checklist was 22 (i.q.r. 20–24), and systematic reviews published in 2017 had a significantly higher median score than those from 2007 (22 (i.q.r. 21–24) versus 20 (17–22); P < 0·001). Among the 27 items, 15 were reported adequately and three were reported poorly (in less than 50 per cent of reports). The proportion of other items reported ranged from 54·4 to 77·9 per cent. In multivariable analysis, systematic reviews published in 2017 (coefficient 0·59, 95 per cent c.i. 0·50 to 0·69) and Cochrane reviews (coefficient 0·67, 0·55 to 0·81) were associated with better reporting.

Conclusion

The quality of reporting of systematic reviews of surgical randomized trials has improved in the past 10 years. Some information relating to specific surgical interventions is, however, still reported poorly.

Improving site selection in clinical studies: a standardised, objective, multistep method and first experience results

INTRODUCTION

A considerable number of clinical studies experience delays, which result in increased duration and costs. In multicentre studies, patient recruitment is among the leading causes of delays. Poor site selection can result in low recruitment and bad data quality. Site selection is therefore crucial for study quality and completion, but currently no specific guidelines are available.

MATERIAL AND METHODS

Selection of sites adequate to participate in a prospective multicentre cohort study was performed through an open call using a newly developed objective multistep approach. The method is based on use of a network, definition of objective criteria and a systematic screening process.

ILLUSTRATIVE EXAMPLE OF THE METHOD AT WORK

Out of 266 interested sites, 24 were shortlisted and finally 12 sites were selected to participate in the study. The steps in the process included an open call through a network, use of selection questionnaires tailored to the study, evaluation of responses using objective criteria and scripted telephone interviews. At each step, the number of candidate sites was quickly reduced leaving only the most promising candidates. Recruitment and quality of data went according to expectations in spite of the contracting problems faced with some sites.

CONCLUSION

The results of our first experience with a standardised and objective method of site selection are encouraging. The site selection method described here can serve as a guideline for other researchers performing multicentre studies.

TRIAL REGISTRATION NUMBER

ClinicalTrials.gov: NCT02297581.

Quality of reporting in surgical randomized clinical trials

BACKGROUND

RCTs testing surgical interventions can change clinical practice. The adequate reporting of surgical trials is an important issue.

METHODS

A cross-sectional survey was undertaken by searching PubMed for two-arm parallel randomized trials assessing surgical interventions published in 2003 or 2013. Quality of reporting was evaluated against the CONSORT checklist containing 29 items (standard CONSORT plus CONSORT Extension for Trials Assessing Non-Pharmacological Treatments). Univariable and multivariable linear regression was undertaken to explore factors associated with quality of reporting measured with CONSORT scores.

RESULTS

Some 120 trials were identified and included. The mean(s.d.) CONSORT score was 12·7(4·0). Trials published in 2013 achieved a higher CONSORT score than those published in 2003 (mean 14·5(3·8) versus 10·8(3·4) respectively; P < 0·001). The extent to which these trials met the requirement for CONSORT reporting differed substantially across items: four of 29 items were reported adequately across trials, and seven were reported adequately in less than 20 per cent of trials. Less than 40 per cent of the trials described the additional items related to surgical interventions and care providers (such as nursing care and anaesthetic management). In multivariable regression analysis, trials published in 2013 (coefficient 3·05, 95 per cent c.i. 1·89 to 4·20) and multicentre studies (coefficient 2·08, 0·79 to 3·37) were associated with statistically higher quality of reporting.

CONCLUSION

The quality of reporting in surgical trials has improved in the past decade. Overall quality, however, remains suboptimal, particularly in relation to details regarding surgical interventions and management.

Trials based on specific fracture configuration and surgical procedures likely to be more relevant for decision making in the management of fractures of the proximal humerus: Findings of a meta-analysis

Objectives

The objective of this study was to perform a meta-analysis of all randomised controlled trials (RCTs) comparing surgical and non-surgical management of fractures of the proximal humerus, and to determine whether further analyses based on complexity of fracture, or the type of surgical intervention, produced disparate findings on patient outcomes.

Methods

A systematic review of the literature was performed identifying all RCTs that compared surgical and non-surgical management of fractures of the proximal humerus. Meta-analysis of clinical outcomes was performed where possible. Subgroup analysis based on the type of fracture, and a sensitivity analysis based on the type of surgical intervention, were also performed.

Results

Seven studies including 528 patients were included. The overall meta-analysis found that there was no difference in clinical outcomes. However, subgroup and sensitivity analyses found improved patient outcomes for more complex fractures managed surgically. Four-part fractures that underwent surgery had improved long-term health utility scores (mean difference, MD 95% CI 0.04 to 0.28; p = 0.007). They were also less likely to result in osteoarthritis, osteonecrosis and non/malunion (OR 7.38, 95% CI 1.97 to 27.60; p = 0.003). Another significant subgroup finding was that secondary surgery was more common for patients that underwent internal fixation compared with conservative management within the studies with predominantly three-part fractures (OR 0.15, 95% CI 0.04 to 0.63; p = 0.009).

Conclusion

This meta-analysis has demonstrated that differences in the type of fracture and surgical treatment result in outcomes that are distinct from those generated from analysis of all types of fracture and surgical treatments grouped together. This has important implications for clinical decision making and should highlight the need for future trials to adopt more specific inclusion criteria.

Cite this article: S. Sabharwal, N. K. Patel, D. Griffiths, T. Athanasiou, C. M. Gupte, P. Reilly. Trials based on specific fracture configuration and surgical procedures likely to be more relevant for decision making in the management of fractures of the proximal humerus: Findings of a meta-analysisBone Joint Res 2016;5:470–480. DOI: 10.1302/2046-3758.510.2000638.

Curved versus Straight Stem Uncemented Total Hip Arthroplasty Osteoarthritis Multicenter trial (CUSTOM): design of a prospective blinded randomised controlled multicentre trial

INTRODUCTION

Answering the demands of an increasingly young and active patient population, recent developments in total hip arthroplasty (THA) have shifted towards minimising tissue damage. The Collum Femoris Preserving (CFP) stem was developed to preserve the trochanteric region of the femur, which potentially preserves the insertion of the gluteus musculature. This might accelerate early postoperative rehabilitation and improve functional outcome. Currently the functional results of the CFP stem have not been compared with conventional straight stems in a randomised controlled trial (RCT). The primary purpose of this trial is to compare the functional result of CFP stem THA with conventional uncemented straight stem THA, measured by the Dutch Hip disability and Osteoarthritis Outcome Score (HOOS) at 3-month follow-up.

METHODS

A prospective blinded multicentre RCT will be performed. We aim to recruit 150 patients. The patients will be randomly allocated to a THA with a straight or a curved stem. All patients, research assistants, clinical assessors and investigators will be blinded for the type of prosthesis for 5 years. Clinical assessments and roentgenograms will be taken preoperative, at 6 weeks after surgery, at 1, 2, 3, 4 and 5 years after surgery. Patient reported outcome measures (PROMs) will be obtained at the same follow-up moments. In addition, the PROMs will also be sent to the patients at 3 and 6 months after surgery. The HOOS at 3-month follow-up will be our primary outcome.

ETHICS AND DISSEMINATION

This trial will be performed in accordance with the Declaration of Helsinki. A local ethics committee has approved this trial. Written informed consent will be obtained from all participating patients. All serious adverse events will be reported to the ethics committee.

RESULTS

Results will be submitted for publication to an orthopaedics related journal.

TRIAL REGISTRATION NUMBER

NTR1560.

Development of appropriateness criteria for the surgical treatment of symptomatic lumbar degenerative spondylolisthesis (LDS)

PURPOSE

Spine surgery rates are increasing worldwide. Treatment failures are often attributed to poor patient selection and inappropriate treatment, but for many spinal disorders there is little consensus on the precise indications for surgery. With an aging population, more patients with lumbar degenerative spondylolisthesis (LDS) will present for surgery. The aim of this study was to develop criteria for the appropriateness of surgery in symptomatic LDS.

METHODS

A systematic review was carried out to summarize the current level of evidence for the treatment of LDS. Clinical scenarios were generated comprising combinations of signs and symptoms in LDS and other relevant variables. Based on the systematic review and their own clinical experience, twelve multidisciplinary international experts rated each scenario on a 9-point scale (1 highly inappropriate, 9 highly appropriate) with respect to performing decompression only, fusion, and instrumented fusion. Surgery for each theoretical scenario was classified as appropriate, inappropriate, or uncertain based on the median ratings and disagreement in the ratings.

RESULTS

744 hypothetical scenarios were generated; overall, surgery (of some type) was rated appropriate in 27%, uncertain in 41% and inappropriate in 31%. Frank panel disagreement was low (7% scenarios). Face validity was shown by the logical relationship between each variable's subcategories and the appropriateness ratings, e.g., no/mild disability had a mean appropriateness rating of 2.3 ± 1.5, whereas the rating for moderate disability was 5.0 ± 1.6 and for severe disability, 6.6 ± 1.6. Similarly, the average rating for no/minimal neurological abnormality was 2.3 ± 1.5, increasing to 4.3 ± 2.4 for moderate and 5.9 ± 1.7 for severe abnormality. The three variables most likely (p < 0.0001) to be components of scenarios rated "appropriate" were: severe disability, no yellow flags, and severe neurological deficit.

CONCLUSION

This is the first study to report criteria for determining candidacy for surgery in LDS developed by a multidisciplinary international panel using a validated method (RAM). The panel ratings followed logical clinical rationale, indicating good face validity. The work refines clinical classification and the phenotype of degenerative spondylolisthesis. The predictive validity of the criteria should be evaluated prospectively to examine whether patients treated "appropriately" have better clinical outcomes.

Finnish Degenerative Meniscal Lesion Study (FIDELITY): a protocol for a randomised, placebo surgery controlled trial on the efficacy of arthroscopic partial meniscectomy for patients with degenerative meniscus injury with a novel 'RCT within-a-cohort' study design

INTRODUCTION

Arthroscopic partial meniscectomy (APM) to treat degenerative meniscus injury is the most common orthopaedic procedure. However, valid evidence of the efficacy of APM is lacking. Controlling for the placebo effect of any medical intervention is important, but seems particularly pertinent for the assessment of APM, as the symptoms commonly attributed to a degenerative meniscal injury (medial joint line symptoms and perceived disability) are subjective and display considerable fluctuation, and accordingly difficult to gauge objectively.

METHODS AND ANALYSIS

A multicentre, parallel randomised, placebo surgery controlled trial is being carried out to assess the efficacy of APM for patients from 35 to 65 years of age with a degenerative meniscus injury. Patients with degenerative medial meniscus tear and medial joint line symptoms, without clinical or radiographic osteoarthritis of the index knee, were enrolled and then randomly assigned (1 : 1) to either APM or diagnostic arthroscopy (placebo surgery). Patients are followed up for 12 months. According to the prior power calculation, 140 patients were randomised. The two randomised patient groups will be compared at 12 months with intention-to-treat analysis. To safeguard against bias, patients, healthcare providers, data collectors, data analysts, outcome adjudicators and the researchers interpreting the findings will be blind to the patients' interventions (APM/placebo). Primary outcomes are Lysholm knee score (a generic knee instrument), knee pain (using a numerical rating scale), and WOMET score (a disease-specific, health-related quality of life index). The secondary outcome is 15D (a generic quality of life instrument). Further, in one of the five centres recruiting patients for the randomised controlled trial (RCT), all patients scheduled for knee arthroscopy due to a degenerative meniscus injury are prospectively followed up using the same protocol as in the RCT to provide an external validation cohort. In this article, we present and discuss our study design, focusing particularly on the internal and external validity of our trial and the ethics of carrying out a placebo surgery controlled trial.

ETHICS AND DISSEMINATION

The protocol has been approved by the institutional review board of the Pirkanmaa Hospital District and the trial has been duly registered at ClinicalTrials.gov. The findings of this study will be disseminated widely through peer-reviewed publications and conference presentations.

TRIAL REGISTRATION

ClinicalTrials.gov, number NCT00549172.

Future research in gynaecological surgery

Gynaecological surgery is constantly evolving. To inform practice with high-impact research, clinicians need to focus on areas of importance. Surveys of specialist members of the British Society of Gynaecological Endoscopy have revealed a range of areas for research: diagnostic performance of laparoscopies; therapeutic laparoscopies in endometriosis; laparoscopic versus hysteroscopic sterilisation; and laparoscopic surgical techniques, among others. Clinical and economic outcomes are important in evaluating effectiveness and use of surgical health technology. For studies to be valid, reliable and generalisable, they would have to be free of bias, large and multi-centred. In a time of financial constraints, it is important to encourage clinicians and trainees to participate in important research studies to improve outcomes for patients.

Methodological and ethical quality of randomized controlled clinical trials in gastrointestinal surgery

BACKGROUND

The randomized controlled trial (RCT) is the gold standard tool used to evaluate therapeutic interventions. Methodological and ethical aspects should be adequately reported to enable readers to make informed and justified judgments regarding the validity of a trial and the treatment effectiveness.

OBJECTIVE

The aim of this study was to evaluate the methodological and ethical qualities of randomized clinical trials in gastrointestinal surgery and to assess the relationship between these two qualities.

STUDY DESIGN

All of the articles chosen for review reported on phase III randomized controlled gastrointestinal surgical trials were published in 12 international journals during 2006 and 2007. The eligible studies were identified, selected, and then evaluated based on a broad set of predetermined criteria. The methodological quality was evaluated using the Jadad scale, and the ethical quality was evaluated using the Berdeu score.

RESULTS

The mean Jadad score was 9.7 ± 1.78. The methodological quality was insufficient in 64 RCTs (37.4 %; Jadad score <9). The mean Berdeu score was 0.36 ± 0.08. The journal impact factor, number of randomized patients, and number of centers correlated with the outcome of the Jadad score, and the journal impact factor, industry funding, and year in which the trial began correlated with the outcome of the Berdeu score. Informed consent from patients was not obtained in 7 % (n = 12) of the RCTs, and research ethics committee approval was not mentioned in 14.6 % (n = 25) of the RCTs.

CONCLUSIONS

The reporting of gastrointestinal surgery RCTs is less than optimal. In our study, the trials of higher methodological quality were more likely to provide information about their ethical aspects. These results suggest the need for more attention to be paid to the conduct of clinical research and the reporting of ethical aspects. The appropriation of the ethical rules by surgeons involved in human clinical trials could improve the methodology and reporting of RCTs in gastrointestinal surgery.

A statistical framework for quantifying clinical equipoise for individual cases during randomized controlled surgical trials

Background

Randomised controlled trials are being increasingly used to evaluate new surgical interventions. There are a number of problematic methodological issues specific to surgical trials, the most important being identifying whether patients are eligible for recruitment into the trial. This is in part due to the diversity in practice patterns across institutions and the enormous range of available interventions that often leads to a low level of agreement between clinicians about both the value and the appropriate choice of intervention. We argue that a clinician should offer patients the option of recruitment into a trial, even if the clinician is not individually in a position of equipoise, if there is collective (clinical) equipoise amongst the wider clinical community about the effectiveness of a proposed intervention (the clinical equipoise principle). We show how this process can work using data collected from an ongoing trial of a surgical intervention.

Results

We describe a statistical framework for the assessment of uncertainty prior to patient recruitment to a clinical trial using a panel of expert clinical assessors and techniques for eliciting, pooling and modelling of expert opinions. The methodology is illustrated using example data from the UK Heel Fracture Trial. The statistical modelling provided results that were clear and simple to present to clinicians and showed how decisions regarding recruitment were influenced by both the collective opinion of the expert panel and the type of decision rule selected.

Conclusions

The statistical framework presented has potential to identify eligible patients and assist in the simplification of eligibility criteria which might encourage greater participation in clinical trials evaluating surgical interventions.

Scientific production and impact of national registers: the example of orthopaedic national registers

OBJECTIVE

National arthroplasty registers are often cited as examples of a non-randomized design that have made an essential contribution to advances in assessing arthroplasty procedures. We aimed to compare national registers to randomized controlled trials (RCTs) and meta-analyses in the field of arthroplasty in terms of scientific production and impact.

METHOD

We systematically searched Medline via PubMed and the registers' websites to select all articles from national registers, RCTs and meta-analyses assessing hip and knee arthroplasty. The scientific production and impact were evaluated by number of publications, number of citations (total and the 3-year citation counts), and information on the 2008 journal impact factor (IF), for each design and identified articles. We also contacted representatives of all the selected registers to determine the availability of the data for external research projects.

RESULTS

We retrieved information on 13 active national hip or knee arthroplasty registers; for 9, data were available for research projects under specific conditions. Overall, 190 publications in peer-reviewed journals resulted from national arthroplasty registers, 476 from RCTs, and 40 from meta-analyses. We found 4,112 citations for national register reports, 7,328 for RCT reports and 552 for meta-analysis reports. The median [interquartile [IQR] range] number of citations for register, RCT and meta-analysis reports in the 3-year period after publication was 3.5 [1.0-6.0], 2.0 [1.0-6.0], and 2.5 [0.5-7.5], respectively.

CONCLUSION

Publications from national registers may have the highest impact among the 3 designs in terms of median citation counts, but data from RCTs remain the most productive evidence in the arthroplasty field. Because of the number of patients recruited by registers, the quality of data collected, and the potential availability of data, scientific production and impact from national registers should be improved.

High-flexion total knee arthroplasty: a systematic review

This study is a systematic literature review of outcomes following total knee arthroplasty with implants specifically designed to enable increased knee flexion. English language comparative studies without date restriction were identified through a computerised literature search and bibliography review. Nine studies met the inclusion criteria representing a total of 399 high-flexion knee arthroplasties in 370 patients. Five studies reported greater flexion or range of motion; however, the methodological rigour was questionable with inadequate blinding, flawed participant selection, short follow-up periods and functional outcomes which lacked sensitivity. There was insufficient evidence of improved range of motion or functional performance after high-flexion knee arthroplasty.

Design considerations for randomised trials in orthopaedic fracture surgery

In the hierarchy of research designs, the results of randomised controlled trials are considered the highest level of evidence. Randomisation is the only method for controlling for both known and unknown prognostic factors between comparison groups. However, there are a number of challenges to conducting trials to evaluate surgical interventions. These include patient and surgeon preferences, inability to blind surgeons and difficulties blinding patients, difficulties in obtaining adequate sample sizes, and a lack of standardisation of surgical procedures. In this paper we address these issues and offer potential solutions within the context of conducting fracture trials in orthopaedics. Careful planning can help identify methodological issues, promote adaptive study designs, and lower the risk of bias to objectively assess new or existing surgical therapies.

How good is the orthopaedic literature?

Randomized trials constitute approximately 3% of the orthopaedic literature Concerns regarding quality of the orthopaedic literature stem from a widespread notion that the overall quality of the surgical literature is in need of improvement. Limitations in surgical research arises primarily from two pervasive issues: 1) A reliance on low levels of evidence to advance surgical knowledge, and 2) Poor reporting quality among the high level surgical evidence that is available. The scarcity of randomized trials may be largely attributable to several unique challenges which make them difficult to conduct. We present characteristics of the orthopaedic literature and address the challenges of conducting randomized trials in surgery.

A guide to planning and executing a surgical randomized controlled trial

Evidence-based medicine requires that treatments given to patients demonstrate effectiveness. The randomized controlled trial (RCT) has become the preeminent study design to assess the efficacy of treatments. Randomized controlled trials are frequently used to evaluate pharmaceutical treatments but are less often used in surgery. The lack of surgical RCTs is partly due to ethical and methodological concerns associated with surgical interventions. We provide a guide to planning and conducting a surgical RCT.