Quality of reporting in surgical randomized clinical trials

A Systematic Review to Inform the Development of a Reporting Guideline for Concept Mapping Research

Concept mapping is a phased, mixed-method approach that is increasingly used in health research to develop an understanding of complex phenomena. The six phases of concept mapping are preparation, idea generation, structuring (clustering and prioritization), data analysis, interpretation, and utilization of the map. The reporting of concept mapping research requires the development of a specific reporting guideline. We conducted a systematic review to identify candidate reporting items for inclusion in a reporting guideline. Three databases (MEDLINE, CINAHL, and PsycInfo) were searched to identify studies that used concept mapping methodology. We included 75 concept mapping studies published since 2019 from which we extracted information about the quality of reporting. A third of the studies focused on public health. We identified 71 candidate items that relate to the quality of reporting concept mapping research. The rationale for the study, the focus prompt, procedures for brainstorming, and structuring statements were consistently reported across the included studies. The process for developing the focus prompt, the rationale for the size of the stakeholder groups, and the process for determining the final concept map were generally not reported. The findings from the review will be used to inform the development of our reporting guideline for concept mapping research.

Two decades of surgical randomized controlled trials: worldwide trends in volume and methodological quality

BACKGROUND

RCTs are essential in guiding clinical decision-making but are difficult to perform, especially in surgery. This review assessed the trend in volume and methodological quality of published surgical RCTs over two decades.

METHODS

PubMed was searched systematically for surgical RCTs published in 1999, 2009, and 2019. The primary outcomes were volume of trials and RCTs with a low risk of bias. Secondary outcomes were clinical, geographical, and funding characteristics.

RESULTS

Some 1188 surgical RCTs were identified, of which 300 were published in 1999, 450 in 2009, and 438 in 2019. The most common subspecialty in 2019 was gastrointestinal surgery (50.7 per cent). The volume of surgical RCTs increased mostly in Asia (61, 159, and 199 trials), especially in China (7, 40, and 81). In 2019, countries with the highest relative volume of published surgical RCTs were Finland and the Netherlands. Between 2009 and 2019, the proportion of RCTs with a low risk of bias increased from 14.7 to 22.1 per cent (P = 0.004). In 2019, the proportion of trials with a low risk of bias was highest in Europe (30.5 per cent), with the UK and the Netherlands as leaders in this respect.

CONCLUSION

The volume of published surgical RCTs worldwide remained stable in the past decade but their methodological quality improved. Considerable geographical shifts were observed, with Asia and especially China leading in terms of volume. Individual European countries are leading in their relative volume and methodological quality of surgical RCTs.

Methods of quality assurance in multicenter trials in laparoscopic fundoplication for gastroesophageal reflux disease

BACKGROUND

Operative performance may affect the internal and external validity of randomized trials. The aim of this study was to review the use of surgical quality assurance mechanisms of published trials on laparoscopic anti-reflux surgery, with the objective to appraise their internal (research quality) and external validity (applicability to the clinical setting).

METHODS

Building upon a previous systematic review and network meta-analysis published by the authors, Medline, Embase, AMED, CINAHL, CENTRAL, and OpenGrey databases were searched for randomized control trials comparing different methods of laparoscopic anti-reflux surgery for the management of gastroesophageal disease. Quality assurance in individual studies was appraised using a specified framework addressing surgeon accreditation, procedure standardization, and performance monitoring.

RESULTS

In total, 2276 articles were screened to obtain 43 publications reporting 29 randomized controlled trials. Twenty-five out of 43 (58.1%) articles reported the number of participating centers and surgeons involved. Additionally, only 21/43 (48.8%) of articles reported consistent use of a bougie, while 23/43 (53.5%) of articles reported consistent division of the short gastric arteries during fundoplication. Surgical experience and credentials were stated in half of the studies. Standardization of the technique was reported in almost 70% of cases, whereas operative notes or video was submitted in one fourth of the studies. Monitoring of the operative performance during the trial was not documented in most of the trials (62%).

CONCLUSION

Surgical quality assurance in randomized trials on laparoscopic anti-reflux surgery is insufficient, which does not allow appraisal of the internal and external validity of this research. With improved reporting, trials assessing the use of laparoscopic anti-reflux surgery will enable surgeons to make informed treatment decisions to enhance patient care in the surgical management of GERD.

Examining the empirical evidence for IDEAL 2b studies: the effects of preceding prospective collaborative cohort studies on the quality and impact of subsequent randomized controlled trials of surgical innovations – protocol for a systematic review and case–control analysis

Randomized controlled trials (RCTs) in surgery face methodological challenges, which often result in low quality or failed trials. The Idea, Development, Exploration, Assessment and Long-term (IDEAL) framework proposes preliminary prospective collaborative cohort studies with specific properties (IDEAL 2b studies) to increase the quality and feasibility of surgical RCTs. Little empirical evidence exists for this proposition, and specifically designed 2b studies are currently uncommon. Prospective collaborative cohort studies are, however, relatively common, and might provide similar benefits. We will, therefore, assess the association between prior ‘IDEAL 2b-like’ cohort studies and the quality and impact of surgical RCTs.

We propose a systematic review using two parallel case–control analyses, with surgical RCTs as subjects and study quality and journal impact factor (IF) as the outcomes of interest. We will search for surgical RCTs published between 2015 and 2019 and and prior prospective collaborative cohort studies authored by any of the RCT investigators. RCTs will be categorized into cases or controls by (1) journal (IF ≥or <5) and (2) study quality (PEDro score ≥or < 7). The case/control OR of exposure to a prior ‘2b like’ study will be calculated independently for quality and impact. Cases will be matched 1: 1 with controls by year of publication, and confounding by peer-reviewed funding, author academic affiliation and trial protocol registration will be examined using multiple logistic regression analysis.

This study will examine whether preparatory IDEAL 2b-like studies are associated with higher quality and impact of subsequent RCTs.

The methodological quality on systematic reviews of surgical randomised controlled trials: A cross-sectional survey

BACKGROUND

Systematic reviews of RCTs have been developed to address end-users' needs and are regarded as the highest level of evidence. Flaws in the design, conduct and analyses of a systematic review can lead to erroneous conclusions and increase the research waste.

OBJECTIVE

We undertook a cross-sectional survey to identify the critical areas of weakness in systematic reviews for surgical interventions by AMSTAR 2.

METHODS

We searched PubMed, EMbase and Cochrane Library to summarize systematic reviews of surgical RCTs published in 2017. The information regarding general characteristics and methodological characteristics were gathered. We conducted descriptive analyses of study characteristics of included systematic reviews and explored the difference among varied methodological quality.

RESULTS

Totally 141 systematic reviews were identified. We found only four reviews (2.8%) were high quality, 3 (2.1%) were moderate quality, 8 (5.7%) were low quality, and the remaining 126 (89.4%) were of critical low quality. The critical weaknesses were lack of pre-registration or published protocols (29.1%), comprehensive literature search (17.7%), lists of excluded studies and reasons for exclusion (19.1%), description of detailed interventions (8.5%), extraction of funding source from included trials (10.6%), and consideration of the risk of bias of included trials when synthesized (16.3%) and interpret (20.6%) the results. Higher methodological quality was only positively associated with Cochrane systematic review.

CONCLUSION

Although two-thirds of included systematic reviews in the field of surgery were published in journals ranking Q1, the methodological quality is suboptimal and needs to be substantially improved. More efforts of multi-disciplinary teams' collaboration, continual education and training, integrally connection between primary studies and systematic review and contributing surgical research to practice should be imperative.

Reporting quality of surgical randomised controlled trials in head and neck cancer: a systematic review

PURPOSE

Randomised controlled trials (RCTs) are considered the gold standard for evaluating the efficacy of an intervention. However, previous research has shown that RCTs in several surgical specialities are poorly reported, making it difficult to ascertain if various biases have been appropriately minimised. This systematic review assesses the reporting quality of surgical head and neck cancer RCTs.

METHODS

A literature search of PubMed and Embase was performed. Papers were included if they reported RCTs which assessed a surgical technique used to treat or diagnose head and neck cancer published during or after 2011. The CONSORT 2010 checklist was used to evaluate the reporting quality of these trials.

RESULTS

41 papers were included. The mean CONSORT score was 16.5/25 (66% adherence) and the scores ranged from 7.5 (30%) to 25. The most common omissions were full trial protocol (found in 14.6%), participant recruitment method (22%) and effect size with a precision estimate for all outcome measures (29.3%). The full design and implementation of the randomisation methods were reported in 6 (14.6%). Papers published in journals which endorsed CONSORT had significantly higher scores (p = 0.02) and the journal impact factor was significantly correlated with CONSORT score (p = 0.01).

CONCLUSION

We have identified several pieces of information that are underreported in surgical head and neck cancer RCTs. These omissions make understanding and comparing the methodologies and conclusions of RCTs more difficult. The endorsement of CONSORT by journals improved adherence, suggesting that wider adoption of the checklist may improve reporting.

The Value of Systematic Reviews and Meta-Analyses in Surgery

BACKGROUND

Systematic reviews and meta-analyses are generally regarded as sitting atop the hierarchy of clinical evidence. The unbiased summary of current evidence that a systematic review provides, along with the increased statistical power from larger numbers of patients, is invaluable in guiding clinical decision-making and development of practice guidelines. Surgical specialties have historically lagged behind other areas of medicine in the application of evidence-based medicine, perhaps due to the unique challenges faced in the conduct of surgical clinical trials. These challenges extend to the conduct of systematic reviews, due to issues with the quality and heterogeneity of the underlying literature.

SUMMARY

Recent years have seen an improvement in the quality of randomized controlled trials in surgical topics and an explosion in the publication of systematic reviews. This review explores recent trends in systematic reviews in surgery and discussed some of the aspects in conducting and interpreting reviews that are unique to surgical topics, including blinding, surgical heterogeneity and learning curves, patient and clinician preference, and industry involvement. Key Messages: Clinical trials, and therefore systematic reviews, of surgical interventions pose unique challenges which are important to consider when conducting them or applying the findings to clinical practice. Despite the challenges, systematic reviews still represent the best level of evidence for development of surgical practice guidelines.

Adherence to the CONSORT statement and extension for nonpharmacological treatments in randomized controlled trials of bariatric surgery: A systematic survey

Reporting is critical for establishing the value of randomized controlled trials (RCTs). This study evaluated the adherence of bariatric surgery RCT reporting to the CONsolidated Standards Of Reporting Trials (CONSORT) statement 2010 and its 2017 extension for non-pharmacologic treatments (NPT extension). We identified all RCTs comparing bariatric surgery with conservational therapy or alternative bariatric surgery up to June 30, 2020. Reporting quality was assessed using criteria developed from the CONSORT statement and the NPT extension and scored as a percentage. The factors associated with reporting quality were explored by univariate and multivariate analysis. In total, 102 RCTs of bariatric surgery were included. The median scores according to the CONSORT statement and NPT extension were 63.3 and 26.8 of a maximum possible 100, respectively. Two-thirds of NPT extension items were reported in less than 25% of the RCTs. The median score improved over time for the CONSORT statement but not the NPT extension. A higher CONSORT score was associated with publication in core clinical journals, protocol registration, and funding. No factors associated with the NPT extension score were identified. Substantial efforts are warranted from authors, journals, registration platforms, and funders to overcome the flaws in the reporting of bariatric surgery RCTs.

Assessment of the reporting quality of randomised controlled trials of massage

Objective

To assess the reporting quality of randomised controlled trials (RCTs) of massage, particularly whether necessary elements related to massage interventions were adequately reported.

Methods

A total of 8 electronic databases were systematically searched for massage RCTs published in English and Chinese from the date of their inception to June 22, 2020. Quality assessment was performed using three instruments, namely the CONSORT (Consolidated Standards of Reporting Trials) 2010 Checklist (37 items), the CONSORT Extension for NPT (Nonpharmacologic Treatments) 2017 checklist (18 items), and a self-designed massage-specific checklist (16 items) which included massage rationale, intervention and control group details. Descriptive statistics were additionally used to analyse the baseline characteristics of included trials.

Results

A total of 2,447 massage RCTs were identified, of which most (96.8%) were distributed in China. For the completeness of CONSORT, NPT Extension, and massage-specific checklists, the average reporting percentages were 50%, 10% and 45%, respectively. Of 68 assessed items in total (exclusion of 3 repeated items on intervention), 42 were poorly presented, including 18 CONSORT items, 15 NPT items, and 9 massage-specific items. Although the overall quality of reporting showed slightly improvement in articles published after 2010, the international (English) journals presented a higher score of the CONSORT and NPT items, while the Chinese journals were associated with the increased score of massage-specific items.

Conclusion

The quality of reporting of published massage RCTs is variable and in need of improvement. Reporting guideline “CONSORT extension for massage” should be developed.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13020-021-00475-6.

Surgical Clinical Trials in India: Underutilized Opportunities

Clinical trials in Surgery are central to research; however, very few surgical clinical trials are conducted in India. Such paucity of surgical trials is a cause for concern, and prompted us to explore the recent landscape of surgical trials in India. We reviewed all clinical trials from general surgery or subspecialties of general surgery registered with the Clinical Trials Registry of India website between 2018 to 15 th May 2021. Specific details such as the surgical subspecialty, study design, multicentric or single institution and funding were obtained. We found a total of 16,710 trials, out of these 4119 (24.6%) were related to all surgical fields. Only 136 (0.8%) trials were found from general surgery and its subspecialties. Most trials were registered from Central Government Institutions (48%), followed by State Government Medical Colleges (11%). Most number of trials was registered from GI surgery (32%). Most (90.5%) trials were single centre based. Common barriers to research are well known; if the State Government Medical Colleges can mentor a culture of research from an early stage of surgical training it can improve research productivity. Multicentre trials, involving smaller hospitals from tier 2 and tier 3 cities, are a potential solution to one of the major obstacles of surgical trials i.e. small number of patients; especially in this pandemic induced draught of elective surgical operations. A positive change in attitude of surgeons and provision of necessary funding can encourage more surgical clinical trials in India.

Randomized Trials in Cardiac Surgery: JACC Review Topic of the Week

Compared with randomized controlled trials (RCTs) in medical specialties, RCTs in cardiac surgery face specific issues. Individual and collective equipoise, rapid evolution of the surgical techniques, as well as difficulties in obtaining funding, and limited education in clinical epidemiology in the surgical community are among the most important challenges in the design phase of the trial. Use of complex interventions and learning curve effect, differences in individual operators' expertise, difficulties in blinding, and slow recruitment make the successful completion of cardiac surgery RCTs particularly challenging. In fact, over the course of the last 20 years, the number of cardiac surgery RCTs has declined significantly. In this review, a team of surgeons, trialists, and epidemiologists discusses the most important challenges faced by RCTs in cardiac surgery and provides a list of suggestions for the successful design and completion of cardiac surgery RCTs.

A Review of Adherence and Predictors of Adherence to the CONSORT Statement in the Reporting of Tuberculosis Vaccine Trials

The statement on Consolidated Standards of Reporting Trials (CONSORT) ensures transparency in the reporting of randomized trials. However, it is unclear if the statement has led to improvement in the quality of reporting of tuberculosis (TB) vaccine trials. We explored the quality of reporting of TB vaccine trials according to the latest version of the CONSORT statement, released in 2010. We searched PubMed and the Cochrane Central Register of Controlled Trials in August 2019. We conducted screening, study selection, and data extraction in duplicate; and resolved differences through discussion. We assessed reporting to be adequate if trials reported at least 75% of the CONSORT 2010 items. We conducted a trend analysis to assess if there was improvement in reporting over time. We also used logistic regression to assess factors associated with adequate reporting. We included 124 trials in the analyses. The mean proportion of adherence was 67.3% (95% confidence interval 64.4% to 70.1%), with only 46 (37%) trials having adequate reporting. There was a significant improvement in the quality of reporting over time (p < 0.0001). Trials published in journals with impact factors between 10 and 20 were more likely to have adequate reporting (odds ratio 9.4; 95% confidence interval 1.30 to 67.8), compared to lower-impact-factor journals. Despite advances over time, the reporting of TB vaccine trials is still inadequate and requires improvement.

Identifying competing interest disclosures in systematic reviews of surgical interventions and devices: a cross-sectional survey

Background

A competing interest is an important source of bias in research and disclosure is frequently employed as a strategy to manage it. Considering the importance of systematic reviews (SRs) and the varying prevalence of competing interests in different research fields, we conducted a survey to identify the range of competing interests in SRs assessing surgical interventions or devices and explored the association between the competing interest disclosures and authors’ conclusions.

Methods

We retrieved SRs of surgical interventions and devices published in 2017 via PubMed. Information regarding general characteristics, funding sources, and competing interest disclosures were extracted. We conducted a descriptive analysis of the studies’ characteristics and compared them between Cochrane SRs (CSRs) and non-Cochrane SRs using the Chi-square test. Results were expressed as odds ratio and their 95% confidence interval.

Results

One hundred fifty-five SRs published in 2017 were included in the study. More than half of the SRs (58.7%) reported their funding sources and 94.2% reported authors’ competing interest disclosures. Among 146 SRs that stated competing interest disclosures, only 35 (22.6%) SRs declared at least one author had a competing interest. More than 40 terms were used to describe competing interests. Cochrane SRs (CSRs) were more likely to provide a detailed description of competing interests compared to those in non-CSRs (48.0% versus 25.4%, P = 0.023). No association between positive conclusions and competing interest disclosures was found (P = 0.484, OR = 0.43, 95%CI: 0.08, 2.16). In the subgroup analyses, SRs stating no competing interest disclosure were more likely to report positive conclusions than those stating at least one type of competing interest, but the difference is not significantly different (P = 0.406, OR = 1.38, 95%CI: 0.64, 2.98).

Conclusion

In surgical SRs, there is a high percentage of competing interest disclosures but without detailed information. The identification and statement of competing interests with a detailed description, particularly the non-financial ones, needs improvement. Some efficient and effective methods/tools for identifying, quantifying, and minimizing potential competing interests in systematic reviews remains valuable.

How to set up a clinical trial

Clinical trials are considered the gold-standard method for the evaluation of healthcare interventions. However, randomised control trials are complex to perform and many researchers, especially those in the early stages of their career, can find it challenging to know where to start set up, contribute to or lead a trial. This guide provides an introduction to trials and also practical advice to help potential investigators complete their clinical trial to time and to budget by signposting the pathway through the complex regulatory landscape. The authors draw on their own recent experiences of running clinical trials and provide tips and tricks for troubleshooting common problems encountered including trial design and documentation.

Design, Conduct, and Analysis of Surgical Randomized Controlled Trials: A Cross-sectional Survey

BACKGROUND

Randomized controlled trial (RCT) testing surgical intervention faced challenges due to complexities of surgery and made it more difficult for surgeons and methodologists than pharmaceutical providers to build a well-design, conduct RCT.

OBJECTIVE

We conducted a cross-sectional survey to address the methodological challenges of RCTs on surgical intervention and offer potential solutions.

METHODS

We searched PubMed in order to summarize 2-arm parallel randomized trials for surgical interventions published in 2013. The information regarding general characteristics, general methodological and special surgical characteristics related to surgical trials comparing alternative procedures was gathered.

RESULTS

Some 200 surgical trials were identified. The extent to which these trials in design, conduct and analysis differed substantially across items. The general information about sample size calculation (77.0%), lost to follow-up (71.5%), trial registration (55.5%), protocols of trials (56.0%), implementation of randomization (59.5%), concealment of randomization (56.0%); reporting of primary outcome as P value (67.0%). Surgery special information revealed that only 21.0% of trials considered surgeons' preference, approximately 12% to 50% of them controlled the quality of surgical interventions and none evaluated the effect of the learning curve.

CONCLUSION

There is much room for improvement concerning the reported designs, conduct, and analysis of surgical RCTs. Considering the difficulty of surgical RCTs, some other approaches, such as surgeons' eligibility, performance of pilot studies, or implementation of pragmatic RCTs/expertise-based trials, should be feasibly implemented to overcome the presented challenges.

Reporting of key methodological issues in placebo-controlled trials of surgery needs improvement: a systematic review

OBJECTIVES

To examine key methodological considerations for using a placebo intervention in randomized controlled trials (RCTs) evaluating invasive procedures, including surgery.

STUDY DESIGN AND SETTING

RCTs comparing an invasive procedure with a placebo were included in this systematic review. Articles published from database inception to December 31, 2017, were retrieved from Ovid MEDLINE, Ovid EMBASE and CENTRAL electronic databases, by handsearching references and expert knowledge. Data on trial characteristics (clinical area, nature of invasive procedure, number of patients and centers) and key methodological (rationale for using placebos, minimization of risk, information provision, offering the treatment intervention to patients randomized to placebo, delivery of cointerventions, and intervention standardization and fidelity) were extracted and summarized descriptively.

RESULTS

One hundred thirteen articles reporting 96 RCTs were identified. Most were conducted in gastrointestinal surgery (n = 40, 42%) and evaluated minimally invasive procedures (n = 44, 46%). Over two-thirds randomized fewer than 100 patients (n = 65, 68%) and a third were single center (n = 31, 32%). A third (n = 33, 34%) did not report a rationale for using a placebo. Most common strategies to minimize patient risk were operator skill (n = 22, 23%) and independent data monitoring (n = 28, 29%). Provision of patient information regarding placebo use was infrequently reported (n = 11, 11%). Treatment interventions were offered to patients randomized to placebo in 43 trials (45%). Cointerventions were inconsistently reported, but 64 trials (67%) stated that anesthesia was matched between groups. Attempts to standardize interventions and monitor their delivery were reported in n = 7, (7%) and n = 4, (4%) trials, respectively.

CONCLUSION

Most placebo-controlled trials in surgery evaluate minor surgical procedures and currently there is inconsistent reporting of key trial methods. There is a need for guidance to optimize the transparency of trial reporting in this area.

Quality of reporting of systematic reviews and meta-analyses of surgical randomized clinical trials

Background

Well designed and conducted systematic reviews are essential to clinical practice. Surgical intervention is more complex than medical intervention when considering special items related to procedures. There has been no cross‐sectional study of the reporting quality of systematic reviews of surgical randomized trials focused on special items relating to surgical interventions.

Methods

A cross‐sectional survey of systematic reviews of surgical randomized trials published in 2007 and 2017 was undertaken via a PubMed search. Quality of reporting was assessed by the PRISMA checklist, with intervention details containing 27 items. Univariable and multivariable linear regression was used to explore factors in the checklist as indicators of reporting quality.

Results

A total of 204 systematic reviews were identified. The median score for the PRISMA checklist was 22 (i.q.r. 20–24), and systematic reviews published in 2017 had a significantly higher median score than those from 2007 (22 (i.q.r. 21–24) versus 20 (17–22); P < 0·001). Among the 27 items, 15 were reported adequately and three were reported poorly (in less than 50 per cent of reports). The proportion of other items reported ranged from 54·4 to 77·9 per cent. In multivariable analysis, systematic reviews published in 2017 (coefficient 0·59, 95 per cent c.i. 0·50 to 0·69) and Cochrane reviews (coefficient 0·67, 0·55 to 0·81) were associated with better reporting.

Conclusion

The quality of reporting of systematic reviews of surgical randomized trials has improved in the past 10 years. Some information relating to specific surgical interventions is, however, still reported poorly.

Challenges and potential solutions to the evaluation, monitoring, and regulation of surgical innovations

Background

As it may be argued that many surgical interventions provide obvious patient benefits, formal, staged assessment of the efficacy and safety of surgical procedures has historically been and remains uncommon. The majority of innovative surgical procedures have therefore often been developed based on anatomical and pathophysiological principles in an attempt to better manage clinical problems.

Main Body

In this manuscript, we sought to review and contrast the models for pharmaceutical and surgical innovation in North America, including their stages of development and methods of evaluation, monitoring, and regulation. We also aimed to review the present structure of academic surgery, the role of methodological experts and funding in conducting surgical research, and the current system of regulation of innovative surgical procedures. Finally, we highlight the influence that evidence and surgical history, education, training, and culture have on elective and emergency surgical decision-making. The above discussion is used to support the argument that the model used for assessment of innovative pharmaceuticals cannot be applied to that for evaluating surgical innovations. It is also used to support our position that although the evaluation and monitoring of innovative surgical procedures requires a rigorous, fit-for-purpose, and formal system of assessment to protect patient safety and prevent unexpected adverse health outcomes, it will only succeed if it is supported and championed by surgical practice leaders and respects surgical history, education, training, and culture.

Conclusion

We conclude the above debate by providing a recommended approach to the evaluation, monitoring, and regulation of surgical innovations, which we hope may be used as a guide for all stakeholders involved in interpreting and/or conducting future surgical research.

Quality of Randomized Controlled Trials Published By Plastic Surgeons: Long-Term Follow-Up

INTRODUCTION

In two previous studies, the quality of randomized controlled trials (RCTs) with the participation of at least one plastic surgeon was evaluated in two periods: 1966-2003 and 2004-2008.

OBJECTIVE

To evaluate the evolution of the quality of RCT publications by plastic surgeons in the subsequent 5-year period, from 2009 to 2013.

METHODS

RCTs published from 2009 to 2013 in English with the participation of at least one plastic surgeon were identified by electronic search and classified for concealment of allocation by two independent evaluators. The studies with adequate allocation concealment had their quality evaluated by two evaluators using the Delphi List and the Jadad Quality Scale.

RESULTS

Of the 6997 identified studies, 261 were classified as to concealment of allocation. Of these, 43 (16.47%) had adequate allocation concealment. According to the evaluation in the Delphi List, there was an improvement, in relation to 1966-2003, in the items "most important characteristics of the prognosis" (p < 0.001), "use of independent evaluator" (p = 0.0029), and "measures of variability and point estimate for the primary variable" (p = 0.0057); there was no difference in relation to 2004-2008. Regarding the Jadad Quality Scale, there was an increase in scores in relation to 1966-2003 (p < 0.0004) but not in relation to the 2004-2008 period.

CONCLUSION

There was no difference in the quality of publications of RCTs by plastic surgeons in the period 2009-2013 compared to the previous 5 years (2004-2008), but both periods presented higher quality than the period 1966-2003.

LEVEL OF EVIDENCE III

This journal requires that authors assign a level of evidence to each article. For a full description of these evidence-based medicine ratings, please refer to the Table of Contents or the online Instructions to Authors www.springer.com/00266 .

Quality of Randomized Controlled Trials for Surgical Treatment of Carpal Tunnel Syndrome: A Systematic Review

BACKGROUND

Randomized controlled trials are considered the gold standard in evidence-based medicine. The authors conducted a systematic review to evaluate the quantity, quality, and trends of randomized controlled trials that assess surgical treatment of carpal tunnel syndrome.

METHODS

The authors identified randomized controlled trials comparing two or more surgical interventions for carpal tunnel syndrome in PubMed, Cochrane, Scopus, Google Scholar, and Clinicaltrials.gov. Two independent reviewers evaluated articles for inclusion, extracted data, and assessed randomized controlled trial quality using the Jadad score.

RESULTS

Of 2253 identified studies, 58 met full inclusion criteria. They were published between 1985 and 2015, with a significant increase over time (p = 0.003). They were most frequently published in Journal of Hand Surgery (European Volume) [n = 15 (25.9 percent)]. Most randomized controlled trials were single-center studies [n = 54 (93.1 percent)] conducted in the United Kingdom [n = 13 (22.4 percent)] or the United States [n = 10 (17.2 percent)], with a mean study size of 80.1 ± 55.5 patients. Funding source was unknown in 62.1 percent (n = 36). Three-quarters [n = 44 (75.9 percent)] of randomized controlled trials did not define the primary outcome measure(s). Less than 30 percent (n = 17) of randomized controlled trials conducted a power analysis. Only four studies with patients reported lost to follow-up provided an explanation for each patient. Six randomized controlled trials (10.3 percent) conducted intention-to-treat analysis. The mean Jadad score was 2.14 ± 1.26, with no significant improvement over time (p = 0.245).

CONCLUSIONS

Despite the significant increase in the number of randomized controlled trials published studying surgical treatment of carpal tunnel syndrome over time, a mean Jadad score of 2.14 with no change over time indicates a need for improvement in quality. Proper study design is key to avoiding introduction of bias and ensuring the validity of conclusions drawn.

Quality of reporting in randomized controlled trials of therapeutic cardiovascular medical devices

BACKGROUND

Therapeutic medical devices play an important role in the treatment of cardiovascular diseases. The reliability of the randomized controlled trial, which is the best design for assessing treatment effects, largely depends on the information found in published reports. Limited information regarding the quality of reporting about therapeutic medical devices in trials was provided.

METHOD

A cross-sectional study was conducted to assess the reporting quality of randomized controlled trials that tested the effects of therapeutic cardiovascular medical devices. The quality of reporting was assessed against a modified Consolidated Standards of Reporting Trials checklist, including 47 items from the Consolidated Standards of Reporting Trials statement and Consolidated Standards of Reporting Trials extension. We also examined the specific items regarding medical devices. Univariable and multivariable linear regressions were undertaken to explore potential factors associated with Consolidated Standards of Reporting Trials scores.

RESULT

Some 115 randomized controlled trials were identified. The mean (standard deviation) Consolidated Standards of Reporting Trials score was 20.5 (5.0). The extent of compliance with the Consolidated Standards of Reporting Trials reporting guideline differed substantially across items: 5 of the 47 items were reported adequately across trials (more than 90%), and 10 were reported adequately in less than 5% of trials. Less than 50% of the trials reported additional items related to the medical device. Multivariable regression analysis showed that trials published in general journals (coefficient 7.44, 95% confidence interval [CI]: 5.50-9.38), with larger sample sizes (coefficient 2.30, 95% CI: 0.76-3.83), and multiple-center studies (coefficient 3.14, 95% CI: 1.27-5.01) were associated with a higher quality of reporting.

CONCLUSION

The overall reporting quality in randomized controlled trials of therapeutic medical device trials is suboptimal, particularly in terms of items regarding surgeons and hospitals. We suggest that the existing Consolidated Standards of Reporting Trials and extension should be modified to be more applicable to therapeutic medical devices.

Quality of surgical randomized controlled trials in hand surgery: a systematic review

We assessed the quantity, quality and trends of randomized controlled trials comparing hand surgical interventions. Study characteristics were collected for 125 randomized controlled trials comparing hand surgical interventions. The Jadad scale (0-5), which assesses methodological quality of trials, was calculated. Logistic regressions were conducted to determine associations with the Jadad score. The studies were published between 1981 and 2015, with an increase over time, most often in Journal of Hand Surgery (European). Mean study size was 68 patients. Mean Jadad score was 2.1, without improvement over time. Thirty percent conducted a power analysis and 8% an intention-to-treat analysis. Studies conducted in the United Kingdom and with smaller sample sizes, power analysis and intention-to-treat analysis were associated with a higher Jadad score. The quantity of trials has increased over time while methodological quality has remained low, indicating a need to improve quality of trials in hand surgery literature.

The reporting of safety among drug systematic reviews was poor before the implementation of the PRISMA harms checklist

OBJECTIVES

To examine, through a cross-sectional survey, how well safety information was reported among drug systematic reviews predating PRISMA harms checklist and explore factors associated with better reporting.

STUDY DESIGN AND SETTING

We searched PubMed to identify all systematic reviews published in the Cochrane Database of Systematic Review or the core clinical journals in 2015, one year before the PRISMA harms checklist was published. We randomly selected, in a 1:1 ratio, Cochrane and non-Cochrane systematic reviews assessing drug effects (including both efficacy and safety). We used the PRISMA harms checklist published in 2016 to assess the quality of reporting of drug safety information. Multivariable linear regression analyses were used to explore the association of six prespecified variables with more complete reporting of PRISMA harms items.

RESULTS

We included 120 systematic reviews, including 60 Cochrane and 60 non-Cochrane reviews. Scores on the PRISMA harms checklist (23 items) were low (median 4, [first, third quartile: 2, 6]), with no difference between Cochrane and non-Cochrane reviews (4.5 [2, 7] vs. 4 [2.5, 5]; P = 0.29). Among all eligible reviews, only one item (i.e., state conclusions in coherence with the review findings) was reported adequately (proportion of adherence 81.6%); proportion of reporting for other items ranged from 1.7% to 68.3%. The four essential reporting items from PRISMA harms checklist were also poorly complied (proportion of adherence ranged from 1.7% to 9.2%). Multivariable linear regression analyses found no significant associations between any study characteristic and reporting on the PRISMA harms, likely because of limited variability in scores across studies.

CONCLUSIONS

The reporting of safety information was poor both for Cochrane and non-Cochrane drug systematic reviews predating PRISMA harms checklist. The findings suggested a strong need to use the PRISMA harms checklist for reporting safety among drug systematic reviews.

Description of complex interventions: analysis of changes in reporting in randomised trials since 2002

Background

Inadequate description of non-pharmacological complex interventions in trial publications means that they cannot be replicated or assessed for generalisability. There are published guidelines on how to describe an intervention, such as those from the CONSORT Group. However, there have been few evaluations of whether intervention reporting is improving.

Methods

We aimed to assess whether descriptions of multicomponent, non-pharmacological interventions evaluated in randomised trials are improving. To do so, we chose trials of educational and psychotherapeutic interventions to promote adherence to therapy, and compared those published between 2002 and 2007 (Time-1) with those between 2010 and 2015 (Time-2). These time periods were chosen to concord with the publication in 2008 of the CONSORT extension statement of reporting guidelines for non-pharmacological treatment which included items on intervention description. We assessed 19 items, based on the CONSORT Statement and the more recent Template for Intervention Description and Replication Checklist (TIDieR). Two reviewers independently extracted data. We created a quality score of the eight items we considered key information for replication and assessment of generalisability (setting, provider, recipient, comparator, intervention intensity, how it was conducted, existence of a manual or protocol, and detail of whether there was an assessment of fidelity). Score per item was ‘1’ if reported adequately and ‘0’ if not.

Results

Of the eligible trials, 42 were published in Time-1 and 134 published in Time-2. The trials included were published in 112 peer-reviewed journals, 52 of these journals currently require authors to follow the CONSORT Statements, while only one recommended adherence to the TIDieR.

Most items of CONSORT and TIDieR were reported by more than half of the trials at both time points. Few trials reported fidelity. A large proportion of the trials did not report the existence of a manual or protocol, or what the comparator group received. We found no statistically significant improvement in the eight-item quality score (Time-1: mean 5.71 (standard deviation (SD) 1.09), Time-2: 5.87 (SD 1.28), p = 0.49).

Conclusions

We found no overall evidence that reporting the specifics of multicomponent, non-pharmacological interventions is improving. Details to replicate interventions remain lacking, impairing best implementation or meaningful further research. Editorial endorsement of reporting checklists needs to be more extensive.