Neglected external validity in reports of randomized trials: the example of hip and knee osteoarthritis

Deficiencies in reporting inclusion/exclusion criteria and characteristics of patients in randomized controlled trials of therapeutic interventions in pressure injuries: a systematic methodological review

Wound care is a complex procedure and the related research may include many variables. Deficiencies in the sample inclusion and exclusion criteria may limit the generalizability of randomized controlled trials (RCTs) for wound patients in the real world. This study aimed to evaluate deficiencies in reporting the inclusion and exclusion criteria and the characteristics of patients in RCTs of pressure injuries (PI) therapeutic interventions. We conducted a systematic methodological review in which 40 full text RCTs of PI treatment interventions published in English, from 2008 to 2020, were identified. Data on the general characteristics of the included RCTs and data about inclusion/exclusion criteria and characteristics of patients were collected. The inclusion/exclusion criteria were categorized into five domains (definition of disease, precision, safety, ethical/legal and administrative). Study duration (in weeks) was 8.0 (quartile 1: 2.0; quartile 3: 48.0); only 5.0% of the trials mentioned race, skin colour or ethnicity, and 37.5% reported the duration of the wound. Only 9 (22.5%) studies reported the drugs that the included patients were using and 10 (25.0%) RCTs reported adverse events. The presence of the five domains was observed only in 12.5% of RCTs and only 12 (30.0%) had the precision domain. Much more research is required in systematic assessments of the external validity of trials because there is substantial disparity between the information that is provided by RCTs and the information that is required by clinicians. We concluded that there are deficiencies in reporting of data related to inclusion/exclusion criteria and characteristics of patients of RCTs assessing PI therapeutic interventions.

Herausforderungen von Evidenzlevel-1-Studien in der Unfallchirurgie und Orthopädie

Randomisierte kontrollierte klinische Studien stellen für chirurgische Fachdisziplinen eine Herausforderung dar. Dieser Artikel beschreibt die Besonderheiten und Schwierigkeiten solcher Studien. Wann immer möglich, sollte das höchste Evidenzlevel angestrebt werden. Ist ein hohes Evidenzlevel allerdings methodisch nicht korrekt erreichbar, sollte das bestmögliche nächste Evidenzlevel erzielt werden. Die Bewertung, auch im Hinblick auf die Kostenerstattung, sollte daher gerade in chirurgischen Disziplinen nicht ausschließlich am Evidenzlevel festgemacht werden. Vielmehr sollte im Rahmen von Expertengremien und unter Zuhilfenahme der besten, verfügbaren Evidenz beurteilt werden.

The research burden of randomized controlled trial participation: a systematic thematic synthesis of qualitative evidence

BACKGROUND

Participation in randomized controlled trials (RCTs) may be quite demanding and could represent an important burden for patients. We aimed to explore this research burden (i.e., the psychological, physical, and financial burdens) experienced by patients through their participation in a RCT.

METHODS

We conducted a systematic review of qualitative studies exploring adult patients' experiences with RCT participation. We searched MEDLINE (PubMed), CINAHL, PSYCHINFO, and Embase (search date March 2018) for eligible reports. Qualitative data coding and indexing were assisted by NVivo. The quality of reports was assessed by using the Critical Appraisal Skills Program (CASP) tool.

RESULTS

We included 45 qualitative studies that involved 1732 RCT participants. Important psychological burdens were identified at every stage of the trial process. Participants reported feeling anxiety and being afraid of "being a 'guinea pig'" and described undergoing randomization and allocation to a placebo as particularly difficult resulting in disappointment, anger, and depression. Patients' follow-up and trial closure were also responsible for a wide range of psychological, physical, and financial burdens. Furthermore, factors related to burdensome impacts and consequences were discerned. These factors involved trial information, poorly organized and too-demanding follow-up, and lack of appropriate management when the patient's participation ended. Trial participation was also associated with beneficial effects such as the satisfaction of feeling "useful," gaining "a sense of control," and receiving special attention.

CONCLUSIONS

Our finding provides a detailed description of research burden across the whole RCT process. Many of the burdens described could be anticipated, and some avoided in a movement toward minimally disruptive clinical research. Such an approach could improve trial recruitment and retention.

REVIEW REGISTRATION

PROSPERO CRD42018098994.

External Validity of the HOPE-Trial: Hemiarthroplasty Compared with Total Hip Arthroplasty for Displaced Femoral Neck Fractures in Octogenarians

BACKGROUND

Randomized controlled trials (RCTs) are the most reliable way of evaluating the effect of new treatments by comparing them with previously accepted treatment regimens. The results obtained from an RCT are extrapolated from the study environment to the general health care system. The ability to do so is called external validity. We sought to evaluate the external validity of an RCT comparing the results of total hip arthroplasty with those of hemiarthroplasty for the treatment of displaced femoral neck fractures in patients ≥80 years of age.

METHODS

This prospective, single-center cohort study included 183 patients ≥80 years of age who had a displaced femoral neck fracture. All patients were screened according to the inclusion and exclusion criteria for an RCT comparing total hip arthroplasty and hemiarthroplasty. The population for this study consisted of patients who gave their informed consent and were randomized into the RCT (consenting group, 120 patients) as well as those who declined to give their consent to participate (non-consenting group, 63 patients). The outcome measurements were mortality, complications, and patient-reported outcome measures. Follow-up was carried out postoperatively with use of a mailed survey that included patient-reported outcome questionnaires.

RESULTS

We found a statistically significant and clinically relevant difference between the groups, with the non-consenting group having a higher risk of death compared with the consenting group. (hazard ratio, 4.6; 95% confidence interval, 1.9 to 11.1). No differences were found between the groups in terms of patient-reported outcome measures or surgical complications.

CONCLUSIONS

This cohort study indicates a higher mortality rate but comparable hip function and quality of life among eligible non-consenters as compared with eligible consenters when evaluating the external validity of an RCT in patients ≥80 years of age with femoral neck fracture.

LEVEL OF EVIDENCE

Therapeutic Level II. See Instructions for Authors for a complete description of levels of evidence.

Evidence into practice: protocol for a new mixed-methods approach to explore the relationship between trials evidence and clinical practice through systematic identification and analysis of articles citing randomised controlled trials

INTRODUCTION

Randomised controlled trials (RCTs) provide high-quality evidence to inform practice. However, much routine care is not based on available RCT evidence. Understanding this disconnect may improve trial design, reporting and implementation. Published literature commenting on RCTs may yield relevant insights. This protocol presents a new approach examining how researchers understand, contextualise and use evidence from RCTs, through analysis of letters, editorials and discussion pieces citing individual RCTs. Surgical case studies will illustrate its ability to identify wide-ranging factors influencing application of trials evidence.

METHODS AND ANALYSIS

In-depth study of published literature will explore written responses to RCTs. After purposefully selecting individual RCTs, we will systematically identify all citing articles covered in Web of Science and Scopus. Editorials, discussions and letters will be included. These are considered most likely to provide critiques and opinions about index RCTs. Original articles and reviews will be excluded. Clinical specialty, RCT design, outcomes and bibliographical data will be collected for RCTs and citing articles. Citing articles will be thematically analysed using the constant comparison technique to explore author understanding, contextualisation and relationship to clinical practice for the index trial. Coding will include generic issues relevant to all RCTs, such as sample size or blinding, and features specific to surgery, such as learning curve. Index trial quality will be examined using validated tools. Results will be combined to create a broad overview of the understanding and use of RCT evidence.

ETHICS AND DISSEMINATION

This study involves secondary use of existing articles and does not require ethical approval. Pilot work will establish its feasibility and inform progression to larger scale utilisation across a broad range of RCTs. Findings will be published in a peer-reviewed journal and presented at surgical and methodological conferences. Results will guide future work on trial design to optimise implementation of results.

Total disc arthroplasty versus anterior cervical interbody fusion: use of the Spine Tango registry to supplement the evidence from randomized control trials

BACKGROUND CONTEXT

Several randomized controlled trials (RCTs) have compared patient outcomes of anterior (cervical) interbody fusion (AIF) with those of total disc arthroplasty (TDA). Because RCTs have known limitations with regard to their external validity, the comparative effectiveness of the two therapies in daily practice remains unknown.

PURPOSE

This study aimed to compare patient-reported outcomes after TDA versus AIF based on data from an international spine registry.

STUDY DESIGN AND SETTING

A retrospective analysis of registry data was carried out.

PATIENT SAMPLE

Inclusion criteria were degenerative disc or disc herniation of the cervical spine treated by single-level TDA or AIF, no previous surgery, and a Core Outcome Measures Index (COMI) completed at baseline and at least 3 months' follow-up. Overall, 987 patients were identified.

OUTCOME MEASURES

Neck and arm pain relief and COMI score improvement were the outcome measures.

METHODS

Three separate analyses were performed to compare TDA and AIF surgical outcomes: (1) mimicking an RCT setting, with admission criteria typical of those in published RCTs, a 1:1 matched analysis was carried out in 739 patients; (2) an analysis was performed on 248 patients outside the classic RCT spectrum, that is, with one or more typical RCT exclusion criteria; (3) a subgroup analysis of all patients with additional follow-up longer than 2 years (n=149).

RESULTS

Matching resulted in 190 pairs with an average follow-up of 17 months that had no residual significant differences for any patient characteristics. Small but statistically significant differences in outcome were observed in favor of TDA, which are potentially clinically relevant. Subgroup analyses of atypical patients and of patients with longer-term follow-up showed no significant differences in outcome between the treatments.

CONCLUSIONS

The results of this observational study were in accordance with those of the published RCTs, suggesting substantial pain reduction both after AIF and TDA, with slightly greater benefit after arthroplasty. The analysis of atypical patients suggested that, in patients outside the spectrum of clinical trials, both surgical interventions appeared to work to a similar extent to that shown for the cohort in the matched study. Also, in the longer-term perspective, both therapies resulted in similar benefits to the patients.

RANDOMIZED CONTROLLED CLINICAL TRIALS IN ORTHOPEDICS: DIFFICULTIES AND LIMITATIONS

Randomized controlled clinical trials (RCTs) are considered to be the gold standard for evidence-based medicine nowadays, and are important for directing medical practice through consistent scientific observations. Steps such as patient selection, randomization and blinding are fundamental for conducting a RCT, but some additional difficulties are presented in trials that involve surgical procedures, as is common in orthopedics. The aim of this article was to highlight and discuss some difficulties and possible limitations on RCTs within the field of surgery.

The relationship between external and internal validity of randomized controlled trials: A sample of hypertension trials from China

Objective

To explore the relationship between the external validity and the internal validity of hypertension RCTs conducted in China.

Methods

Comprehensive literature searches were performed in Medline, Embase, Cochrane Central Register of Controlled Trials (CCTR), CBMdisc (Chinese biomedical literature database), CNKI (China National Knowledge Infrastructure/China Academic Journals Full-text Database) and VIP (Chinese scientific journals database) as well as advanced search strategies were used to locate hypertension RCTs. The risk of bias in RCTs was assessed by a modified scale, Jadad scale respectively, and then studies with 3 or more grading scores were included for the purpose of evaluating of external validity. A data extract form including 4 domains and 25 items was used to explore relationship of the external validity and the internal validity. Statistic analyses were performed by using SPSS software, version 21.0 (SPSS, Chicago, IL).

Results

226 hypertension RCTs were included for final analysis. RCTs conducted in university affiliated hospitals (P < 0.001) or secondary/tertiary hospitals (P < 0.001) were scored at higher internal validity. Multi-center studies (median = 4.0, IQR = 2.0) were scored higher internal validity score than single-center studies (median = 3.0, IQR = 1.0) (P < 0.001). Funding-supported trials had better methodological quality (P < 0.001). In addition, the reporting of inclusion criteria also leads to better internal validity (P = 0.004). Multivariate regression indicated sample size, industry-funding, quality of life (QOL) taken as measure and the university affiliated hospital as trial setting had statistical significance (P < 0.001, P < 0.001, P = 0.001, P = 0.006 respectively).

Conclusion

Several components relate to the external validity of RCTs do associate with the internal validity, that do not stand in an easy relationship to each other. Regarding the poor reporting, other possible links between two variables need to trace in the future methodological researches.

CONSORT compliance in surgical randomized trials: are we there yet? A systematic review

OBJECTIVE

We performed a systematic review assessing the reporting quality of trials of surgical interventions, and explored associated trial level variables.

BACKGROUND

Randomized controlled trials (RCTs) provide clinicians with the best evidence for the effects of interventions, but may not be reported with necessary detail.

METHODS

In May 2009, 3 databases (MEDLINE, EMBASE, and CENTRAL) were searched for RCTs that assessed a surgical intervention using a comprehensive electronic strategy developed by the Cochrane Collaboration. The Consolidated Standards of Reporting Trials (CONSORT) checklist was used as a measure of reporting quality. An overall CONSORT score was calculated and expressed as a proportion. This was supplemented with domains related to external validity. We also collected data on characteristics hypothesized to improve reporting quality, and exploratory regression was performed to determine associations.

RESULTS

One hundred fifty recently published RCTs were included. The most commonly represented surgical subspecialties were general (29%), orthopedic (23%), and cardiothoracic (13%). Most (65%) were published in subspecialty surgical journals. Overall reporting quality was low, with only 55% of CONSORT items addressed. Less than half of trials described adequate methods for sample size calculation (45%), random sequence generation (43%), allocation concealment (45%), and blinding (37%). The strongest associations with reporting quality were adequate methods related to methodological domains, an author with an epidemiology/statistics degree, and a longer report length.

CONCLUSIONS

There remains much room for improvement for the reporting of surgical intervention trials. Authors and journal editors should apply existing reporting guidelines, and guidelines specific to the reporting of surgical interventions should be developed.

Improving the external validity of clinical trials: the case of multiple chronic conditions

The U.S. Department of Health and Human Services vision and strategic framework on multiple chronic conditions (MCCs) incorporates recommendations designed to facilitate research that will improve our knowledge about interventions and systems that will benefit individuals with MCCs (or multimorbidity). The evidence base supporting the management of patients with MCCs will be built through intervention trials specifically designed to address multimorbidity and identification of MCCs in participants across the clinical trial range. This article specifically focuses on issues relating to external validity with specific reference to trials involving patients with MCCs. The exclusion of such patients from clinical trials has been well documented. Randomized control trials (RCTs) are considered the “gold standard” of evidence, but may have drawbacks in relation to external validity, particularly in relation to multimorbidity. It may, therefore, be necessary to consider a broader range of research methods that can provide converging evidence on intervention effects to address MCCs. Approaches can also be taken to increase the usefulness of RCTs in general for providing evidence to inform multimorbidity management. Additional improvements to RCTs would include better reporting of inclusion and exclusion criteria and participant characteristics in relation to MCCs. New trials should be considered in terms of how they will add to the existing evidence base and should inform how interventions may work in different settings and patient groups. Research on treatments and interventions for patients with MCCs is badly needed. It is important that this research includes patient-centered measures and that generalizability issues be explicitly addressed.

Journal of Comorbidity 2013;3:30–35

The external validity of randomized controlled trials of hypertension within China: from the perspective of sample representation

OBJECTIVE

To explore external validity of randomized controlled trials (RCTs) of hypertension within China from the view of sample representation.

METHODS

Comprehensive literature searches were performed in Medline, Embase, Cochrane Central Register of Controlled Trials (CCTR) et al and advanced search strategies were used to locate hypertension RCTs as well as observational studies conducted in China during 1996 to 2009 synchronously. The risk of bias in RCTs and observational studies was assessed by two modified scales respectively, and then both types of studies with 3 or more grading scores were included for the purpose of evaluating of external validity. Following that the study characteristics relative to sample representation were extracted from RCTs and observational studies synchronously, and the later were taken as external references for validating sample representation of RCTs.

RESULTS

226 hypertension RCTs and 21 observational studies were included for final analysis. Comparing samples with observational studies, the mean age of samples within RCTs was 54.46 years, significantly lower than that of observational studies (66.35 years) (P=0.002). The average disease course in patients of RCTs was 3.89 years and grade III hypertensive patients accounted for 17%; both were lower than that of the observational studies (12.96 years, P<0.001; 34%, P=0.026 respectively). In addition, the proportions of patients with complications due to heart failure, stroke, diabetes, or coronary heart disease in RCTs were 8%, 5%, 12% and 11% correspondingly, all of which were significantly less than that of observational studies (11%, 18%, 17% and 29%).

CONCLUSION

Sample characteristics within hypertension RCTs were significantly different from those in observational studies. The samples in most RCTs were under-represented. It's feasible to take samples of observational studies as a mirror of the actual composition of hypertension patients in the real world, if the reporting of observational studies is abundant and available.

Comparative effectiveness research in spine surgery

Degenerative spine disorders are a significant cause of patient morbidity and are a prominent factor in healthcare costs in many countries. Pressure for healthcare cost control and the desire for improved outcomes have led to an expanding emphasis on evidence-based medicine methodologies in spine research. Determination of the optimal treatment paradigm for many common degenerative spinal disorders has proven difficult and comparative effectiveness research is increasingly being employed to examine these clinical dilemmas. The Swedish Spine Registry and the Registry of the Scoliosis Research Society are two of the long-standing databases compiling data for spine patients. Spine surgery professional organizations have recently taken a prominent role in assembling procedural- and diagnosis-based registries, specifically addressing therapeutic outcomes for spine patients. As healthcare systems continue to evolve, comparative effectiveness research driven by spine registries may better elucidate the appropriate clinical choices for patients with these challenging illnesses.

ASSIST applicability scoring of surgical trials. an investigator-reported assessment tool

Context

We aimed to develop a new tool for assessing and depicting the applicability of the results of surgical randomized controlled trials (RCTs) from the trial investigators' perspective.

Methods

We identified all items related to applicability by a systematic methodological review, and then a sample of surgeons used these items in a web-based survey to evaluate the applicability of their own trial results. For each applicability item, participants had to indicate on a numerical scale that was simplified as a three-item scale: 1) items essential to consider, 2) items requiring attention, and 3) items inconsequential to the applicability of the results of their own RCT to clinical practice. For the final tool, we selected only items that were rated as being essential or requiring attention for at least 25% of the trials evaluated. We propose a specific process to construct the tool and to depict applicability in a graph. We identified all investigators of published and registered ongoing RCTs assessing surgery and invited them to participate in the web-based survey.

Results

148 surgeons assessed applicability for their own trial and participated in the process of item selection. The final tool contains 22 items (4 dedicated to patients, 5 to centers, 5 to surgeons and 8 to the intervention). We proposed a straightforward process of constructing the graphical tool: 1) a multidisciplinary team of investigators or other care providers participating in the trial could independently assess each item, 2) a consensus method could be used, and 3) the investigators could depict their assessment of the applicability of the trial results in 4 graphs related to patients, centers, surgeons and the intervention.

Conclusions

This investigator-reported assessment tool could help readers define under what conditions they could reasonably apply the results of a surgical RCT to their clinical practice.

External validity in randomised controlled trials of acupuncture for osteoarthritis knee pain

OBJECTIVES

To assess two aspects of the external validity of acupuncture research for osteoarthritis knee pain and determine the common acupoints and treatment parameters used.

METHODS

The external validity of 16 randomised controlled trials (RCTs) was investigated using a scale consisting of two aspects: reporting and performance. The reporting aspect included acupuncturist's background, study location, treatment detailed, patient characteristics, positive trial results, adverse effects and between-group statistical differences, whereas treatment appropriateness, appropriate controls and outcomes were classified as the performance aspect. Acupuncture treatment in RCTs was compared with common practice according to the literature sources and survey of acupuncturists working in different parts of Thailand.

RESULTS

The levels of external validity for the reporting and performance aspects were in the range of 31.3% to 100%. Statistic values such as mean difference and confidence interval were reported by the minority of trials (43.8%). Patient satisfaction and quality of life were seldom used (31.3%). There were minor differences between research and practice in terms of the points used (25.0%), number of treatment sessions (6.3%) and frequency (12.5%). The most frequently used points were ST34, ST35, ST36, SP6, SP9, SP10, GB34, Xiyan and ah shi points, and the commonly used treatment parameters were 20 minutes, 10-15 sessions and two treatments weekly.

CONCLUSIONS

Reporting of the external validity of acupuncture RCTs for knee pain was notably inadequate in terms of trial setting, treatment provider and statistical reporting. The majority of studies involved appropriate controls and outcomes and applied acupuncture treatments in line with practice.

Reporting of eligibility criteria of randomised trials: cohort study comparing trial protocols with subsequent articles

OBJECTIVE

To determine whether and how eligibility criteria of participants prespecified in protocols of randomised trials are reported in subsequent articles.

DESIGN

Cohort study.

SETTING

Protocols submitted to the ethics committee of a German medical faculty.

DATA SOURCES

52 trial protocols and 78 subsequent publications published between 2000 and 2006.

MAIN OUTCOME MEASURE

Proportion of matching, missing, modified, or newly added eligibility criteria between trial protocols and subsequent publications.

RESULTS

Differences were found between protocols and subsequent publications for all 52 trials. Information on eligibility criteria was missing in the publications for all 52 trials (100%, 95% confidence interval 93% to 100%), modified for 44 (85%, 72% to 93%), and newly added for 21 (41%, 27% to 55%). The mean number of eligibility criteria for each trial was 25 (range 7-43) and the mean proportion of matching eligibility criteria per trial was 50% (95% confidence interval 44% to 55%, range 13-93). Of 1248 eligibility criteria prespecified in the protocols, 606 (49%, 46% to 51%) were matching in subsequent publications, 479 (38%, 36% to 41%) were missing, and 163 (13%, 11% to 15%) were modified. 51 eligibility criteria were added to publications. Most prespecified eligibility criteria were about comorbidity (42%, 39% to 45%), treatment (20%, 18% to 22%), or type or severity of illness (17%, 15% to 19%). Most of the missing eligibility criteria (96%, 94% to 97%) and modified eligibility criteria (54%, 46% to 62%) suggested broader study populations and most of the added eligibility criteria (86%, 74% to 94%) suggested narrower study populations.

CONCLUSIONS

Many users of trial information rely on published journal articles. These articles generally do not reflect the exact definition of the study population as prespecified in the protocol. Incomplete or inadequate reporting of eligibility criteria hampers a proper assessment of the applicability of trial results.

Development of a documentation instrument for the conservative treatment of spinal disorders in the International Spine Registry, Spine Tango

Spine Tango is the first and only International Spine Registry in operation to date. So far, only surgical spinal interventions have been recorded and no comparable structured and comprehensive documentation instrument for conservative treatments of spinal disorders is available. This study reports on the development of a documentation instrument for the conservative treatment of spinal disorders by using the Delphi consensus method. It was conducted with a group of international experts in the field. We also assessed the usability of this new assessment tool with a prospective feasibility study on 97 outpatients and inpatients with low back or neck pain undergoing conservative treatment. The new 'Spine Tango conservative' questionnaire proved useful and suitable for the documentation of pathologies, conservative treatments and outcomes of patients with low back or neck problems. A follow-up questionnaire seemed less important in the predominantly outpatient setting. In the feasibility study, between 43 and 63% of patients reached the minimal clinically important difference in pain relief and Core Outcome Measures Index at 3 months after therapy; 87% of patients with back pain and 85% with neck pain were satisfied with the received treatment. With 'Spine Tango conservative' a first step has been taken to develop and implement a complementary system for documentation and evaluation of non-surgical spinal interventions and outcomes within the framework of the International Spine Registry. It proved useful and feasible in a first pilot study, but it will take the experience of many more cases and therapists to develop a version similarly mature as the surgical instruments of Spine Tango.

Applicability and generalisability of the results of systematic reviews to public health practice and policy: a systematic review

BACKGROUND

The purpose of the study was to evaluate systematic reviews of research into two public health priorities, tobacco consumption and HIV infection, in terms of the reporting of data related to the applicability of trial results (i.e., whether the results of a trial can be reasonably applied or generalized to a definable group of patients in a particular setting in routine practice, also called external validity or generalisability).

METHODS

All systematic reviews of interventions aimed at reducing or stopping tobacco use and treating or preventing HIV infection published in the Cochrane database of systematic reviews and in journals indexed in MEDLINE between January 1997 and December 2007 were selected. We used a standardized data abstraction form to extract data related to applicability in terms of the context of the trial, (country, centres, settings), participants (recruitment, inclusion and exclusion criteria, baseline characteristics of participants such as age, sex, ethnicity, coexisting diseases or co-morbidities, and socioeconomic status), treatment (duration, intensity/dose of treatment, timing and delivery format), and the outcomes assessment from selected reviews.

RESULTS

A total of 98 systematic reviews were selected (57 Cochrane reviews and 41 non-Cochrane reviews); 49 evaluated interventions aimed at reducing or stopping tobacco use and 49 treating or preventing HIV infection. The setting of the individual studies was reported in 45 (46%) of the systematic reviews, the number of centres in 21 (21%), and the country where the trial took place in 62 (63%). Inclusion and exclusion criteria of the included studies were reported in 16 (16%) and 13 (13%) of the reviews, respectively. Baseline characteristics of participants in the included studies were described in 59 (60%) of the reviews. These characteristics concerned age in about half of the reviews, sex in 46 (47%), and ethnicity in 9 (9%).Applicability of results was discussed in 13 (13%) of the systematic reviews. The reporting was better in systematic reviews by the Cochrane Collaboration than by non-Cochrane groups.

CONCLUSIONS

Our study highlighted the lack of consideration of applicability of results in systematic reviews of research into 2 public health priorities: tobacco consumption and HIV infection.

Applicability and generalisability of published results of randomised controlled trials and non-randomised studies evaluating four orthopaedic procedures: methodological systematic review

OBJECTIVE

To compare the reporting of essential applicability data from randomised controlled trials and non-randomised studies evaluating four new orthopaedic surgical procedures.

DATA SOURCES

Medline and the Cochrane central register of controlled trials.

STUDY SELECTION

All articles of comparative studies assessing total hip or knee arthroplasty carried out by a minimally invasive approach or computer assisted navigation system.

DATA EXTRACTION

Items judged to be essential for interpreting the applicability of findings about such procedures were identified by a survey of a sample of orthopaedic surgeons (77 of 512 completed the survey). Reports were evaluated for data describing these "essential" items and the number of centres and surgeons involved in the trials. When data on the number of centres and surgeons were not reported, the corresponding author of the selected trials was contacted. Results 84 articles were identified (38 randomised controlled trials, 46 non-randomised studies). The median percentage (interquartile range) of essential items reported for non-randomised studies compared with randomised controlled trials was 38% (25-63%) versus 44% (38-45%) for items about patients, 71% (43-86%) versus 71% (57-86%) for items considered essential for all interventions, and 38% (25-50%) versus 50% (25-50%) for items about the context of care. More than 80% of both study types were single centre studies, with one or two participating surgeons.

CONCLUSION

The reporting of data related to the applicability of results was poor in published articles of both non-randomised studies and randomised controlled trials and did not differ by study design. The applicability of results from the trials and studies was similar in terms of number of centres and surgeons involved and the reproducibility of the intervention.