Subgroup analysis and covariate adjustment in randomized clinical trials of traumatic brain injury: a systematic review

Refining outcome prediction after traumatic brain injury with machine learning algorithms

Outcome after traumatic brain injury (TBI) is typically assessed using the Glasgow outcome scale extended (GOSE) with levels from 1 (death) to 8 (upper good recovery). Outcome prediction has classically been dichotomized into either dead/alive or favorable/unfavorable outcome. Binary outcome prediction models limit the possibility of detecting subtle yet significant improvements. We set out to explore different machine learning methods with the purpose of mapping their predictions to the full 8 grade scale GOSE following TBI. The models were set up using the variables: age, GCS-motor score, pupillary reaction, and Marshall CT score. For model setup and internal validation, a total of 866 patients could be included. For external validation, a cohort of 369 patients were included from Leuven, Belgium, and a cohort of 573 patients from the US multi-center ProTECT III study. Our findings indicate that proportional odds logistic regression (POLR), random forest regression, and a neural network model achieved accuracy values of 0.3-0.35 when applied to internal data, compared to the random baseline which is 0.125 for eight categories. The models demonstrated satisfactory performance during external validation in the data from Leuven, however, their performance were not satisfactory when applied to the ProTECT III dataset.

Overcoming combination fatigue: Addressing high-dimensional effect measure modification and interaction in clinical, biomedical, and epidemiologic research using multilevel analysis of individual heterogeneity and discriminatory accuracy (MAIHDA)

Growing interest in precision medicine, gene-environment interactions, health equity, expanding diversity in research, and the generalizability results, requires researchers to evaluate how the effects of treatments or exposures differ across numerous subgroups. Evaluating combination complexity, in the form of effect measure modification and interaction, is therefore a common study aim in the biomedical, clinical, and epidemiologic sciences. There is also substantial interest in expanding the combinations of factors analyzed to include complex treatment protocols (e.g., multiple study arms or factorial randomization), comorbid medical conditions or risk factors, and sociodemographic and other subgroup identifiers. However, expanding the number of subgroup category combinations creates combination fatigue problems, including concerns over small sample size, reduced power, multiple testing, spurious results, and design and analytic complexity. Creative new approaches for managing combination fatigue and evaluating high-dimensional effect measure modification and interaction are needed. Intersectional MAIHDA (multilevel analysis of individual heterogeneity and discriminatory accuracy) has already attracted substantial interest in social epidemiology, and has been hailed as the new gold standard for investigating health inequities across complex intersections of social identity. Leveraging the inherent advantages of multilevel models, a more general multicategorical MAIHDA can be used to study statistical interactions and predict effects across high-dimensional combinations of conditions, with important advantages over alternative approaches. Though it has primarily been used thus far as an analytic approach, MAIHDA should also be used as a framework for study design. In this article, I introduce MAIHDA to the broader health sciences research community, discuss its advantages over conventional approaches, and provide an overview of potential applications in clinical, biomedical, and epidemiologic research.

Estimating the prevalence of discrepancies between study registrations and publications: a systematic review and meta-analyses

OBJECTIVES

Prospectively registering study plans in a permanent time-stamped and publicly accessible document is becoming more common across disciplines and aims to reduce risk of bias and make risk of bias transparent. Selective reporting persists, however, when researchers deviate from their registered plans without disclosure. This systematic review aimed to estimate the prevalence of undisclosed discrepancies between prospectively registered study plans and their associated publication. We further aimed to identify the research disciplines where these discrepancies have been observed, whether interventions to reduce discrepancies have been conducted, and gaps in the literature.

DESIGN

Systematic review and meta-analyses.

DATA SOURCES

Scopus and Web of Knowledge, published up to 15 December 2019.

ELIGIBILITY CRITERIA

Articles that included quantitative data about discrepancies between registrations or study protocols and their associated publications.

DATA EXTRACTION AND SYNTHESIS

Each included article was independently coded by two reviewers using a coding form designed for this review (osf.io/728ys). We used random-effects meta-analyses to synthesise the results.

RESULTS

We reviewed k=89 articles, which included k=70 that reported on primary outcome discrepancies from n=6314 studies and, k=22 that reported on secondary outcome discrepancies from n=1436 studies. Meta-analyses indicated that between 29% and 37% (95% CI) of studies contained at least one primary outcome discrepancy and between 50% and 75% (95% CI) contained at least one secondary outcome discrepancy. Almost all articles assessed clinical literature, and there was considerable heterogeneity. We identified only one article that attempted to correct discrepancies.

CONCLUSIONS

Many articles did not include information on whether discrepancies were disclosed, which version of a registration they compared publications to and whether the registration was prospective. Thus, our estimates represent discrepancies broadly, rather than our target of undisclosed discrepancies between prospectively registered study plans and their associated publications. Discrepancies are common and reduce the trustworthiness of medical research. Interventions to reduce discrepancies could prove valuable.

REGISTRATION

osf.io/ktmdg. Protocol amendments are listed in online supplemental material A.

Racial and ethnic subgroup reporting in diabetes randomized controlled trials published from 2000 to 2020: A survey

BACKGROUND

It remained unknown about the status of and trends in racial/ethnic subgroup reporting in the diabetes trials over the past two decades.

OBJECTIVES

In this survey, we aimed to evaluate the current state of and temporal trends in subgroup reporting by race/ethnicity regarding the effects of interventions in diabetes randomized controlled trials (RCTs) from year 2000-2020 and to explore the potential trial factors in relation to racial/ethnic subgroup reporting.

METHODS

We searched electronic databases for eligible diabetes RCTs. The outcome was whether the trials had the event of racial/ethnic subgroup reporting regarding the intervention effects on trial primary outcomes. Poisson regression was used to assess the temporal trends in racial/ethnic subgroup reporting, and univariable logistic regression models were employed for evaluating trial factors related to racial/ethnic subgroup reporting.

RESULTS

A total of 405 diabetes RCTs were eligible for inclusion. There were 26 (6.42%) trials with racial/ethnic subgroup reporting. A chronological trend towards increased rates of racial/ethnic subgroup reporting was observed; however, the trend was not statistically significant (p = 0.07). Advanced patients' age (Odds ratio [OR] = 2.92, 95% confidence interval [CI]: 1.24-6.88), follow-up duration (OR = 3.53, 95% CI: 1.13-11.00), and BIPOC (Black, Indigenous, and People of Colour) enrolment (OR = 2.39, 95% CI: 1.01-5.62) were found to positively relate with racial/ethnic subgroup reporting, while the industrial funding was associated with decreased reporting (OR = 0.43, 95% CI: 0.19-0.97). Less than one fourth of the trials with racial/ethnic subgroup reporting predefined the subgroup analysis.

CONCLUSIONS

The majority of diabetes RCTs did not report intervention effects by racial/ethnic subgroup, which was not temporally improved over the past two decades. More efforts and strategies are needed to improve the racial/ethnic subgroup consideration and reporting in diabetes trials.

Silver dressing in the treatment of diabetic foot: A protocol for systematic review and meta-analysis

BACKGROUND

Diabetic foot (DF) is one of the most common and serious chronic complications of diabetes. At present, there are many dressings used in the treatment of the diabetic foot. Among them, silver dressings are widely used, but the conclusion has not yet been formed. The purpose of this study is to search for relevant studies on the treatment of DF with silver dressings through evidence-based medicine methods and to draw conclusions with higher levels of evidence to provide a basis for the clinical treatment of DF.

METHODS

Computer search of databases such as CNKI, SinoMed, VIP, Wanfang, PubMed, Embase, and Cochrane Library. The search time is from the establishment of the database to January 23, 2021. Two researchers will independently select studies, collect data, and assess the methodology quality by the Cochrane risk of bias tool. The meta-analysis will be completed by RevMan 5.3 software.

RESULTS

This systematic review will provide an assessment of the current state of DF, aiming to assess the efficacy of silver dressings for patients with DF.

CONCLUSION

This systematic review will provide a credible evidence-based for the clinical treatment of DF with silver dressings.

Complete intraureteral stent placement relieves daytime urinary frequency compared with conventional placement in patients with an indwelling ureteral stent: post-hoc analysis of a randomized, controlled trial

A previous randomized, controlled trial had demonstrated that complete intraureteral stent placement (CIU-SP) was superior to conventional stent placement (C-SP) in terms of improvement of stent-related urinary symptoms. However, it is unclear as to which subdomain symptom and cohort could benefit the most from CIU-SP compared to C-SP in urinary symptoms while considering the baseline urinary status. To determine this, a post-hoc analysis was performed using data from a previous study (CIU-SP group, n = 39; C-SP group, n = 41). We assessed the mean changes in the International Prostate Symptom Score (I-PSS) and the Overactive Bladder Symptom Score (OABSS) from baseline to day 14. Statistical comparison between the two groups was performed using analysis of covariance with adjustment of baseline urinary status as a covariate. Among 80 patients, the total I-PSS was significantly lower in the CIU-SP group than in the C-SP group in the cohort with mild urinary symptoms (P = 0.005), but not in those with moderate/severe symptoms (P = 0.521). The CIU-SP group showed significantly improved I-PSS and OABSS daytime frequencies, with the highest t statistic (2.47 and 2.10, respectively) among subdomains of both symptom scores compared with the C-SP group (both P < 0.001). In multivariate regression analysis, the stent placement method (CIU-SP vs. C-SP) was independently associated with the I-PSS daytime frequency on day 14 (P = 0.017). This study suggests that CIU-SP significantly improved stent-related daytime frequency compared with C-SP, and it may benefit especially those patients who have mild urinary symptoms before the placement of ureteral stents.

Has the reporting quality of published randomised controlled trial protocols improved since the SPIRIT statement? A methodological study

OBJECTIVES

To determine the reporting quality of published randomised controlled trial (RCT) protocols before and after the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) statement (2013), and any association with author, trial or journal factors.

DESIGN

Methodological study.

DATA SOURCES

MEDLINE, Embase and CENTRAL were electronically searched using optimised search strategies.

ELIGIBILITY CRITERIA

Protocols written for an RCT of living humans, published in full text in a peer-reviewed journal and published in the English language.

MAIN OUTCOME

Primary outcome was the overall proportion of checklist items which were adequately reported in RCT protocols published before and after the SPIRIT statement.

RESULTS

300 RCT protocols were retrieved; 150 from the period immediately before the SPIRIT statement (9 July 2012 to 28 December 2012) and 150 from a recent period after the SPIRIT statement (25 January 2019 to 20 March 2019). 47.9% (95% CI, 46.5% to 49.3%) of checklist items were adequately reported in RCT protocols before the SPIRIT statement and 56.7% (95% CI, 54.9% to 58.5%) after the SPIRIT statement. This represents an 8.8% (95% CI, 6.6% to 11.1%; p<0.0001) mean improvement in the overall proportion of checklist items adequately reported since the SPIRIT statement. While 40% of individual checklist items had a significant improvement in adequate reporting after the SPIRIT statement, 11.3% had a significant deterioration and there were no RCT protocols in which all individual checklist items were complete. The factors associated with higher reporting quality of RCT protocols in multiple regression analysis were author expertise or experience in epidemiology or statistics, multicentre trials, longer protocol word length and publicly reported journal policy of compliance with the SPIRIT statement.

CONCLUSION

The overall reporting quality of RCT protocols has significantly improved since the SPIRIT statement, although a substantial proportion of individual checklist items remain poorly reported. Continued and concerted efforts are required by journals, editors, reviewers and investigators to improve the completeness and transparency of RCT protocols.

A systematic review of comparisons between protocols or registrations and full reports in primary biomedical research

BACKGROUND

Prospective study protocols and registrations can play a significant role in reducing incomplete or selective reporting of primary biomedical research, because they are pre-specified blueprints which are available for the evaluation of, and comparison with, full reports. However, inconsistencies between protocols or registrations and full reports have been frequently documented. In this systematic review, which forms part of our series on the state of reporting of primary biomedical, we aimed to survey the existing evidence of inconsistencies between protocols or registrations (i.e., what was planned to be done and/or what was actually done) and full reports (i.e., what was reported in the literature); this was based on findings from systematic reviews and surveys in the literature.

METHODS

Electronic databases, including CINAHL, MEDLINE, Web of Science, and EMBASE, were searched to identify eligible surveys and systematic reviews. Our primary outcome was the level of inconsistency (expressed as a percentage, with higher percentages indicating greater inconsistency) between protocols or registration and full reports. We summarized the findings from the included systematic reviews and surveys qualitatively.

RESULTS

There were 37 studies (33 surveys and 4 systematic reviews) included in our analyses. Most studies (n = 36) compared protocols or registrations with full reports in clinical trials, while a single survey focused on primary studies of clinical trials and observational research. High inconsistency levels were found in outcome reporting (ranging from 14% to 100%), subgroup reporting (from 12% to 100%), statistical analyses (from 9% to 47%), and other measure comparisons. Some factors, such as outcomes with significant results, sponsorship, type of outcome and disease speciality were reported to be significantly related to inconsistent reporting.

CONCLUSIONS

We found that inconsistent reporting between protocols or registrations and full reports of primary biomedical research is frequent, prevalent and suboptimal. We also identified methodological issues such as the need for consensus on measuring inconsistency across sources for trial reports, and more studies evaluating transparency and reproducibility in reporting all aspects of study design and analysis. A joint effort involving authors, journals, sponsors, regulators and research ethics committees is required to solve this problem.

Effects of yoga, strength training and advice on back pain: a randomized controlled trial

Background

Among the working population, non-specific low-back pain and neck pain are one of the most common reasons for sickness absenteeism. The aim was to evaluate the effects of an early intervention of yoga - compared with strength training or evidence-based advice - on sickness absenteeism, sickness presenteeism, back and neck pain and disability among a working population.

Methods

A randomized controlled trial was conducted on 159 participants with predominantly (90%) chronic back and neck pain. After screening, the participants were randomized to kundalini yoga, strength training or evidence-based advice. Primary outcome was sickness absenteeism. Secondary outcomes were sickness presenteeism, back and neck pain and disability. Self-reported questionnaires and SMS text messages were completed at baseline, 6 weeks, 6 and 12 months.

Results

The results did not indicate that kundalini yoga and strength training had any statistically significant effects on the primary outcome compared with evidence-based advice. An interaction effect was found between adherence to recommendations and sickness absenteeism, indicating larger significant effects among the adherers to kundalini yoga versus evidence-based advice: RR = 0.47 (CI 0.30; 0.74, p = 0.001), strength training versus evidence-based advice: RR = 0.60 (CI 0.38; 0.96, p = 0.032). Some significant differences were also found for the secondary outcomes to the advantage of kundalini yoga and strength training.

Conclusions

Guided exercise in the forms of kundalini yoga or strength training does not reduce sickness absenteeism more than evidence-based advice alone. However, secondary analyses reveal that among those who pursue kundalini yoga or strength training at least two times a week, a significantly reduction in sickness absenteeism was found. Methods to increase adherence to treatment recommendations should be further developed and applied in exercise interventions.

Trial Registration

Clinicaltrials.gov NCT01653782, date of registration: June, 28, 2012, retrospectively registered.

Improving precision by adjusting for prognostic baseline variables in randomized trials with binary outcomes, without regression model assumptions

In randomized clinical trials with baseline variables that are prognostic for the primary outcome, there is potential to improve precision and reduce sample size by appropriately adjusting for these variables. A major challenge is that there are multiple statistical methods to adjust for baseline variables, but little guidance on which is best to use in a given context. The choice of method can have important consequences. For example, one commonly used method leads to uninterpretable estimates if there is any treatment effect heterogeneity, which would jeopardize the validity of trial conclusions. We give practical guidance on how to avoid this problem, while retaining the advantages of covariate adjustment. This can be achieved by using simple (but less well-known) standardization methods from the recent statistics literature. We discuss these methods and give software in R and Stata implementing them. A data example from a recent stroke trial is used to illustrate these methods.

Developing a Cognition Endpoint for Traumatic Brain Injury Clinical Trials

Cognitive impairment is a core clinical feature of traumatic brain injury (TBI). After TBI, cognition is a key determinant of post-injury productivity, outcome, and quality of life. As a final common pathway of diverse molecular and microstructural TBI mechanisms, cognition is an ideal endpoint in clinical trials involving many candidate drugs and nonpharmacological interventions. Cognition can be reliably measured with performance-based neuropsychological tests that have greater granularity than crude rating scales, such as the Glasgow Outcome Scale-Extended, which remain the standard for clinical trials. Remarkably, however, there is no well-defined, widely accepted, and validated cognition endpoint for TBI clinical trials. A single cognition endpoint that has excellent measurement precision across a wide functional range and is sensitive to the detection of small improvements (and declines) in cognitive functioning would enhance the power and precision of TBI clinical trials and accelerate drug development research. We outline methodologies for deriving a cognition composite score and a research program for validation. Finally, we discuss regulatory issues and the limitations of a cognition endpoint.

Predicting outcome in traumatic brain injury: Sharing experience of pilot traumatic brain injury registry

Background:

A reliable prediction of outcome for the victims of traumatic brain injury (TBI) on admission is possible from concurrent data analysis from any systematic real-time registry.

Objective:

To determine the clinical relevance of the findings from our TBI registry to develop prognostic futuristic models with readily available traditional and novel predictors.

Materials and Methods:

Prospectively collected data using predesigned pro forma were analyzed from the first phase of a trauma registry from a South Indian Trauma Centre, compatible with computerized management system at electronic data entry and web data entry interface on demographics, clinical, management, and discharge status.

Statistical Analysis:

On univariate analysis, the variables with P < 0.15 were chosen for binary logistic model. On regression model, variables were selected with test of coefficient 0.001 and with Nagelkerke R² with alpha error of 5%.

Results:

From 337 cases, predominantly males from rural areas in their productive age, road traffic injuries accounted for two-thirds cases, one-fourths occurred during postmonsoon while two-wheeler was the most common prerequisite. Fifty percent of patients had moderate to severe brain injury; the most common finding was unconsciousness followed by vomiting, ear bleed, seizures, and traumatic amnesia. Fifteen percent required intracranial surgery. Patients with severe Glasgow coma scale score were 4.5 times likely to have the fatal outcome (P = 0.003). Other important clinical variables accountable for fatal outcomes were oral bleeds and cervical spine injury while imperative socio-demographic risk correlates were age and seasons.

Conclusion:

TBI registry helped us finding predictors of clinical relevance for the outcomes in victims of TBI in search of prognostic futuristic models in TBI victims.

Clinical trial metadata: defining and extracting metadata on the design, conduct, results and costs of 125 randomised clinical trials funded by the National Institute for Health Research Health Technology Assessment programme

BACKGROUND

By 2011, the Health Technology Assessment (HTA) programme had published the results of over 100 trials with another 220 in progress. The aim of the project was to develop and pilot 'metadata' on clinical trials funded by the HTA programme.

OBJECTIVES

The aim of the project was to develop and pilot questions describing clinical trials funded by the HTA programme in terms of it meeting the needs of the NHS with scientifically robust studies. The objectives were to develop relevant classification systems and definitions for use in answering relevant questions and to assess their utility.

DATA SOURCES

Published monographs and internal HTA documents.

REVIEW METHODS

A database was developed, 'populated' using retrospective data and used to answer questions under six prespecified themes. Questions were screened for feasibility in terms of data availability and/or ease of extraction. Answers were assessed by the authors in terms of completeness, success of the classification system used and resources required. Each question was scored to be retained, amended or dropped.

RESULTS

One hundred and twenty-five randomised trials were included in the database from 109 monographs. Neither the International Standard Randomised Controlled Trial Number nor the term 'randomised trial' in the title proved a reliable way of identifying randomised trials. Only limited data were available on how the trials aimed to meet the needs of the NHS. Most trials were shown to follow their protocols but updates were often necessary as hardly any trials recruited as planned. Details were often lacking on planned statistical analyses, but we did not have access to the relevant statistical plans. Almost all the trials reported on cost-effectiveness, often in terms of both the primary outcome and quality-adjusted life-years. The cost of trials was shown to depend on the number of centres and the duration of the trial. Of the 78 questions explored, 61 were well answered, 33 fully with 28 requiring amendment were the analysis updated. The other 17 could not be answered with readily available data.

LIMITATIONS

The study was limited by being confined to 125 randomised trials by one funder.

CONCLUSIONS

Metadata on randomised controlled trials can be expanded to include aspects of design, performance, results and costs. The HTA programme should continue and extend the work reported here.

FUNDING

The National Institute for Health Research HTA programme.

Assessing Heterogeneity of Treatment Effects: Are Authors Misinterpreting Their Results?

OBJECTIVE

To determine whether investigations of heterogeneity of treatment effects (HTE) in randomized-controlled trials (RCTs) are prespecified and whether authors' interpretations of their analyses are consistent with the objective evidence.

DATA SOURCES/STUDY SETTING

Trials published in Annals of Internal Medicine, British Medical Journal, Journal of the American Medical Association, Lancet, and New England Journal of Medicine in 1994, 1999, and 2004.

STUDY DESIGN

We reviewed 87 RCTs that reported formal tests for statistical interaction or heterogeneity (HTE analyses), derived from a probability sample of 541 articles.

DATA COLLECTION/EXTRACTION

We recorded reasons for performing HTE analysis; an objective classification of evidence for HTE (termed "clinicostatistical divergence" [CSD]); and authors' interpretations of findings. Authors' interpretations, compared with CSD, were coded as understated, overstated, or adequately stated.

PRINCIPLE FINDINGS

Fifty-three RCTs (61 percent) claimed prespecified covariates for HTE analyses. Trials showed strong (6), moderate (11), weak (25), or negligible (16) evidence for CSD (29 could not be classified due to inadequate information). Authors stated that evidence for HTE was sufficient to support differential treatment in subgroups (10); warranted more research (31); was absent (21); or provided no interpretation (25). HTE was overstated in 22 trials, adequately stated in 57 trials, and understated in 8 trials.

CONCLUSIONS

Inconsistencies in performance and reporting may limit the potential of HTE analysis as a tool for identifying HTE and individualizing care in diverse populations. Recommendations for future studies on the reporting and interpretation of HTE analyses are provided.

Evidence for the selective reporting of analyses and discrepancies in clinical trials: a systematic review of cohort studies of clinical trials

BACKGROUND

Most publications about selective reporting in clinical trials have focussed on outcomes. However, selective reporting of analyses for a given outcome may also affect the validity of findings. If analyses are selected on the basis of the results, reporting bias may occur. The aims of this study were to review and summarise the evidence from empirical cohort studies that assessed discrepant or selective reporting of analyses in randomised controlled trials (RCTs).

METHODS AND FINDINGS

A systematic review was conducted and included cohort studies that assessed any aspect of the reporting of analyses of RCTs by comparing different trial documents, e.g., protocol compared to trial report, or different sections within a trial publication. The Cochrane Methodology Register, Medline (Ovid), PsycInfo (Ovid), and PubMed were searched on 5 February 2014. Two authors independently selected studies, performed data extraction, and assessed the methodological quality of the eligible studies. Twenty-two studies (containing 3,140 RCTs) published between 2000 and 2013 were included. Twenty-two studies reported on discrepancies between information given in different sources. Discrepancies were found in statistical analyses (eight studies), composite outcomes (one study), the handling of missing data (three studies), unadjusted versus adjusted analyses (three studies), handling of continuous data (three studies), and subgroup analyses (12 studies). Discrepancy rates varied, ranging from 7% (3/42) to 88% (7/8) in statistical analyses, 46% (36/79) to 82% (23/28) in adjusted versus unadjusted analyses, and 61% (11/18) to 100% (25/25) in subgroup analyses. This review is limited in that none of the included studies investigated the evidence for bias resulting from selective reporting of analyses. It was not possible to combine studies to provide overall summary estimates, and so the results of studies are discussed narratively.

CONCLUSIONS

Discrepancies in analyses between publications and other study documentation were common, but reasons for these discrepancies were not discussed in the trial reports. To ensure transparency, protocols and statistical analysis plans need to be published, and investigators should adhere to these or explain discrepancies.

The risks and rewards of covariate adjustment in randomized trials: an assessment of 12 outcomes from 8 studies

Background

Adjustment for prognostic covariates can lead to increased power in the analysis of randomized trials. However, adjusted analyses are not often performed in practice.

Methods

We used simulation to examine the impact of covariate adjustment on 12 outcomes from 8 studies across a range of therapeutic areas. We assessed (1) how large an increase in power can be expected in practice; and (2) the impact of adjustment for covariates that are not prognostic.

Results

Adjustment for known prognostic covariates led to large increases in power for most outcomes. When power was set to 80% based on an unadjusted analysis, covariate adjustment led to a median increase in power to 92.6% across the 12 outcomes (range 80.6 to 99.4%). Power was increased to over 85% for 8 of 12 outcomes, and to over 95% for 5 of 12 outcomes. Conversely, the largest decrease in power from adjustment for covariates that were not prognostic was from 80% to 78.5%.

Conclusions

Adjustment for known prognostic covariates can lead to substantial increases in power, and should be routinely incorporated into the analysis of randomized trials. The potential benefits of adjusting for a small number of possibly prognostic covariates in trials with moderate or large sample sizes far outweigh the risks of doing so, and so should also be considered.

SPIRIT 2013 statement: defining standard protocol items for clinical trials

The protocol of a clinical trial serves as the foundation for study planning, conduct, reporting, and appraisal. However, trial protocols and existing protocol guidelines vary greatly in content and quality. This article describes the systematic development and scope of SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) 2013, a guideline for the minimum content of a clinical trial protocol.The 33-item SPIRIT checklist applies to protocols for all clinical trials and focuses on content rather than format. The checklist recommends a full description of what is planned; it does not prescribe how to design or conduct a trial. By providing guidance for key content, the SPIRIT recommendations aim to facilitate the drafting of high-quality protocols. Adherence to SPIRIT would also enhance the transparency and completeness of trial protocols for the benefit of investigators, trial participants, patients, sponsors, funders, research ethics committees or institutional review boards, peer reviewers, journals, trial registries, policymakers, regulators, and other key stakeholders.

SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials

High quality protocols facilitate proper conduct, reporting, and external review of clinical trials. However, the completeness of trial protocols is often inadequate. To help improve the content and quality of protocols, an international group of stakeholders developed the SPIRIT 2013 Statement (Standard Protocol Items: Recommendations for Interventional Trials). The SPIRIT Statement provides guidance in the form of a checklist of recommended items to include in a clinical trial protocol. This SPIRIT 2013 Explanation and Elaboration paper provides important information to promote full understanding of the checklist recommendations. For each checklist item, we provide a rationale and detailed description; a model example from an actual protocol; and relevant references supporting its importance. We strongly recommend that this explanatory paper be used in conjunction with the SPIRIT Statement. A website of resources is also available (www.spirit-statement.org). The SPIRIT 2013 Explanation and Elaboration paper, together with the Statement, should help with the drafting of trial protocols. Complete documentation of key trial elements can facilitate transparency and protocol review for the benefit of all stakeholders.

Guidelines for randomized clinical trial protocol content: a systematic review

BACKGROUND

All randomized clinical trials (RCTs) require a protocol; however, numerous studies have highlighted protocol deficiencies. Reporting guidelines may improve the content of research reports and, if developed using robust methods, may increase the utility of reports to stakeholders. The objective of this study was to systematically identify and review RCT protocol guidelines, to assess their characteristics and methods of development, and to compare recommendations.

METHODS

We conducted a systematic review of indexed literature (MEDLINE, EMBASE and the Cochrane Methodology Register from inception to September 2010; reference lists; related article features; forward citation searching) and a targeted search of supplementary sources, including a survey of major trial funding agencies in six countries. Records were eligible if they described a content guideline in English or French relevant to RCT protocols. Guidelines were excluded if they specified content for protocols for trials of specific procedures or conditions or were intended to assess trial quality. We extracted guideline characteristics and methods. Content was mapped for a subset of guidelines that described development methods or had institutional endorsement.

RESULTS

Forty guidelines published in journals, books and institutional reports were included in the review; seven were specific to RCT protocols. Only eight (20%) described development methods which included informal consensus methods, pilot testing and formal validation; no guideline described all of these methods. No guideline described formal consensus methods or a systematic retrieval of empirical evidence to inform its development. The guidelines included a median of 23 concepts per guideline (interquartile range (IQR) = 14 to 34; range = 7 to 109). Among the subset of guidelines (n = 23) for which content was mapped, approximately 380 concepts were explicitly addressed (median concepts per guideline IQR = 31 (24,80); range = 16 to 150); most concepts were addressed in a minority of guidelines.

CONCLUSIONS

Existing guidelines for RCT protocol content varied substantially in their recommendations. Few reports described the methods of guideline development, limiting comparisons of guideline validity. Given the importance of protocols to diverse stakeholders, we believe a systematically developed, evidence-informed guideline for clinical trial protocols is needed.

Association between social influences and drinking outcomes across three years

OBJECTIVE

Multiple studies have shown social network variables to mediate and predict drinking outcome, but, because of self-selection biases, these studies cannot reliably determine whether the influence is causal or correlational. The goal of this study was to evaluate evidence for a causal role for social network characteristics in determining long-term outcomes using state-of-the-art statistical methods.

METHOD

Outpatient and aftercare clients enrolled in Project MATCH (N = 1,726) were assessed at intake and at 3, 6, 9, 12, and 15 months; the outpatient sample was also followed to 39 months. Generalized linear modeling with propensity stratification tested whether changes in social network ties (i.e., number of pro-abstainers and pro-drinkers) at Month 9 predicted percentage of days abstinent and drinks per drinking day at 15 and 39 months, covarying for Alcoholics Anonymous (AA) attendance at Month 9.

RESULTS

An increase in the number of pro-drinkers predicted worse drinking outcomes, measured by percentage of days abstinent and drinks per drinking day, at Months 15 and 39 (p < .0001). An increase in the number of pro-abstainers predicted more percentage of days abstinent for both time periods (p < .01). The social network variables uniquely predicted 5%-12% of the outcome variance; AA attendance predicted an additional 1%-6%.

CONCLUSIONS

Network composition following treatment is an important and plausibly causal predictor of alcohol outcome across 3 years, adjusting for multiple confounders. The effects are consistent across patients exhibiting a broad range of alcohol-related impairment. Results support the further development of treatments that promote positive social changes and highlight the need for additional research on the determinants of social network changes.

Baseline matters: the importance of covariation for baseline severity in the analysis of clinical trials

BACKGROUND

Clinical trials testing the effectiveness of interventions for addictions, HIV transmission risk, and other behavioral health problems are important to advancing evidence-based treatment. Such trials are expensive and time-consuming to conduct, but the underlying effect sizes tend to be modest, and often findings are disappointing, failing to show evidence of treatment effects.

OBJECTIVES

To demonstrate how appropriate covariation for baseline severity can enhance detection of treatment effects.

METHODS

Explication and case example.

RESULTS

Baseline severity (the score of the outcome measure at baseline, prior to randomization) is often strongly associated with outcome in such studies. Covariation for baseline score may enhance detection of treatment effects, because the variance explained by the baseline score is removed from the error variance in the estimate of the difference in outcome between treatments. Alternatively, the effect of treatment may manifest in the form of a baseline-by-treatment interaction. Common interaction patterns include that treatment may be more effective among patients with higher levels of baseline severity, or treatment may be more effective among patients with low severity at baseline ('relapse prevention' effect). Such effects may be important to developing treatment guidelines and offer clues toward understanding the mechanisms of action of treatments and of the disorders.

CONCLUSIONS AND SCIENTIFIC SIGNIFICANCE

This article illustrates principles of covariation for baseline and the baseline-by-treatment interaction in nontechnical graphical terms, and discusses examples from clinical trials. Implications for the design and analysis of clinical trials are discussed, and it is argued that covariation for baseline severity of the outcome measure and testing of the baseline-by-treatment interaction should be considered for inclusion in the primary outcome analyses of treatment effectiveness trials of substantial size.

A comparison of subgroup analyses in grant applications and publications

In this paper, the authors compare subgroup analyses as outlined in grant applications and their related publications. Grants awarded by the Netherlands Organization for Health Research and Development (ZonMw) from 2001 onward that were finalized before March 1, 2010, were studied. Of the 79 grant proposals, 50 (63%) were intervention studies, 18 (23%) were diagnostic studies, and 6 (8%) were prognostic studies. Subgroups were mentioned in 49 (62%) grant applications and in 53 (67%) publications. In 20 of the 79 projects (25%), the publications were completely in agreement with the grant proposal; that is, subgroups that were prespecified in the grant proposal were reported and no new subgroup analyses were introduced in the publications. Of the 149 prespecified subgroups, 46 (31%) were reported in the final report or scientific publications, and 143 of the 189 (76%) reported subgroups were based on post-hoc findings. For 77% of the subgroup analyses in the publications, there was no mention of whether these were prespecified or post hoc. Justification for subgroup analysis and methods to study subgroups were rarely reported. The authors conclude that there is a large discrepancy between grant applications and final publications regarding subgroup analyses. Both nonreporting prespecified subgroup analyses and reporting post-hoc subgroup analyses are common. More guidance is clearly needed.

The problem of subgroup analyses: an example from a trial on ruptured intracranial aneurysms

The randomized ISAT demonstrated the superiority of endovascular treatment in patients with ruptured intracranial aneurysms considered suitable for either clipping or coiling. A later publication proposed a second look at the results, demonstrating that older patients with ruptured MCA aneurysms appeared to benefit from clipping, in disagreement with the general findings of the trial. Subgroup analyses in randomized trials and observational studies examine whether effects of interventions differ between subgroups according to the characteristics of patients. However, many apparent subgroup effects have been shown to be spurious. Misleading subgroup effects can result in withholding efficacious treatment from patients who would benefit or can encourage ineffective or potentially harmful treatments for patients who would fare better without. Some guidelines for the prudent interpretation of subgroup findings are reviewed.

Prediction of two month modified Rankin Scale with an ordinal prediction model in patients with aneurysmal subarachnoid haemorrhage

Background

Aneurysmal subarachnoid haemorrhage (aSAH) is a devastating event with a frequently disabling outcome. Our aim was to develop a prognostic model to predict an ordinal clinical outcome at two months in patients with aSAH.

Methods

We studied patients enrolled in the International Subarachnoid Aneurysm Trial (ISAT), a randomized multicentre trial to compare coiling and clipping in aSAH patients.

Several models were explored to estimate a patient's outcome according to the modified Rankin Scale (mRS) at two months after aSAH. Our final model was validated internally with bootstrapping techniques.

Results

The study population comprised of 2,128 patients of whom 159 patients died within 2 months (8%). Multivariable proportional odds analysis identified World Federation of Neurosurgical Societies (WFNS) grade as the most important predictor, followed by age, sex, lumen size of the aneurysm, Fisher grade, vasospasm on angiography, and treatment modality. The model discriminated moderately between those with poor and good mRS scores (c statistic = 0.65), with minor optimism according to bootstrap re-sampling (optimism corrected c statistic = 0.64).

Conclusion

We presented a calibrated and internally validated ordinal prognostic model to predict two month mRS in aSAH patients who survived the early stage up till a treatment decision. Although generalizability of the model is limited due to the selected population in which it was developed, this model could eventually be used to support clinical decision making after external validation.

Trial Registration

International Standard Randomised Controlled Trial, Number ISRCTN49866681

Reporting on covariate adjustment in randomised controlled trials before and after revision of the 2001 CONSORT statement: a literature review

OBJECTIVES

To evaluate the use and reporting of adjusted analysis in randomised controlled trials (RCTs) and compare the quality of reporting before and after the revision of the CONSORT Statement in 2001.

DESIGN

Comparison of two cross sectional samples of published articles.

DATA SOURCES

Journal articles indexed on PubMed in December 2000 and December 2006.

STUDY SELECTION

Parallel group RCTs with a full publication carried out in humans and published in English

MAIN OUTCOME MEASURES

Proportion of articles reported adjusted analysis; use of adjusted analysis; the reason for adjustment; the method of adjustment and the reporting of adjusted analysis results in the main text and abstract.

RESULTS

In both cohorts, 25% of studies reported adjusted analysis (84/355 in 2000 vs 113/422 in 2006). Compared with articles reporting only unadjusted analyses, articles that reported adjusted analyses were more likely to specify primary outcomes, involve multiple centers, perform stratified randomization, be published in general medical journals, and recruit larger sample sizes. In both years a minority of articles explained why and how covariates were selected for adjustment (20% to 30%). Almost all articles specified the statistical methods used for adjustment (99% in 2000 vs 100% in 2006) but only 5% and 10%, respectively, reported both adjusted and unadjusted results as recommended in the CONSORT guidelines.

CONCLUSION

There was no evidence of change in the reporting of adjusted analysis results five years after the revision of the CONSORT Statement and only a few articles adhered fully to the CONSORT recommendations.

Early prognosis in traumatic brain injury: from prophecies to predictions

Traumatic brain injury (TBI) is a heterogeneous condition that encompasses a broad spectrum of disorders. Outcome can be highly variable, particularly in more severely injured patients. Despite the association of many variables with outcome, prognostic predictions are notoriously difficult to make. Multivariable analysis has identified age, clinical severity, CT abnormalities, systemic insults (hypoxia and hypotension), and laboratory variables as relevant factors to include in models to predict outcome in individual patients. Advances in statistical modelling and the availability of large datasets have facilitated the development of prognostic models that have greater performance and generalisability. Two prediction models are currently available, both of which have been developed on large datasets with state-of-the-art methods, and offer new opportunities. We see great potential for their use in clinical practice, research, and policy making, as well as for assessment of the quality of health-care delivery. Continued development, refinement, and validation is advocated, together with assessment of the clinical impact of prediction models, including treatment response.

Subgroup Analysis of Trials Is Rarely Easy (SATIRE): a study protocol for a systematic review to characterize the analysis, reporting, and claim of subgroup effects in randomized trials

BACKGROUND

Subgroup analyses in randomized trials examine whether effects of interventions differ between subgroups of study populations according to characteristics of patients or interventions. However, findings from subgroup analyses may be misleading, potentially resulting in suboptimal clinical and health decision making. Few studies have investigated the reporting and conduct of subgroup analyses and a number of important questions remain unanswered. The objectives of this study are: 1) to describe the reporting of subgroup analyses and claims of subgroup effects in randomized controlled trials, 2) to assess study characteristics associated with reporting of subgroup analyses and with claims of subgroup effects, and 3) to examine the analysis, and interpretation of subgroup effects for each study's primary outcome.

METHODS

We will conduct a systematic review of 464 randomized controlled human trials published in 2007 in the 118 Core Clinical Journals defined by the National Library of Medicine. We will randomly select journal articles, stratified in a 1:1 ratio by higher impact versus lower impact journals. According to 2007 ISI total citations, we consider the New England Journal of Medicine, JAMA, Lancet, Annals of Internal Medicine, and BMJ as higher impact journals. Teams of two reviewers will independently screen full texts of reports for eligibility, and abstract data, using standardized, pilot-tested extraction forms. We will conduct univariable and multivariable logistic regression analyses to examine the association of pre-specified study characteristics with reporting of subgroup analyses and with claims of subgroup effects for the primary and any other outcomes.

DISCUSSION

A clear understanding of subgroup analyses, as currently conducted and reported in published randomized controlled trials, will reveal both strengths and weaknesses of this practice. Our findings will contribute to a set of recommendations to optimize the conduct and reporting of subgroup analyses, and claim and interpretation of subgroup effects in randomized trials.

Discrepancies in sample size calculations and data analyses reported in randomised trials: comparison of publications with protocols

OBJECTIVE

To evaluate how often sample size calculations and methods of statistical analysis are pre-specified or changed in randomised trials.

DESIGN

Retrospective cohort study. Data source Protocols and journal publications of published randomised parallel group trials initially approved in 1994-5 by the scientific-ethics committees for Copenhagen and Frederiksberg, Denmark (n=70).

MAIN OUTCOME MEASURE

Proportion of protocols and publications that did not provide key information about sample size calculations and statistical methods; proportion of trials with discrepancies between information presented in the protocol and the publication.

RESULTS

Only 11/62 trials described existing sample size calculations fully and consistently in both the protocol and the publication. The method of handling protocol deviations was described in 37 protocols and 43 publications. The method of handling missing data was described in 16 protocols and 49 publications. 39/49 protocols and 42/43 publications reported the statistical test used to analyse primary outcome measures. Unacknowledged discrepancies between protocols and publications were found for sample size calculations (18/34 trials), methods of handling protocol deviations (19/43) and missing data (39/49), primary outcome analyses (25/42), subgroup analyses (25/25), and adjusted analyses (23/28). Interim analyses were described in 13 protocols but mentioned in only five corresponding publications.

CONCLUSION

When reported in publications, sample size calculations and statistical methods were often explicitly discrepant with the protocol or not pre-specified. Such amendments were rarely acknowledged in the trial publication. The reliability of trial reports cannot be assessed without having access to the full protocols.

Predicting outcome after traumatic brain injury: development and international validation of prognostic scores based on admission characteristics

BACKGROUND

Traumatic brain injury (TBI) is a leading cause of death and disability. A reliable prediction of outcome on admission is of great clinical relevance. We aimed to develop prognostic models with readily available traditional and novel predictors.

METHODS AND FINDINGS

Prospectively collected individual patient data were analyzed from 11 studies. We considered predictors available at admission in logistic regression models to predict mortality and unfavorable outcome according to the Glasgow Outcome Scale at 6 mo after injury. Prognostic models were developed in 8,509 patients with severe or moderate TBI, with cross-validation by omission of each of the 11 studies in turn. External validation was on 6,681 patients from the recent Medical Research Council Corticosteroid Randomisation after Significant Head Injury (MRC CRASH) trial. We found that the strongest predictors of outcome were age, motor score, pupillary reactivity, and CT characteristics, including the presence of traumatic subarachnoid hemorrhage. A prognostic model that combined age, motor score, and pupillary reactivity had an area under the receiver operating characteristic curve (AUC) between 0.66 and 0.84 at cross-validation. This performance could be improved (AUC increased by approximately 0.05) by considering CT characteristics, secondary insults (hypotension and hypoxia), and laboratory parameters (glucose and hemoglobin). External validation confirmed that the discriminative ability of the model was adequate (AUC 0.80). Outcomes were systematically worse than predicted, but less so in 1,588 patients who were from high-income countries in the CRASH trial.

CONCLUSIONS

Prognostic models using baseline characteristics provide adequate discrimination between patients with good and poor 6 mo outcomes after TBI, especially if CT and laboratory findings are considered in addition to traditional predictors. The model predictions may support clinical practice and research, including the design and analysis of randomized controlled trials.

Primer: the fallacy of subgroup analysis

The identification of subgroups of patients from randomized clinical trials that are of specific interest for guiding clinical decisions can be an attractive idea; however, since such trials are designed for the comparison of groups of patients, performing subgroup analyses can result in misinterpretation of the data. Such analyses must, therefore, be performed and evaluated with caution: these should be pre-planned and included in the design of a suitably powered trial. Data obtained should be analyzed using formal statistical tests of interaction on proper subgroups rather than improper subgroups of patients, the results obtained should be delineated carefully, and details of how these analyses were performed, and how the data should be interpreted, should be reported in the trial paper. The caveats associated with this approach, such as the occurrence of false positive or false negative effects, chance differences in observed effects, lack of power to perform the analysis, floor or ceiling effects, issues relating to multiple statistical testing, and over-reporting and under-reporting are discussed in this review. Subgroup analyses can, however, provide valuable, albeit predominantly exploratory, information on which to base clinical decisions if they are performed in accordance with recommendations and guidelines, and do, therefore, have a legitimate place in rheumatology clinical trials.

Selection in reported epidemiological risks: an empirical assessment

BACKGROUND

Epidemiological studies may be subject to selective reporting, but empirical evidence thereof is limited. We empirically evaluated the extent of selection of significant results and large effect sizes in a large sample of recent articles.

METHODS AND FINDINGS

We evaluated 389 articles of epidemiological studies that reported, in their respective abstracts, at least one relative risk for a continuous risk factor in contrasts based on median, tertile, quartile, or quintile categorizations. We examined the proportion and correlates of reporting statistically significant and nonsignificant results in the abstract and whether the magnitude of the relative risks presented (coined to be consistently > or =1.00) differs depending on the type of contrast used for the risk factor. In 342 articles (87.9%), > or =1 statistically significant relative risk was reported in the abstract, while only 169 articles (43.4%) reported > or =1 statistically nonsignificant relative risk in the abstract. Reporting of statistically significant results was more common with structured abstracts, and was less common in US-based studies and in cancer outcomes. Among 50 randomly selected articles in which the full text was examined, a median of nine (interquartile range 5-16) statistically significant and six (interquartile range 3-16) statistically nonsignificant relative risks were presented (p = 0.25). Paradoxically, the smallest presented relative risks were based on the contrasts of extreme quintiles; on average, the relative risk magnitude was 1.41-, 1.42-, and 1.36-fold larger in contrasts of extreme quartiles, extreme tertiles, and above-versus-below median values, respectively (p < 0.001).

CONCLUSIONS

Published epidemiological investigations almost universally highlight significant associations between risk factors and outcomes. For continuous risk factors, investigators selectively present contrasts between more extreme groups, when relative risks are inherently lower.

Pharmacological enhancement of cognitive and behavioral deficits after traumatic brain injury

PURPOSE OF REVIEW

To provide the clinician with a reasonable overview of the modern pharmacological alternatives to treat the cognitive and behavioral sequels of traumatic brain injury.

RECENT FINDINGS

Original research in this area is sparse and more than half of the articles published on the subject recently have been reviews. Of the three randomized controlled trials, one studied methylphenidate (n = 18), one methylphenidate and sertraline (n = 30) and one amantadine (n = 27). All these studies reported beneficial effects on various cognitive measures, but because of the study protocols, the evidence provided may be questioned. The various reviews, uncontrolled studies and case reports suggest that at least psychostimulants, cholinergic agents, dopaminergic agents and antidepressants may be beneficial in treating the cognitive and behavioral symptoms of traumatic brain injury.

SUMMARY

The clinician trying to ameliorate the cognitive and behavioral symptoms of traumatic brain injury has to make decisions about pharmacotherapy that are still based mainly on clinical experience. Large randomized controlled trials giving high-quality evidence are so far missing. This review discusses the problems facing both the clinician and the scientist treating the cognitive and behavioral sequels of traumatic brain injury. A symptom-based approach is suggested for current practice.

Adjustment for Strong Predictors of Outcome in Traumatic Brain Injury Trials: 25% Reduction in Sample Size Requirements in the IMPACT Study

The aim of this study was to quantify the potential reduction in sample size that can be achieved by adjustment for predictors of outcome in traumatic brain injury (TBI) trials. We used individual patient data from seven therapeutic phase III randomized clinical trials (RCTs; n = 6166) in moderate or severe TBI, and three TBI surveys (n = 2238). The primary outcome was the dichotomized Glasgow Outcome Scale at 6 months (favorable/unfavorable). Baseline predictors of outcome considered were age, motor score, pupillary reactivity, computed tomography (CT) classification, traumatic subarachnoid hemorrhage, hypoxia, hypotension, glycemia, and hemoglobin. We calculated the potential sample size reduction obtained by adjustment of a hypothetical treatment effect for one to seven predictors with logistic regression models. The distribution of predictors was more heterogeneous in surveys than in trials. Adjustment of the treatment effect for the strongest predictors (age, motor score, and pupillary reactivity) yielded a reduction in sample size of 16-23% in RCTs and 28-35% in surveys. Adjustment for seven predictors yielded a reduction of about 25% in most studies: 20-28% in RCTs and 32-39% in surveys. A major reduction in sample size can be obtained with covariate adjustment in TBI trials. Covariate adjustment for strong predictors should be incorporated in the analysis of future TBI trials.