Quality of Abstracts Describing Randomized Trials in the Proceedings of American Society of Clinical Oncology Meetings: Guidelines for Improved Reporting

Highlights from ecancer Choosing Wisely Nepal 2022: critical appraisal skills for evidence-based practice, 24th-25th September 2022, Kathmandu, Nepal

The ecancer Kathmandu 2022 workshop on the 24th-25th September 2022 was the first ecancer conference organised in Nepal, a Southeast Asian nation sandwiched between India and China. It was focused on critical appraisal skills for evidence-based practice and was organised in partnership with the Karnali Academy of Health Sciences and the Civil Service Hospital from Nepal, and the Queen's Global Oncology Program from Canada. The workshop emphasised the need for critical thinking in understanding clinical research, and also motivated the delegates to undertake meaningful clinical research relevant to the local setting. The sessions highlighted the features of a good clinical research, identify pitfalls in the reporting of clinical trials, implementation of the research into locally relevant practice and development of local clinical guidelines. Furthermore, the faculty also discussed how to write a good scientific paper, the do's and don'ts of a systematic review and meta-analysis, the role of peer-review and how to do one properly and what do editors look for in evaluating papers submitted for publication. The audience learned the importance of finding a good mentor and fostering local and international collaboration. The local faculty also highlighted their own personal journeys and how mentorship and global collaboration played an important role in their own academic career. The enthusiastic panel discussion was a highlight of the programme where the delegates learned about several important topics from the faculties, such as work-life balance, the role of mentorship in building careers and building networks.

Availability of results of interventional studies assessing colorectal cancer from 2013 to 2020

Colorectal cancer (CRC) is one of the most frequent cancers worldwide. Our aim was to evaluate the availability of results of interventional studies studying CRC. We searched the ClinicalTrials.gov registry for all interventional studies on CRC management in adults completed or terminated between 01/01/2013 and 01/01/2020. To identify results, we searched for results posted on the ClinicalTrials.gov registry and/or published in a full-text article. Our primary outcome was the proportion of CRC interventional studies with available results (i.e. posted on the ClinicalTrials.gov registry and/or published in a full-text article). Secondary outcomes were 1) median time between primary completion and earliest date of results availability, 2) the cumulative percentage of interventional studies with results available over time 3) the cumulative percentage of interventional studies with results posted on the ClinicalTrials.gov registry over time and 4) the percentage of results available in open access. We identified 763 eligible interventional studies in ClinicalTrials.gov, which included 679 198 patients. Of these, 286 (37%) trials, including 270 845 (40%) patients, did not have any results available. Median time for results availability was 32.6 months (IQ 16.1-unreached). The cumulative percentage of interventional studies with available results was 17% at 12 months, 39% at 24 months and 55% at 36 months. Results were more likely available for trials that were randomized, completed, had one trial site in the United States, and with mixed funding. The cumulative percentage of interventional studies with results posted on ClinicalTrials.gov was 2% at 12 months. Results were available in open access for 420 (420/477 = 88%) trials. Our results highlight an important waste in research for interventional studies studying CRC.

Results of trials assessing transarterial chemoembolization for treating hepatocellular carcinoma are critically underreported

OBJECTIVE

We aimed to evaluate to what extent the results of registered randomized controlled trials (RCTs) assessing transarterial chemoembolization (TACE) for treating hepatocellular carcinoma (HCC) are publicly available.

METHODS

We searched the Cochrane Central Register of Controlled Trials, the International Clinical Trials Registry Platform, and primary registries for RCTs assessing TACE for treating HCC, registered between January 2008 and August 2018, that had exceeded their completion date by more than 1 year. We systematically searched PubMed, EMBASE, and Google Scholar for a publication as well as the registry for results. The main outcomes were the availability of results, and the time to the first availability of results (i.e., posted on the registry or published). Secondary outcomes were the proportion of results available at 12 and 36 months after completion.

RESULTS

Among 67 identified RCTs, including a total target number to 11,599 participants, 26 had publicly available results (39%; i.e., 42% of total target number of participants). Results of 25 RCTs (37%) were published, with only 3 having results posted on the registry and 2 with both published and posted results. The median (Q1-Q3) time from completion to the first public availability of results was 18 months (11-29). The cumulative percentages of RCTs with results available were 10% (95% CI, 3-17%) and 29% (95% CI, 17-39%) at 12 and 36 months, respectively, after completion.

CONCLUSIONS

Despite the ethical commitments and societal expectations for disclosure of results, the availability of results of RCTs on TACE for treating HCC is very limited.

KEY POINTS

• Underreporting of trial results is a major cause of wasted medical research since inaccessible research results fail to help both patients and clinicians. • Transarterial chemoembolization (TACE) is the most common treatment for hepatocellular carcinoma (HCC) and has called for considerable research efforts. • Yet, almost two-thirds of randomized controlled trials assessing TACE for treating HCC did not yield any public results, either on the registry platform or in scientific journals.

Quality of reporting and fragility index for randomized controlled trials in the vesicoureteral reflux literature: where do we stand?

BACKGROUND/INTRODUCTION

Randomized controlled trials (RCTs) are considered the "gold standard" methodology for examining the effects of clinical interventions, yet only 1% of urology literature employs this design. The Consolidated Standards of Reporting Trials (CONSORT) statement contains a standardized checklist of 37 items to be included when reporting RCTs to ensure transparency and completeness of information [2]. Despite the robust design of RCTs, the number of events can greatly change the significance of the results, which can be represented by the fragility index (FI).

OBJECTIVE

The objective was to assess the quality of reporting of RCTs in the pediatric vesicoureteral reflux (VUR) literature using the 2010 CONSORT statement and, for studies with significant positive findings, to determine the FI as a measure of robustness of the results.

STUDY DESIGN

A comprehensive search was conducted through MEDLINE® and Embase® to identify RCTs in VUR literature from 2000 to 2016. Two reviewers independently selected articles, which were evaluated using the CONSORT checklist. An overall quality of reporting score (OQS) (%) was calculated by dividing the number of checklist items present in each study by the maximum possible score (34) and expressed as a percentage. Studies were classified as low (<40%), moderate (40-70%) and high quality (>70%) based on the modified assessing the methodological quality of systematic reviews (AMSTAR) checklist. Of the 2052 initial matches, 98% were excluded due to methodology or content, a further 28 studies were found not to meet inclusion criteria after full text review. The FI was calculated for the 7 studies that met inclusion criteria with significantly different results by manually adding events to the study groups until p>0.05.

RESULTS

Twenty-two studies met inclusion criteria. The mean OQS was 46+17% with 9 (41%) identified as low quality (score <40%), 11 (50%) as moderate (40-70%) and 2 (9%) as high quality (>70%). There was no significant difference in OQS between RCTs with a sample size > 100 (n=15) versus <100 patients (n=7) (45+17% vs. 47+17%, p=0.7). However, we noted a difference when we compared RCTs with biostatistician support (n=4) vs. those without (n=18) (59+20% vs. 43+15%, p<0.05). Seven studies reported significant positive results making calculation of FI possible. The mean FI was 5.8+5.1 indicating that most studies were fragile. There was no correlation between the OQS and FI.

DISCUSSION AND CONCLUSION

The mean OQS of VUR RCTs was suboptimal (46%), with most studies having low FI scores indicating the instability of the findings. The only variable that significantly impacted the OQS was biostatistician support. Implementation of the CONSORT checklist with a minimum of 50% inclusion as a prerequisite for submission of manuscripts may improve the quality and transparency of reporting. Calculation of the FI could provide readers with an objective measure of robustness for the published trials, allowing for appropriate interpretation of the results.

Meta-Research on Oncology Trials: A Toolkit for Researchers with Limited Resources

"Meta-research" is a discipline that investigates research practices. Meta-research on clinical trials is an attempt to summarize descriptive and methodological features of published or ongoing clinical trials, including aspects of their implementation, design, analysis, reporting, and interpretation. In this type of investigation, the unit of analysis is a primary source of information about a clinical trial (e.g., published reports, study protocols, or abstracts), with meta-research being a second layer of information that summarizes what is known from various primary sources. After the formulation of the primary research question, the methodology of meta-research resembles that of other research projects, with predefined eligibility criteria, exposure variables, primary and secondary outcomes of interest, and an analysis plan. This type of study usually provides a high-level picture of the literature on a specific topic, always accompanied by a critical evaluation of the methodology and/or the quality of reporting of the studies included. Because relatively few resources are consumed to produce meta-research, these studies offer a great opportunity for clinical scientists working in settings with limited resources. In this article, we present the principles of designing and conducting meta-research and use our experience to suggest recommendations on how to perform and how to report this type of potentially very creative study. IMPLICATIONS FOR PRACTICE: The term meta-research pertains to a type of study in which the unit of analysis is, in most cases, the publication of a clinical trial. This type of study usually provides a high-level picture of the literature on a specific topic, always accompanied by a critical evaluation of the methodology, design, and/or the quality of reporting of the studies included. Because relatively few resources are consumed to produce meta-research, these studies offer a great opportunity for clinical scientists who work in low-income countries. This article presents the principles of designing and conducting meta-research and proposes practical recommendations on how to perform and report this type of potentially very creative study.

Quality of reporting for randomized controlled trials in the hypospadias literature: Where do we stand?

INTRODUCTION

To assess the quality of reporting of randomized controlled trials (RCTs) in the hypospadias literature using the 2010 Consolidated Standards of Reporting Trials (CONSORT) statement. It was hypothesized that hypospadias RCTs that contained clear descriptions of key methodological items, allocation concealment, blinding, and sample size justification would have higher overall quality of reporting scores (OQS).

MATERIALS AND METHODS

A comprehensive search was conducted through MEDLINE to identify RCTs in hypospadias surgical techniques and postoperative management during the period 1990-2014. Two reviewers independently selected articles, which were evaluated using the CONSORT checklist. An overall quality score (%) was calculated to assess the quality of reporting. In addition, a methodological index score out of 4 was calculated based on the following items: use of intention to treat/sample size justification, allocation concealment, specification of randomization type, and blinding of outcome assessors.

RESULTS

Of the 76 initial results, 39 (51%) were excluded due to their predominant focus on anesthesia. After full-text screening, 10 (13%) citations were further excluded because they were case control studies or did not focus on hypospadias techniques, resulting in 27 (36%) studies included for analysis. The mean overall quality score was 37 ± 12% and a median of 36% (range: 14-61%). Fifteen (56%) studies were identified as low quality (score <40%) and 12 (44%) as moderate quality (40-70%). No studies were classified as high quality (>70%). Hypospadias RCTs published between 2007 and 2014 versus those reported before 2007 (44 ± 9% vs 33 ± 11%, P = 0.01), RCTs with a sample size >100 patients versus those <100 (47 ± 8% vs 36 ± 11%, P = 0.01), RCTs that disclosed having received funding versus those that did not (56 ± 4% vs 38 ± 10%, P < 0.01) and RCTs that had proof of biostatistician/epidemiologist support versus those that did not (58 ± 5% vs 36 ± 11%, P = 0.01) had a higher mean OQS. The number of articles that met specific 2010 CONSORT criteria is illustrated in Summary Fig.

DISCUSSION

It was found that the contemporary hypospadias literature continues to suffer from suboptimal reporting standards. There seems to be an improvement in the OQS for studies published after 2007 and those with larger sample sizes, usually >100 patients. Nevertheless, none of the studies obtained high quality of reporting (OQS >70%) as per the CONSORT statement checklist. The inadequacies in reporting were related to sample size justifications, randomization method, allocation concealment strategy, blinding, description of subjects lost to follow-up and Intention To Treat (ITT) analysis. These findings were consistent with the poor quality of reporting observed in other surgical fields.

CONCLUSIONS

The current overall quality score in hypospadias literature is suboptimal and efforts must be made to improve quality.

Inflammatory Bowel Disease-Related Abstracts Presented at National Conferences in the USA Are Frequently Unpublished as Full Manuscripts

BACKGROUND

Numerous abstracts related to inflammatory bowel disease (IBD) are presented at national conferences in the USA. The overall rate of publication of these abstracts as complete manuscripts is unknown .

METHODS

Abstracts submitted to the 2010 American College of Gastroenterology (ACG), Advances in Inflammatory Bowel Diseases (AIBD), and the American Gastroenterological Association abstracts at Digestive Disease Week (DDW) were reviewed. Each abstract was reviewed manually by two authors for type of research, study design, patient population, and outcome. Both PubMed and Google were then searched to determine whether the abstract was published as a full manuscript within five years of the conference. Univariate and multivariate logistic regression analysis was carried out using Stata 14.1.

RESULTS

In total, 872 abstracts were reviewed. 49% (426/872) were published as complete manuscripts within five years of the conference. The average length of time to publication was 1.87 years (range 0-5). 42% of abstracts from ACG, 58% from AIBD, and 23% from DDW were eventually published (p < 0.0001). However, abstracts presented at DDW had the shortest time to publication compared to the other conferences (p = 0.002). Factors predictive of eventual publication include: number of authors (mean 7.5 for published vs 6.4 for unpublished p = 0.0001), clinical research compared to basic and translational (p = 0.026), and studies assessing drug safety with no adverse effects reported (p = 0.006).

CONCLUSION

Nearly 50% of the abstracts presented at major gastroenterology conferences in the USA are published as full manuscripts 5 years after the conference. Further studies are needed to assess why so many abstracts are not published.

Assessing the Eventual Publication of Clinical Trial Abstracts Submitted to a Large Annual Oncology Meeting

BACKGROUND

Despite the ethical imperative to publish clinical trials when human subjects are involved, such data frequently remain unpublished. The objectives were to tabulate the rate and ascertain factors associated with eventual publication of clinical trial results reported as abstracts in the Proceedings of the American Society of Clinical Oncology (American Society of Clinical Oncology).

MATERIALS AND METHODS

Abstracts describing clinical trials for patients with breast, lung, colorectal, ovarian, and prostate cancer from 2009 to 2011 were identified by using a comprehensive online database (http://meetinglibrary.asco.org/abstracts). Abstracts included reported results of a treatment or intervention assessed in a discrete, prospective clinical trial. Publication status at 4-6 years was determined by using a standardized search of PubMed. Primary outcomes were the rate of publication for abstracts of randomized and nonrandomized clinical trials. Secondary outcomes included factors influencing the publication of results.

RESULTS

A total of 1,075 abstracts describing 378 randomized and 697 nonrandomized clinical trials were evaluated. Across all years, 75% of randomized and 54% of nonrandomized trials were published, with an overall publication rate of 61%. Sample size was a statistically significant predictor of publication for both randomized and nonrandomized trials (odds ratio [OR] per increase of 100 participants = 1.23 [1.11-1.36], p < .001; and 1.64 [1.15-2.34], p = .006, respectively). Among randomized studies, an industry coauthor or involvement of a cooperative group increased the likelihood of publication (OR 2.37, p = .013; and 2.21, p = .01, respectively). Among nonrandomized studies, phase II trials were more likely to be published than phase I (p < .001). Use of an experimental agent was not a predictor of publication in randomized (OR 0.76 [0.38-1.52]; p = .441) or nonrandomized trials (OR 0.89 [0.61-1.29]; p = .532).

CONCLUSION

This is the largest reported study examining why oncology trials are not published. The data show that 4-6 years after appearing as abstracts, 39% of oncology clinical trials remain unpublished. Larger sample size and advanced trial phase were associated with eventual publication; among randomized trials, an industry-affiliated author or a cooperative group increased likelihood of publication. Unfortunately, we found that, despite widespread recognition of the problem and the creation of central data repositories, timely publishing of oncology clinical trials results remains unsatisfactory.

Public availability of results of observational studies evaluating an intervention registered at ClinicalTrials.gov

BACKGROUND

Observational studies are essential for assessing safety. The aims of this study were to evaluate whether results of observational studies evaluating an intervention with safety outcome(s) registered at ClinicalTrials.gov were published and, if not, whether they were available through posting on ClinicalTrials.gov or the sponsor website.

METHODS

We identified a cohort of observational studies with safety outcome(s) registered on ClinicalTrials.gov after October 1, 2007, and completed between October 1, 2007, and December 31, 2011. We systematically searched PubMed for a publication, as well as ClinicalTrials.gov and the sponsor website for results. The main outcomes were the time to the first publication in journals and to the first public availability of the study results (i.e. published or posted on ClinicalTrials.gov or the sponsor website). For all studies with results publicly available, we evaluated the completeness of reporting (i.e. reported with the number of events per arm) of safety outcomes.

RESULTS

We identified 489 studies; 334 (68%) were partially or completely funded by industry. Results for only 189 (39%, i.e. 65% of the total target number of participants) were published at least 30 months after the study completion. When searching other data sources, we obtained the results for 53% (n = 158; i.e. 93% of the total target number of participants) of unpublished studies; 31% (n = 94) were posted on ClinicalTrials.gov and 21% (n = 64) on the sponsor website. As compared with non-industry-funded studies, industry-funded study results were less likely to be published but not less likely to be publicly available. Of the 242 studies with a primary outcome recorded as a safety issue, all these outcomes were adequately reported in 86% (114/133) when available in a publication, 91% (62/68) when available on ClinicalTrials.gov, and 80% (33/41) when available on the sponsor website.

CONCLUSIONS

Only 39% of observational studies evaluating an intervention with safety outcome(s) registered at ClinicalTrials.gov had their results published at least 30 months after study completion. The registration of these observational studies allowed searching other sources (results posted at ClinicalTrials.gov and sponsor website) and obtaining results for half of unpublished studies and 93% of the total target number of participants.

Reporting quality of abstracts in phase III clinical trials of systemic therapy in metastatic solid malignancies

Background

Manuscript abstracts represent a critical source of information for oncology practitioners. Practitioners may utilize the information contained in abstracts as a basis for treatment decisions particularly when full-text articles are not accessible. In 2007, the Consolidated Standards of Reporting Trials (CONSORT) extension statement for abstracts provided a minimum list of elements that should be included in abstracts. In this study we evaluate the degree of adherence to these recommendations and accessibility of full text publications in oncology publications.

Methods

A systematic review of abstracts of randomized, controlled, phase III trials in metastatic solid malignancies published between January 2009 and December 2011 in PubMed, Medline, and Embase was completed. Abstracts were assigned a completeness score of 0–18 based on the number of CONSORT-recommended elements. Accessibility through open access was recorded.

Results

174 abstracts with data for 95,956 patients were reviewed. The median completeness score was 9 (range, 3–17). Open access to full text articles was available for 80 % of abstracts. The remaining 20 % (35 out of 174) had a median cost of 38 USD (range: $22–49.95). The least frequently reported elements were: trial design description (20 %), participant allocation method (13 %), blinding (24 %), trial enrollment status (22 %), registration and name of trial (26 %) and funding source (18 %). The most frequently reported elements were eligibility criteria (98 %), study interventions (100 %), and primary endpoint (87 %).

Conclusion

There is poor adherence to the CONSORT recommendations for abstract reporting in publications of randomized cancer clinical trials which could negatively impact clinical decision-making. Full-text articles are frequently available through open access.

An introduction to systematic reviews in animal health, animal welfare, and food safety

In this paper, we provide an introduction to systematic reviews and discuss the process for conducting systematic reviews in animal health, animal welfare, and food safety. The research synthesis need that can be addressed by a systematic review is discussed. The use of systematic reviews to address questions about intervention effects, etiology, diagnostic tests evaluation and disease burden are discussed. The steps included in a systematic review are described.

Impact of spin in the abstracts of articles reporting results of randomized controlled trials in the field of cancer: the SPIIN randomized controlled trial

PURPOSE

We aimed to assess the impact of spin (ie, reporting to convince readers that the beneficial effect of the experimental treatment is greater than shown by the results) on the interpretation of results of abstracts of randomized controlled trials (RCTs) in the field of cancer.

METHODS

We performed a two-arm, parallel-group RCT. We selected a sample of published RCTs with statistically nonsignificant primary outcome and with spin in the abstract conclusion. Two versions of these abstracts were used-the original with spin and a rewritten version without spin. Participants were clinician corresponding authors of articles reporting RCTs, investigators of trials, and reviewers of French national grants. The primary outcome was clinicians' interpretation of the beneficial effect of the experimental treatment (0 to 10 scale). Participants were blinded to study hypothesis.

RESULTS

Three hundred clinicians were randomly assigned using a Web-based system; 150 clinicians assessed an abstract with spin and 150 assessed an abstract without spin. For abstracts with spin, the experimental treatment was rated as being more beneficial (mean difference, 0.71; 95% CI, 0.07 to 1.35; P = .030), the trial was rated as being less rigorous (mean difference, -0.59; 95% CI, -1.13 to 0.05; P = .034), and clinicians were more interested in reading the full-text article (mean difference, 0.77; 95% CI, 0.08 to 1.47; P = .029). There was no statistically significant difference in the clinicians' rating of the importance of the study or the need to run another trial.

CONCLUSION

Spin in abstracts can have an impact on clinicians' interpretation of the trial results.

Relation of completeness of reporting of health research to journals' endorsement of reporting guidelines: systematic review

OBJECTIVE

To assess whether the completeness of reporting of health research is related to journals' endorsement of reporting guidelines.

DESIGN

Systematic review.

DATA SOURCES

Reporting guidelines from a published systematic review and the EQUATOR Network (October 2011). Studies assessing the completeness of reporting by using an included reporting guideline (termed "evaluations") (1990 to October 2011; addendum searches in January 2012) from searches of either Medline, Embase, and the Cochrane Methodology Register or Scopus, depending on reporting guideline name.

STUDY SELECTION

English language reporting guidelines that provided explicit guidance for reporting, described the guidance development process, and indicated use of a consensus development process were included. The CONSORT statement was excluded, as evaluations of adherence to CONSORT had previously been reviewed. English or French language evaluations of included reporting guidelines were eligible if they assessed the completeness of reporting of studies as a primary intent and those included studies enabled the comparisons of interest (that is, after versus before journal endorsement and/or endorsing versus non-endorsing journals).

DATA EXTRACTION

Potentially eligible evaluations of included guidelines were screened initially by title and abstract and then as full text reports. If eligibility was unclear, authors of evaluations were contacted; journals' websites were consulted for endorsement information where needed. The completeness of reporting of reporting guidelines was analyzed in relation to endorsement by item and, where consistent with the authors' analysis, a mean summed score.

RESULTS

101 reporting guidelines were included. Of 15,249 records retrieved from the search for evaluations, 26 evaluations that assessed completeness of reporting in relation to endorsement for nine reporting guidelines were identified. Of those, 13 evaluations assessing seven reporting guidelines (BMJ economic checklist, CONSORT for harms, PRISMA, QUOROM, STARD, STRICTA, and STROBE) could be analyzed. Reporting guideline items were assessed by few evaluations.

CONCLUSIONS

The completeness of reporting of only nine of 101 health research reporting guidelines (excluding CONSORT) has been evaluated in relation to journals' endorsement. Items from seven reporting guidelines were quantitatively analyzed, by few evaluations each. Insufficient evidence exists to determine the relation between journals' endorsement of reporting guidelines and the completeness of reporting of published health research reports. Journal editors and researchers should consider collaborative prospectively designed, controlled studies to provide more robust evidence.

SYSTEMATIC REVIEW REGISTRATION

Not registered; no known register currently accepts protocols for methodology systematic reviews.

Public availability of results of trials assessing cancer drugs in the United States

PURPOSE

To evaluate to what extent results of completed trials of cancer drugs conducted in the United States are publicly available at ClinicalTrials.gov, as required by the Food and Drug Administration Amendments Act (FDAAA), or are published in journals.

METHODS

We searched ClinicalTrials.gov for cancer trials governed by the FDAAA: phase II to IV trials assessing drugs in the United States with a primary completion date between December 26, 2007, and May 31, 2010. For each trial, we also searched PubMed to identify the publication of results. We assessed the cumulative percentages of posted or published results over time by using the Kaplan-Meier method.

RESULTS

We identified 646 trials, including 209 randomized controlled trials (RCTs). At 12 months after completion of the trials, the cumulative percentages of trials with results posted at ClinicalTrials.gov, published in journals, and available either at ClinicalTrials.gov or in journals were 9% (95% CI, 7% to 11%), 12% (95% CI, 10% to 15%), and 20% (95% CI, 17% to 23%), respectively, and for RCTs, the percentages were 12% (95% CI, 8% to 16%), 5% (95% CI, 2% to 8%), and 17% (95% CI, 12% to 22%), respectively. At 36 months, these percentages were 31% (95% CI, 28% to 35%), 35% (95% CI, 31% to 39%), and 55% (95% CI, 51% to 59%), respectively, and for RCTs, they were 38% (95% CI, 31% to 45%), 32% (95% CI, 25% to 39%), and 56% (95% CI, 48% to 62%), respectively. Public availability of phase III trials was 15% (95% CI, 7% to 23%) at 12 months, 39% (95% CI, 27% to 49%) at 24 months, and 64% (95% CI, 50% to 73%) at 36 months.

CONCLUSION

Despite the FDAAA, results for nearly half the trials of cancer drugs in the United States were not publicly available 3 years after completion of the trials.

Improving the quality of abstract reporting for economic analyses in oncology

BACKGROUND

The increasing cost of cancer drugs underscores the importance of economic analyses. Although guidelines for abstract reporting of randomized controlled studies and phase i trials are available, similar recommendations for conference abstracts of economic analyses are lacking. Our objectives were to identify items considered to be essential in abstracts of economic analyses;to evaluate the quality of abstracts submitted to the American Society of Clinical Oncology (asco), the American Society of Hematology (ash), and the International Society for Pharmacoeconomics and Outcomes Research (ispor) meetings; andto propose guidelines for future abstract reporting at conferences.

METHODS

Health economic experts were surveyed and asked to rate each of 24 possible abstract elements on a 5-point Likert scale. A scoring system for abstract quality was devised based on elements with an average expert rating of 3.5 or greater. Abstracts for economic analyses from asco, ash, and ispor meetings were reviewed and assigned a quality score.

RESULTS

Of 99 experts, 50 (51%) responded to the survey (average age: 53 years; 78% men; 54% from the United States, 28% from Europe, 18% from Canada). In total, 216 abstracts were reviewed: asco, 53%; ash, 14%; and ispor, 33%. The median quality score was 75, but notable deficiencies were observed. Cost perspective was reported in only 61% of abstracts, and time horizon was described in only 47%. Abstracts from recent years demonstrated better quality scores. We also observed disparities in quality scores for various cancer sites (p = 0.005).

CONCLUSIONS

The quality of conference abstracts for economic analyses in oncology has room for improvement. Abstracts may be enhanced using the guidelines derived from our survey of experts.

Has the quality of abstracts for randomised controlled trials improved since the release of Consolidated Standards of Reporting Trial guideline for abstract reporting? A survey of four high-profile anaesthesia journals

BACKGROUND AND OBJECTIVE

Randomised controlled trial (RCT) abstracts published in journal articles have traditionally been deficient of crucial information. To improve the quality of RCT abstracts, in January 2008, the Consolidated Standards of Reporting Trial (CONSORT) group published a checklist of essential information for inclusion. The current study assessed whether there has been an improvement in the quality of RCT abstracts published in main anaesthesia journals since this new guideline was introduced.

METHODS

Articles involving human RCTs published in four high-profile anaesthesia journals (Anaesthesia, Anesthesia & Analgesia, Anesthesiology and the European Journal of Anaesthesiology) were reviewed, comparing those published from October 2005 to September 2006 (pre-CONSORT abstracts) with those published from October 2008 to September 2009 (post-CONSORT abstracts). Trials involving healthy volunteers or cadavers, cost-effectiveness studies, meta-analyses and letters were excluded. Abstracts from remaining RCTs were randomly assigned to four reviewers in a blinded fashion and reviewed for content using the new CONSORT checklist.

RESULTS

In total, 527 RCT abstracts (pre-CONSORT RCTs, n = 275 and post-CONSORT RCTs, n = 252) were analysed. The majority of abstracts in both groups provided an appropriate description of study interventions (73.1 and 73.8%, pre-CONSORT abstracts versus post-CONSORT abstracts, respectively), objective (91.3 and 90.1%) and conclusions (72.4 and 66.3%). From pre-CONSORT to post-CONSORT guidelines for abstract reporting, there were significant improvements in correctly identifying blinding (18.2-29%) and harmful effects (31.6-42.1%). The improvement in reporting the nature of the trial in abstract titles (20.1-29%) and primary outcome measure in the methods section (22.9-30.6%) did not reach significance. There was no clear improvement in the already poor reporting of trial design, participants, randomisation, recruitment, outcomes, trial registration and funding sources.

CONCLUSIONS

Despite some promising improvements and inter-journal differences, the overall quality of RCT abstracts and adherence to the CONSORT checklist for abstracts remains poor.

Compendium of unpublished phase III trials in oncology: characteristics and impact on clinical practice

PURPOSE

Many phase III trials presented at annual meetings of the American Society of Clinical Oncology (ASCO) remain unpublished. The results of these unpublished trials, if more generally known, might have an impact on clinical practice.

METHODS

Abstracts of large phase III trials evaluating systemic cancer treatment were identified from conference proceedings of the 1989 to 2003 ASCO annual meetings. PubMed, Medline, and Embase were searched for corresponding publications. A compendium of unpublished phase III trials was assembled. Clinical significance of nonpublication was determined by disease site-specific oncology experts from two academic cancer centers in Canada.

RESULTS

A total of 709 phase III trials were identified of which 66 (9.3%) remain without a subsequent publication at a minimum of 6.5 years of follow-up and 94 (13%) were published after a delay of ≥ 5 years from their initial presentation. Of the unpublished trials, 48% were presented as oral sessions at an ASCO meeting, and 71% of the abstracts reported negative results. The experts judged that 70% of the unpublished trials addressed an important question and 59% might have had clinical impact if their results had been published promptly.

CONCLUSION

A substantial number of cancer clinical trials with potential influence on clinical practice remain unpublished and many other trials are published after a substantial delay. Nonpublication of clinical trials breaks an implicit contract with participants, institutional review boards, and sponsors.

Randomised controlled trials in oncology closed early for benefit: trends in methodology, results, and interpretation

PURPOSE

To assess methodology, results and interpretation of oncology randomised controlled trials closed early for benefit (RCTCEB).

METHODS

Structured literature search (1950-2008) to identify all published oncology RCTCEB. We then searched for related follow-up articles and conference abstracts to evaluate whether study results and conclusions changed with longer follow-up. A standardised data abstraction process captured information related to statistical methodology, details of interim analyses, results and conclusions. Original articles and follow-up reports were compared for results of primary end-point and author conclusions.

RESULTS

We identified 71 RCTCEB. In 16 articles (23%) the study primary end-point was not explicitly stated. Most trials were open to accrual (47/71, 66%) at the time of closure. Formal interim analysis was performed in 65 (92%) trials of which 72% (47/65) was reported as planned; 82% (53/65) reported stopping rules. Trials on average accrued 75% of the planned sample size. Amongst the 23 (32%) RCTCEB with follow-up reports, in only one case did the study results or conclusions change substantially.

CONCLUSIONS

While the majority of oncology RCTCEB follows rigourous methodological principles, an important percentage includes limitations in design and/or analysis. Amongst the 23 studies with subsequent follow-up reports, initial results were confirmed in 22 (96%).

The quality of reporting and publication status of vaccines trials presented at veterinary conferences from 1988 to 2003

Conference proceedings, although an importance source to learn about new interventions, are brief and not subject to external evaluation prior to publication. Studies in human medicine suggest that trial results from conference proceedings may be poorly reported. The primary objective of this study was to evaluate characteristics of veterinary vaccine studies published in conference proceedings. 154 cattle or swine vaccine trials presented at veterinary practitioner conferences held in the United States from 1988 to 2003 were the test base to evaluate the following: abstract-to-publication ratio, positive-outcome ratios, differences between proceeding and subsequent journal articles, and compliance with the CONSORT Statement extension for abstracts. The abstract-to-publication ratio was 5/89 for swine trials and 6/65 for cattle trials. The positive-outcome ratio for swine conference proceeding and journal articles was 57/89 and 4/5 respectively. The positive-outcome ratio for bovine conference proceeding and journal articles was 34/65 and 4/6 respectively. No major differences were found between conference proceedings and matching journal articles. Fewer than 10% of conference proceedings included: identification of the trials as randomized in the title, study design as field or challenge; the primary outcome; trial status; results for primary outcome; information about harms and funding source (2/89). When conference proceedings are subsequently published in journals; there is no significant difference in the data, however subsequent publication is uncommon. For many conference proceedings it would be difficult to assess the internal and external validity of the trial based on the information reported.

Randomized clinical trials presented at the World Congress of Endourology: how is the quality of reporting?

PURPOSE

To assess the quality of reporting of randomized controlled trials (RCTs) presented in abstract form at the annual World Congress of Endourology (WCE) and evaluate their course of subsequent publication.

MATERIALS AND METHODS

All RCTs presented in abstract form at the 2004, 2005, and 2006 WCE annual meetings were identified for review. Quality of reporting was assessed by applying a standardized 14-item evaluation tool based on the Consolidated Standards for the Reporting of Trials (CONSORT) statement. The subsequent publication rate for the corresponding studies by scanning Medline was also evaluated. Appropriate statistical analysis was performed.

RESULTS

A total of 94 RCTs (3.5% of 2669) were identified for review: 21 in 2004, 36 in 2005, and 37 in 2006. Overall, 45 (47.3% of the total) were subsequently published as a full length indexed manuscript with a mean time to publication of 16.4 ± 13.2 months. Approximately 61 (60%) identified the study design as RCT in the abstract title. None reported the method of randomization. In studies that reported blinding (seven, 11% of 62), five were double blinded and two single blinded. Adverse events were reported in 38% of cases. Only 10% of the abstracts complied fully with more than 10 items according to our CONSORT-based checklist, whereas the majority of them failed to comply with most of the CONSORT requirements.

CONCLUSIONS

Although representing a small portion of the overall number of abstracts, there has been a steady increase of presentation of RCTs at the WCE over the assessed 3-year period. Most of the time they are recognized as RCTs in the abstract title. When applying the CONSORT criteria, necessary information to assess their methodologic quality is incomplete in some cases.

The use of research abstracts in formulary decision making by the Joint Oncology Drug Review of Canada

BACKGROUND AND OBJECTIVE

Opinions on the use of research abstracts in policy decision making are conflicting. We sought to evaluate the influence of research abstracts in guiding decisions of the Joint Oncology Drug Review of Canada (JODR), which conducts clinical and economic reviews of new cancer treatment drugs for formulary listing.

METHODS

The minutes of the monthly meetings of the JODR between 2005 and 2007 were reviewed. One submission per drug indication was included. Elements evaluated included the level of evidence supporting each decision, the year the study was published and subsequent publication of any abstracts.

RESULTS

There were 73 recommendations reviewed over the 36 months. Ten recommendations were deferred and eight recommendations were re-submissions, thus 55 recommendations underwent analysis. There were 31 recommendations based to some extent on abstracts, of which 14 (45%) were in favour of formulary listing and 17 (55%) were opposed. Twelve recommendations were based exclusively on abstracts, seven (58%) of which were in favour of formulary listing. As a comparison, published randomized controlled trials were part of the evidentiary base in 30 committee recommendations (55%). Of these, 17 (57%) were in favour of formulary listing, while 13 (43%) were opposed.

CONCLUSIONS

Research abstracts are commonly involved in evidence-based decision making for formulary listing. The rates of approving cancer drugs for funding by the JODR were similar among recommendations based on abstracts and other levels of evidence. Abstracts can play an important role in guiding decision making.

Conference abstracts of a new oncology drug do not always lead to full publication: proceed with caution

BACKGROUND

Conference abstracts, often the first public record of a study, serve as a catalyst to initiate clinical and policy change. On average, 45% of all conference abstracts subsequently appear as full publications; however, the generalizability of this finding to studies of one intervention, in one population, is unknown. Our objectives were to determine the full publication rate of a cohort of abstracts, median time to publication, and predictors of these relationships.

METHODS

We included the first 5 years of clinical abstract reports of rituximab for non-Hodgkin's lymphoma (NHL) from American Society of Hematology (ASH) meetings (1997-2001), identified all unique studies, and used electronic databases to identify full publications. We determined the full publication rate, median time to publication, and predictors of these outcomes.

RESULTS

Of 109 abstracts representing 86 unique studies, the publication rate was 52.3% (45, 95% confidence interval [CI]: 41.3, 63.2), and the median time to publication, 1.4 years with 6.8 years' follow-up. Author affiliation with industry (odds ratio [OR] [95% CI]=4.60 [1.32, 16.08] and presentation type (oral OR=5.94 [1.31, 26.88], poster OR=3.39 [1.24, 9.25]; reference, publication in conference abstract book only) independently predicted subsequent full publication in the adjusted analysis. We identified no predictors of time to publication.

INTERPRETATION

We suggest cautious consideration of data from conference proceedings to inform new technology clinical or policy decisions. Future work needs to examine the generalizability of our results to other diseases and technologies.

Evolution of the randomized controlled trial in oncology over three decades

PURPOSE

The randomized controlled trial (RCT) is the gold standard for establishing new therapies in clinical oncology. Here we document changes with time in design, sponsorship, and outcomes of oncology RCTs.

METHODS

Reports of RCTs evaluating systemic therapy for breast, colorectal (CRC), and non-small-cell lung cancer (NSCLC) published 1975 to 2004 in six major journals were reviewed. Two authors abstracted data regarding trial design, results, and conclusions. Conclusions of authors were graded using a 7-point Likert scale. For each study the effect size for the primary end point was converted to a summary measure.

RESULTS

A total of 321 eligible RCTs were included (48% breast, 24% CRC, 28% NSCLC). Over time, the number and size of RCTs increased considerably. For-profit/mixed sponsorship increased substantially during the study period (4% to 57%; P < .001). There was increasing use of time-to-event measures (39% to 78%) and decreasing use of response rate (54% to 14%) as primary end point (P < .001). Effect size remained stable over the study period. Authors have become more likely to strongly endorse the experimental arm (P = .017). A significant P value for the primary end point and industry sponsorship were each independently associated with endorsement of the experimental agent (odds ratio [OR] = 19.6, 95% CI, 8.9 to 43.1, and OR = 3.5, 95% CI, 1.6 to 7.5, respectively).

CONCLUSION

RCTs in oncology have become larger and are more likely to be sponsored by industry. Authors of modern RCTs are more likely to strongly endorse novel therapies. For-profit sponsorship and statistically significant results are independently associated with endorsement of the experimental arm.

The Completeness of Reporting (CORE) index identifies important deficiencies in observational study conference abstracts

OBJECTIVE

The aim was to develop, test, and apply an index to assess the completeness of reporting in a cohort of observational studies of conference abstracts.

STUDY DESIGN AND SETTING

Using rigorous methods, we reduced 245 items generated by literature review to 48 candidate items. In a random sample of 30 conference abstracts of rituximab for nonHodgkin lymphoma, we developed an item impact score using a survey of abstract stakeholders combined with the prevalence of each of the 48 items. We retained 14 independent items representing completeness of reporting, the CORE-14. Two raters determined the reliability of the instrument. We then applied the CORE-14 in another 78 studies to determine the prevalence of each feature.

RESULTS

Our survey response rate was 83.9% (47/56). Interrater reliability (95% CI) of the CORE-14 instrument was 0.56 (0.25, 0.77), which improved by averaging across scores provided by two raters (0.72 [0.49, 0.86]). Applying the CORE-14 in an additional set of 78 abstracts, six items occurred > or =85% and four items occurred < or =40% of the time.

CONCLUSION

Opportunities to improve conference abstract reporting exist. This scale could guide future conference abstract submissions and aid individuals considering conference abstract data to inform clinical practice, systematic reviews, guidelines, or policy.

Consistency of phase III clinical trial abstracts presented at an annual meeting of the American Society of Clinical Oncology compared with their subsequent full-text publications

PURPOSE

This study aimed to determine the consistency of phase III clinical trial abstracts presented at American Society of Clinical Oncology (ASCO) Annual Meetings compared with their subsequent full-text publications.

METHODS

We identified abstracts describing phase III clinical trials of chemotherapy, chemoradiotherapy, immunotherapy, and hormone therapy presented at the 36th ASCO Annual Meeting in May 2000. We searched MEDLINE and PubMed for all corresponding publications. Data were extracted from the abstracts and publications that met our inclusion criteria.

RESULTS

A total of 192 abstracts were identified. Seventy-four abstracts met our inclusion criteria. Six years after the 2000 ASCO Meeting, 74% of abstracts had corresponding publications. The primary end point was stated in 34% of abstracts and 100% of published papers. The primary end point result differed by more than 5% between the abstract and publication in 42% of comparisons. The statistical significance of the primary end point and study conclusions were consistent between abstracts and subsequent publications in 89% and 91% of the comparisons, respectively. Abstracts selected as plenary or oral presentations were significantly more likely to be published. No factors predicted consistency for primary end point significance and overall conclusion between ASCO abstracts and their journal publications.

CONCLUSION

When carefully selected, ASCO Annual Meeting abstracts of phase III trials consistently reflect final published results, but some differences were observed that warrant caution in using abstract results to shape treatment decisions before full publication.

Statistical Power of Negative Randomized Controlled Trials Presented at American Society for Clinical Oncology Annual Meetings

Purpose

To investigate the prevalence of underpowered randomized controlled trials (RCTs) presented at American Society of Clinical Oncology (ASCO) annual meetings.

Methods

We surveyed all two-arm phase III RCTs presented at ASCO annual meetings from 1995 to 2003 for which negative results were obtained. Post hoc calculations were performed using a power of 80% and an α level of .05 (two sided) to determine sample sizes required to detect small, medium, and large effect sizes. For studies reporting a proportion or time-to-event as primary end point, effect size was expressed as an odds ratio (OR) or hazard ratio (HR), respectively, with a small effect size defined as OR/HR ≥ 1.3, medium effect size defined as OR/HR ≥ 1.5, and large effect size defined as OR/HR ≥ 2.0. Logistic regression was used to identify factors associated with lack of statistical power.

Results

Of 423 negative RCTs for which post hoc sample size calculations could be performed, 45 (10.6%), 138 (32.6%), and 233 (55.1%) had adequate sample size to detect small, medium, and large effect sizes, respectively. Only 35 negative RCTs (7.1%) reported a reason for inadequate sample size. In a multivariable model, studies that were presented at oral sessions (P = .0038), multicenter studies supported by a cooperative group (P < .0001), and studies with time to event as primary outcome (P < .0001) were more likely to have adequate sample size.

Conclusion

More than half of negative RCTs presented at ASCO annual meetings do not have an adequate sample to detect a medium-size treatment effect.

Better reporting of randomized trials in biomedical journal and conference abstracts

Well reported research published in conference and journal abstracts is important as individuals reading these reports often base their initial assessment of a study based on information reported in the abstract. However, there is growing concern about the reliability and quality of information published in these reports. This article provides an overview of research evidence underpinning the need for better reporting of abstracts reported in conference proceedings and abstracts of journal articles; with a particular focus in the area of health care. Where available we highlight evidence which refers specifically to abstracts reporting randomized trials. We seek to identify current initiatives aimed at improving the reporting of these reports and recommend that an extension of the CONSORT Statement (Consolidated Standards of Reporting Trials), CONSORT for Abstracts, be developed. This checklist would include a list of essential items to be reported in any conference or journal abstract reporting the results of a randomized trial.

What is the quality of reporting in weight loss intervention studies? A systematic review of randomized controlled trials

BACKGROUND

Despite the large number of randomized controlled trials (RCTs) assessing weight loss interventions, no study has assessed the quality of reporting in these trials.

PURPOSE

To assess the quality of reporting of RCTs of weight loss interventions and to identify predictors of reporting quality.

METHODS

The RCTs assessed were derived from a published systematic review of trials investigating the efficacy of weight loss interventions. For our study, two reviewers independently rated the quality of reporting in these trials, based on the Consolidated Standards for Reporting of Trials (CONSORT) Statement. We describe the quality of reporting using number (percent) of studies satisfying each of the 44 CONSORT criteria. We use generalized estimating equations (GEE) to fit a multivariable regression model to determine factors that are associated with the overall quality reporting score.

RESULTS

We assessed 63 RCTs, of which 25 were dietary-lifestyle trials, 22 were pharmacological trials and 16 were behavior-cognitive, exercise-lifestyle, or surgical trials. Less than half (46%) of the trials defined the primary outcome of the study; about 10% provided the description of the method of allocation concealment. Multivariable GEE results showed that the sample size, type of intervention (non-pharmacologic trials having lower scores than pharmacologic trials), and publication time relative to the CONSORT Statement publication in 1996 (publications after 1996 having higher scores) were strong predictors of the quality reporting score. Reporting a statistically significant result on the primary outcome was not significantly associated with the quality score.

CONCLUSION

While the overall quality in reporting seemed to have improved since the publication of the revised CONSORT Statement in 1996, the reporting of some key methodologic aspects, such as clear description of primary outcome and method of allocation concealment, still requires improvements. Factors that are significantly associated with the overall quality reporting score can be used as surrogates in the review of protocols to enhance the quality of the final reports.

The International Society for the Study of the Lumbar Spine: research trends over time and publication rates

STUDY DESIGN AND OBJECTIVES

A retrospective review was conducted to assess trends in research selected for podium presentation at ISSLS.

SUMMARY OF BACKGROUND DATA

Abstracts of past papers presented to ISSLS provide quantified documentation of trends in research as well as insight into future directions.

METHODS

A systematic review of abstracts of research presented was performed. Graphical plots in each category of research and aspects of study quality were used to identify any trends. Univariate and multivariate logistical regression analyses were used to identify factors related to eventual publication.

RESULTS

A total of 43% of selected papers focused on the disc, representing the single largest area of research. There is a linear trend over time toward increased selection of basic science papers and decreased clinical papers. Experimental research design is becoming more frequently selected. In 1978, over 80% of papers were purely descriptive. By 2002, 42% of papers were descriptive, whereas 40% incorporated an experimental design. However, over the past decade, there has been a linear increase in the proportion of randomized controlled trials among clinical papers compared with basic science or biomechanical papers. A total of 54% of all abstracts presented recently went on to publication. Independent factors that were associated with significantly (P < 0.05) higher publication rates included use of blinded or independent observers (84.6%), experimental design (68.4%), basic science or biomechanical papers (62.3%), and statistically positive result (61.8%).

CONCLUSION

Overall, the type of research selected for presentation at ISSLS has improved over the years in regards to greater use of experimental and randomized study design. The eventual publication rates compare favorably to other research societies. Factors related to publication have been identified and should be considered in future research.

Case study of the comparison of data from conference abstracts and full-text articles in health technology assessment of rapidly evolving technologies: does it make a difference?

OBJECTIVES

The aim of this study was to examine (i) the consistency of reporting research findings presented in conference abstracts and presentations and subsequent full publications, (ii) the ability to judge methodological quality of trials from conference abstracts and presentations, and (iii) the effect of inclusion or exclusion of data from these sources on the pooled effect estimates in a meta-analysis.

METHODS

This report is a case study of a selected health technology assessment review (TAR) of a rapidly evolving technology that had identified and included a meta-analysis of trial data from conference abstracts and presentations.

RESULTS

The overall quality of reporting in abstracts and presentations was poor, especially in abstracts. There was incomplete or inconsistent reporting of data in the abstract/presentations. Most often inconsistencies were between conference slide presentations and data reported in published full-text articles. Sensitivity analyses indicated that using data only from published papers would not have altered the direction of any of the results when compared with those using published and abstract data. However, the statistical significance of three of ten results would have changed. If conference abstracts and presentations were excluded from the early analysis, the direction of effect and statistical significance would have changed in one result. The overall conclusions of the original analysis would not have been altered.

CONCLUSIONS

There are inconsistencies in data presented as conference abstracts/presentations and those reported in subsequent published reports. These inconsistencies could impact the final assessment results. Data discrepancies identified across sources included in TARs should be highlighted and their impact assessed and discussed. Sensitivity analyses should be carried out with and without abstract/presentation data included in the analysis. Incomplete reporting in conference abstracts and presentations limits the ability of reviewers to assess confidently the methodological quality of trials.

Quality of randomized controlled trials reporting in the primary treatment of brain tumors

PURPOSE

To assess the reporting quality of randomized controlled trials (RCTs) in the primary treatment of brain tumors and to identify significant predictors of quality.

PATIENTS AND METHODS

Two investigators searched MEDLINE, EMBASE, and bibliographies of retrieved articles for RCTs in the primary treatment of brain tumors published between January 1990 and December 2004. We assessed the quality of overall reporting and key methodologic factors reporting (allocation concealment, blinding, and intention to treat [ITT]). Two investigators also rated articles independently using items from the revised Consolidated Standards of Reporting Trials statement. A generalized estimated equation was used to generate regression models that identified significant factors associated with quality of reporting.

RESULTS

We retrieved 74 relevant RCTs that randomly assigned 14,498 brain tumor patients. The quality of overall reporting has improved during the last 15 years, but eight of the 15 methodologic items were reported in less than 50% of trials. In the appraisal of the reporting quality of key methodologies, allocation concealment, blinding, and adherence to the ITT principle were reported in less than 30% of articles. Multivariable regression models revealed that an impact factor more than 1.66, publication after 1995, and sample size more than 280 were significant factors associated with better overall reporting, whereas complete industrial funding, impact factors more than 2.64, and positive primary outcomes were predictors of higher ratings of the three most important methodologic qualities.

CONCLUSION

Despite improvement in general reporting quality, key methodologies that safeguard against biases may still benefit from better description. Significant factors associated with better reporting may act as surrogates for other characteristics.

Quality and content of abstracts in papers reporting about drug exposures during pregnancy

BACKGROUND

Most clinicians read only the abstract of papers in scientific journals. Therefore, it is very important that abstracts contain as much information as possible, to summarize the data succinctly. Our objectives were to evaluate the quality of information in abstracts reporting human fetal outcomes following drug exposure during pregnancy.

METHODS

We developed quality criteria based on previous work, modifying them for use with pregnancy outcomes. Quality scores were calculated as present/absent for all of the equally weighted criteria, then expressed as percentages (present/[present + absent]). We examined a random sample of 100 abstracts obtained through searches of MEDLINE, EMBASE, and the Web of Science databases from 1990 to 2005. Average quality scores were compared across designs (cohort, case-control, meta-analysis, and mixed design) Using Kruskal-Wallis ANOVA and structured/unstructured formats using Student's t test.

RESULTS

The overall average quality was 59.2% +/- 14% (median, 61.5%; range, 15.4-83.3%). Quality was not significantly different across designs (P = .16) or between structured and unstructured abstracts (P = .44). Quality scores increased over time (Rho = 0.23, P = .02). Most frequently absent were baseline risk (94%), drug dose (91%), nonsignificant P values (72%), confounders (69%), significant P values (57%), and risk difference (48%).

CONCLUSIONS

Abstracts provide insufficient information, particularly baseline risk values, for readers to make evidence-based decisions regarding drug use during pregnancy. Efforts need to be made to improve the quality of abstracts and include critical information such as baseline risk.

Abstract reporting in randomized clinical trials of acute lung injury: an audit and assessment of a quality of reporting score

OBJECTIVES

To evaluate the quality of reporting among abstracts of randomized controlled trials (RCTs) in acute lung injury and to highlight important trial information for abstract inclusion.

DESIGN

Audit of published RCT abstracts.

SAMPLE

A total of 56 RCTs, identified in MEDLINE, EMBASE, HEALTHSTAR, CINAHL, and the Cochrane Central Register of Controlled Trials.

MEASUREMENTS AND MAIN RESULTS

We used a research focus group and published literature on suggested content for abstracts of original articles to generate a list of 32 recommended items. The focus group reduced this list to a 20-item long form list of highly relevant criteria and a 12-item short form list of essential criteria for inclusion in abstracts of RCTs in acute lung injury. After pilot testing the abstract appraisal form, we evaluated abstracts independently and in duplicate. We scored the quality of reporting of each abstract by dividing the number of criteria fulfilled by the number applicable. Although abstracts described the study objectives and interventions well and the participants, outcomes, and conclusions to an intermediate extent, key deficiencies were noted in reporting the study methods, setting, and results. Mean quality of reporting scores were significantly higher for structured compared with unstructured abstracts using the 32-item, 20-item, and 12-item lists (p = .008, .014, and <.0001, respectively), especially for abstracts published after 1990 (p = .004, .017, and .001, respectively). The 20-item and 12-item lists correlated well with the 32-item list (r = .89 and .62, respectively) and with one another (r = .73).

CONCLUSIONS

Key design features and results are frequently under-reported in RCT abstracts, particularly among unstructured abstracts. Checklists may aid authors and editors in prioritizing important criteria for inclusion in RCT abstracts.

Quality of cancer care

Quality of cancer care is difficult to define and to evaluate. Here we consider three components represented by the questions: (i) Is the right question treatment being given? (ii) Is it being done well? and (iii) Is the patient being treated as well as the disease? Determining the right treatment requires a hierarchy of evidence from clinical trials, and high quality clinical trials to determine and supplement that evidence. Evidence-based guidelines are then useful in increasing compliance with evidence-based treatment. Quality of delivery of treatment is dependent on volume of practice: patients with rare or complex diseases should be referred to cancer centres. Treatment of the whole patient requires attention to communication, and to the management of symptoms such as pain and fatigue. Cancer care must be of high quality and delivered with compassion.