Comparing data accuracy between structured abstracts and full-text journal articles: implications in their use for informing clinical decisions

Spin within systematic review abstracts on antiplatelet therapies after acute coronary syndrome: a cross-sectional study

OBJECTIVES

Spin is a reporting practice in which study results are misrepresented by overestimating efficacy or underestimating harm. Prevalence of spin varies between clinical specialties, and estimates are based almost entirely on clinical trials. Little is known about spin in systematic reviews.

DESIGN

We performed a cross-sectional analysis searching MEDLINE and Embase for systematic reviews and meta-analyses pertaining to antiplatelet therapies following acute coronary syndrome on 2 June 2020. Data were extracted evaluating the presence of spin and study characteristics, including methodological quality as rated by A MeaSurement Tool to Assess systematic Reviews (AMSTAR-2). All data extraction was conducted in a masked, duplicate manner from 2 June 2020 to 26 June 2020.

PARTICIPANTS AND SETTING

Not applicable.

PRIMARY AND SECONDARY OUTCOME MEASURES

We assessed abstracts of systematic reviews on antiplatelet therapy following acute coronary syndrome and evaluated the prevalence of the nine most severe types of spin. We additionally explored associations between spin and certain study characteristics, including quality.

RESULTS

Our searches returned 15 263 articles, and 185 systematic reviews met inclusion criteria. Of these 185 reviews, 31.9% (59/185) contained some form of spin in the abstract. Seven forms of spin (1, 2, 3, 4, 5, 7 and 9) among the nine most severe were identified. No instances of types 6 or 8 were found. There were no statistically significant relationships between spin and the evaluated study characteristics or AMSTAR-2 appraisals.

CONCLUSIONS

Spin was present in abstracts for systematic reviews and meta-analyses; subsequent studies are needed to identify correlations between spin and specific study characteristics. There were no statistically significant associations between spin and study characteristics or AMSTAR-2 ratings; however, implementing changes will ensure that spin is reduced in the field of cardiology as well as other fields of medicine.

Evaluation of reporting quality of abstracts of randomized controlled trials regarding patients with COVID-19 using the CONSORT statement for abstracts

OBJECTIVE

To evaluate the reporting quality of randomized controlled trial (RCT) abstracts regarding patients with coronavirus disease 2019 (COVID-19) and to analyze the factors influencing the quality.

METHODS

The PubMed, Embase, Web of Science, and Cochrane Library databases were searched to collect RCTs on patients with COVID-19. The retrieval time was from inception to December 1, 2020. The CONSORT statement for abstracts was used to evaluate the reporting quality of RCT abstracts.

RESULTS

A total of 53 RCT abstracts were included. The CONSORT statement for abstracts showed that the average reporting rate of all items was 50.2%. The items with a lower reporting quality were mainly the trial design and the details of randomization and blinding (<10%). The mean overall adherence score across all studies was 8.68 ± 2.69 (range 4-13.5). Multivariate linear regression analysis showed that the higher reporting scores were associated with higher journal impact factor (P < 0.01), international collaboration (P = 0.04), and structured abstract format (P < 0.01).

CONCLUSIONS

Although many RCTs on patients with COVID-19 have been published in different journals, the overall quality of reporting in the included RCT abstracts was suboptimal, thus diminishing their potential usefulness, and this may mislead clinical decision-making. In order to improve the reporting quality, it is necessary to promote and actively apply the CONSORT statement for abstracts.

Evaluation of spin in the abstracts of systematic reviews and meta-analyses of treatments and interventions for smoking cessation

INTRODUCTION

Smoking cessation treatments and available evidence continue to evolve. To stay current with the latest research, physicians often refer to abstracts of systematic reviews. Because abstracts of systematic reviews may have direct effects on patient care, the information within them should be free of 'spin'. Spin is a specific way of reporting, intentional or not, to highlight that the beneficial effect of the experimental treatment in terms of efficacy or safety is greater than that shown by the results (i.e. overstate efficacy and/or understate harm).

METHODS

We searched systematic reviews and meta-analyses focused on interventions and treatments for smoking cessation. Full-text screening, data extraction, evaluation of spin, and quality assessment were conducted in masked, duplicate fashion. Study and journal characteristics were also recorded to determine whether they were associated with the presence of spin.

RESULTS

A total of 200 systematic reviews that met inclusion criteria were included in the final analyses. Spin occurred in 3.5% (7/200) of the systematic review abstracts included in our sample. No study characteristics were significantly associated with spin.

CONCLUSIONS

Of the reviewed abstracts in systematic reviews and meta-analyses, 96.5% of those that focused on smoking cessation were free of spin. However, the existence of spin warrants further steps to improve the scientific accuracy of abstracts on smoking cessation treatments. By identifying and acknowledging the presence of spin in systematic reviews, we hope to increase awareness about reporting practices in an ultimate effort to improve the integrity of scientific research as a whole.

Update on the global neurosurgery movement: A systematic review of international vernacular, research trends, and authorship

In 2015, key global and neurosurgical organizations increased collaboration to improve neurosurgical care access, delivery, and outcomes, particularly in low- to middle-income countries (LMICs); sparking what has been termed the global neurosurgery movement. The authors sought to assess trends in usage of the term 'global neurosurgery' in academic literature with particular focus on author affiliations, world regions most frequently discussed, and topics of research performed. A PubMed search for articles indexed as 'global neurosurgery' was completed yielding 277 articles which met inclusion criteria. It was found that over time, use of the term 'global neurosurgery' has increased, with increasing growth notable starting in the year 2008 and continuing into October 2019. Statistical comparisons showed authors with affiliated global neurosurgery centers were more likely to publish studies related to the continent of Africa (47.4% vs 15.9%, p < 0.001), and less likely to focus on countries in Asia (2.6% vs 20.9%, p = 0.023). Use of the term 'global neurosurgery' in the article abstract/title/keywords was associated with focus on LMICs (18.6% vs. 5.1%, p = 0.006). Use of the term 'global neurosurgery' was associated with workforce and capacity as research topics (41.9% vs 22.6%, p = 0.036). While fairly new, the global neurosurgery movement has seen a rapid increase in publications utilizing the term 'global neurosurgery.' Articles frequently have focused on collaborative, targeted workforce capacity building in LMICs. We encourage the development of more global neurosurgery academic centers, especially in non-USA countries, to continue this momentum.

How about the reporting quality of case reports in nursing field?

BACKGROUND

As a significantly important part of clinical practice, the professional nursing process can be advanced in many ways. Despite the fact that case reports are regarded to be of a lower quality grade in the hierarchy of evidence, one of the principles of evidence-based medicine is that decision-making should be based on a systematic summary of evidence. However, the evidence on the reporting characteristics of case reports in the nursing field is deficient.

AIM

To use the CARE guidelines to assess reporting quality and factors influencing the quality of case reports in the nursing field.

METHODS

Nursing science citation indexed (SCI-indexed) journals were identified from the professional website. Each of the identified journals was searched on their website for articles published before December 2017. Twenty-one sub-items on the CARE checklist were recorded as “YES”, “PARTLY”, or “NO” according to information reported by the included studies. The responses were assigned corresponding scores of 1, 0.5, and 0, respectively. The overall score was the sum of the 21 sub-items and was defined as “high” (more than 15), “medium” (10.5 to 14.5), and “low” (less than 10). The means, standard deviations, odds ratios (OR), and the associated 95% confidence interval (CI) were determined using Stata 12.0 software.

RESULTS

Ultimately, 184 case reports from 16 SCI-indexed journals were identified, with overall scores ranging from 6.5 to 18 (mean = 13.6 ± 2.3). Of the included case reports, 10.3% were regarded low-quality, 52.7% were considered middle-quality, and 37% were regarded high-quality. There were statistical differences in the mean overall scores of the included case reports with funding versus those without funding (14.2 ± 1.7 vs 13.6 ± 2.4, respectively; P = 0.4456) and journal impact factor < 1.8 versus impact factor ≥ 1.8 (13.3 ± 2.3 vs 13.6 ± 2.4, respectively; P = 0.4977). Five items from the CARE guidelines, 5a (Patient), 6 (Clinical findings), 8c (Diagnostic reasoning), 9 (Therapeutic intervention), and 11d (The main take-away lessons) were well-reported (Reporting rate more than 90%) in most of the included case reports. However, only three items, 2 (Keywords, OR = 0.42, 95%CI: 0.19-0.92, P = 0.03), 4 (Introduction, OR = 0.35, 95%CI: 0.15-0.83, P = 0.017), and 11b (The relevant medical literature, OR = 0.19, 95%CI: 0.06-0.56, P = 0.003) were considered better-reported after the CARE guidelines published in 2013.

CONCLUSION

The reporting quality of case reports in the nursing field apparently has not improved since the publication of the CARE guidelines.

Abstracts of low back pain trials are poorly reported, contain spin of information and are inconsistent with the full text: An overview study

OBJECTIVE

To investigate trials abstracts evaluating treatments for low back pain with regards to completeness of reporting, spin (i.e., interpretation of study results that overemphasizes the beneficial effects of the intervention), and inconsistencies in data with the full text.

DATA SOURCES

The search was performed on Physiotherapy Evidence Database (PEDro) in February 2016.

STUDY SELECTION

This is an overview study of a random sample of 200 low back pain trials published between 2010 and 2015. The languages of publication were restricted to English, Spanish and Portuguese.

DATA EXTRACTION

Completeness of reporting was assessed using the CONSORT for Abstracts checklist (CONSORT-A). Spin was assessed using a SPIN-checklist. Consistency between abstract and full text were assessed by applying the assessment tools to both the abstract and full text of each trial and calculating inconsistencies in the summary score (paired t test) and agreement in the classification of each item (Kappa statistics). Methodological quality was analyzed using the total PEDro score.

DATA SYNTHESIS

The mean number of fully reported items for abstracts using the CONSORT-A was 5.1 (SD 2.4) out of 15 points and the mean number of items with spin was 4.9 (SD 2.6) out of 7 points. Abstract and full text scores were statistically inconsistent (P=0.01). There was slight to moderate agreement between items of the CONSORT-A in the abstracts and full text (mean Kappa 0.20 SD 0.13) and fair to moderate agreement for items of the SPIN-checklist (mean Kappa 0.47 SD 0.09).

CONCLUSIONS

The abstracts were incomplete, with spin and inconsistent with the full text. We advise health care professionals to avoid making clinical decisions based solely upon abstracts. Journal editors, reviewers and authors are jointly responsible for improving abstracts, which could be guided by amended editorial policies.

Clinicians' perceptions of usefulness of the PubMed4Hh mobile device application for clinical decision making at the point of care: a pilot study

BACKGROUND

Although evidence-based practice in healthcare has been facilitated by Internet access through wireless mobile devices, research on the effectiveness of clinical decision support for clinicians at the point of care is lacking. This study examined how evidence as abstracts and the bottom-line summaries, accessed with PubMed4Hh mobile devices, affected clinicians' decision making at the point of care.

METHODS

Three iterative steps were taken to evaluate the usefulness of PubMed4Hh tools at the NIH Clinical Center. First, feasibility testing was conducted using data collected from a librarian. Next, usability testing was carried out by a postdoctoral research fellow shadowing clinicians during rounds for one month in the inpatient setting. Then, a pilot study was conducted from February, 2016 to January, 2017, with clinicians using a mobile version of PubMed4Hh. Invitations were sent via e-mail lists to clinicians (physicians, physician assistants and nurse practitioners) along with periodic reminders. Participants rated the usefulness of retrieved bottom-line summaries and abstracts and indicated their usefulness on a 7-point Likert scale. They also indicated location of use (office, rounds, etc.).

RESULTS

Of the 166 responses collected in the feasibility phase, more than half of questions (57%, n = 94) were answerable by both the librarian using various resources and by the postdoctoral research fellow using PubMed4Hh. Sixty-six questions were collected during usability testing. More than half of questions (60.6%) were related to information about medication or treatment, while 21% were questions regarding diagnosis, and 12% were specific to disease entities. During the pilot study, participants reviewed 34 abstracts and 40 bottom-line summaries. The abstracts' usefulness mean scores were higher (95% CI [6.12, 6.64) than the scores of the bottom-line summaries (95% CI [5.25, 6.10]). The most frequent reason given was that it confirmed current or tentative diagnostic or treatment plan. The bottom-line summaries were used more in the office (79.3%), and abstracts were used more at point of care (51.9%).

CONCLUSIONS

Clinicians reported that retrieving relevant health information from biomedical literature using the PubMed4Hh was useful at the point of care and in the office.

Reporting Quality of Randomized Controlled Trials of Periodontal Diseases in Journal Abstracts-A Cross-sectional Survey and Bibliometric Analysis

OBJECTIVE

Randomized controlled trials (RCTs) by proper design, conduct, analysis, and reporting provide reliable information in clinical care. Reporting of RCT abstracts is of equal importance as there is evidence that many clinicians will change their clinical decisions based on RCT abstracts. The reporting quality of RCT abstracts has been suboptimal. It is not clear whether the reporting quality is related to the journal metrics. The main objective of this study is to conduct a cross-sectional survey to evaluate the reporting quality of RCTs of periodontal diseases in journal abstracts and to perform a bibliometric analysis. The null hypothesis was that there is no association between the journal metrics (5-year impact factor, Eigenfactor score, and Article Influence Score), abstract metrics (word count, and number of authors), journal endorsement of Consolidated Standards of Reporting Trials (CONSORT), and the overall quality of reporting of CONSORT RCT abstract-modified checklist questions.

MATERIALS

CONSORT RCT abstract extension checklist with explanation and elaboration was used and modified to assess the quality of reporting of RCT abstracts of periodontal diseases in the journal abstracts in the year 2012. Bibliometric analysis of journal metrics (5-year impact factor, Eigenfactor score, and Article Influence Score) and abstract metrics (number of authors and abstract word count), the geographic distribution, and the CONSORT-endorsing journal abstracts was compared with the reporting quality of RCT abstracts in periodontal diseases. Calibration and intrarater agreement were done before the data collection and analysis. A second reviewer was consulted for independent evaluation and clarification as needed. For descriptive analysis, the values of continuous variables were expressed as median and interquartile ranges (IQRs) and as proportion percent for binary categorical variables. For association analysis between the binary (yes/no) response variable and the continuous variable, the Mann-Whitney test (for independent samples) was used. For examining the association between 2 categorical variables, Fisher's exact test was used. The chi-square test was performed to examine the association between 2 sets of binary response variables (yes/no). A P value of < .05 was considered statistically significant. All analyses were conducted using SAS, version 9.4.

RESULTS

A total of 198 RCT abstracts of periodontal diseases in the year 2012 from 57 journals were included in the study. Fifteen journals, listed as endorsers of CONSORT, contributed 108 RCT abstracts. Four journals (Journal of Periodontology, Journal of Clinical Periodontology, Clinical Oral Implants Research, and European Journal of Oral Implantology) contributed 84 of 198 RCT abstracts in 2012. European countries contributed the majority (n = 81, 40.91%) of RCT abstracts. Among 31 countries in this study, United States contributed the most RCTs (n = 28, 14.14%) followed by India (24, 12.12%), Italy (n = 22, 11.11%), and Brazil (n = 20, 10.1%). The frequency of journal metrics were 5-year impact factor (median 2.316; IQR: 1.439-2.970); Eigenfactor score (0.00474; 0.00202-0.01395); and Article Influence Score (0.553; 0.382-0.755). The number of authors in 198 RCT abstracts ranged between 2 and 20 (median n = 5, IQR: 4-6), whereas the word count ranged between 48 and 569 (median 235, IQR: 205-269). All RCT abstracts reported the experimental interventions (checklist question #5, frequency 100%). Some items were almost always reported-participant eligibility criteria (#3, 99%); comparison interventions (#6, 99.5%); specific objective or hypothesis (#7, 99.5%); primary outcome (#8, 99.5%); and reporting trial results as a summary (#16, 98.5%). All RCT abstracts never reported how the allocations were concealed (#11, 0) and the source of funding for the trials (#23, 0). Some items were almost always never reported-the number of participants included in the analysis for each intervention (#15, 2%); trial registration number (#21, 2.5%); name of trial register (#22, 2.5%); and how the randomization or sequence generation was done (#22). Dismal reporting was noted in many checklist questions including the identification of the study as randomized in the title #1, 51%; design of the trial #2, 32.8%; trial setting #4, 3.5%; randomization #10, 3.5%; blinding #12, 21.7%; details about blinding #13, 8.1%; number of participants randomized to each intervention #14, 26.3%; effect size #17, 13.6%; precision of the estimate of the effect #18, 6.1%; and adverse effects #19, 14.1%. Strikingly, there was a very high reporting of statistical significance #25, 92.4%. European countries, in particular, reported relatively better than other countries in essential questions such as #17 effect size reporting, and #18 precision (uncertainty), which have been largely unreported by rest of the countries. Finally, despite the majority of RCTs published in 2012 were by CONSORT-endorsing journals, there was no difference in the quality of reporting in majority of checklist items when compared with journals not listed as CONSORT endorsers. With few exceptions, there was no statistically significant association between the majority of the CONSORT RCT abstract checklist questions and the journal metrics and abstract metrics analyzed in this study. Unexpectedly, lower ranking journals in journal metrics reported certain essential checklist questions relatively better.

CONCLUSION

The reporting quality of RCT of periodontal diseases in the journal abstracts published in 2012 needs substantial improvement. These items have been laid out in this study to help all stakeholders-authors, clinicians, researchers, peer reviewers, journal editors, and publishers to take note and help with the improvement of the same. Despite few significant associations in the bibliometric factors analyzed with better reporting, the results overall led to the failure to reject the null hypothesis that there is no association between the journal metrics, word count, and number of authors and the quality of reporting of CONSORT RCT abstract-modified checklist questions.

The over-representation of significant p values in abstracts compared to corresponding full texts: A systematic review of surgical randomized trials

Background

Abstracts are often the only read summaries of research findings, and it is essential that they accurately represent of the contents of the full text of the randomised control trial (RCT). We investigated whether outcomes in surgical trials were selectively reported in abstracts based on their statistical significance.

Objective

To compare the proportion of significant p-values reported in abstracts to their corresponding full texts in surgical RCTs.

Method

A Meta-analysis of 350 full text RCTs conducted on humans that compared a surgical intervention to any other intervention. An electronic search of MEDLINE, EMBASE, and the Cochrane Central Register of Controlled Trials (CENTRAL) was conducted. All outcomes were extracted from the abstract and the full text. Frequency histograms were used to plot the distribution of numerically reported p-values across the statistical significance spectrum. For each RCT, a 2 × 2 table was populated with that trial's outcomes and whether the outcome was statistically significant (p < 0.05). From each 2 × 2 table, an odds ratio (OR) was calculated describing the association between statistical significance, and reporting in the abstract. ORs were pooled in random effects meta-analysis for an overall estimate of the association.

Results

A total of 8258 reported outcomes were included. Outcomes reported in a surgical RCT abstract had three times the odds of being significant when compared to the corresponding full text (OR = 3.0, 95% confidence interval 2.5–3.6, p < 0.001). This finding was consistent and not subject to heterogeneity (I² = 0%). Both histograms demonstrated a large drop in the frequency of reported p values between 0.04 and 0.05, and after the 0.06 thresholds.

Conclusions

Data presented in abstracts is biased to statistically significant outcomes. Clinicians and policy makers should do not rely solely on information presented in abstracts for their decision-making.

Cancer studies based on secondary data analysis of the Taiwan's National Health Insurance Research Database: A computational text analysis and visualization study

There has been a surge in the academic publication output based on secondary analyses of the data from the Taiwan's National Health Insurance claim records. It has become a challenge to comprehend such a rapid expansion of the literature. Therefore, this study aimed to explore the conceptual content of National Health Insurance Research Database-based cancer research, using the abstract of articles extracted from PubMed between 2002 and 2015. Search terms including "National Health Insurance Research Database (NHIRD) AND Taiwan," "Taiwan AND population-based," and "Taiwan AND nationwide" were used to search in PubMed with the publication date limited to between 1997 and 2015. The retrieved articles were manually screened to retain only those that were cancer-related and were based on secondary data analysis of the NHIRD. A total 589 articles were selected for subsequent text mining using the R software. Among the 589 articles, the top 5 most studied cancer types were breast (16.3%), lung (11.4%), colorectal (10.4%), liver (8.3%), and prostate (7.5%). The articles that received the highest number of citations by PubMed Central articles were cited 92 times. The top 3 most frequently occurred keywords in the abstracts of the 589 articles were cancer, patient, and risk, with 3670, 2535, and 1652 times, respectively. Analysis of key conception indicated that the most common conceptions were diabetes, survival, breast cancer, lung cancer, and colorectal cancer. In conclusion, in this study of 589 published articles on secondary data analysis of the NHIRD, indexed by PubMed between 2002 and 2015, we found that while the risk factors of cancer, treatment of cancer, and survival of cancer patients were popular research topics, end-of-life cancer care issues were less studied. Further studies should explore these areas since they are as important as treatment of the disease itself for many patients.

Trial Designs and Characteristics in Laser Studies in Dermatology: A Systematic Review

BACKGROUND

Lasers are increasingly used in Dermatology. Clinical studies are among the best means to assess their efficacy.

OBJECTIVE

To obtain an overview of therapeutic laser studies in the field of Dermatology, the authors conducted a systematic review to describe the types of study published over the past 25 years, and their reporting in the abstracts.

METHODS

The authors selected interventional studies that evaluated laser treatment on human subjects with skin diseases. Reviews or single-case reports were excluded. Journal characteristics, study design, and trial characteristics were recorded and analyzed according to trends over time and journal impact factor ratings.

RESULTS

Of the 681 studies included, 57% were comparative, of which 46% were randomized, controlled studies. Trial duration was clearly mentioned in 63% of the abstracts. Trial duration was generally under 6 months (78%). Some parameters were rarely mentioned as follows: the number of patients lost to follow-up (only 12%), blinding (21%), and a clearly stated main outcome (36%). Over the 25 years, there was a significant trend toward an increased proportion of randomized, controlled trials, from 15% between 1990 and 2000 to 41% between 2010 and 2015 (p < .001).

CONCLUSION

This systematic review highlights areas for improvement in laser studies in Dermatology.

If the results of an article are noteworthy, read the entire article; do not rely on the abstract alone

Clinicians typically update their knowledge by reading articles on the Internet. Easy access to the articles' abstracts and a lack of time to access other information sources creates a risk that therapeutic or diagnostic decisions will be made after reading just the abstracts. Occasionally, however, the abstracts of articles from clinical trials that have not obtained statistically significant differences in the primary study endpoint have reported other positive results, for example, of a secondary endpoint or a subgroup analysis. The article, however, correctly reports all results, including those of the primary endpoint. In the abstract, the safety information of the experimental treatment is usually deficient. The whole article should be read if a clinical decision is to be made.

Improvement in the quality of abstracts in major clinical journals since CONSORT extension for abstracts: a systematic review

BACKGROUND

We sought to determine if the publication of the Consolidated Standards of Reporting Trials (CONSORT)(1) extension for abstracts in 2008 had led to an improvement in reporting abstracts of randomized controlled trials (RCTs).(2) METHODS: We searched PubMed for RCTs published in 2007 and 2012 in top-tier general medicine journals. A random selection of 100 trial abstracts was obtained for each year. Data were extracted in duplicate on the adherence to the CONSORT extension for abstracts. The primary outcome was the mean number of items reported and the secondary outcome was the odds of reporting each item. We also estimated incidence rate ratios (IRRs).(3) RESULTS: Significantly more checklist items were reported in 2012 than in 2007: adjusted mean difference was 2.91 (95% confidence interval [CI](4) 2.35, 3.41; p<0.001). In 2012 there were significant improvements in reporting the study as randomized in the title, describing the trial design, the participants, and objectives and blinding. In the Results section, trial status and numbers analyzed were also reported better. The IRRs were significantly higher for 2012 (IRR 1.32; 95% CI 1.25, 1.39; p<0.001) and in multisite studies compared to single site studies (IRR 1.08; 95% CI 1.03, 1.15; p=0.006).

CONCLUSIONS

There was a significant improvement in the reporting of abstracts of RCTs in 2012 compared to 2007. However, there is still room for improvement as some items remain under-reported.