Systematic review finds that appraisal tools for medical research studies address conflicts of interest superficially

A structured approach to information retrieval improved identification of funding and researchers' conflicts of interest in trials included in Cochrane reviews

OBJECTIVES

To compare the contemporary Cochrane review approach for retrieving information on trial funding and researchers' conflicts of interest with a structured approach for information retrieval.

STUDY DESIGN AND SETTING

Methodological study of 100 Cochrane reviews from August to December 2020 and one randomly selected trial from each review. Reporting of trial funding and researchers' conflicts of interest in reviews was compared with information identified using a structured retrieval process, and time to retrieve information was noted. We also formulated a guide to systematic reviewers for efficient information retrieval.

RESULTS

Sixty-eight of 100 Cochrane reviews reported trial funding and 24 reported trial researchers' conflicts of interest. A simple structured approach, searching only trial publications (including conflicts of interest disclosure forms), identified funding for 16 additional trials and conflicts of interest information for 39 additional trials. A comprehensive structured approach, searching multiple information sources, identified funding for two additional trials and conflicts of interest for 14 additional trials. The median time to retrieve information was 10 minutes per trial (interquartile range: 7-15) for the simple approach and 20 minutes (11-43) for the comprehensive approach.

CONCLUSION

A structured information retrieval approach improves identification of funding and researchers' conflicts of interest in trials included in Cochrane reviews.

Panoramic quality assessment tool for investigator initiated trials

Objectives

Quality can be a challenge for Investigator initiated trials (IITs) since these trials are scarcely overseen by a sponsor or monitoring team. Therefore, quality assessment for departments managing clinical research grants program is important and urgently needed. Our study aims at developing a handy quality assessment tool for IITs that can be applied by both departments and project teams.

Methods

The framework of the quality assessment tool was developed based on the literature studies, accepted guidelines and the Delphi method. A total of 272 ongoing IITs funded by Shanghai non-profit organizations in 2015 and 2016 were used to extract quality indexes. Confirmatory factor analysis (CFA) was used to further evaluate the validity and feasibility of the conceptual quality assessment tool.

Results

The tool consisted of 4 critical quality attributes, including progress, quality, regulation, scientificity, and 13 observed quality indexes. A total of 257 IITs were included in the validity and feasibility assessment. The majority (60.29%) were Randomized Controlled Trial (RCT), and 41.18% were multi-center studies. In order to test the validity and feasibility of IITs quality assessment tool, CFA showed that the model fit the data adequately. (CMIN/DF = 1.868, GFI = 0.916; CFI = 0.936; TLI = 0.919; RMSEA = 0.063; SRMR = 0.076). Different types of clinical studies fit well in the tool. However, RCT scored lower than prospective cohort and retrospective study in enrollment progress (7.02 vs. 7.43, 9.63, respectively).

Conclusion

This study established a panoramic quality assessment tool based on the Delphi method and CFA, and the feasibility and effectiveness of the tool were verified through clinical research examples. The use of this tool can help project management departments effectively and dynamically manage research projects, rationally allocate resources, and ensure the quality of IITs.

Broad Medical Uncertainty and the ethical obligation for openness

This paper argues that there exists a collective epistemic state of 'Broad Medical Uncertainty' (BMU) regarding the effectiveness of many medical interventions. We outline the features of BMU, and describe some of the main contributing factors. These include flaws in medical research methodologies, bias in publication practices, financial and other conflicts of interest, and features of how evidence is translated into practice. These result in a significant degree of uncertainty regarding the effectiveness of many medical treatments and unduly optimistic beliefs about the benefit/harm profiles of such treatments. We argue for an ethical presumption in favour of openness regarding BMU as part of a 'Corrective Response'. We then consider some objections to this position (the 'Anti-Corrective Response'), including concerns that public honesty about flaws in medical research could undermine trust in healthcare institutions. We suggest that, as it stands, the Anti-Corrective Response is unconvincing.

The methodological quality on systematic reviews of surgical randomised controlled trials: A cross-sectional survey

BACKGROUND

Systematic reviews of RCTs have been developed to address end-users' needs and are regarded as the highest level of evidence. Flaws in the design, conduct and analyses of a systematic review can lead to erroneous conclusions and increase the research waste.

OBJECTIVE

We undertook a cross-sectional survey to identify the critical areas of weakness in systematic reviews for surgical interventions by AMSTAR 2.

METHODS

We searched PubMed, EMbase and Cochrane Library to summarize systematic reviews of surgical RCTs published in 2017. The information regarding general characteristics and methodological characteristics were gathered. We conducted descriptive analyses of study characteristics of included systematic reviews and explored the difference among varied methodological quality.

RESULTS

Totally 141 systematic reviews were identified. We found only four reviews (2.8%) were high quality, 3 (2.1%) were moderate quality, 8 (5.7%) were low quality, and the remaining 126 (89.4%) were of critical low quality. The critical weaknesses were lack of pre-registration or published protocols (29.1%), comprehensive literature search (17.7%), lists of excluded studies and reasons for exclusion (19.1%), description of detailed interventions (8.5%), extraction of funding source from included trials (10.6%), and consideration of the risk of bias of included trials when synthesized (16.3%) and interpret (20.6%) the results. Higher methodological quality was only positively associated with Cochrane systematic review.

CONCLUSION

Although two-thirds of included systematic reviews in the field of surgery were published in journals ranking Q1, the methodological quality is suboptimal and needs to be substantially improved. More efforts of multi-disciplinary teams' collaboration, continual education and training, integrally connection between primary studies and systematic review and contributing surgical research to practice should be imperative.

Association between conflicts of interest and favourable recommendations in clinical guidelines, advisory committee reports, opinion pieces, and narrative reviews: systematic review

OBJECTIVE

To investigate the association between conflicts of interest and favourable recommendations in clinical guidelines, advisory committee reports, opinion pieces, and narrative reviews.

DESIGN

Systematic review.

ELIGIBILITY CRITERIA

Studies that compared the association between conflicts of interest and favourable recommendations of drugs or devices (eg, recommending a drug) in clinical guidelines, advisory committee reports, opinion pieces (eg, editorials), or narrative reviews.

DATA SOURCES

PubMed, Embase, Cochrane Methodology Register (from inception to February 2020), reference lists, Web of Science, and grey literature.

DATA EXTRACTION AND ANALYSIS

Two authors independently extracted data and assessed the methodological quality of the studies. Pooled relative risks and 95% confidence intervals were estimated using random effects models (relative risk >1 indicates that documents with conflicts of interest more often had favourable recommendations than documents with no conflicts of interest). Financial and non-financial conflicts of interest were analysed separately, and the four types of documents were analysed separately (preplanned) and combined (post hoc).

RESULTS

21 studies that analysed 106 clinical guidelines, 1809 advisory committee reports, 340 opinion pieces, and 497 narrative reviews were included. Unpublished data were received for 11 studies (eight full datasets and three summary datasets). 15 studies showed risk of confounding because the compared documents could differ in factors other than conflicts of interest (eg, different drugs used for different populations). The relative risk for associations between financial conflicts of interest and favourable recommendations for clinical guidelines was 1.26 (95% confidence interval 0.93 to 1.69; four studies of 86 clinical guidelines), for advisory committee reports was 1.20 (0.99 to 1.45; four studies of 629 advisory committee reports), for opinion pieces was 2.62 (0.91 to 7.55; four studies of 284 opinion pieces), and for narrative reviews was 1.20 (0.97 to 1.49; four studies of 457 narrative reviews). An analysis of all four types of documents combined supported these findings (1.26, 1.09 to 1.44). In one study that investigated specialty interests, the association between including radiologists as authors of guidelines and recommending routine breast cancer was: relative risk 2.10, 95% confidence interval 0.92 to 4.77; 12 clinical guidelines).

CONCLUSIONS

We interpret our findings to indicate that financial conflicts of interest are associated with favourable recommendations of drugs and devices in clinical guidelines, advisory committee reports, opinion pieces, and narrative reviews. Limitations of this review were risk of confounding in the included studies and the statistical imprecision of individual analyses of each document type. It is not certain whether non-financial conflicts of interest influence recommendations.

SYSTEMATIC REVIEW REGISTRATION

Cochrane Methodology Review Protocol MR000040.

Conflicts of interest in clinical guidelines, advisory committee reports, opinion pieces, and narrative reviews: associations with recommendations

BACKGROUND

Treatment and diagnostic recommendations are often made in clinical guidelines, reports from advisory committee meetings, opinion pieces such as editorials, and narrative reviews. Quite often, the authors or members of advisory committees have industry ties or particular specialty interests which may impact on which interventions are recommended. Similarly, clinical guidelines and narrative reviews may be funded by industry sources resulting in conflicts of interest.

OBJECTIVES

To investigate to what degree financial and non-financial conflicts of interest are associated with favourable recommendations in clinical guidelines, advisory committee reports, opinion pieces, and narrative reviews.

SEARCH METHODS

We searched PubMed, Embase, and the Cochrane Methodology Register for studies published up to February 2020. We also searched reference lists of included studies, Web of Science for studies citing the included studies, and grey literature sources.

SELECTION CRITERIA

We included studies comparing the association between conflicts of interest and favourable recommendations of drugs or devices (e.g. recommending a particular drug) in clinical guidelines, advisory committee reports, opinion pieces, or narrative reviews.

DATA COLLECTION AND ANALYSIS

Two review authors independently included studies, extracted data, and assessed risk of bias. When a meta-analysis was considered meaningful to synthesise our findings, we used random-effects models to estimate risk ratios (RRs) with 95% confidence intervals (CIs), with RR > 1 indicating that documents (e.g. clinical guidelines) with conflicts of interest more often had favourable recommendations. We analysed associations for financial and non-financial conflicts of interest separately, and analysed the four types of documents both separately (pre-planned analyses) and combined (post hoc analysis).

MAIN RESULTS

We included 21 studies analysing 106 clinical guidelines, 1809 advisory committee reports, 340 opinion pieces, and 497 narrative reviews. We received unpublished data from 11 studies; eight full data sets and three summary data sets. Fifteen studies had a risk of confounding, as they compared documents that may differ in other aspects than conflicts of interest (e.g. documents on different drugs used for different populations). The associations between financial conflicts of interest and favourable recommendations were: clinical guidelines, RR: 1.26, 95% CI: 0.93 to 1.69 (four studies of 86 clinical guidelines); advisory committee reports, RR: 1.20, 95% CI: 0.99 to 1.45 (four studies of 629 advisory committee reports); opinion pieces, RR: 2.62, 95% CI: 0.91 to 7.55 (four studies of 284 opinion pieces); and narrative reviews, RR: 1.20, 95% CI: 0.97 to 1.49 (four studies of 457 narrative reviews). An analysis combining all four document types supported these findings (RR: 1.26, 95% CI: 1.09 to 1.44). One study investigating specialty interests found that the association between including radiologist guideline authors and recommending routine breast cancer screening was RR: 2.10, 95% CI: 0.92 to 4.77 (12 clinical guidelines).

AUTHORS' CONCLUSIONS

We interpret our findings to indicate that financial conflicts of interest are associated with favourable recommendations of drugs and devices in clinical guidelines, advisory committee reports, opinion pieces, and narrative reviews. However, we also stress risk of confounding in the included studies and the statistical imprecision of individual analyses of each document type. It is not certain whether non-financial conflicts of interest impact on recommendations.

Influence and management of conflicts of interest in randomised clinical trials: qualitative interview study

OBJECTIVE

To characterise and analyse the experiences of trial researchers of if and how conflicts of interest had unduly influenced clinical trials they had worked on, what management strategies they had used to minimise any potential influence, and their experiences and views on conflicts of interest more generally.

DESIGN

Qualitative interview study.

PARTICIPANTS

Trial researchers who had participated in at least 10 clinical trials with methodological or statistical expertise. Researchers differed by geographical location, educational background, and experience with different types of funders. Interviewees were identified by searches on Web of Science and snowball sampling. 52 trial researchers were approached by email; 20 agreed to be interviewed.

SETTING

Interviews conducted by telephone, recorded, transcribed verbatim, imported to NVivo 12, and analysed by systematic text condensation. Semistructured interviews focused on financial and non-financial conflicts of interest.

RESULTS

The interviewees had participated in a median of 37.5 trials and were mainly male physicians who had experience with commercial and non-commercial trial funders. Two predefined themes (influence of conflicts of interest and management strategies) and two additional themes (definition and reporting of conflicts of interest) emerged. Examples of perceived influence of conflicts of interest were: choice of inferior comparator, manipulation of the randomisation process, prematurely stopping the trials, fabrication of data, blocking access to data, and spin (eg, overly favourable interpretation of the results). Examples of strategies to manage conflicts of interest were: disclosure procedures, exclusion of the funder from design and analysis, independent committees, contracts ensuring complete access to the data, and no restriction by the funder on analysis and reporting. Interviewees used different definitions or thresholds for what they considered to be conflicts of interest, and they described different criteria for when to report them. Some interviewees considered non-commercial financial conflicts of interest (eg, funding of trials by governmental health agencies with a political agenda) to be equally or more important than commercial financial conflicts of interest (eg, funding by drug and device companies), but more challenging to report and manage.

CONCLUSION

This study described how trial researchers perceive conflicts of interest unduly influencing clinical trials they had worked on, and the management strategies they used to prevent these influences. The results indicated considerable variability in researchers' understanding of what conflicts of interest are and when they should be reported.