Association between conflicts of interest and favourable recommendations in clinical guidelines, advisory committee reports, opinion pieces, and narrative reviews: systematic review

OBJECTIVE

To investigate the association between conflicts of interest and favourable recommendations in clinical guidelines, advisory committee reports, opinion pieces, and narrative reviews.

DESIGN

Systematic review.

ELIGIBILITY CRITERIA

Studies that compared the association between conflicts of interest and favourable recommendations of drugs or devices (eg, recommending a drug) in clinical guidelines, advisory committee reports, opinion pieces (eg, editorials), or narrative reviews.

DATA SOURCES

PubMed, Embase, Cochrane Methodology Register (from inception to February 2020), reference lists, Web of Science, and grey literature.

DATA EXTRACTION AND ANALYSIS

Two authors independently extracted data and assessed the methodological quality of the studies. Pooled relative risks and 95% confidence intervals were estimated using random effects models (relative risk >1 indicates that documents with conflicts of interest more often had favourable recommendations than documents with no conflicts of interest). Financial and non-financial conflicts of interest were analysed separately, and the four types of documents were analysed separately (preplanned) and combined (post hoc).

RESULTS

21 studies that analysed 106 clinical guidelines, 1809 advisory committee reports, 340 opinion pieces, and 497 narrative reviews were included. Unpublished data were received for 11 studies (eight full datasets and three summary datasets). 15 studies showed risk of confounding because the compared documents could differ in factors other than conflicts of interest (eg, different drugs used for different populations). The relative risk for associations between financial conflicts of interest and favourable recommendations for clinical guidelines was 1.26 (95% confidence interval 0.93 to 1.69; four studies of 86 clinical guidelines), for advisory committee reports was 1.20 (0.99 to 1.45; four studies of 629 advisory committee reports), for opinion pieces was 2.62 (0.91 to 7.55; four studies of 284 opinion pieces), and for narrative reviews was 1.20 (0.97 to 1.49; four studies of 457 narrative reviews). An analysis of all four types of documents combined supported these findings (1.26, 1.09 to 1.44). In one study that investigated specialty interests, the association between including radiologists as authors of guidelines and recommending routine breast cancer was: relative risk 2.10, 95% confidence interval 0.92 to 4.77; 12 clinical guidelines).

CONCLUSIONS

We interpret our findings to indicate that financial conflicts of interest are associated with favourable recommendations of drugs and devices in clinical guidelines, advisory committee reports, opinion pieces, and narrative reviews. Limitations of this review were risk of confounding in the included studies and the statistical imprecision of individual analyses of each document type. It is not certain whether non-financial conflicts of interest influence recommendations.

SYSTEMATIC REVIEW REGISTRATION

Cochrane Methodology Review Protocol MR000040.

Conflicts of interest in clinical guidelines, advisory committee reports, opinion pieces, and narrative reviews: associations with recommendations

BACKGROUND

Treatment and diagnostic recommendations are often made in clinical guidelines, reports from advisory committee meetings, opinion pieces such as editorials, and narrative reviews. Quite often, the authors or members of advisory committees have industry ties or particular specialty interests which may impact on which interventions are recommended. Similarly, clinical guidelines and narrative reviews may be funded by industry sources resulting in conflicts of interest.

OBJECTIVES

To investigate to what degree financial and non-financial conflicts of interest are associated with favourable recommendations in clinical guidelines, advisory committee reports, opinion pieces, and narrative reviews.

SEARCH METHODS

We searched PubMed, Embase, and the Cochrane Methodology Register for studies published up to February 2020. We also searched reference lists of included studies, Web of Science for studies citing the included studies, and grey literature sources.

SELECTION CRITERIA

We included studies comparing the association between conflicts of interest and favourable recommendations of drugs or devices (e.g. recommending a particular drug) in clinical guidelines, advisory committee reports, opinion pieces, or narrative reviews.

DATA COLLECTION AND ANALYSIS

Two review authors independently included studies, extracted data, and assessed risk of bias. When a meta-analysis was considered meaningful to synthesise our findings, we used random-effects models to estimate risk ratios (RRs) with 95% confidence intervals (CIs), with RR > 1 indicating that documents (e.g. clinical guidelines) with conflicts of interest more often had favourable recommendations. We analysed associations for financial and non-financial conflicts of interest separately, and analysed the four types of documents both separately (pre-planned analyses) and combined (post hoc analysis).

MAIN RESULTS

We included 21 studies analysing 106 clinical guidelines, 1809 advisory committee reports, 340 opinion pieces, and 497 narrative reviews. We received unpublished data from 11 studies; eight full data sets and three summary data sets. Fifteen studies had a risk of confounding, as they compared documents that may differ in other aspects than conflicts of interest (e.g. documents on different drugs used for different populations). The associations between financial conflicts of interest and favourable recommendations were: clinical guidelines, RR: 1.26, 95% CI: 0.93 to 1.69 (four studies of 86 clinical guidelines); advisory committee reports, RR: 1.20, 95% CI: 0.99 to 1.45 (four studies of 629 advisory committee reports); opinion pieces, RR: 2.62, 95% CI: 0.91 to 7.55 (four studies of 284 opinion pieces); and narrative reviews, RR: 1.20, 95% CI: 0.97 to 1.49 (four studies of 457 narrative reviews). An analysis combining all four document types supported these findings (RR: 1.26, 95% CI: 1.09 to 1.44). One study investigating specialty interests found that the association between including radiologist guideline authors and recommending routine breast cancer screening was RR: 2.10, 95% CI: 0.92 to 4.77 (12 clinical guidelines).

AUTHORS' CONCLUSIONS

We interpret our findings to indicate that financial conflicts of interest are associated with favourable recommendations of drugs and devices in clinical guidelines, advisory committee reports, opinion pieces, and narrative reviews. However, we also stress risk of confounding in the included studies and the statistical imprecision of individual analyses of each document type. It is not certain whether non-financial conflicts of interest impact on recommendations.

Antibiotics for recurrent acute pharyngo-tonsillitis: systematic review

The purpose was to determine the current evidence for preferable antibiotic treatment in three common clinical situations with insufficient consensus: Q1: Can antibiotic treatment prevent future attacks of acute pharyngo-tonsillitis (APT) in patients with recurrent APT (RAPT)? Q2: Which antibiotic regimen is preferable in the treatment of APT in patients with RAPT? Q3: Which antibiotic regimen is preferable in the treatment of relapsing APT? Five databases were searched systematically for randomized clinical trials on patients with RAPT with or without current APT or with relapse of APT. Of the unique publications, 643 were found. Five studies addressing Q1 (n = 3) and Q2 (n = 2) met the eligibility criteria. No studies reporting on Q3 were included. Q1: Two studies found that clindamycin and cefpodoxime, respectively, were effective in preventing future APT episodes and in eradicating group A streptococci from the tonsils of RAPT patients. One study found that long-term azithromycin had no effect on the number of APT episodes. Q2: Two studies reported superior clinical and microbiological effects of clindamycin and amoxicillin with clavulanate, respectively, compared to penicillin. The four studies showing superior effects of clindamycin and amoxicillin with clavulanate were assessed to have high risk of bias. Hence, the level of evidence was moderate. There is considerable evidence to suggest that clindamycin and amoxicillin with clavulanate are superior to penicillin with preferable effects on the microbiological flora and the number of future attacks of APT in patients with RAPT. Antibiotic treatment is an option in patients with RAPT, who has contraindications for tonsillectomy.

Prevalence of various etiologies of hearing loss among cochlear implant recipients: Systematic review and meta-analysis

OBJECTIVE

To investigate the etiology of deafness in cochlear implanted children and to address the question whether there is a need for more thorough diagnostics, especially concerning genetics.

DESIGN

Systematic review. Four databases were searched for studies (year 2000-2014) on cochlear implanted children (n > 100). Studies were excluded if etiology had influenced their inclusion criteria. Eligibility and methodological quality were assessed independently by three authors. The studies' description of diagnostic evaluation was categorized in three groups.

STUDY SAMPLE

Sixteen studies were included (5069 children).

RESULTS

The most common etiological categories were 'Unknown' 40.3% (95% CI 32.8 to 48.0), 'Non-syndromic' 22.4% (95% CI 17.1 to 28.2), and 'Postnatal' 11.3% (95% CI 7.2 to 16.2). Studies published after 2006 had a lower proportion of 'Unknown' etiology 35.3% (95% CI 28.0 to 42.8) than older 45.5% (95% CI 31.0 to 60.4). Important information was missing from several studies: 11 (69%) studies did not provide detailed description on diagnostic evaluation of the etiology of deafness and had a higher proportion of 'Unknown' etiology.

CONCLUSIONS

In order to ensure a higher level of comparability in future studies, we recommend agreement upon an international standard of diagnostics and the introduction of an international standard for reporting etiology.

Comparison of results from different imputation techniques for missing data from an anti-obesity drug trial

Background

In randomised trials of medical interventions, the most reliable analysis follows the intention-to-treat (ITT) principle. However, the ITT analysis requires that missing outcome data have to be imputed. Different imputation techniques may give different results and some may lead to bias. In anti-obesity drug trials, many data are usually missing, and the most used imputation method is last observation carried forward (LOCF). LOCF is generally considered conservative, but there are more reliable methods such as multiple imputation (MI).

Objectives

To compare four different methods of handling missing data in a 60-week placebo controlled anti-obesity drug trial on topiramate.

Methods

We compared an analysis of complete cases with datasets where missing body weight measurements had been replaced using three different imputation methods: LOCF, baseline carried forward (BOCF) and MI.

Results

561 participants were randomised. Compared to placebo, there was a significantly greater weight loss with topiramate in all analyses: 9.5 kg (SE 1.17) in the complete case analysis (N = 86), 6.8 kg (SE 0.66) using LOCF (N = 561), 6.4 kg (SE 0.90) using MI (N = 561) and 1.5 kg (SE 0.28) using BOCF (N = 561).

Conclusions

The different imputation methods gave very different results. Contrary to widely stated claims, LOCF did not produce a conservative (i.e., lower) efficacy estimate compared to MI. Also, LOCF had a lower SE than MI.

International or national publication of case reports

INTRODUCTION

Case reports are often regarded as second-class research, but are an important part of medical science as they often present first evidence of new discoveries. We here describe the type of case reports published in a Danish general medical journal.

MATERIAL AND METHODS

We included all case reports published in Ugeskrift for Laeger in 2009. For each report, two authors extracted information on study characteristics and classified the relevance and the role of the report.

RESULTS

We included 139 case reports written in Danish. Thirty-nine (28%) were of general relevance and 100 (72%) of speciality relevance. The median number of authors was three (range: 1-7). The first author was a non-specialist physician in 119 (86%) of the reports and the last author a specialist in 103 (78%). A total of 124 (89%) reports had an educational role, six (4%) dealt with new diseases, two (1%) with new side effects, three (2%) with new mechanisms and four (3%) were curiosities. A total of 59 (42%) reports were surgical, 64 (46%) non-surgical and 16 (12%) paraclinical.

CONCLUSION

We found that most case reports published in Ugeskrift for Laeger were of speciality relevance and had an educational perspective. The journal may consider focusing on cases of more general educational relevance and should also consider whether the current form and language suit the aim and role of the various types of case reports.

Disagreements in meta-analyses using outcomes measured on continuous or rating scales: observer agreement study

OBJECTIVE

To study the inter-observer variation related to extraction of continuous and numerical rating scale data from trial reports for use in meta-analyses.

DESIGN

Observer agreement study.

DATA SOURCES

A random sample of 10 Cochrane reviews that presented a result as a standardised mean difference (SMD), the protocols for the reviews and the trial reports (n=45) were retrieved.

DATA EXTRACTION

Five experienced methodologists and five PhD students independently extracted data from the trial reports for calculation of the first SMD result in each review. The observers did not have access to the reviews but to the protocols, where the relevant outcome was highlighted. The agreement was analysed at both trial and meta-analysis level, pairing the observers in all possible ways (45 pairs, yielding 2025 pairs of trials and 450 pairs of meta-analyses). Agreement was defined as SMDs that differed less than 0.1 in their point estimates or confidence intervals.

RESULTS

The agreement was 53% at trial level and 31% at meta-analysis level. Including all pairs, the median disagreement was SMD=0.22 (interquartile range 0.07-0.61). The experts agreed somewhat more than the PhD students at trial level (61% v 46%), but not at meta-analysis level. Important reasons for disagreement were differences in selection of time points, scales, control groups, and type of calculations; whether to include a trial in the meta-analysis; and data extraction errors made by the observers. In 14 out of the 100 SMDs calculated at the meta-analysis level, individual observers reached different conclusions than the originally published review.

CONCLUSIONS

Disagreements were common and often larger than the effect of commonly used treatments. Meta-analyses using SMDs are prone to observer variation and should be interpreted with caution. The reliability of meta-analyses might be improved by having more detailed review protocols, more than one observer, and statistical expertise.

Industry-supported meta-analyses compared with meta-analyses with non-profit or no support: differences in methodological quality and conclusions

Background

Studies have shown that industry-sponsored meta-analyses of drugs lack scientific rigour and have biased conclusions. However, these studies have been restricted to certain medical specialities. We compared all industry-supported meta-analyses of drug-drug comparisons with those without industry support.

Methods

We searched PubMed for all meta-analyses that compared different drugs or classes of drugs published in 2004. Two authors assessed the meta-analyses and independently extracted data. We used a validated scale for judging the methodological quality and a binary scale for judging conclusions. We divided the meta-analyses according to the type of support in 3 categories: industry-supported, non-profit support or no support, and undeclared support.

Results

We included 39 meta-analyses. Ten had industry support, 18 non-profit or no support, and 11 undeclared support. On a 0–7 scale, the median quality score was 6 for meta-analyses with non-profit or no support and 2.5 for the industry-supported meta-analyses (P < 0.01). Compared with industry-supported meta-analyses, more meta-analyses with non-profit or no support avoided bias in the selection of studies (P = 0.01), more often stated the search methods used to find studies (P = 0.02), searched comprehensively (P < 0.01), reported criteria for assessing the validity of the studies (P = 0.02), used appropriate criteria (P = 0.04), described methods of allocation concealment (P = 0.05), described methods of blinding (P = 0.05), and described excluded patients (P = 0.08) and studies (P = 0.15). Forty percent of the industry-supported meta-analyses recommended the experimental drug without reservations, compared with 22% of the meta-analyses with non-profit or no support (P = 0.57).

In a sensitivity analysis, we contacted the authors of the meta-analyses with undeclared support. Eight who replied that they had not received industry funding were added to those with non-profit or no support, and 3 who did not reply were added to those with industry support. This analysis did not change the results much.

Conclusion

Transparency is essential for readers to make their own judgment about medical interventions guided by the results of meta-analyses. We found that industry-supported meta-analyses are less transparent than meta-analyses with non-profit support or no support.

[Cochrane reviews compared with industry-supported and other meta-analyses of the same drugs--secondary publication]

24 Cochrane reviews were compared with other, closely matched, meta-analyses of the same drugs. The quality score was highest for Cochrane reviews (7 vs. 2, p < 0.01). Of 8 industry-supported reviews, 7 with conclusions all recommended the experimental drug without reservations, compared with none of the Cochrane reviews (p = 0.02) although the estimated treatment effect was similar on average (p = 0.64). Industry-supported reviews were less transparent, had few reservations about methodological limitations of the included trials, and should be read with caution.

Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic review

OBJECTIVE

To compare the methodological quality and conclusions in Cochrane reviews with those in industry supported meta-analyses and other meta-analyses of the same drugs.

DESIGN

Systematic review comparing pairs of meta-analyses that studied the same two drugs in the same disease and were published within two years of each other.

DATA SOURCES

Cochrane Database of Systematic Reviews (2003, issue 1), PubMed, and Embase.

DATA EXTRACTION

Two observers independently extracted data and used a validated scale to judge the methodological quality of the reviews.

RESULTS

175 of 1596 Cochrane reviews had a meta-analysis that compared two drugs. Twenty four meta-analyses that matched the Cochrane reviews were found: eight were industry supported, nine had undeclared support, and seven had no support or were supported by non-industry sources. On a 0-7 scale, the median quality score was 7 for Cochrane reviews and 3 for other reviews (P < 0.01). Compared with industry supported reviews and reviews with undeclared support, Cochrane reviews had more often considered the potential for bias in the review--for example, by describing the method of concealment of allocation and describing excluded patients or studies. The seven industry supported reviews that had conclusions recommended the experimental drug without reservations, compared with none of the Cochrane reviews (P = 0.02), although the estimated treatment effect was similar on average (z = 0.46, P = 0.64). Reviews with undeclared support and reviews with not for profit support or no support had conclusions that were similar in cautiousness to the Cochrane reviews.

CONCLUSIONS

Industry supported reviews of drugs should be read with caution as they were less transparent, had few reservations about methodological limitations of the included trials, and had more favourable conclusions than the corresponding Cochrane reviews.