1
|
Lan RH, Paranjpe I, Saeed M, Perez MV. Characteristics of contemporary atrial fibrillation clinical trials and their association with industry sponsorship. Heart Rhythm 2024:S1547-5271(24)00236-4. [PMID: 38453036 DOI: 10.1016/j.hrthm.2024.03.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/23/2024] [Revised: 02/28/2024] [Accepted: 03/01/2024] [Indexed: 03/09/2024]
Abstract
BACKGROUND Industry sponsorship is an important source of funding for atrial fibrillation (AF) clinical trials, the implications of which have not been analyzed. OBJECTIVE The purpose of this study was to determine the characteristics of contemporary AF clinical trials and to evaluate their association with funding source. METHODS We systematically assessed all completed AF trials registered in the ClinicalTrials.gov database between conception to October 31, 2023, and extracted publicly available information including funding source, trial size, demographic distribution, intervention, location, and publication status. Trial characteristics were compared using the Wilcoxon rank-sum test and Fisher exact test for continuous and categorical variables, respectively. RESULTS Of the 253 clinical trials assessed, 171 (68%) reported industry funding. Industry funding was associated with a greater median number of patients enrolled (172 vs 80; P <.001), publication rate (56.7% vs 42.7%; P = .04), probability of being product-focused (48.0% vs 24.4%; P <.001), and multicontinental recruitment location (25.2% vs 2.4%; P <.001) when compared to nonindustry-funded trials. However, industry funding was not associated with a significant difference in median impact factor (7.7 vs 7.7; P = .723). The overall proportion of industry-funded trials did not change over time (P = 1). CONCLUSION Industry-funded clinical trials in AF often are larger, more frequently published, multicontinental, and product-focused. Industry funding was found to be associated with significant differences in study enrollment and publication metrics.
Collapse
Affiliation(s)
- Roy H Lan
- Department of Medicine, Stanford University School of Medicine, Stanford, California
| | - Ishan Paranjpe
- Department of Medicine, Stanford University School of Medicine, Stanford, California
| | - Mohammad Saeed
- Department of Cardiology, Texas Heart Institute, Houston, Texas; Center for Cardiac Arrhythmias and Electrophysiology, Texas Heart Institute, Houston, Texas
| | - Marco V Perez
- Cardiovascular Institute, Stanford University, Stanford, California; Stanford Center for Inherited Cardiovascular Disease, Stanford, California; Division of Cardiovascular Medicine, Department of Medicine, Stanford University School of Medicine, Stanford, California.
| |
Collapse
|
2
|
Urlings MJE, Duyx B, Swaen GMH, Bouter LM, Zeegers MPA. Determinants of Citation in Epidemiological Studies on Phthalates: A Citation Analysis. SCIENCE AND ENGINEERING ETHICS 2020; 26:3053-3067. [PMID: 32789752 PMCID: PMC7755632 DOI: 10.1007/s11948-020-00260-y] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/31/2020] [Accepted: 08/06/2020] [Indexed: 05/04/2023]
Abstract
Citing of previous publications is an important factor in knowledge development. Because of the great amount of publications available, only a selection of studies gets cited, for varying reasons. If the selection of citations is associated with study outcome this is called citation bias. We will study determinants of citation in a broader sense, including e.g. study design, journal impact factor or the funding source of the publication. As a case study we assess which factors drive citation in the human literature on phthalates, specifically the metabolite mono(2-ethylhexyl) phthalate (MEHP). A systematic literature search identified all relevant publications on human health effect of MEHP. Data on potential determinants of citation were extracted in duplo. Specialized software was used to create a citation network, including all potential citation pathways. Random effect logistic regression was used to assess whether these determinants influence the likelihood of citation. 112 Publications on MEHP were identified, with 5684 potential citation pathways of which 551 were actual citations. Reporting of a harmful point estimate, journal impact factor, authority of the author, a male corresponding author, research performed in North America and self-citation were positively associated with the likelihood of being cited. In the literature on MEHP, citation is mostly driven by a number of factors that are not related to study outcome. Although the identified determinants do not necessarily give strong indications of bias, it shows selective use of published literature for a variety of reasons.
Collapse
Affiliation(s)
- Miriam J E Urlings
- Nutrition and Translational Research in Metabolism, School NUTRIM, Maastricht University, Maastricht, The Netherlands.
| | - Bram Duyx
- Nutrition and Translational Research in Metabolism, School NUTRIM, Maastricht University, Maastricht, The Netherlands
| | - Gerard M H Swaen
- Nutrition and Translational Research in Metabolism, School NUTRIM, Maastricht University, Maastricht, The Netherlands
| | - Lex M Bouter
- Department of Epidemiology and Data Science, Amsterdam University Medical Centers, Amsterdam, The Netherlands
- Department of Philosophy, Faculty of Humanities, Vrije Universiteit, Amsterdam, The Netherlands
| | - Maurice P A Zeegers
- Nutrition and Translational Research in Metabolism, School NUTRIM, Maastricht University, Maastricht, The Netherlands
- Care and Public Health Research Institute, School CAPHRI, Maastricht University, Maastricht, The Netherlands
| |
Collapse
|
3
|
Duyx B, Urlings MJE, Swaen GMH, Bouter LM, Zeegers MP. Determinants of citation in the literature on diesel exhaust exposure and lung cancer: a citation analysis. BMJ Open 2020; 10:e033967. [PMID: 33033008 PMCID: PMC7542959 DOI: 10.1136/bmjopen-2019-033967] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/02/2022] Open
Abstract
OBJECTIVES Epidemiological research on the association between diesel exhaust exposure and lung cancer risk has some methodological challenges that give rise to different conclusions and intense debates. This raises the question about the role of selective citation and of citation bias in particular. Our aim was to investigate the occurrence and prevalence of selective citation in this field. DESIGN Citation analysis. SETTING Web of Science Core Collection. PARTICIPANTS We identified 96 publications in this network, with 4317 potential citations. For each publication, we extracted characteristics such as study conclusion and funding source. Some of these characteristics are related to the study content: study design, sample size, method of diesel exposure assessment, type of diesel technology under investigation, and whether smoking had been adjusted for. PRIMARY AND SECONDARY OUTCOME MEASURES Whether a citation occurs or not, measured and analysed according to the preregistered protocol. Exploratively we analysed the association between funding source and study conclusion. RESULTS Methodological content of a study was clearly related to citation, studies using more sophisticated methods were more likely to be cited. There was some evidence for citation bias: supportive publications had a higher chance of being cited than non-supportive ones, but after adjustment for study quality, this effect decreased substantially (adjusted OR 1.3, 95% CI 1.0 to 1.7). Explorative analyses indicated that three quarters of non-profit funded publications had a supportive study conclusion against only one quarter of the industry-funded publications. CONCLUSIONS There is evidence for selective citation within this field, but the evidence for citation bias was weak. It seems that factors related to the methodology had more impact on citation than the conclusion of a study. Interestingly, publications that were funded by industry were more skeptical about a causal relationship between diesel exhaust and lung cancer compared to non-profit-funded publications.
Collapse
Affiliation(s)
- Bram Duyx
- CAPHRI School for Public Health and Primary Care, Department of Genetics and Cell Biology, Maastricht University, Maastricht, Limburg, The Netherlands
| | - Miriam J E Urlings
- CAPHRI School for Public Health and Primary Care, Department of Genetics and Cell Biology, Maastricht University, Maastricht, Limburg, The Netherlands
| | - Gerard M H Swaen
- CAPHRI School for Public Health and Primary Care, Department of Genetics and Cell Biology, Maastricht University, Maastricht, Limburg, The Netherlands
| | - Lex M Bouter
- Department of Epidemiology and Biostatistic, Amsterdam University Medical Centres, Location VUmc, Amsterdam, Noord-Holland, The Netherlands
| | - Maurice P Zeegers
- CAPHRI School for Public Health and Primary Care, Department of Genetics and Cell Biology, Maastricht University, Maastricht, Limburg, The Netherlands
| |
Collapse
|
4
|
Abstract
The number of citations a publication receives has been regarded as one measure of its importance and clinical impact. However, studies have yet to investigate which characteristics are predictors of citation rates within the spine subspecialty literature. To explore this topic, all articles published in 2010 in Spine and from 2010 to 2011 in The Spine Journal and the Journal of Neurosurgery: Spine were reviewed. The Web of Science search engine was used to determine the number of times each article was cited in the 5 years following its publication. Sample characteristics were collected and were compared with a χ test for differences Multivariate logistic regression was utilized to determine if collected study characteristics were associated with achievement of citation frequency higher than the median for the entire study sample. Among the 927 articles analyzed, the 5-year citation number ranged from 0 to 125, with a median of 8 (interquartile range: 4-16). Upon multivariate analysis, the following were identified as predictors of citation number higher than the median: North American origin (P=0.014), sample size >30 (P<0.001), study topic (P<0.050), and publication in the Journal of Neurosurgery: Spine (P<0.001). Practitioners and research personnel can use these findings to help elucidate which factors might affect the potential impact and overall reach of their work in the spine literature.
Collapse
|
5
|
Duyx B, Urlings MJE, Swaen GMH, Bouter LM, Zeegers MP. Selective citation in the literature on the hygiene hypothesis: a citation analysis on the association between infections and rhinitis. BMJ Open 2019; 9:e026518. [PMID: 30782945 PMCID: PMC6377569 DOI: 10.1136/bmjopen-2018-026518] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/28/2022] Open
Abstract
OBJECTIVE Our objective was to assess the occurrence and determinants of selective citation in scientific publications on Strachan's original hygiene hypothesis. His hypothesis states that lack of exposure to infections in early childhood increases the risk of rhinitis. SETTING Web of Science Core Collection. PARTICIPANTS We identified 110 publications in this network, consisting of 5551 potential citations. PRIMARY AND SECONDARY OUTCOME MEASURES Whether a citation occurs or not, measured and analysed according to the preregistered protocol. RESULTS We found evidence for citation bias in this field: publications supportive of the hypothesis were cited more often than non-supportive publications (OR adjusted for study design [adjOR] 2.2, 95% CI 1.6 to 3.1), and the same was the case for publications with mixed findings (adjOR 3.1, 95% CI 2.2 to 4.5). Other relevant determinants for citation were type of exposure, specificity, journal impact factor, authority and self-citation. Surprisingly, prospective cohort studies were cited less often than other empirical studies. CONCLUSIONS There is clear evidence for selective citation in this research field, and particularly for citation bias.
Collapse
Affiliation(s)
- Bram Duyx
- Nutrition and Translational Research in Metabolism (School NUTRIM), Maastricht University, Maastricht, Limburg, The Netherlands
| | - Miriam J E Urlings
- Nutrition and Translational Research in Metabolism (School NUTRIM), Maastricht University, Maastricht, Limburg, The Netherlands
| | - Gerard M H Swaen
- Nutrition and Translational Research in Metabolism (School NUTRIM), Maastricht University, Maastricht, Limburg, The Netherlands
| | - Lex M Bouter
- Department of Epidemiology and Biostatistics, Amsterdam University Medical Center, Amsterdam, The Netherlands
- Department of Philosophy, Faculty of Humanities, Vrije Universiteit, Amsterdam, Noord-Holland, The Netherlands
| | - Maurice P Zeegers
- Nutrition and Translational Research in Metabolism (School NUTRIM), Maastricht University, Maastricht, Limburg, The Netherlands
- Care and Public Health Research Institute (School CAPHRI), Maastricht University, Maastricht, Limburg, The Netherlands
| |
Collapse
|
6
|
Movassagi K, Kunze KN, Beck EC, Fu MC, Nho SJ. Predictors of 5-Year Citation Rate in the Orthopaedic Sports Medicine Literature. Am J Sports Med 2019; 47:206-211. [PMID: 30481045 DOI: 10.1177/0363546518810504] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/31/2023]
Abstract
BACKGROUND The citation rate of a research published article is an indicator of its quality and impact and contributes to the journal's impact factor. Within the orthopaedic sports medicine literature, predictors of citation rates have not been previously described. PURPOSE To identify characteristics of published articles that predict 5-year citation rates of studies in the orthopaedic sports medicine literature. STUDY DESIGN Cross-sectional study. METHODS Research articles published in The American Journal of Sports Medicine (AJSM), Arthroscopy: The Journal of Arthroscopic and Related Surgery, and Knee Surgery, Sports Traumatology, Arthroscopy (KSSTA) from 2012 were analyzed. Extracted characteristics of published articles included journal, author number, origin of study, first author degree, subject of study, study type, sample size, number of references and institutions, conflicts of interest, level of evidence, and 5-year citation rates. Multivariate logistic regression was used to determine predictors of greater than the mean number of citations at 5 years. RESULTS A total of 825 published articles ( AJSM, n = 313; Arthroscopy, n = 173; KSSTA, n = 339) were included in the final analysis. The mean number of 5-year citations was 23.2 (95% CI, 21.6-24.9; range, 1.0-260.0). AJSM had a significantly greater citation rate (32.4) than Arthroscopy (21.7) and KSSTA (15.2) ( P < .001 for both). Arthroscopy had a greater citation rate than KSSTA ( P = .008). Independent predictors of greater than the mean number of citations at 5 years were published articles in AJSM (odds ratio [OR], 5.17; 95% CI, 2.81-9.52; P < .0001), published articles of North American origin (OR, 1.79; 95% CI, 1.25-2.58; P = .002), and published articles regarding the hip (OR, 2.68; 95% CI, 1.08-6.67; P = .035). CONCLUSION Published articles in AJSM, those from North America, and those examining the hip were independent predictors of greater citation rates at 5 years.
Collapse
Affiliation(s)
- Kamran Movassagi
- Department of Orthopaedic Surgery, University of California, San Francisco-Fresno, Fresno, California, USA
| | - Kyle N Kunze
- Department of Orthopaedic Surgery, Rush University Medical Center, Chicago, Illinois, USA
| | - Edward C Beck
- Department of Orthopaedic Surgery, Rush University Medical Center, Chicago, Illinois, USA
| | - Michael C Fu
- Department of Orthopaedic Surgery, Hospital for Special Surgery, New York, New York, USA
| | - Shane J Nho
- Department of Orthopaedic Surgery, Rush University Medical Center, Chicago, Illinois, USA
| |
Collapse
|
7
|
Urlings MJE, Duyx B, Swaen GMH, Bouter LM, Zeegers MPA. Citation bias in the literature on dietary trans fatty acids and serum cholesterol. J Clin Epidemiol 2018; 106:88-97. [PMID: 30342971 DOI: 10.1016/j.jclinepi.2018.10.008] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2017] [Revised: 08/12/2018] [Accepted: 10/09/2018] [Indexed: 11/29/2022]
Abstract
OBJECTIVE Balanced citations are a necessary condition for a sound development of scientific knowledge, whereas selective citations may bias scientific consensus. In this study, we assess which determinants influenced the likelihood of being cited in the literature on trans fatty acids and cholesterol. STUDY DESIGN AND SETTING We conducted a citation network analysis of the literature concerning trans fats and low density cholesterol and high density cholesterol. Each publication was scored on various potential determinants of citation, such as study outcome, study design, sample size, journal impact factor, and funding source. We applied random effect logistic regression to identify determinants of citation. RESULTS A network of 108 publications was identified, containing 5,041 potential citation paths and 669 utilized citation paths. Reporting statistically significant results was found to be a strong predictor of citation, together with sample size, journal impact factor, and the authority of the authors. CONCLUSION Within the literature on trans fat intake and cholesterol, selective citations are based on several grounds. Especially the effect of reporting significant results on citation requires special attention because disproportionate attention is paid to publications suggesting a harmful effect of trans fat on cholesterol.
Collapse
Affiliation(s)
- Miriam J E Urlings
- CAPHRI School for Public Health and Primary Care, Maastricht University, Maastricht, The Netherlands; NUTRIM School of Nutrition and Translational Research in Metabolism, Maastricht University, Maastricht, The Netherlands.
| | - Bram Duyx
- CAPHRI School for Public Health and Primary Care, Maastricht University, Maastricht, The Netherlands; NUTRIM School of Nutrition and Translational Research in Metabolism, Maastricht University, Maastricht, The Netherlands
| | - Gerard M H Swaen
- CAPHRI School for Public Health and Primary Care, Maastricht University, Maastricht, The Netherlands; NUTRIM School of Nutrition and Translational Research in Metabolism, Maastricht University, Maastricht, The Netherlands
| | - Lex M Bouter
- Department of Epidemiology and Biostatistics, Amsterdam University Medical Centers, The Netherlands; Department of Philosophy, Faculty of Humanities, Vrije Universiteit, Amsterdam, The Netherlands
| | - Maurice P A Zeegers
- CAPHRI School for Public Health and Primary Care, Maastricht University, Maastricht, The Netherlands; NUTRIM School of Nutrition and Translational Research in Metabolism, Maastricht University, Maastricht, The Netherlands
| |
Collapse
|
8
|
|
9
|
Duyx B, Urlings MJ, Swaen GM, Bouter LM, Zeegers MP. Scientific citations favor positive results: a systematic review and meta-analysis. J Clin Epidemiol 2017; 88:92-101. [DOI: 10.1016/j.jclinepi.2017.06.002] [Citation(s) in RCA: 74] [Impact Index Per Article: 10.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/26/2016] [Revised: 05/15/2017] [Accepted: 06/03/2017] [Indexed: 10/19/2022]
|
10
|
Lopez J, Calotta N, Doshi A, Soni A, Milton J, May JW, Tufaro AP. Citation Rate Predictors in the Plastic Surgery Literature. JOURNAL OF SURGICAL EDUCATION 2017; 74:191-198. [PMID: 27651051 DOI: 10.1016/j.jsurg.2016.08.005] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/05/2016] [Revised: 07/29/2016] [Accepted: 08/07/2016] [Indexed: 06/06/2023]
Abstract
BACKGROUND The purpose of this study is to determine and characterize the scientific and nonscientific factors that influence the rate of article citation in the field of plastic surgery. DESIGN Cross-sectional study. SETTING We reviewed all entries in Annals of Plastic Surgery and Journal of Plastic, Reconstructive, and Aesthetic Surgery from January 1, 2007 to December 31, 2007; and Plastic and Reconstructive Surgery from January 1, 2007 to December 31, 2008. All scientific articles were analyzed and several article characteristics were extracted. The number of citations at 5 years was collected as the outcome variable. A multivariable analysis was performed to determine which variables were associated with higher citations rates. RESULTS A total of 2456 articles were identified of which only 908 fulfilled the inclusion criteria. Most studies were publications in the fields of reconstructive (26.3%) or pediatric/craniofacial (17.6%) surgery. The median number of citations 5 years from publication was 8. In the multivariable analysis, factors associated with higher citations rates were subspecialty field (p = 0.0003), disclosed conflict of interest (p = 0.04), number of authors (p = 0.04), and journal (p = 0.02). CONCLUSION We have found that higher level of evidence (or other study methodology factors) is not associated with higher citation rates. Instead, conflict of interest, subspecialty topic, journal, and number of authors are strong predictors of high citation rates in plastic surgery.
Collapse
Affiliation(s)
- Joseph Lopez
- Department of Plastic and Reconstructive Surgery, Johns Hopkins Hospital, Baltimore, Maryland.
| | - Nicholas Calotta
- Department of Plastic and Reconstructive Surgery, Johns Hopkins Hospital, Baltimore, Maryland
| | - Ankur Doshi
- Department of Plastic and Reconstructive Surgery, Johns Hopkins Hospital, Baltimore, Maryland
| | - Ashwin Soni
- Division of Plastic Surgery, University of Washington Medical Center, Seattle, Washington
| | - Jacqueline Milton
- Department of Biostatistics, Boston University School of Public Health, Boston, Massachusetts
| | - James W May
- Division of Plastic and Reconstructive Surgery, Massachusetts General Hospital, Boston, Massachusetts
| | - Anthony P Tufaro
- Department of Plastic and Reconstructive Surgery, Johns Hopkins Hospital, Baltimore, Maryland
| |
Collapse
|
11
|
Abstract
BACKGROUND Clinical research affecting how doctors practice medicine is increasingly sponsored by companies that make drugs and medical devices. Previous systematic reviews have found that pharmaceutical-industry sponsored studies are more often favorable to the sponsor's product compared with studies with other sources of sponsorship. A similar association between sponsorship and outcomes have been found for device studies, but the body of evidence is not as strong as for sponsorship of drug studies. This review is an update of a previous Cochrane review and includes empirical studies on the association between sponsorship and research outcome. OBJECTIVES To investigate whether industry sponsored drug and device studies have more favorable outcomes and differ in risk of bias, compared with studies having other sources of sponsorship. SEARCH METHODS In this update we searched MEDLINE (2010 to February 2015), Embase (2010 to February 2015), the Cochrane Methodology Register (2015, Issue 2) and Web of Science (June 2015). In addition, we searched reference lists of included papers, previous systematic reviews and author files. SELECTION CRITERIA Cross-sectional studies, cohort studies, systematic reviews and meta-analyses that quantitatively compared primary research studies of drugs or medical devices sponsored by industry with studies with other sources of sponsorship. We had no language restrictions. DATA COLLECTION AND ANALYSIS Two assessors screened abstracts and identified and included relevant papers. Two assessors extracted data, and we contacted authors of included papers for additional unpublished data. Outcomes included favorable results, favorable conclusions, effect size, risk of bias and whether the conclusions agreed with the study results. Two assessors assessed risk of bias of included papers. We calculated pooled risk ratios (RR) for dichotomous data (with 95% confidence intervals (CIs)). MAIN RESULTS Twenty-seven new papers were included in this update and in total the review contains 75 included papers. Industry sponsored studies more often had favorable efficacy results, RR: 1.27 (95% CI: 1.17 to 1.37) (25 papers) (moderate quality evidence), similar harms results RR: 1.37 (95% CI: 0.64 to 2.93) (four papers) (very low quality evidence) and more often favorable conclusions RR: 1.34 (95% CI: 1.19 to 1.51) (29 papers) (low quality evidence) compared with non-industry sponsored studies. Nineteen papers reported on sponsorship and efficacy effect size, but could not be pooled due to differences in their reporting of data and the results were heterogeneous. We did not find a difference between drug and device studies in the association between sponsorship and conclusions (test for interaction, P = 0.98) (four papers). Comparing industry and non-industry sponsored studies, we did not find a difference in risk of bias from sequence generation, allocation concealment, follow-up and selective outcome reporting. However, industry sponsored studies more often had low risk of bias from blinding, RR: 1.25 (95% CI: 1.05 to 1.50) (13 papers), compared with non-industry sponsored studies. In industry sponsored studies, there was less agreement between the results and the conclusions than in non-industry sponsored studies, RR: 0.83 (95% CI: 0.70 to 0.98) (six papers). AUTHORS' CONCLUSIONS Sponsorship of drug and device studies by the manufacturing company leads to more favorable efficacy results and conclusions than sponsorship by other sources. Our analyses suggest the existence of an industry bias that cannot be explained by standard 'Risk of bias' assessments.
Collapse
Affiliation(s)
- Andreas Lundh
- Odense University Hospital and University of Southern DenmarkCenter for Evidence‐Based MedicineSdr. Boulevard 29, Entrance 50 (Videncentret)OdenseDenmark5000
| | - Joel Lexchin
- York UniversitySchool of Health Policy and Management121 Walmer RdTorontoONCanadaM5R 2X8
| | - Barbara Mintzes
- The University of SydneyCharles Perkins Centre and Faculty of PharmacyRoom 6W75, 6th FloorThe Hub, Charles Perkins Centre D17SydneyNSWAustralia2006
| | - Jeppe B Schroll
- Herlev HospitalDepartment of Obstetrics and GynaecologyHerlev Ringvej 75HerlevDenmark2730
| | - Lisa Bero
- Charles Perkins Centre and Faculty of Pharmacy, University of Sydney6th Floor (6W76)The University of SydneySydneyNew South Wales 2006Australia
| | | |
Collapse
|
12
|
van Lent M, Overbeke J, Out HJ. Role of editorial and peer review processes in publication bias: analysis of drug trials submitted to eight medical journals. PLoS One 2014; 9:e104846. [PMID: 25118182 PMCID: PMC4130599 DOI: 10.1371/journal.pone.0104846] [Citation(s) in RCA: 47] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2014] [Accepted: 06/25/2014] [Indexed: 12/03/2022] Open
Abstract
BACKGROUND Publication bias is generally ascribed to authors and sponsors failing to submit studies with negative results, but may also occur after submission. We evaluated whether submitted manuscripts on randomized controlled trials (RCTs) with drugs are more likely to be accepted if they report positive results. METHODS Manuscripts submitted from January 2010 through April 2012 to one general medical journal (BMJ) and seven specialty journals (Annals of the Rheumatic Diseases, British Journal of Ophthalmology, Gut, Heart, Thorax, Diabetologia, and Journal of Hepatology) were included, if at least one study arm assessed the efficacy or safety of a drug and a statistical test was used to evaluate treatment effects. Publication status was retrospectively retrieved from submission systems or provided by journals. Sponsorship and trial results were extracted from manuscripts and classified according to predefined criteria. Main outcome measure was acceptance for publication. RESULTS Of 15,972 manuscripts submitted, 472 (3.0%) were drug RCTs, of which 98 (20.8%) were published. Among submitted drug RCTs, 287 (60.8%) had positive and 185 (39.2%) negative results. Of these, 60 (20.9%) and 38 (20.5%), respectively, were published. Manuscripts on non-industry trials (n = 213) reported positive results in 138 (64.8%) manuscripts, compared to 71 (47.7%) on industry-supported trials (n = 149), and 78 (70.9%) on industry-sponsored trials (n = 110). Twenty-seven (12.7%) non-industry trials were published, compared to 27 (18.1%) industry-supported and 44 (40.0%) industry-sponsored trials. After adjustment for other trial characteristics, manuscripts reporting positive results were not more likely to be published (OR, 1.00; 95% CI, 0.61 to 1.66). Submission to specialty journals, sample size, multicentre status, journal impact factor, and corresponding authors from Europe or US were significantly associated with publication. CONCLUSIONS For the selected journals, there was no tendency to preferably publish manuscripts on drug RCTs that reported positive results, suggesting that publication bias may occur mainly prior to submission.
Collapse
Affiliation(s)
- Marlies van Lent
- Clinical Research Centre Nijmegen, Department of Pharmacology – Toxicology, Radboud University Medical Centre, Nijmegen, The Netherlands
| | - John Overbeke
- Department of Primary and Community Care, Radboud University Medical Centre, Nijmegen, The Netherlands
| | - Henk Jan Out
- Clinical Research Centre Nijmegen, Department of Pharmacology – Toxicology, Radboud University Medical Centre, Nijmegen, The Netherlands
- Teva Pharmaceuticals, Amsterdam, The Netherlands
| |
Collapse
|
13
|
Gasparyan AY, Ayvazyan L, Akazhanov NA, Kitas GD. Conflicts of interest in biomedical publications: considerations for authors, peer reviewers, and editors. Croat Med J 2013; 54:600-8. [PMID: 24382859 PMCID: PMC3893982 DOI: 10.3325/cmj.2013.54.600] [Citation(s) in RCA: 36] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/14/2023] Open
Abstract
This article overviews evidence on common instances of conflict of interest (COI) in research publications from general and specialized fields of biomedicine. Financial COIs are viewed as the most powerful source of bias, which may even distort citation outcomes of sponsored publications. The urge to boost journal citation indicators by stakeholders of science communication is viewed as a new secondary interest, which may compromize the interaction between authors, peer reviewers and editors. Comprehensive policies on disclosure of financial and non-financial COIs in scholarly journals are presented as proxies of their indexing in evidence-based databases, and examples of successful medical journals are discussed in detail. Reports on clinical trials, systematic reviews, meta-analyses and clinical practice guidelines may be unduly influenced by author-pharmaceutical industry relations, but these publications do not always contain explicit disclosures to allow the readers to judge the reliability of the published conclusions and practice-changing recommendations. The article emphasizes the importance of adhering to the guidance on COI from learned associations such as the International Committee of Medical Journal Editors (ICMJE). It also considers joint efforts of authors, peer reviewers and editors as a foundation for appropriately defining and disclosing potential COIs.
Collapse
Affiliation(s)
- Armen Yuri Gasparyan
- Departments of Rheumatology and Research and Development, Dudley Group NHS Foundation Trust (A Teaching Trust of The University of Birmingham), Russells Hall Hospital, Dudley, United Kingdom
| | - Lilit Ayvazyan
- Department of Medical Chemistry, Yerevan State Medical University, Yerevan, Armenia
| | - Nurbek A. Akazhanov
- Department of Internship and Residency for General Practitioners N3, Kazakh National Medical University, Almaty, Kazakhstan
| | - George D. Kitas
- Departments of Rheumatology and Research and Development, Dudley Group NHS Foundation Trust (A Teaching Trust of The University of Birmingham), Russells Hall Hospital, Dudley, United Kingdom
- Arthritis Research UK Epidemiology Unit, University of Manchester, Manchester, United Kingdom
| |
Collapse
|
14
|
Authors' self-declared financial conflicts of interest do not impact the results of major cardiovascular trials. J Am Coll Cardiol 2013; 61:1137-43. [PMID: 23395075 DOI: 10.1016/j.jacc.2012.10.056] [Citation(s) in RCA: 19] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/09/2012] [Revised: 10/01/2012] [Accepted: 10/08/2012] [Indexed: 11/22/2022]
Abstract
OBJECTIVES This study assessed whether the results of major, potentially practice-altering cardiovascular trials were influenced by the authors' self-declared financial conflicts of interest (FCOI). Secondary objectives included assessment of trial outcomes by source of funding, by FCOI subtype, and by trial endpoints. BACKGROUND Financial conflicts of interest, ubiquitous in cardiovascular medicine because of significant investigator-industry collaborations, potentially can influence trial outcomes. METHODS A MEDLINE search was performed using the MeSH term cardiovascular disease limited to randomized controlled trials and clinical trials published from January 1, 2000, through April 15, 2008, in 3 high-impact journals. Two reviewers independently abstracted data from the published article. Chi-square tests, Fisher exact tests, and multivariate logistic regression were used to assess the associations between FCOI and study characteristics and between FCOI and trial outcomes. RESULTS Of the 550 articles reviewed, 51.1% satisfied FCOI criteria, including at least one of the following: stock ownership, employee, speaker's bureau, and consultant). Of the 538 articles providing sponsorship information, 34.6% reported funding solely by nonprofit organizations, 48.3% reported funding solely by industry, and 17.1% reported funding by a combination. Prevalence of FCOI significantly increased with level of industry funding: 21.5% (none), 50.0% (shared), 75.0% (industry solely, n = 281, p < 0.0001). However, no differences in reporting of favorable results were detected when articles were analyzed by self-declared FCOI (60.5% vs. 59.5% in those with and without, odds ratio: 1.04, p = 0.81). This result was upheld in multivariate analysis. CONCLUSIONS Authors' self-declared FCOI and source of funding do not seem to impact outcomes in major cardiovascular clinical trials.
Collapse
|
15
|
Abstract
BACKGROUND Clinical research affecting how doctors practice medicine is increasingly sponsored by companies that make drugs and medical devices. Previous systematic reviews have found that pharmaceutical industry sponsored studies are more often favorable to the sponsor's product compared with studies with other sources of sponsorship. This review is an update using more stringent methodology and also investigating sponsorship of device studies. OBJECTIVES To investigate whether industry sponsored drug and device studies have more favorable outcomes and differ in risk of bias, compared with studies having other sources of sponsorship. SEARCH METHODS We searched MEDLINE (1948 to September 2010), EMBASE (1980 to September 2010), the Cochrane Methodology Register (Issue 4, 2010) and Web of Science (August 2011). In addition, we searched reference lists of included papers, previous systematic reviews and author files. SELECTION CRITERIA Cross-sectional studies, cohort studies, systematic reviews and meta-analyses that quantitatively compared primary research studies of drugs or medical devices sponsored by industry with studies with other sources of sponsorship. We had no language restrictions. DATA COLLECTION AND ANALYSIS Two assessors identified potentially relevant papers, and a decision about final inclusion was made by all authors. Two assessors extracted data, and we contacted authors of included papers for additional unpublished data. Outcomes included favorable results, favorable conclusions, effect size, risk of bias and whether the conclusions agreed with the study results. Two assessors assessed risk of bias of included papers. We calculated pooled risk ratios (RR) for dichotomous data (with 95% confidence intervals). MAIN RESULTS Forty-eight papers were included. Industry sponsored studies more often had favorable efficacy results, risk ratio (RR): 1.24 (95% confidence interval (CI): 1.14 to 1.35), harms results RR: 1.87 (95% CI: 1.54 to 2.27) and conclusions RR: 1.31 (95% CI: 1.20 to 1.44) compared with non-industry sponsored studies. Ten papers reported on sponsorship and effect size, but could not be pooled due to differences in their reporting of data. The results were heterogeneous; five papers found larger effect sizes in industry sponsored studies compared with non-industry sponsored studies and five papers did not find a difference in effect size. Only two papers (including 120 device studies) reported separate data for devices and we did not find a difference between drug and device studies on the association between sponsorship and conclusions (test for interaction, P = 0.23). Comparing industry and non-industry sponsored studies, we did not find a difference in risk of bias from sequence generation, allocation concealment and follow-up. However, industry sponsored studies more often had low risk of bias from blinding, RR: 1.32 (95% CI: 1.05 to 1.65), compared with non-industry sponsored studies. In industry sponsored studies, there was less agreement between the results and the conclusions than in non-industry sponsored studies, RR: 0.84 (95% CI: 0.70 to 1.01). AUTHORS' CONCLUSIONS Sponsorship of drug and device studies by the manufacturing company leads to more favorable results and conclusions than sponsorship by other sources. Our analyses suggest the existence of an industry bias that cannot be explained by standard 'Risk of bias' assessments.
Collapse
Affiliation(s)
- Andreas Lundh
- The Nordic Cochrane Centre, Rigshospitalet, Copenhagen, Denmark.
| | | | | | | | | |
Collapse
|
16
|
Lai NM, Teng CL, Lee ML. Interpreting systematic reviews: are we ready to make our own conclusions? A cross-sectional study. BMC Med 2011; 9:30. [PMID: 21450083 PMCID: PMC3100234 DOI: 10.1186/1741-7015-9-30] [Citation(s) in RCA: 32] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/11/2011] [Accepted: 03/30/2011] [Indexed: 11/21/2022] Open
Abstract
BACKGROUND Independent evaluation of clinical evidence is advocated in evidence-based medicine (EBM). However, authors' conclusions are often appealing for readers who look for quick messages. We assessed how well a group of Malaysian hospital practitioners and medical students derived their own conclusions from systematic reviews (SRs) and to what extent these were influenced by their prior beliefs and the direction of the study results. METHODS We conducted two cross-sectional studies: one with hospital practitioners (n = 150) attending an EBM course in June 2008 in a tertiary hospital and one with final-year medical students (n = 35) in November 2008. We showed our participants four Cochrane SR abstracts without the authors' conclusions. For each article, the participants chose a conclusion from among six options comprising different combinations of the direction of effect and the strength of the evidence. We predetermined the single option that best reflected the actual authors' conclusions and labelled this as our best conclusion. We compared the participants' choices with our predetermined best conclusions. Two chosen reviews demonstrated that the intervention was beneficial ("positive"), and two others did not ("negative"). We also asked the participants their prior beliefs about the intervention. RESULTS Overall, 60.3% correctly identified the direction of effect, and 30.1% chose the best conclusions, having identified both the direction of effect and the strength of evidence. More students (48.2%) than practitioners (22.2%) chose the best conclusions (P < 0.001). Fewer than one-half (47%) correctly identified the direction of effect against their prior beliefs. "Positive" SRs were more likely than "negative" SRs to change the participants' beliefs about the effect of the intervention (relative risk (RR) 1.8, 95% confidence interval 1.3 to 2.6) and "convert" those who were previously unsure by making them choose the appropriate direction of effect (RR 1.9, 95% confidence interval 1.3 to 2.8). CONCLUSIONS The majority of our participants could not generate appropriate conclusions from SRs independently. Judicious direction from the authors' conclusions still appears crucial to guiding our health care practitioners in identifying appropriate messages from research. Authors, editors and reviewers should ensure that the conclusions of a paper accurately reflect the results. Similar studies should be conducted in other settings where awareness and application of EBM are different.
Collapse
Affiliation(s)
- Nai Ming Lai
- Department of Paediatrics, Monash University Sunway Campus, Jeffrey Cheah School of Medicine and Health Sciences, JKR 1235, Bukit Azah, 80100, Johor Bahru, Johor, Malaysia.
| | | | | |
Collapse
|
17
|
Martin A. Intérêts et conflits d’intérêts: le point de vue d’un expert. ONCOLOGIE 2010. [DOI: 10.1007/s10269-010-1956-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/01/2022]
|
18
|
|
19
|
Lundh A, Barbateskovic M, Hróbjartsson A, Gøtzsche PC. Conflicts of interest at medical journals: the influence of industry-supported randomised trials on journal impact factors and revenue - cohort study. PLoS Med 2010; 7:e1000354. [PMID: 21048986 PMCID: PMC2964336 DOI: 10.1371/journal.pmed.1000354] [Citation(s) in RCA: 86] [Impact Index Per Article: 6.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/26/2010] [Accepted: 09/14/2010] [Indexed: 11/24/2022] Open
Abstract
BACKGROUND transparency in reporting of conflict of interest is an increasingly important aspect of publication in medical journals. Publication of large industry-supported trials may generate many citations and journal income through reprint sales and thereby be a source of conflicts of interest for journals. We investigated industry-supported trials' influence on journal impact factors and revenue. METHODS AND FINDINGS we sampled six major medical journals (Annals of Internal Medicine, Archives of Internal Medicine, BMJ, JAMA, The Lancet, and New England Journal of Medicine [NEJM]). For each journal, we identified randomised trials published in 1996-1997 and 2005-2006 using PubMed, and categorized the type of financial support. Using Web of Science, we investigated citations of industry-supported trials and the influence on journal impact factors over a ten-year period. We contacted journal editors and retrieved tax information on income from industry sources. The proportion of trials with sole industry support varied between journals, from 7% in BMJ to 32% in NEJM in 2005-2006. Industry-supported trials were more frequently cited than trials with other types of support, and omitting them from the impact factor calculation decreased journal impact factors. The decrease varied considerably between journals, with 1% for BMJ to 15% for NEJM in 2007. For the two journals disclosing data, income from the sales of reprints contributed to 3% and 41% of the total income for BMJ and The Lancet in 2005-2006. CONCLUSIONS publication of industry-supported trials was associated with an increase in journal impact factors. Sales of reprints may provide a substantial income. We suggest that journals disclose financial information in the same way that they require them from their authors, so that readers can assess the potential effect of different types of papers on journals' revenue and impact.
Collapse
Affiliation(s)
- Andreas Lundh
- The Nordic Cochrane Centre, Rigshospitalet, Copenhagen, Denmark.
| | | | | | | |
Collapse
|
20
|
Abstract
Harvey Marcovitch discusses new research findings from Andreas Lundh and colleagues that examined the effect of publishing industry-funded clinical trials on journal citations and reprint income at six major medical journals.
Collapse
|
21
|
Abstract
Despite remarkable growth in the clinical neurology literature, there is little research on peer review and biomedical publication in neurology. Biomedical publication research encompasses every step of the research process, from the methodology to the publication of research findings. Some general medical journals have served as leaders in improving scientific publication. Many medical fields have taken it upon themselves to characterize journals and peer reviewers within their own fields. Not all of these data can be applied to the clinical neurology literature; research methodologies, article types, and the journals themselves are unique to every medical field. This article reviews current publication research in the neurology literature and concludes that, to sustain and improve upon the integrity of clinical neurology research, further study is needed of the journals and the peer review process in neurology. Otherwise, the value of clinical research findings and patient care guidelines will be diminished.
Collapse
Affiliation(s)
- Victoria S S Wong
- Department of Neurology, University of California, Davis Medical Center, Sacramento, California 95817, USA.
| |
Collapse
|
22
|
Gundle KR, Dingel MJ, Koenig BA. 'To prove this is the industry's best hope': big tobacco's support of research on the genetics of nicotine addiction. Addiction 2010; 105:974-83. [PMID: 20659058 PMCID: PMC2911634 DOI: 10.1111/j.1360-0443.2010.02940.x] [Citation(s) in RCA: 26] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/29/2023]
Abstract
BACKGROUND New molecular techniques focus a genetic lens upon nicotine addiction. Given the medical and economic costs associated with smoking, innovative approaches to smoking cessation and prevention must be pursued; but can sound research be manipulated by the tobacco industry? METHODOLOGY The chronological narrative of this paper was created using iterative reviews of primary sources (the Legacy Tobacco Documents), supplemented with secondary literature to provide a broader context. The empirical data inform an ethics and policy analysis of tobacco industry-funded research. FINDINGS The search for a genetic basis for smoking is consistent with industry's decades-long plan to deflect responsibility away from the tobacco companies and onto individuals' genetic constitutions. Internal documents reveal long-standing support for genetic research as a strategy to relieve the tobacco industry of its legal responsibility for tobacco-related disease. CONCLUSIONS Industry may turn the findings of genetics to its own ends, changing strategy from creating a 'safe' cigarette to defining a 'safe' smoker.
Collapse
Affiliation(s)
- Kenneth R. Gundle
- Medical Student, Harvard Medical School, 4285 SW 78 Ave, Portland, OR 97225
| | - Molly J. Dingel
- Assistant Professor, Center for Learning Innovation, University of Minnesota, Rochester, 300 University Square, 111 South Broadway, Rochester, MN 55904
| | - Barbara A. Koenig
- Professor of Biomedical Ethics and Medicine, Mayo Clinic, 200 First Street SW, Rochester, MN 55905
| |
Collapse
|
23
|
Schott G, Pachl H, Limbach U, Gundert-Remy U, Lieb K, Ludwig WD. The financing of drug trials by pharmaceutical companies and its consequences: part 2: a qualitative, systematic review of the literature on possible influences on authorship, access to trial data, and trial registration and publication. DEUTSCHES ARZTEBLATT INTERNATIONAL 2010; 107:295-301. [PMID: 20490338 PMCID: PMC2872821 DOI: 10.3238/arztebl.2010.0295] [Citation(s) in RCA: 28] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/29/2009] [Accepted: 02/23/2010] [Indexed: 11/27/2022]
Abstract
BACKGROUND In recent years, a number of studies have shown that clinical drug trials financed by pharmaceutical companies yield favorable results for company products more often than independent trials do. Moreover, pharmaceutical companies have been found to influence drug trials in various ways. This overview of current, systematic studies on this topic is intended to identify and characterize the particular aspects of the performance of a drug trial that can be affected by financial support from a pharmaceutical company. METHODS Publications retrieved from a systematic Medline search on this topic from 1 November 2002 to 16 December 2009 were independently evaluated and selected by two of the authors. These publications were supplemented by further ones found in their references sections. RESULTS 57 publications were included for evaluation in Parts 1 and 2 of this article. A number of studies revealed that many trials financed by pharmaceutical companies-in some cases, as many as half of all such trials-are never published. Moreover, multiple publications of the same findings were found, and some reports were found to include selectively published data. Further studies revealed evidence of other problems including incomplete trial registration, constraints on publishing rights, withheld knowledge of adverse drug reactions, and the use of ghostwriters who were supplied by the pharmaceutical companies. CONCLUSION Financial support from a pharmaceutical company influences multiple aspects of the performance of drug trials and often leads to a favorable result for the corporate sponsor of the trial. Public access to trial protocols and results must be ensured. Moreover, more effort should be made to carry out drug trials independently, without the financial support of pharmaceutical companies.
Collapse
Affiliation(s)
- Gisela Schott
- Arzneimittelkommission der deutschen Arzteschaft, Herbert-Lewin-Platz 1, 10623 Berlin, Germany.
| | | | | | | | | | | |
Collapse
|
24
|
|
25
|
Pandolfini C, Bonati M. Children's presence in research. A review of online registers. Eur J Clin Pharmacol 2009; 65:873-80. [PMID: 19590865 DOI: 10.1007/s00228-009-0687-7] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/10/2009] [Accepted: 06/22/2009] [Indexed: 10/20/2022]
Abstract
OBJECTIVES To determine how many online clinical trial registers include paediatric trial data, how much information is provided, ease of searching for paediatric trials and the accessibility of paediatric trial data in general. METHODS Medline and Google were searched for mention of clinical trial registers in July 2008. All registers considered to be eligible were evaluated for trial information provided, search options available, and number of trials, both total and paediatric. A meta-analytic weighted average of the presence of paediatric trials was calculated and compared to the percentage of published paediatric trial articles in Medline. The paediatric trials in the registers were searched for in the World Health Organization's International Clinical Trials Registry Platform (ICTRP). All online, freely accessible registers including ongoing trials on different drugs or therapeutic areas, were eligible for review. RESULTS Twelve registers were included in the review. All except one provided detailed trial data, while search options varied between registers: seven provided free-text searching, two listed their trials by condition, two provided elaborate search options (age group, condition, study purpose, etc) and one simply listed its trials. Nine of the 12 registers' search facilities made it possible to search for paediatric trials, and these were analysed further: the percentage of paediatric trials in the single registers ranged from 4.8 to 33.3%, and the weighted average was 15% (95% confidence interval 8.2-21.8). The percentage of published, paediatric trial articles was 25%. Of the paediatric trials also searched for in the ICTRP, 66% were found. CONCLUSIONS Great difficulty was found in retrieving paediatric trials due to the limited and inadequate search functions of the registers reviewed but, in general, the registers seem to represent fewer paediatric trials than those reported in the literature. The ICTRP portal is important for trial accessibility, but it is still in an initial phase and far from representative of the global research situation, especially in the field of paediatrics.
Collapse
Affiliation(s)
- C Pandolfini
- Laboratory for Mother and Child Health, Department of Public Health, "Mario Negri" Pharmacological Research Institute, Via Giuseppe La Masa 19, 20156 Milan, Italy
| | | |
Collapse
|