The impact of cancer research: how publications influence UK cancer clinical guidelines

Exploring the clinical translation intensity of papers published by the world's top scientists in basic medicine

The extent to which basic medical research is translated into clinical practice is a topic of interest to all stakeholders. In this study, we assessed the clinical translation intensity of papers published by scientists who have made outstanding contributions to the field of basic medicine (Lasker Prize winners for Basic Medical Research). Approximate Potential for Translation (APT), Translational science scores (TS), and Citations by clinical research (Cited by Clin.) were analyzed as dependent variables. A traditional citation indicator was used as a reference (relative citation ratio, RCR). In order to examine the correlation between these different indicators and the characteristics of the paper, the author, and the institution. we used nonparametric tests, Spearman correlations, ordinal least squares regressions (OLS), quantile regressions, and zero-inflated negative binomial regression methods. We found that among the basic medical research papers published by Lasker Basic Medicine Award winners, (1) 20% are cited by clinical research; 11.6% of the papers were more valuable for clinical research than basic research; 12.8% have a probability of more than 50% to be cited in future clinical studies; (2) Spearman correlations were conducted among APT, TS, Cited by Clin., RCR, and all of the other continuous variables. There is a significant, positive, low to moderate correlation between APT, TS, and Cited by Clin (APT and TS: r = 0.549, p < 0.01; APT and Cited by Clin: r = 0.530, p < 0.01; TS and Cited by Clin: r = 0.383, p < 0.01). However, the relationship between RCR and the three indicators of clinical translation intensity was not consistent. APT was positively correlated with RCR (r = 0.553, p < 0.01). Cited by Clin. is weakly positively correlated with RCR (r = 0.381, p < 0.01). There is almost no correlation between TS and RCR (r = 0.184, p < 0.01). (3) Publication age, primary research paper, multidisciplinary science, number of disciplines, authors, institutions, funded projects, references, length of the title, length of paper, physical age, gender, nationality, institutional type, Nobel Prize have a significant relationship with 1 to 3 types of clinical translation intensity measures. In a sample of basic medical research papers published by the world's top scientists in basic medicine, we came to the following conclusions: the three indicators, APT, TS and Cited by Clin., measured the clinical translation intensity of the papers from different perspectives. They are both related to each other and have their own characteristics. In a sample of basic medical research papers published by the world's top scientists in basic medicine, characteristics at the paper, winner, and institution level significantly correlated with the measures of clinical translation intensity. Gender effect on the clinical translation intensity of papers was confirmed. Traditional citation-based indicators and translational-focused indicators measure academic impact and clinical impact respectively. There is a certain degree of disconnect between them. Two types of indicators should be used in combination in future assessments of basic medical research.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11192-023-04634-4.

Altmetric score, reads, and citations in paediatric cardiology: do they correlate and what do they mean?

INTRODUCTION

The primary objective of this study was to determine whether Altmetric score, number of reads, and citations for paediatric cardiology manuscripts correlate with one another. A secondary objective was to determine the extent to which factors mediated citation number for paediatric cardiology manuscripts.

METHODS

Data for this study came from manuscripts published in Cardiology in the Young (2010-2021). Data were extracted by using data shared on the journal website. Spearman's correlation analyses were conducted between manuscript reads, citations, and Altmetric score. Regression analyses were conducted with number of citations as the dependent variable and year of publication, publication type, number of reads, and Altmetric score as independent variables.

RESULTS

A total of 2642 manuscripts were included in the final analyses. Reads and citations had poor correlation (r-value 0.32); reads and Altmetric score had negligible correlation (r-value 0.26); and Altmetric score and citations had negligible correlation (r-value 0.07). Year of publication was independently associated with number of citations (β -0.95, p-value <0.01). Manuscript type was independently associated with number of citations (β 1.04, p-value <0.01). Number of reads was independently associated with citations (β 0.01, p-value <0.01). Altmetric score was independently associated with number of citations (β 0.05, p-value <0.01).

CONCLUSION

This study describes the correlation of reads, citations, and Altmetric score in manuscripts published in Cardiology in the Young, demonstrating poor correlation, at best, between these metrics. Each bibliometric index seems to represent a different phenomenon of manuscript consumption. No single bibliometric index in isolation offers ample representation of manuscript consumption.

Uniting Latin America Through Research: How Regional Research Can Strengthen Local Policies, Networking, and Outcomes for Patients With Lung Cancer

Lung cancer represents a considerable global health threat, leading the list in terms of cancer-related deaths worldwide. An important proportion of lung cancer cases occur within Latin America, and current projections show that over the next decade, the number of deaths due to lung cancer will double in the region, underscoring the need to implement evidence-based interventions to improve outcomes. Several challenges have limited the progress in lung cancer research in Latin America for many years, though recently the surge of multidisciplinary, transnational, and transcultural research groups have overcome many of these limitations. The increase in region-specific knowledge has improved cancer care in the area, providing clinicians with a specific demographic and molecular profile for Hispanic patients with lung cancer; as a result, the implementation of precision oncology has benefited from a profound knowledge of the patient profile. Nonetheless, there are still challenges to improve research in Latin America, including stabilizing funding sources to continue independent research, supporting mentoring programs and an early immersion in clinical research for early career fellows, and overcoming barriers for publishing.

Evaluating cancer research impact: lessons and examples from existing reviews on approaches to research impact assessment

BACKGROUND

Performing cancer research relies on substantial financial investment, and contributions in time and effort from patients. It is therefore important that this research has real life impacts which are properly evaluated. The optimal approach to cancer research impact evaluation is not clear. The aim of this study was to undertake a systematic review of review articles that describe approaches to impact assessment, and to identify examples of cancer research impact evaluation within these reviews.

METHODS

In total, 11 publication databases and the grey literature were searched to identify review articles addressing the topic of approaches to research impact assessment. Information was extracted on methods for data collection and analysis, impact categories and frameworks used for the purposes of evaluation. Empirical examples of impact assessments of cancer research were identified from these literature reviews. Approaches used in these examples were appraised, with a reflection on which methods would be suited to cancer research  impact evaluation going forward.

RESULTS

In total, 40 literature reviews were identified. Important methods to collect and analyse data for impact assessments were surveys, interviews and documentary analysis. Key categories of impact spanning the reviews were summarised, and a list of frameworks commonly used for impact assessment was generated. The Payback Framework was most often described. Fourteen examples of impact evaluation for cancer research were identified. They ranged from those assessing the impact of a national, charity-funded portfolio of cancer research to the clinical practice impact of a single trial. A set of recommendations for approaching cancer research impact assessment was generated.

CONCLUSIONS

Impact evaluation can demonstrate if and why conducting cancer research  is worthwhile. Using a mixed methods, multi-category assessment organised within a framework, will provide a robust evaluation, but the ability to perform this type of assessment may be constrained by time and resources. Whichever approach is used, easily measured, but inappropriate metrics should be avoided. Going forward, dissemination of the results of cancer research impact assessments will allow the cancer research community to learn how to conduct these evaluations.

A characterization of professional media and its links to research

The publishing industry is a vast system whose elements form a metaphorical ecosystem with knowledge flowing through connections between heterogeneous elements. In this paper we seek a more robust understanding of different types of literature, and whether and how they support one another in the diffusion of knowledge. We analyze a corpus comprising professional electronic media in US dentistry and its relation to the peer reviewed journal literature. Our corpus includes full text from magazines, news sites and blogs that provide information to clinicians. We find links to research are made through several mechanisms: articles describing new clinical guidelines, referencing, summaries of recently published journal articles and crossover authoring. There is little to no apparent time lag in the diffusion of information from research literature to professional media.

A new database of the references on international clinical practice guidelines: a facility for the evaluation of clinical research

Although there are now several bibliographic databases of research publications, such as Google Scholar, Pubmed, Scopus, and the Web of Science (WoS), and some also include counts of citations, there is at present no similarly comprehensive database of the rapidly growing number of clinical practice guidelines (CPGs), with their references, which sometimes number in the hundreds. CPGs have been shown to be useful for the evaluation of clinical (as opposed to basic) biomedical research, which often suffers from relatively low counts of citations in the serial literature. The objectives were to introduce a new citation database, clinical impact®, and demonstrate how it can be used to evaluate research impact of clinical research publications by exploring the characteristics of CPG citations of two sets of papers, as well as show temporal variation of clinical impact® and the WoS. The paper includes the methodology used to retain the data and also the rationale adopted to achieve data quality. The analysis showed that although CPGs tend preferentially to cite papers from their own country, this is not always the case. It also showed that cited papers tend to have a more clinical research level than uncited papers. An analysis of diachronous citations in both clinical impact® and the WoS showed that although the WoS citations showed a decreasing trend after a peak at 2–3 years after publication, this was less clear for CPG citations and a longer timescale would be needed to evaluate their impact on these documents.

Association of National Cancer Institute-Sponsored Clinical Trial Network Group Studies With Guideline Care and New Drug Indications

Importance

National Cancer Institute Clinical Trial Network (NCTN) groups serve a vital role in identifying effective new antineoplastic regimens. However, the downstream clinical effect of their trials has not been systematically examined.

Objective

To examine the association of NCTN trials with guideline care and new drug indications.

Design, Setting, and Participants

This retrospective cohort study evaluated phase 3 SWOG Cancer Research Network clinical trials from January 1, 1980, through June 30, 2017. Only completed trials with published results were included. To be considered practice influential (PI), a trial must have been associated with guideline care through its inclusion in National Comprehensive Cancer Network (NCCN) clinical guidelines or US Food and Drug Administration (FDA) new drug approvals in favor of a recommended treatment. Data were analyzed from June 15, 2018, through March 29, 2019.

Main Outcomes and Measures

Estimated overall rate of PI trials, as well as trends over time. The total federal investment supporting the set of trials was also determined.

Results

In total, 182 trials consisting of 148 028 patients were studied. Eighty-two studies (45.1%; 95% CI, 37.7%-52.6%) were PI, of which 70 (38.5%) influenced NCCN guidelines, 6 (3.3%) influenced FDA new drug approvals, and 6 (3.3%) influenced both. The number of PI trials was 47 of 65 (72.3%) among those with positive findings and 35 of 117 (29.9%) among those with negative findings. Thus, 35 of 82 PI trials (42.7%) were based on studies with negative findings, with nearly half of these studies (17 of 35 [48.6%]) reaffirming standard of care compared with experimental therapy. The total federal investment spent in conducting the trials was $1.36 billion (2017 US dollars), a rate of $7.5 million per study or $16.6 million per PI trial.

Conclusions and Relevance

Nearly half of all phase 3 trials by one of the NCTN's largest groups were associated with guideline care or new drug indications, including those with positive and negative findings. Compared with the costs of a new drug approval in pharmaceutical companies, typically estimated at more than $1 billion, the amount of federal funds invested to provide this valuable evidence was modest. These results suggest that the NCTN program contributes clinically meaningful, cost-efficient evidence to guide patient care.

Does citation matter? Research citation in policy documents as an indicator of research impact - an Australian obesity policy case-study

Background

Citation of research in policy documents has been suggested as an indicator of the potential longer-term impacts of research. We investigated the use of research citations in childhood obesity prevention policy documents from New South Wales (NSW), Australia, considering the feasibility and value of using research citation as a proxy measure of research impact.

Methods

We examined childhood obesity policy documents produced between 2000 and 2015, extracting childhood obesity-related references and coding these according to reference type, geographical origin and type of research. A content analysis of the policy documents examined where and how research was cited in the documents and the context of citation for individual research publications.

Results

Over a quarter (28%) of the policy documents (n = 86) were not publicly available, almost two-thirds (63%) contained references, half (47%) cited obesity-related research and over a third (41%) of those containing references used unorthodox referencing styles, making reference extraction laborious. No patterns, in terms of the types of documents more likely to cite research, were observed and the number of obesity research publications cited per document was highly variable. In total, 263 peer-reviewed and 94 non-peer-reviewed obesity research publications were cited. Research was most commonly cited to support a policy argument or choice of solution. However, it was not always possible to determine how or why individual publications were cited or whether the cited research itself had influenced the policy process. Content analysis identified circumstances where research was mentioned or considered, but not directly cited.

Conclusions

Citation of research in policy documents in this case did not always provide evidence that the cited research had influenced the policy process, only that it was accessible and relevant to the content of the policy document. Research citation across these public health policy documents varied greatly and is unlikely to be an accurate reflection of actual research use by the policy agencies involved. The links between citation and impact may be more easily drawn in specific policy areas or types of documents (e.g. clinical guidelines), where research appraisal feeds directly into policy recommendations.

Integrative Whole-Person Oncology Care in the UK

The term "whole-person cancer care"-an approach that addresses the needs of the person as well as treating the disease-is more widely understood in the United Kingdom than its synonym "integrative oncology." The National Health Service (NHS) provides free access to care for all, which makes it harder to prioritize NHS funding of whole-person medicine, where interventions may be multimodal and lacking in cost-effectiveness data. Despite this, around 30% of cancer patients are known to use some form of complementary or alternative medicine (CAM). This is virtually never medically led, and usually without the support or even the knowledge of their oncology teams, with the exception of one or two large cancer centers. UK oncology services are, however, starting to be influenced from three sides; first, by well-developed and more holistic palliative care services; second, by directives from central government via the sustainable health care agenda; and third, by increasing pressure from patient-led groups and cancer charities. CAM remains unlikely to be provided through the NHS, but nutrition, physical activity, mindfulness, and stress management are already becoming a core part of the NHS "Living With and Beyond Cancer" agenda. This supports cancer survivors into stratified pathways of care, based on individual, self-reported holistic needs and risk assessments, which are shared between health care professionals and patients. Health and well-being events are being built into cancer care pathways, designed to activate patients into self-management and support positive lifestyle change. Those with greater needs can be directed toward appropriate external providers, where many examples of innovative practice exist. These changes in policy and vision for the NHS present an opportunity for integrative oncology to develop further and to reach populations who would, in many other countries, remain underserved or hard to reach by whole-person approaches.

Differential research impact in cancer practice guidelines' evidence base: lessons from ESMO, NICE and SIGN

Background

This is an appraisal of the impact of cited research evidence underpinning the development of cancer clinical practice guidelines (CPGs) by the professional bodies of the European Society for Medical Oncology (ESMO), the National Institute for Health and Care Excellence (NICE) and the Scottish Intercollegiate Guidelines Network (SIGN).

Methods

A total of 101 CPGs were identified from ESMO, NICE and SIGN websites across 13 cancer sites. Their 9486 cited references were downloaded from the Web of Science Clarivate Group database, analysed on Excel (2016) using Visual Basic Application macros and imported onto SPSS (V.24.0) for statistical tests.

Results

ESMO CPGs mostly cited research from Western Europe, while the NICE and SIGN ones from the UK, Canada, Australia and Scandinavian countries. The ESMO CPGs cited more recent and basic research (eg, drugs treatment), in comparison with NICE and SIGN CPGs where older and more clinical research (eg, surgery) papers were referenced. This chronological difference in the evidence base is also in line with that ESMO has a shorter gap between the publication of the research and its citation on the CPGs. It was demonstrated that ESMO CPGs report more chemotherapy research, while the NICE and SIGN CPGs report more surgery, with the results being statistically significant.

Conclusions

We showed that ESMO, NICE and SIGN differ in their evidence base of CPGs. Healthcare professionals should be aware of this heterogeneity in effective decision-making of tailored treatments to patients, irrespective of geographic location across Europe.

Policy documents as sources for measuring societal impact: how often is climate change research mentioned in policy-related documents?

In the current UK Research Excellence Framework (REF) and the Excellence in Research for Australia (ERA), societal impact measurements are inherent parts of the national evaluation systems. In this study, we deal with a relatively new form of societal impact measurements. Recently, Altmetric-a start-up providing publication level metrics-started to make data for publications available which have been mentioned in policy documents. We regard this data source as an interesting possibility to specifically measure the (societal) impact of research. Using a comprehensive dataset with publications on climate change as an example, we study the usefulness of the new data source for impact measurement. Only 1.2 % (n = 2341) out of 191,276 publications on climate change in the dataset have at least one policy mention. We further reveal that papers published in Nature and Science as well as from the areas "Earth and related environmental sciences" and "Social and economic geography" are especially relevant in the policy context. Given the low coverage of the climate change literature in policy documents, this study can be only a first attempt to study this new source of altmetrics data. Further empirical studies are necessary, because mentions in policy documents are of special interest in the use of altmetrics data for measuring target-oriented the broader impact of research.

Impact of NIHR HTA Programme funded research on NICE clinical guidelines: a retrospective cohort

BACKGROUND

It is vitally important that there is a connection between health research and clinical practice. Indications as to the impact of the research on evidence-based practice and policy can be obtained by tracking the use of outputs of health research, especially its use in clinical guidelines (CGs). This study aims to assess the proportion of National Institute for Health and Care Excellence (NICE) CGs citing National Institute for Health Research Health Technology Assessment (NIHR HTA) studies and the impact of evidence from those studies on the included NICE CGs.

METHODS

This is a retrospective cohort study assessing the proportion of NICE CGs from all NICE CGs issued between April 2001 and April 2012, which cited evidence from studies funded by the NIHR HTA Programme and the impact of those studies on the CGs as the primary and secondary outcome measures.

RESULTS

Of the cohort of NICE CGs (n = 122), 3 (2%) CGs were based on previous NIHR HTA reports and would not have been issued in that form without those NIHR HTA studies, 90 (74%) included evidence from NIHR HTA studies, and 29 (24%) did not include evidence from NIHR HTA studies. The impact of NIHR HTA evidence on NICE CGs varied in the type and quantity of data used.

CONCLUSIONS

Findings suggest that NIHR HTA funded research impacts on clinical guidance from NICE and hence is well connected to both clinical practice and policy.

Characteristics of Spanish articles of "scientific quality" cited in clinical practice guidelines on mental health

INTRODUCTION

The study aims to illustrate the impact of Spanish research in clinical decision making. To this end, we analysed the characteristics of the most significant Spanish publications cited in clinical practice guidelines (CPG) on mental health.

MATERIAL AND METHODS

We conducted a descriptive qualitative study on the characteristics of ten articles cited in Spanish CPG on mental health, and selected for their "scientific quality". We analysed the content of the articles on the basis of the following characteristics: topics, study design, research centres, scientific and practical relevance, type of funding, and area or influence of the reference to the content of the guidelines.

RESULTS

Among the noteworthy studies, some basic science studies, which have examined the establishment of genetic associations in the pathogenesis of mental illness are included, and others on the effectiveness of educational interventions. The content of those latter had more influence on the GPC, because they were cited in the summary of the scientific evidence or in the recommendations. Some of the outstanding features in the selected articles are the sophisticated designs (experimental or analytical), and the number of study centres, especially in international collaborations. Debate or refutation of previous findings on controversial issues may have also contributed to the extensive citation of work.

CONCLUSIONS

The inclusion of studies in the CPG is not a sufficient condition of "quality", but their description can be instructive for the design of future research or publications.

Tracking the impact of research on policy and practice: investigating the feasibility of using citations in clinical guidelines for research evaluation

OBJECTIVES

To investigate the feasibility of using research papers cited in clinical guidelines as a way to track the impact of particular funding streams or sources.

SETTING

In recent years, medical research funders have made efforts to enhance the understanding of the impact of their funded research and to provide evidence of the 'value' of investments in particular areas of research. One of the most challenging areas of research evaluation is around impact on policy and practice. In the UK, the National Institute of Health and Clinical Excellence (NICE) provide clinical guidelines, which bring together current high-quality evidence on the diagnosis and treatment of clinical problems. Research referenced in these guidelines is an indication of its potential to have real impact on health policy and practice.

DESIGN

This study is based on analysis of the authorship and funding attribution of research cited in two NICE clinical guidelines: dementia and chronic obstructive pulmonary disease.

RESULTS

Analysis identified that around a third of papers cited in the two NICE guidelines had at least one author based in the UK. In both cases, about half of these UK attributed papers contained acknowledgements which allowed the source of funding for the research to be identified. The research cited in these guidelines was found to have been supported by a diverse set of funders from different sectors. The study also investigated the contribution of research groups based in universities, industry and the public sector.

CONCLUSIONS

The study found that there is great potential for guidelines to be used as sources of information on the quality of the research used in their development and that it is possible to track the source of the funding of the research. The challenge is in harnessing the relevant information to track this in an efficient way.

Funding and Performance on Clinical Guidelines: The Cases of Dementia and Chronic Obstructive Pulmonary Disease

In September 2009, the United Kingdom (UK) Medical Research Council, the Wellcome Trust and the Department of Health (England) commissioned RAND Europe to investigate the characteristics of research cited in two UK clinical guidelines: Dementia and Chronic Obstructive Pulmonary Disease (COPD). The exploratory work is part of an overall drive among funders to understand better how research reaches policy and practice. The objectives of our exploratory study were to understand the following questions.What are the characteristics of publications cited in these clinical guidelines?On those papers with a UK affiliation, who funded the research cited in these clinical guidelines?

How may clinical research improve healthcare outcomes?

Healthcare outcomes such as overall survival or quality of life are the end results of a complex interaction between the patient, treatment and the healthcare system. Research may identify superior interventions but their dissemination and changing the behaviour of healthcare providers is challenging. Demonstrating and measuring the benefits of clinical research on healthcare outcomes is an important issue but there is remarkably little empiric work to date in this area. In this chapter we explore benefits that may arise in healthcare from contributing to clinical research, and consider the mechanisms which may be relevant. Improvements in infrastructure, the processes of care and workforces are important. Complex adaptive systems theory provides a framework for considering the many feedback loops that relate research, health outcomes and the behaviour of healthcare providers. Given the costs of research, additional studies to examine the impact of research on healthcare outcomes and to explore the mechanisms are justified and highly desirable.

An analysis of research activity in major UK cancer centres

The organisation of cancer research is critical to its overall creativity and productivity. Cancer centres are a major organisational structure for this research, however, little is known about their effect on research or how national policy-making intersects with this complex policy nexus. This study of the evolution of United Kingdom cancer centres (UKCC), part of a wider European and United States programme, uses a bibliometric analysis of research activity prior to the creation of the NCRI and after its formation (1995-2004/5). In terms of critical research mass UKCC are very heterogeneous with a fourfold difference between the top and bottom quintiles. UK centres published just over one eighth of the total UKCC in 1995 but almost a quarter by 2004. This centrification occurred in the absence of any national strategy. Overall these centres conduct more fundamental (laboratory-based) research than that being conducted in the wider network but this hides major heterogeneity. UKCC collaborate with European investigators in 5-28% of all their outputs and with USA the range is between 6% and 21%. We have also derived new measures of research impact on clinical management and the general public as well as the impact of national policy on research assessment for certain types of cancer research.

Assessing the scientific research productivity of Puerto Rican cancer researchers: bibliometric analysis from the Science Citation Index

OBJECTIVE

The analysis of cancer scientific production in Puerto Rico is largely unexplored. The objective of this study was to characterize trends in cancer-related research publications by authors affiliated to Puerto Rican institutions in recent decades.

METHODS

Manuscripts were retrieved from the Science Citation Index (SCI) database from 1982 to 2009. Search criterions were that the author's affiliation field contained some institution located in Puerto Rico and that the manuscripts were related to cancer research (according to keywords from the National Cancer Institute' cancer definition). Indexes measured in our analysis included number and type of manuscript, scientific collaboration, author's affiliation, and journal visibility. All the analyses were conducted using ProCite for bibliographic information management and STATA and SEER Joinpoint for the statistical inquiry.

RESULTS

From 1982-2009, cancer-related papers authored by scientists located in Puerto Rico came to 451. Over the last three decades the scientific production underwent significant growth (APC = 6.4%, p < 0.05) with the highest peak between 2000 and 2009 (61.4% of all articles). Universities are the local institutional sector with the highest number of authors (81.4%), and the University of Puerto Rico is the most active center in this regard (68.5%). Forty-three percent of the manuscripts (n = 195) were published in 20 journals from which 14 are observed to have high visibility when compared to similar thematic journals.

CONCLUSIONS

Cancer-scientific production in Puerto Rico underwent constant growth during the last three decades. A complete understanding of citing, publishing, and collaboration patterns in Puerto Rico is critical to researchers, policy makers, and health-care professionals in order to make informed decisions about cancer research priorities.