Directing research funds to the right research projects: a review of criteria used by research organisations in Australia in prioritising health research projects for funding

Embedding stakeholder preferences in setting priorities for health research: Using a discrete choice experiment to develop a multi-criteria tool for evaluating research proposals

We determined weights for a multi-criteria tool for assessing the relative merits of clinical-trial research proposals, and investigated whether the weights vary across relevant stakeholder groups. A cross-sectional, adaptive discrete choice experiment using 1000minds online software was administered to consumers, researchers and funders affiliated with the Australian Clinical Trials Alliance (ACTA). We identified weights for four criteria-Appropriateness, Significance, Relevance, Feasibility-and their levels, representing their relative importance, so that research proposals can be scored between 0% (nil or very low merit) and 100% (very high merit). From 220 complete survey responses, the most important criterion was Appropriateness (adjusted for differences between stakeholder groups, mean weight 28.9%) and the least important was Feasibility (adjusted mean weight 19.5%). Consumers tended to weight Relevance more highly (2.7% points difference) and Feasibility less highly (3.1% points difference) than researchers. The research or grant writing experience of researchers or consumers was not associated with the weights. A multi-criteria tool for evaluating research proposals that reflects stakeholders' preferences was created. The tool can be used to assess the relative merits of clinical trial research proposals and rank them, to help identify the best proposals for funding.

Mortality and mental health funding-do the dollars add up? Eating disorder research funding in Australia from 2009 to 2021: a portfolio analysis

Background

Eating Disorders (EDs) are among the deadliest of the mental disorders and carry a sizeable public health burden, however their research and treatment is consistently underfunded, contributing to protracted illness and ongoing paucity of treatment innovation.

Methods

We compare absolute levels and growth rates of Australian mental health research funding by illness group for the years 2009-2021, with a specific focus on eating disorders analysed at the portfolio level.

Findings

Actual and adjusted data obtained from Australia's three national medical research funding bodies (NHMRC, ARC and MRFF) shows eating disorders receive a disproportionately low allocation of mental health research funding despite having amongst the highest mortality rates. Forty-one category one research grants totalling $AUD28.1 million were funded for eating disorders over the period. When adjusted for inflation, this equates to $2.05 per affected individual, compared with $19.56 for depression, $32.11 for autism, and $176.19 for schizophrenia. Half of all research funded for eating disorders was 'basic' research (e.g., illness underpinning), with little investment in the development of innovative treatment models, novel therapeutics or translation, well reflected by recovery rates of less than 50% in individuals with Anorexia Nervosa.

Interpretation

Significant discrepancy remains between research funding dollars and disease burden associated with the mental health disorders. The extent to which eating disorders are underfunded may in part be attributable to inaccuracies in epidemiological and burden of disease data.

Funding

This work was in-part funded by the Australian Government Department of Health and the National Eating Disorder Research & Translation Strategy. The funder was not directly involved in informing the development of the current study.

Approaches to prioritising research for clinical trial networks: a scoping review

BACKGROUND

Prioritisation of clinical trials ensures that the research conducted meets the needs of stakeholders, makes the best use of resources and avoids duplication. The aim of this review was to identify and critically appraise approaches to research prioritisation applicable to clinical trials, to inform best practice guidelines for clinical trial networks and funders.

METHODS

A scoping review of English-language published literature and research organisation websites (January 2000 to January 2020) was undertaken to identify primary studies, approaches and criteria for research prioritisation. Data were extracted and tabulated, and a narrative synthesis was employed.

RESULTS

Seventy-eight primary studies and 18 websites were included. The majority of research prioritisation occurred in oncology and neurology disciplines. The main reasons for prioritisation were to address a knowledge gap (51 of 78 studies [65%]) and to define patient-important topics (28 studies, [35%]). In addition, research organisations prioritised in order to support their institution's mission, invest strategically, and identify best return on investment. Fifty-seven of 78 (73%) studies used interpretative prioritisation approaches (including Delphi surveys, James Lind Alliance and consensus workshops); six studies used quantitative approaches (8%) such as prospective payback or value of information (VOI) analyses; and 14 studies used blended approaches (18%) such as nominal group technique and Child Health Nutritional Research Initiative. Main criteria for prioritisation included relevance, appropriateness, significance, feasibility and cost-effectiveness.

CONCLUSION

Current research prioritisation approaches for groups conducting and funding clinical trials are largely interpretative. There is an opportunity to improve the transparency of prioritisation through the inclusion of quantitative approaches.

A Review of Web-Based Tools for Value-of-Information Analysis

Value-of-information analysis (VOI) is a decision-theoretic approach that is used to inform reimbursement decisions, optimise trial design and set research priorities. The application of VOI analysis for informing policy decisions in practice has been limited due, in part, to the perceived complexity associated with the calculation of VOI measures. Recent efforts have resulted in the development of efficient methods to estimate VOI measures and the development of user-friendly web-based tools to facilitate VOI calculations. We review the existing web-based tools including Sheffield Accelerated Value of Information (SAVI), the web interface to the BCEA (Bayesian Cost-Effectiveness Analysis) R package (BCEAweb), Rapid Assessment of Need for Evidence (RANE), and Value of Information for Cardiovascular Trials and Other Comparative Research (VICTOR). We describe what each tool is designed to do, the inputs they require, and the outputs they produce. Finally, we discuss how tools for VOI calculations might be improved in the future to facilitate the use of VOI analysis in practice.

Cost-effectiveness analysis of telephone-based cognitive behaviour therapy compared to treatment as usual CBT for cancer patients: Evidence from a small, randomised controlled trial

OBJECTIVE

A previous equivalence randomised trial indicated that Telephone-based Cognitive Behaviour Therapy (T-CBT) was not inferior to Treatment as Usual CBT (TAU-CBT) delivered face to face in terms of psychological benefit with both groups showing post-therapy improvements compared to pre-therapy baseline. The aim here is to clarify costs and benefits through an economic evaluation of the two therapy models.

METHOD

The cost-effectiveness analysis (cost per quality-adjusted life year [QALY]) was derived from a single-centre (UK-based), two-arm randomised control trial. Data from 78 patients were available for the main analysis, which includes both an NHS cost perspective and a societal perspective which includes the cost of time off work and any additional private care. Sensitivity analyses were undertaken, which included patients only completing the four core therapy sessions (46 patients) and considering only patients taking both core and the additional therapy sessions which were optional (32 patients).

RESULTS

The base-case analysis, adopting an NHS perspective, showed that T-CBT was associated with an incremental cost of £50 (95% CI: -£759 to £989) and a 0.03 QALY (95% CI: -0.09 to 0.03) decrement per patient when compared to TAU-CBT. The analysis adopting a societal perspective yielded similar results, with T-CBT providing an incremental cost of £171 (95% CI: -£769 to £1112) and a 0.03 QALY (95% CI: -0.08 to 0.03) decrement per patient in comparison to TAU-CBT. The first sensitivity analysis, considering patients only taking the core therapy sessions, showed that T-CBT provided an incremental cost of £100 (95% CI: -£945 to £1247) and yielded a decrement of 0.01 QALY (95% CI: -0.03 to 0.01) per patient compared to TAU-CBT. The second sensitivity analysis, which focused solely on patients who also underwent optional sessions, showed that T-CBT was associated with an incremental cost of £17 (95% CI: -£1307 to £1454) and a 0.04 QALY (95% CI: -0.11 to 0.03) decrement per patient when compared to TAU-CBT.

CONCLUSIONS

Based on this single trial, T-CBT is not cost-effective as a therapy option for cancer patients with high psychological needs when compared to TAU-CBT.

A Framework to Prioritise Health Research Proposals for Funding: Integrating Value for Money

When making funding decisions, research organisations largely consider the merits (e.g. scientific rigour and feasibility) of submitted research proposals; yet, there is often little or no reference to their value for money. This may be attributed to the challenges of assessing and integrating value of research into existing research prioritisation processes. We propose a framework that considers both the merits of research and its value for money to guide health research funding decisions. A practical framework is developed based on current processes followed by funding organizations for assessing investigator-initiated research proposals, and analytical methods for evaluating the expected value of research. We apply the analytical methods to estimate the expected return on investment of two real-world grant applications. The framework comprises four sequential steps: (1) initial screening of applications for eligibility and completeness; (2) merit assessment of eligible proposals; (3) estimating the expected value of research for the shortlisted proposals that pass the first two steps and ranking of proposals based on return on investment; and (4) selecting research proposals for funding. We demonstrate how the expected value for money can be efficiently estimated using certain information provided in funding applications. The proposed framework integrates value-for-money assessment into the existing research prioritisation processes. Considering value for money to inform research funding decisions is vital to achieve efficient utilisation of research budgets and maximise returns on research investments.