Revealed and Stated Preferences of Decision Makers for Priority Setting in Health Technology Assessment: A Systematic Review

Bring Out Your Dead: A Review of the Cost Minimisation Approach in Health Technology Assessment Submissions to the Australian Pharmaceutical Benefits Advisory Committee

OBJECTIVES

Published literature has levied criticism against the cost-minimisation analysis (CMA) approach to economic evaluation over the past two decades, with multiple papers declaring its 'death'. However, since introducing the requirements for economic evaluations as part of health technology (HTA) decision-making in 1992, the cost-minimisation analysis (CMA) approach has been widely used to inform recommendations about the public subsidy of medicines in Australia. This research aimed to highlight the breadth of use of CMA in Australia and assess the influence of preconditions for the approach on subsidy recommendations METHODS: Relevant information was extracted from Public Summary Documents of Pharmaceutical Benefits Advisory Committee (PBAC) meetings in Australia considering submissions for the subsidy of medicines that included a CMA and were assessed between July 2005 and December 2022. A generalised linear model was used to explore the relationship between whether medicines were recommended and variables that reflected the primary preconditions for using CMA set out in the published PBAC Methodology Guidelines. Other control variables were selected through the Bolasso Method. Subgroup analysis was undertaken which replicated this modelling process.

RESULTS

While the potential for inferior safety or efficacy reduced the likelihood of recommendation (p < 0.01), the effect sizes suggest that the requirements for CMA were not requisite for recommendation.

CONCLUSION

The Australian practice of CMA does not strictly align with the PBAC Methodology Guidelines and the theoretically appropriate application of CMA. However, within the confines of a deliberative HTA decision-making process that balances values and judgement with available evidence, this may be considered acceptable, particularly if stakeholders consider the current approach delivers sufficient clarity of process and enables patients to access medicines at an affordable cost.

What is the economic and social return on investment for telephone cancer information and support services in Australia? An evaluative social return on investment study protocol

INTRODUCTION

Over 50% of people affected by cancer report unmet support needs. To address unmet information and psychological needs, non-government organisations such as Cancer Councils (Australia) have developed state-based telephone cancer information and support services. Due to competing demands, evidence of the value of these services is needed to ensure that future investment makes the best use of scarce resources. This research aims to determine the costs and broader economic and social value of a telephone support service, to inform future funding and service provision.

METHODS AND ANALYSIS

A codesigned, evaluative social return on investment analysis (SROI) will be conducted to estimate and compare the costs and monetised benefits of Cancer Council Victoria's (CCV) telephone support line, 13 11 20, over 1-year and 3-year benefit periods. Nine studies will empirically estimate the parameters to inform the SROI and calculate the ratio (economic and social value to value invested): step 1 mapping outcomes (in-depth analysis of CCV's 13 11 20 recorded call data; focus groups and interviews); step 2 providing evidence of outcomes (comparative survey of people affected by cancer who do and do not call CCV's 13 11 20; general public survey); step 3 valuing the outcomes (financial proxies, value games); step 4 establishing the impact (Delphi); step 5 calculating the net benefit and step 6 service improvement (discrete choice experiment (DCE), 'what if' analysis). Qualitative (focus groups, interviews) and quantitative studies (natural language processing, cross-sectional studies, Delphi) and economic techniques (willingness-to-pay, financial proxies, value games, DCE) will be applied.

ETHICS AND DISSEMINATION

Ethics approval for each of the studies will be sought independently as the project progresses. So far, ethics approval has been granted for the first two studies. As each study analysis is completed, results will be disseminated through presentation, conferences, publications and reports to the partner organisations.

Systematic Review of the Relative Social Value of Child and Adult Health

OBJECTIVES

We aimed to synthesise knowledge on the relative social value of child and adult health.

METHODS

Quantitative and qualitative studies that evaluated the willingness of the public to prioritise treatments for children over adults were included. A search to September 2023 was undertaken. Completeness of reporting was assessed using a checklist derived from Johnston et al. Findings were tabulated by study type (matching/person trade-off, discrete choice experiment, willingness to pay, opinion survey or qualitative). Evidence in favour of children was considered in total, by length or quality of life, methodology and respondent characteristics.

RESULTS

Eighty-eight studies were included; willingness to pay (n = 9), matching/person trade-off (n = 12), discrete choice experiments (n = 29), opinion surveys (n = 22) and qualitative (n = 16), with one study simultaneously included as an opinion survey. From 88 studies, 81 results could be ascertained. Across all studies irrespective of method or other characteristics, 42 findings supported prioritising children, while 12 provided evidence favouring adults in preference to children. The remainder supported equal prioritisation or found diverse or unclear views. Of those studies considering prioritisation within the under 18 years of age group, nine findings favoured older children over younger children (including for life saving interventions), six favoured younger children and five found diverse views.

CONCLUSIONS

The balance of evidence suggests the general public favours prioritising children over adults, but this view was not found across all studies. There are research gaps in understanding the public's views on the value of health gains to very young children and the motivation behind the public's views on the value of child relative to adult health gains.

CLINICAL TRIAL REGISTRATION

The review is registered at PROSPERO number: CRD42021244593. There were two amendments to the protocol: (1) some additional search terms were added to the search strategy prior to screening to ensure coverage and (2) a more formal quality assessment was added to the process at the data extraction stage. This assessment had not been identified at the protocol writing stage.

Resource allocation in public sector programmes: does the value of a life differ between governmental departments?

BACKGROUND

The value of a life is regularly monetised by government departments for informing resource allocation. Guidance documents indicate how economic evaluation should be conducted, often specifying precise values for different impacts. However, we find different values of life and health are used in analyses by departments within the same government despite commonality in desired outcomes. This creates potential inconsistencies in considering trade-offs within a broader public sector spending budget. We provide evidence to better inform the political process and to raise important issues in assessing the value of public expenditure across different sectors.

METHODS

Our document analysis identifies thresholds, explicitly or implicitly, as observed in government-related publications in the following public sectors: health, social care, transport, and environment. We include both demand-side and supply-side thresholds, understood as societies' and governments' willingness to pay for health gains. We look at key countries that introduced formal economic evaluation processes early on and have impacted other countries' policy development: Australia, Canada, Japan, New Zealand, the Netherlands, and the United Kingdom. We also present a framework to consider how governments allocate resources across different public services.

RESULTS

Our analysis supports that identifying and describing the Value of a Life from disparate public sector activities in a manner that facilitates comparison is theoretically meaningful. The optimal allocation of resources across sectors depends on the relative position of benefits across different attributes, weighted by the social value that society puts on them. The value of a Quality-Adjusted Life Year is generally used as a demand-side threshold by Departments of transport and environment. It exceeds those used in health, often by a large enough proportion to be a multiple thereof. Decisions made across departments are generally based on an unspecified rationing rule.

CONCLUSIONS

Comparing government expenditure across different public sector departments, in terms of the value of each department outcome, is not only possible but also desirable. It is essential for an optimal resource allocation to identify the relevant social attributes and to quantify the value of these attributes for each department.

Stakeholders' Decisions and Preferences for the Provision and Use of Medicinal Cannabis: A Scoping Review

Background: The aim of this scoping review was to examine the extent that stakeholder's decisions about and preferences for the provision and use of medicinal cannabis (MC) had been investigated. We sought to identify which populations were examined, the methods used for eliciting preferences and exploring decisions, and the reported outcomes of studies. Methods: Electronic databases (PubMed, CINAHL, Embase, BSC and PsycINFO) and the reference lists of relevant articles were searched for studies published up to March 2022. Studies were included if stakeholder preferences for MC were (1) the primary focus of the research, or (2) an aspect of a larger preference focus. Studies that (3) described the decisions to use MC were also included. Results: Thirteen studies were reviewed. The population focus of these was primarily patient, with seven studies focused on general patient populations and five studies targeting specific patient populations such as cancer survivors, and people experiencing depression. Methods included health economics preference methods, qualitative interviews, and a single multicriteria decision-making study. Four categories of outcomes were defined and included comparisons of MC with a therapeutic alternative (n=5), preferences for MC attributes (n=5), administration preferences (n=4), and the decision process of users (n=2). Motivation differences in preference were found. Purely medicinal users and novice users place more importance on cannabidiol (CBD) than tetrahydrocannabinol. Overall, inhalation methods of administration were preferred due to quick onset of symptom relief. Price was the greatest influence on choice for recreational/medicinal users, whereas purely medicinal users were less price sensitive for products with higher CBD content. Conclusion: Studies examining public preferences for the provision and use of MC were absent. Revealed preference methods are a useful technique for understanding preferences for characteristics that are difficult to visibly assess such as cannabinoid or strain. The outcomes of symptom-specific multicriteria decision method studies that compare the benefit-safety profiles of commonly used treatments and MC may be a useful decision support tool for health practitioners. Studies with representative samples are needed to understand the impact of age, gender, and race on preferences for MC.

Pregnant women's and policymakers' preferences for the expansion of noninvasive prenatal screening: A discrete choice experiment approach study

Background and Aims

Quantitative approaches for eliciting preferences for new interventions are mostly conducted by patients and rarely by policymakers. This study aimed to quantify the preferences of pregnant women and policymakers regarding the addition of a new test to prenatal screening programs for detecting chromosomal abnormalities.

Methods

A discrete choice experiment was conducted to measure the respondents' preferences for a new prenatal test. A seven-attribute instrument was built based on interviews with pregnant women and policymakers. The data were analyzed using robust conditional logistic regression and nested logit models.

Results

In total, 272 pregnant women and 24 policymakers completed the questionnaire (response rates of 48% and 55%, respectively). Overall, all attributes were statistically significant in the pregnant women group, whereas only three attributes (test performance, degree of test result certainty, and cost) were statistically significant in the policymakers group. Statistically significant differences in test performance and information were observed between the two groups.

Conclusion

Policymakers differed from pregnant women in their appraisal of attributes related to their preference for a new prenatal screening intervention. The low response rates observed in both groups suggest that further investigation of the relevance of this approach must be conducted.

Barriers and facilitators to implementing priority setting and resource allocation tools in hospital decisions: A systematic review

Health care budgets in high-income countries are having issues coping with unsustainable growth in demand, particularly in the hospital setting. Despite this, implementing tools systematising priority setting and resource allocation decisions has been challenging. This study answers two questions: (1) what are the barriers and facilitators to implementing priority setting tools in the hospital setting of high-income countries? and (2) what is their fidelity? A systematic review using the Cochrane methods was conducted including studies of hospital-related priority setting tools reporting barriers or facilitators for implementation, published after the year 2000. Barriers and facilitators were classified using the Consolidated Framework for Implementation Research (CFIR). Fidelity was assessed using priority setting tool's standards. Out of thirty studies, ten reported program budgeting and marginal analysis (PBMA), twelve multi-criteria decision analysis (MCDA), six health technology assessment (HTA) related frameworks, and two, an ad hoc tool. Barriers and facilitators were outlined across all CFIR domains. Implementation factors not frequently observed, such as 'evidence of previous successful tool application', 'knowledge and beliefs about the intervention' or 'external policy and incentives' were reported. Conversely, some constructs did not yield any barrier or facilitator including 'intervention source' or 'peer pressure'. PBMA studies satisfied the fidelity criteria between 86% and 100%, for MCDA it varied between 36% and 100%, and for HTA it was between 27% and 80%. However, fidelity was not related to implementation. This study is the first to use an implementation science approach. Results represent the starting point for organisations wishing to use priority setting tools in the hospital setting by providing an overview of barriers and facilitators. These factors can be used to assess readiness for implementation or to serve as the foundation for process evaluations. Through our findings, we aim to improve the uptake of priority setting tools and support their sustainable use.

Conjoint Analysis: A Research Method to Study Patients’ Preferences and Personalize Care

This article aims to describe the conjoint analysis (CA) method and its application in healthcare settings, and to provide researchers with a brief guide to conduct a conjoint study. CA is a method for eliciting patients’ preferences that offers choices similar to those in the real world and allows researchers to quantify these preferences. To identify literature related to conjoint analysis, a comprehensive search of PubMed (MEDLINE), EMBASE, Web of Science, and Google Scholar was conducted without language or date restrictions. To identify the trend of publications and citations in conjoint analysis, an online search of all databases indexed in the Web of Science Core Collection was conducted on the 8th of December 2021 without time restriction. Searching key terms covered a wide range of synonyms related to conjoint analysis. The search field was limited to the title, and no language or date limitations were applied. The number of published documents related to CA was nearly 900 during the year 2021 and the total number of citations for CA documents was approximately 20,000 citations, which certainly shows that the popularity of CA is increasing, especially in the healthcare sciences services discipline, which is in the top five fields publishing CA documents. However, there are some limitations regarding the appropriate sample size, quality assessment tool, and external validity of CA.

Preference to Family Doctor Contracted Service of Patients with Chronic Disease in Urban China: A Discrete Choice Experiment

OBJECTIVE

Shanghai is one of the pioneers proposing family doctor contract service (FDCS). However, there is no quantitative research focusing on the Shanghai experience from a demand-side perspective. This study investigated Shanghai chronic patients' relative preferences for FDCS using a discrete choice experiment method.

METHODS

A face-to-face discrete choice experiment (DCE) was performed to elicit the preference with 300 samples. Attributes and levels were extracted from the literature review and focus group consultation with patients. Seven attributes, follow-up frequency, medicine accessibility, family doctor competency, health management, referral convenience, appointment flexibility, and shared decision-making, were decided. Three levels were attached to each attribute. A mixed logit model was used to evaluate the multiple-choice data.

RESULTS

A total of 248 patients completed the survey. Patient valued FDCS medicine accessibility (β=0.57, P < 0.05), and high family doctor competency (β= 0.43, P < 0.05), regular health management activities (β=0.36, P < 0.05), high follow-up frequency (β=0.31, P < 0.05) the most. The good doctor-patient shared decision-making atmosphere (β=0.12, P < 0.05), high referral convenience (β=0.06, P < 0.05) and high appointment flexibility (β=0.04, P < 0.05) are valued as less important. No significant preference heterogeneity was identified for patients with different sociodemographic characteristics. Respondents reported other FDCS needs, including online health consultation, specialist services in local institutes, higher reimbursement rates, free rehabilitation guidance for the disabled and personal health management.

CONCLUSION

This research is the first discrete choice experiment FDCS preference research targeting on Chinese urban population. The results suggested that to increase the quality of FDCS, policy-makers should prioritize follow-up frequency, medicine accessibility, family doctor competency and health management. The service package should consider a higher reimbursement rate and rehabilitation guidance for the disabled if extra health-care resources available. Future FDCS policy should consider stated societal preference and be congruent with it.

Coverage and Pricing Recommendations of the French National Health Authority for Innovative Drugs: A Retrospective Analysis From 2014 to 2020

OBJECTIVES

This study provides a retrospective analysis of the recommendations of the French National Health Authority on the reimbursement and pricing of innovative drugs.

METHODS

The analysis includes drugs subjected to both economic and clinical evaluations in France from 2014 to 2020. Ordered logistic and quantile regressions are used to estimate the factors associated with the clinical value (SMR), the clinical added value (ASMR), and the incremental cost-utility ratio (ICUR) of innovative drugs. All variables used in the regression analyses are extracted from the Clinical and Economic Opinions for the 146 observations.

RESULTS

Regression analyses indicate that 2 of the 5 official criteria, the efficacy-adverse events balance of the drug and its function, are significantly associated with the SMR rating. The ASMR is positively associated with the disease severity, the quality-adjusted life-year (QALY) gain provided by the drug, and the validation of the ICUR in the Economic Opinion. At the first quartile of the ICUR distribution (approximately €50 000/QALY), higher ICUR levels are observed for drugs with a smaller target population and for drugs claimed as more innovative. Higher ICUR levels are also observed for pediatric drugs and for drugs with no therapeutic alternative at the third quartile of the distribution (approximately €240 000/QALY).

CONCLUSIONS

Not all official criteria of the SMR are associated with actual ratings obtained. Regarding the ASMR, the results support the idea of a convergence between the 2 independent clinical and economic appraisal processes. Finally, the factors influencing the ICUR level vary across the distribution of ICUR.

Healthcare Funding Decisions and Real-World Benefits: Reducing Bias by Matching Untreated Patients

Governments and health insurers often make funding decisions based on health gains from randomised controlled trials. These decisions are inherently uncertain because health gains in trials may not translate to practice owing to differences in the population, treatment use and setting. Post-market analysis of real-world data can provide additional evidence but estimates from standard matching methods may be biased when unobserved characteristics explain whether a patient is treated and their outcomes. We propose a new untreated matching approach that can reduce this bias. Our approach utilises the outcomes of contemporaneous untreated patients to improve the matching of treated and historical control patients. We assess the performance of this new approach compared to standard matching using a simulation study and demonstrate the steps required using a funding decision for prostate cancer treatments in Australia. Our simulation study shows that our new matching approach eliminates nearly all bias when unobserved treatment selection is related to outcomes, and outperforms standard matching in most scenarios. In our empirical example, standard matching overestimated survival by 15% (95% confidence interval 2-34) compared to our untreated matching approach. The health gains estimated using our approach were slightly lower than expected based on the trial evidence, but we also found evidence that in practice prescribers ceased prior therapies earlier, treated a more vulnerable population and continued treatment for longer. Our untreated matching approach offers researchers a new tool for reducing uncertainty in healthcare funding decisions using real-world data.

Ranking the Criteria Used in the Appraisal of Drugs for Reimbursement: A Stated Preferences Elicitation With Health Technology Assessment Stakeholders Across Jurisdictional Contexts

OBJECTIVES

Our goal was to estimate the relative importance assigned to health technology assessment (HTA) criteria by stakeholders involved in the HTA process. HTA is an increasingly common framework used in the appraisal of drugs for public reimbursement. It identifies clinical, economic, social, and organizational criteria to be considered. The criteria can vary across jurisdictions and are typically appraised by multidisciplinary expert committees. Guidance on the relative weighing of criteria is often absent.

METHODS

We elicited stakeholders' preferences using a single-scenario discrete choice experiment and a best-worst scaling model with conviction scores to assess the weights assigned to selected criteria by HTA stakeholders. We recruited 111 HTA stakeholders across multiple jurisdictions, including members of expert committees, clinical and economic experts, patients, and public payer representatives. Each judged twelve hypothetical cancer drug profiles for suitability for public funding and identified which characteristics were best and worst. In addition to standard discrete choice experiment and best-worst scaling models, we estimated a hybrid model to obtain a ranking of criteria by importance they played in the appraisal.

RESULTS

A strong clinical benefit proved the most important criterion, followed by cost considerations, presence of adverse events, and availability of other treatments. The importance of clinical benefit was moderated by unmet need, adverse events, and number of patients.

CONCLUSION

Policymakers might want to consider providing an explicit weighing scheme, or moving to a 2-stage selection process with an assessment of the quality of clinical evidence as a gatekeeping step for a full HTA review.

Development of the WHO-INTEGRATE evidence-to-decision framework: an overview of systematic reviews of decision criteria for health decision-making

Background

Decision-making in public health and health policy is complex and requires careful deliberation of many and sometimes conflicting normative and technical criteria. Several approaches and tools, such as multi-criteria decision analysis, health technology assessments and evidence-to-decision (EtD) frameworks, have been proposed to guide decision-makers in selecting the criteria most relevant and appropriate for a transparent decision-making process. This study forms part of the development of the WHO-INTEGRATE EtD framework, a framework rooted in global health norms and values as reflected in key documents of the World Health Organization and the United Nations system. The objective of this study was to provide a comprehensive overview of criteria used in or proposed for real-world decision-making processes, including guideline development, health technology assessment, resource allocation and others.

Methods

We conducted an overview of systematic reviews through a combination of systematic literature searches and extensive reference searches. Systematic reviews reporting criteria used for real-world health decision-making by governmental or non-governmental organization on a supranational, national, or programme level were included and their quality assessed through a bespoke critical appraisal tool. The criteria reported in the reviews were extracted, de-duplicated and sorted into first-level (i.e. criteria), second-level (i.e. sub-criteria) and third-level (i.e. decision aspects) categories. First-level categories were developed a priori using a normative approach; second- and third-level categories were developed inductively.

Results

We included 36 systematic reviews providing criteria, of which one met all and another eleven met at least five of the items of our critical appraisal tool. The criteria were subsumed into 8 criteria, 45 sub-criteria and 200 decision aspects. The first-level of the category system comprised the following seven substantive criteria: “Health-related balance of benefits and harms”; “Human and individual rights”; “Acceptability considerations”; “Societal considerations”; “Considerations of equity, equality and fairness”; “Cost and financial considerations”; and “Feasibility and health system considerations”. In addition, we identified an eight criterion “Evidence”.

Conclusion

This overview of systematic reviews provides a comprehensive overview of criteria used or suggested for real-world health decision-making. It also discusses key challenges in the selection of the most appropriate criteria and in seeking to implement a fair decision-making process.

The impact of vaccination on gender equity: conceptual framework and human papillomavirus (HPV) vaccine case study

BACKGROUND

Although the beneficial effects of vaccines on equity by socioeconomic status and geography are increasingly well-documented, little has been done to extend these analyses to examine the linkage between vaccination and gender equity. In this paper, evidence from the published literature is used to develop a conceptual framework demonstrating the potential impact of vaccination on measures of gender equity. This framework is then applied to human papillomavirus (HPV) vaccination in three countries with different economic and disease burden profiles to establish a proof of concept in a variety of contexts.

METHODS

We conducted a literature review examining evidence on the linkage between health outcomes and dimensions of gender equity. We utilized the Papillomavirus Rapid Interface for Modelling and Economics (PRIME) model to estimate cervical cancer incidence and deaths due to HPV types 16/18 by age in each country. We estimated labor force participation and fertility effects from improvements in health, and converted these into inputs consistent with those used to calculate the United Nations Gender Inequality Index to assess gender equity.

RESULTS

In our case study, we found that HPV vaccination among girls could help narrow socioeconomic gender disparities by quantifying the main pathways by which HPV vaccination improves health, which enables improvement in gender equity indicators such as labor force participation and maternal mortality ratios. While these improvements are small when averaged over the entire population, the components measured - labor force participation and maternal mortality ratio - account for 50% of the index scores.

CONCLUSIONS

This proof of concept model is a starting point to inform future health and economic analyses that might incorporate the impact of gender equity as an additional impact of vaccination in improving the health and well-being of the population.

Determining Value in Health Technology Assessment: Stay the Course or Tack Away?

The economic evaluation of new health technologies to assess whether the value of the expected health benefits warrants the proposed additional costs has become an essential step in making novel interventions available to patients. This assessment of value is problematic because there exists no natural means to measure it. One approach is to assume that society wishes to maximize aggregate health, measured in terms of quality-adjusted life-years (QALYs). Commonly, a single 'cost-effectiveness' threshold is used to gauge whether the intervention is sufficiently efficient in doing so. This approach has come under fire for failing to account for societal values that favor treating more severe illness and ensuring equal access to resources, regardless of pre-existing conditions or capacity to benefit. Alternatives involving expansion of the measure of benefit or adjusting the threshold have been proposed and some have advocated tacking away from the cost per QALY entirely to implement therapeutic area-specific efficiency frontiers, multicriteria decision analysis or other approaches that keep the dimensions of benefit distinct and value them separately. In this paper, each of these alternative courses is considered, based on the experiences of the authors, with a view to clarifying their implications.

Health technology assessment (HTA) and economic evaluation: efficiency or fairness first

The economic evaluation which supports Health Technology Assessment (HTA) should inform policy makers of the value to society conferred by a given allocation of resources. However, neither the theory nor practise of economic evaluation satisfactorily reflect social values. Both are primarily concerned with efficiency, commonly conceptualised as the maximisation of utility or quality adjusted life years (QALYs). The focus is upon the service and the benefits obtained from it. This has resulted in an evaluation methodology which discriminates against groups and treatments which the population would like to prioritise. This includes high cost treatments for patients with rare diseases. In contrast with prevailing methods, there is increasing evidence that the public would prefer a fairness-focused framework in which the service was removed from centre stage and replaced by the patient. However methods for achieving fairness are ad hoc and under-developed. The article initially reviews the theory of economic evaluation and argues that its focus upon individual utility and efficiency as defined by the theory omits potentially important social values. Some empirical evidence relating to population values is presented and four studies by the first author are reviewed. These indicate that when people adopt the social perspective of a citizen they have a preference for sharing the health budget in a way which does not exclude patients who require services that are not cost effective, such as orphan medicinal products (OMP's) and treatments for patients with ultra-rare diseases (URD's).

Weighing of Evidence by Health Technology Assessment Bodies: Retrospective Study of Reimbursement Recommendations for Conditionally Approved Drugs

This study assessed whether five Health Technology Assessment (HTA) bodies in Europe were more negative about drugs with a Conditional Marketing Authorization (CMA) that are approved without controlled studies compared to CMA drugs that are approved based on controlled studies. The HTA recommendations were categorized into positive, restricted, and negative. A total of 92 HTA recommendations were available for 27 drugs. Thirty of 62 (48%) and 17 of 30 (57%) of the recommendations were negative for drugs with and without controlled studies, respectively. Overall, only 12 (13%) recommendations were positive. In all jurisdictions, recommendations between drugs with and drugs without controlled data were comparable, which suggests that the presence of controlled data is not decisive in HTA evaluations. The small proportion of unrestricted positive recommendations highlights difficulties with recommending the drugs in this cohort, which may be caused by scientific uncertainty or other factors. Earlier collaboration between stakeholders is advised in order to improve patient access.