Still Elegantly Muddling Through? NICE and Uncertainty in Decision Making About the Rationing of Expensive Medicines in England

We need to talk about values: a proposed framework for the articulation of normative reasoning in health technology assessment

It is acknowledged that health technology assessment (HTA) is an inherently value-based activity that makes use of normative reasoning alongside empirical evidence. But the language used to conceptualise and articulate HTA's normative aspects is demonstrably unnuanced, imprecise, and inconsistently employed, undermining transparency and preventing proper scrutiny of the rationales on which decisions are based. This paper - developed through a cross-disciplinary collaboration of 24 researchers with expertise in healthcare priority-setting - seeks to address this problem by offering a clear definition of key terms and distinguishing between the types of normative commitment invoked during HTA, thus providing a novel conceptual framework for the articulation of reasoning. Through application to a hypothetical case, it is illustrated how this framework can operate as a practical tool through which HTA practitioners and policymakers can enhance the transparency and coherence of their decision-making, while enabling others to hold them more easily to account. The framework is offered as a starting point for further discussion amongst those with a desire to enhance the legitimacy and fairness of HTA by facilitating practical public reasoning, in which decisions are made on behalf of the public, in public view, through a chain of reasoning that withstands ethical scrutiny.

An empirical ethics study of the coherence of NICE technology appraisal policy and its implications for moral justification

BACKGROUND

As the UK's main healthcare priority-setter, the National Institute for Health and Care Excellence (NICE) has good reason to want to demonstrate that its decisions are morally justified. In doing so, it has tended to rely on the moral plausibility of its principle of cost-effectiveness and the assertion that it has adopted a fair procedure. But neither approach provides wholly satisfactory grounds for morally defending NICE's decisions. In this study we adopt a complementary approach, based on the proposition that a priority-setter's claim to moral justification can be assessed, in part, based on the coherence of its approach and that the reliability of any such claim is undermined by the presence of dissonance within its moral system. This study is the first to empirically assess the coherence of NICE's formal approach and in doing so to generate evidence-based conclusions about the extent to which this approach is morally justified.

METHODS

The study is grounded in the theory, methods and standards of empirical bioethics. Twenty NICE policy documents were coded to identify and classify the normative commitments contained within NICE technology appraisal policy as of 31 December 2021. Coherence was systematically assessed by attempting to bring these commitments into narrow reflective equilibrium (NRE) and by identifying sources of dissonance.

FINDINGS

Much of NICE policy rests on coherent values that provide a strong foundation for morally justified decision-making. However, NICE's formal approach also contains several instances of dissonance which undermine coherence and prevent NRE from being fully established. Dissonance arises primarily from four sources: i) NICE's specification of the principle of cost-effectiveness; ii) its approach to prioritising the needs of particular groups; iii) its conception of reasonableness in the context of uncertainty, and iv) its concern for innovation as an independent value.

CONCLUSION

At the time of analysis, the level of coherence across NICE policy provides reason to question the extent to which its formal approach to technology appraisal is morally justified. Some thoughts are offered on why, given these findings, NICE has been able to maintain its legitimacy as a healthcare priority-setter and on what could be done to enhance coherence.

An Analysis of Uncertainties and Data Collection Agreements in the Cancer Drugs Fund

BACKGROUND

Managed Access Agreements (MAAs) are a commercial arrangement that provide patients earlier access to innovative health technologies while uncertainties in the evidence base are resolved through data collection. In the UK, data collection agreements (DCAs) outline the evidence that will be collected during the MAA period and are intended to resolve uncertainties in the clinical- and cost-effectiveness of a technology sufficient for the National Institute of Health and Care Excellence (NICE) committee to make a final decision on reimbursement.

OBJECTIVE

The aim of this study was to identify the primary uncertainties leading to a recommendation for entry to the Cancer Drugs Fund (CDF) and evaluate how the corresponding DCAs attempt to address these.

METHODS

A database of MAAs agreed within the CDF was compiled with coverage between July 2016 and December 2020 (the time during which evidence generation was routinely collected within the CDF up until the time of analysis). Uncertainties in the evidence base for technologies entering the CDF were analysed alongside the outcomes planned for data collection during the MAA. These data provide an overview of the key uncertainties surrounding health technologies in the CDF on entry and the types of evidence targeted by DCAs.

RESULTS

In the assessment of 39 Cancer Drugs Fund (CDF) cases, NICE committees identified a total of 108 key uncertainties in cost-effectiveness estimates. Overall survival was the most commonly identified uncertainty, followed by generalisability of the evidence to the target population. DCAs specified a range of outcomes relevant to understanding the clinical effectiveness of the technology, though fewer than half (43.6%) of the DCAs addressed all the key uncertainties identified by the NICE committee.

CONCLUSION

The analysis indicated that data collection within the CDF is not sufficient to resolve all the uncertainties identified by the NICE committee, meaning that other approaches will be needed at re-appraisal to ensure that the NICE committee can reach a final decision on reimbursement.

Institutional Priority-Setting for Novel Drugs and Therapeutics: A Qualitative Systematic Review

BACKGROUND

There is a lack of guidance on approaches to formulary management and funding for high-cost drugs and therapeutics by individual healthcare institutions. The objective of this review was to assess institutional approaches to resource allocation for such therapeutics, with a particular focus on paediatric and rare disease populations.

METHODS

A search of Embase and MEDLINE was conducted for studies relevant to decision-making for off-formulary, high-cost drugs and therapeutics. Abstracts were evaluated for inclusion based on the Simple Multiple-Attribute Rating Techniques (SMART) criteria. A framework of 30 topics across 4 categories was used to guide data extraction and was based on findings from the initial abstract review and previous health technology assessment (HTA) publications. Reflexive thematic analysis was conducted using QSR NVivo 12 software.

RESULTS

A total of 168 studies were included for analysis. Only 4 (2%) focused on paediatrics, while 21 (12%) centred on adults and the remainder (85%) did not specify. Thirty-two (19%) studies discussed the importance of high-cost therapeutics and 34 (23%) focused on rare/orphan drugs. Five themes were identified as being relevant to institutional decision-making for high-cost therapeutics: institutional strategy, substantive criteria, procedural considerations, guiding principles and frameworks, and operational activities. Each of these themes encompassed several sub-themes and was complemented by a sixth category specific to paediatrics and rare diseases.

CONCLUSION

The rising cost of novel drugs and therapeutics underscores the need for robust, evidence-based, and ethically defensible decision-making processes for health technology funding, particularly at the hospital level. Our study highlights practices and themes to aid decision-makers in thinking critically about institutional, substantive, procedural, and operational considerations in support of legitimate decisions about institutional funding of high-cost drugs and therapeutics, as well as opportunities and challenges that exist for paediatric and rare disease populations.

Good conduct in a context of rationing: A case study of how frontline professionals deal with distributive dilemmas of novel gene therapies

Classical dilemmas of how to distribute limited resources have been rekindled by the rise of advanced, high-cost therapies. Building on a case study of a novel gene therapy in neuropaediatric care, this article explores the dilemmas that explicit priority setting can create for frontline professionals and develops a typology of professionals' responses to these dilemmas. Despite political attempts to centralise priority setting and spare health professionals from having to consider treatment costs at the 'bedside', this study shows that concern for economic efficiency and budget control nonetheless need to be handled and balanced against other accountabilities in the daily work of frontline professionals. Contributing to the sociological debate on priority setting and rationing, this study develops an analytical perspective attuned to the relational aspects of frontline work and the challenges related to the balancing of diverging ideas of good conduct. Further, focussing on an empirical field at the forefront of genomic medicine, this study brings the sociological debate on priority setting and rationing up to date with current developments in precision medicine.

The normative grounds for NICE decision-making: a narrative cross-disciplinary review of empirical studies

The National Institute for Health and Care Excellence (NICE) is the UK's primary health care priority-setter, responsible for advising the National Health Service on its adoption of health technologies. The normative basis for NICE's advice has long been the subject of public and academic interest, but the existing literature does not include any comprehensive summary of the factors observed to have substantively shaped NICE's recommendations. The current review addresses this gap by bringing together 29 studies that have explored NICE decision-making from different disciplinary perspectives, using a range of quantitative and qualitative methods. It finds that although cost-effectiveness has historically played a central role in NICE decision-making, 10 other factors (uncertainty, budget impact, clinical need, innovation, rarity, age, cause of disease, wider societal impacts, stakeholder influence and process factors) are also demonstrably influential and interact with one another in ways that are not well understood. The review also highlights an over-representation in the literature of appraisals conducted prior to 2009, according to methods that have since been superseded. It suggests that this may present a misleading view of the importance of allocative efficiency to NICE's current approach and illustrates the need for further up-to-date research into the normative grounds for NICE's decisions.

The sociology of rationing: Towards increased interdisciplinary dialogue - A critical interpretive literature review

Since the 1990s, the sociology of rationing has developed in explicit opposition to health economic and bioethical approaches to healthcare rationing. This implies a limited engagement with other disciplines and a limited impact on political debates. To bring the sociology of rationing into an interdisciplinary dialogue, it is important to understand the disciplines' analytical differences and similarities. Based on a critical interpretive literature synthesis, this article examines four disciplinary perspectives on healthcare rationing and priority setting: (1) Health economics, which seeks to develop decision models to provide for more rational resource allocation; (2) Bioethics, which seeks to develop normative principles and procedures to facilitate a just allocation of resources; (3) Health policy studies, which focus on issues of legitimacy and implementation of decision models; and lastly (4) Sociology, which analyses the uncertainty of rationing and the resulting value conflicts and negotiations. The article provides an analytical overview and suggestions on how to advance the impact of sociological arguments in future rationing debates: Firstly, we discuss how to develop the concepts and assumptions of the sociology of rationing. Secondly, we identify specific themes relevant for sociological inquiry, including the recurring problem of how to translate administrative priority setting decisions into clinical practice.

Under careful construction: combining findings, arguments, and values into robust health care coverage decisions

Background

Health care coverage decisions deal with health care technology provision or reimbursement at a national level. The coverage decision report, i.e., the publicly available document giving reasons for the decision, may contain various elements: quantitative calculations like cost and clinical effectiveness analyses and formalised and non-formalised qualitative considerations. We know little about the process of combining these heterogeneous elements into robust decisions.

Methods

This study describes a model for combining different elements in coverage decisions. We build on two qualitative cases of coverage appraisals at the Dutch National Health Care Institute, for which we analysed observations at committee meetings (n = 2, with field notes taken) and the corresponding audio files (n = 3), interviews with appraisal committee members (n = 10 in seven interviews) and with Institute employees (n = 5 in three interviews), and relevant documents (n = 4).

Results

We conceptualise decisions as combinations of elements, specifically (quantitative) findings and (qualitative) arguments and values. Our model contains three steps: 1) identifying elements; 2) designing the combinations of elements, which entails articulating links, broadening the scope of designed combinations, and black-boxing links; and 3) testing these combinations and choosing one as the final decision.

Conclusions

Based on the proposed model, we suggest actively identifying a wider variety of elements and stepping up in terms of engaging patients and the public, including facilitating appeals. Future research could explore how different actors perceive the robustness of decisions and how this relates to their perceived legitimacy.

Decision-making in the midst of uncertainty: appraising expensive medicines in England

Decisions need to be made about which services or technologies should be prioritized for provision in the NHS in England .The analysis focuses specifically on the National Institute for Health and Care Excellence (NICE), and on how they appraise expensive medicines. This analysis takes a sociological perspective on decision-making in relation to uncertainty and how uncertainties are managed, drawing on evidence from a scoping study and an ethnographic study. Uncertainties were central to these rationing decisions. Three types of layers of uncertainty -epistemic, procedural and interpersonal - were shown to be salient. Another form of uncertainty was associated with the complexity of the science and that included the level of technicality of the information provided. The analysis highlighted the salience of uncertainties associated with interpersonal relations and the relations between the committees and the drug industry, clinical and patient experts. A key element in these relationships was trust. Decision makers adopted a mixture of formal and informal, collective and individual strategies in making decisions and a need to exercise pragmatism within a more formal institutional framework. The paper concludes by considering more recent policy developments in relation to appraising expensive medicines.

Transforming uncertainties into legitimate regulation? NICE and CONITEC agencies' decisions on rare diseases

As a scientific and technological practice, the evaluation of health technologies (HTA) is, at the same time, a challenge to determine the value of the technologies to be incorporated. This study aimed to explore and compare the results and technical elements of the evaluations issued for rare diseases between the English (NICE) and the Brazilian agency (CONITEC). The first part of the study involved the systematic search for evaluations from 2013 to 2019. In the second stage, the reports were analyzed based on: (i) descriptive narrative review; and (ii) calculation of the absolute and relative frequency according to each domain and component (element) applied in the European HTA network model. Twenty-four medicines were distinctly assessed during the study period. Through 126 questions (elements) distributed among nine domains, the analysis revealed that 67 (53.2%) and 44 (35.0%) were described in the reports, 42 (33.3%) and 59 (47.0 %) were only considered partially, and 17 (13.5%) and 23 (18.0%) were not considered in the NICE and CONITEC reports, respectively. We identified a relatively low agreement between the Brazilian agency with the English agency in the reports issued for rare diseases. It remains to be seen whether the agencies are able to capture the various values ​​of these medicines, as well as manage uncertainties in the evaluations.

Necessity under construction - societal weighing rationality in the appraisal of health care technologies

Health care coverage decisions may employ many different considerations, which are brought together across two phases. The assessment phase examines the available scientific evidence, such as the cost-effectiveness, of the technology. The appraisal then contextualises this evidence to arrive at an (advised) coverage decision, but little is known about how this is done.In the Netherlands, the appraisal is set up to achieve a societal weighing and is the primary place where need- and solidarity-related ('necessity') argumentations are used. To elucidate how the Dutch appraisal committee 'constructs necessity', we analysed observations and recordings of two appraisal committee meetings at the National Health Care Institute, the corresponding documents (five), and interviews with committee members and policy makers (13 interviewees in 12 interviews), with attention to specific necessity argumentations.The Dutch appraisal committee constructs necessity in four phases: (1) allowing explicit criteria to steer the process; (2) allowing patient (representative) contributions to challenge the process; (3) bringing new argumentations in from outside and weaving them together; and (4) formulating recommendations to societal stakeholders. We argue that in these ways, the appraisal committee achieves societal weighing rationality, as the committee actively uses argumentations from society and embeds the decision outcome in society.

The Emerging Social Science Literature on Health Technology Assessment: A Narrative Review

BACKGROUND

Social scientists have paid increasing attention to health technology assessment (HTA). This paper provides an overview of existing social scientific literature on HTA, with a focus on sociology and political science and their subfields.

METHODS

Narrative review of key pieces in English.

RESULTS

Three broad themes recur in the emerging social science literature on HTA: the drivers of the establishment and concrete institutional designs of HTA bodies; the effects of institutionalized HTA on pricing and reimbursement systems and the broader society; and the social and political influences on HTA decisions.

CONCLUSION

Social scientists bring a focus on institutions and social actors involved in HTA, using primarily small-N research designs and qualitative methods. They provide valuable critical perspectives on HTA, at times challenging its otherwise unquestioned assumptions. However, they often leave aside questions important to the HTA practitioner community, including the role of culture and values. Closer collaboration could be beneficial to tackle new relevant questions pertaining to HTA.

Decisions of Value: Going Backstage Comment on "Contextual Factors Influencing Cost and Quality Decisions in Health and Care: A Structured Evidence Review and Narrative Synthesis"

This commentary expands on two of the key themes briefly raised in the paper involving analysis of the evidence about key contextual influences on decisions of value. The first theme focuses on the need to explore in more detail what is called backstage decision-making looking at how actual decisions are made drawing on evidence from ethnographies about decision-making. These studies point to less of an emphasis on instrumental and calculative forms of decision-making with more of an emphasis on more pragmatic rationality. The second related theme picks up on the issue of sources of information as a contextual influence particularly highlighting the salience of uncertainty or information deficits. It is argued that there are a range of different types of uncertainties, not only associated with information deficits, which are found particularly in allocative types of decisions of value. This means that the decision-making process although attempting to be linear and rational, tends to be characterised by a form of navigation where the decision-makers navigate their way through the uncertainties inherent and overtly manifested in the decision-making process.

Decision making in NICE single technological appraisals: How does NICE incorporate patient perspectives?

CONTEXT

The National Institute for Health and Care Excellence (NICE) has an explicit mandate to include patient and public involvement in the appraisal of medicines to be available for funding on the NHS. NICE involves an appraisal committee who are required to take on board experiential evidence from patient experts alongside population-based evidence on clinical and cost-effectiveness when making a decision whether to fund a drug.

OBJECTIVE

This paper considers how NICE Single Technological Appraisal (STA) committees attempt to incorporate the views of patients in making decisions about funding medicines on the NHS.

METHODS

A prospective design was employed to follow three pharmaceutical products involving three different appraisal committees. Three data collection methods were used: analysis of documentary evidence sent by NICE, non-participant unstructured observations of the open and closed sessions of meetings and qualitative interviews.

SETTINGS AND PARTICIPANTS

Unstructured non-participant observations were carried out at nine STA meetings, and 41 semi-structured interviews were undertaken with committee members from NICE's STA committees, patient experts, analysts from NICE's project team and drug manufacturers.

RESULTS

Our analysis showed how the committees displayed a preference for an ideal-type of patient representative, disagreement among the committee when weighing-up patient statements in the STA process and more pre-preparation support for patient involvement.

CONCLUSIONS

Although NICE has attempted to adopt an approach flexible to patients and carers through formal decision-making arrangements that incorporate patient views, nonetheless, the processes of the STAs can in fact undermine the very evidence collected from patient representatives.

Evaluating Quality of Decision-Making Processes in Medicines' Development, Regulatory Review, and Health Technology Assessment: A Systematic Review of the Literature

Introduction: Although pharmaceutical companies, regulatory authorities, and health technology assessment (HTA) agencies have been increasingly using decision-making frameworks, it is not certain whether these enable better quality decision making. This could be addressed by formally evaluating the quality of decision-making process within those organizations. The aim of this literature review was to identify current techniques (tools, questionnaires, surveys, and studies) for measuring the quality of the decision-making process across the three stakeholders.

Methods: Using MEDLINE, Web of Knowledge, and other Internet-based search engines, a literature review was performed to systematically identify techniques for assessing quality of decision making in medicines development, regulatory review, and HTA. A structured search was applied using key words and a secondary review was carried out. In addition, the measurement properties of each technique were assessed and compared. Ten Quality Decision-Making Practices (QDMPs) developed previously were then used as a framework for the evaluation of techniques identified in the review. Due to the variation in studies identified, meta-analysis was inappropriate.

Results: This review identified 13 techniques, where 7 were developed specifically to assess decision making in medicines' development, regulatory review, or HTA; 2 examined corporate decision making, and 4 general decision making. Regarding how closely each technique conformed to the 10 QDMPs, the 13 techniques assessed a median of 6 QDMPs, with a mode of 3 QDMPs. Only 2 techniques evaluated all 10 QDMPs, namely the Organizational IQ and the Quality of Decision Making Orientation Scheme (QoDoS), of which only one technique, QoDoS could be applied to assess decision making of both individuals and organizations, and it possessed generalizability to capture issues relevant to companies as well as regulatory authorities.

Conclusion: This review confirmed a general paucity of research in this area, particularly regarding the development and systematic application of techniques for evaluating quality decision making, with no consensus around a gold standard. This review has identified QoDoS as the most promising available technique for assessing decision making in the lifecycle of medicines and the next steps would be to further test its validity, sensitivity, and reliability.