The Evolving Nature of Health Technology Assessment: A Critical Appraisal of NICE's New Methods Manual

An empirical ethics study of the coherence of NICE technology appraisal policy and its implications for moral justification

BACKGROUND

As the UK's main healthcare priority-setter, the National Institute for Health and Care Excellence (NICE) has good reason to want to demonstrate that its decisions are morally justified. In doing so, it has tended to rely on the moral plausibility of its principle of cost-effectiveness and the assertion that it has adopted a fair procedure. But neither approach provides wholly satisfactory grounds for morally defending NICE's decisions. In this study we adopt a complementary approach, based on the proposition that a priority-setter's claim to moral justification can be assessed, in part, based on the coherence of its approach and that the reliability of any such claim is undermined by the presence of dissonance within its moral system. This study is the first to empirically assess the coherence of NICE's formal approach and in doing so to generate evidence-based conclusions about the extent to which this approach is morally justified.

METHODS

The study is grounded in the theory, methods and standards of empirical bioethics. Twenty NICE policy documents were coded to identify and classify the normative commitments contained within NICE technology appraisal policy as of 31 December 2021. Coherence was systematically assessed by attempting to bring these commitments into narrow reflective equilibrium (NRE) and by identifying sources of dissonance.

FINDINGS

Much of NICE policy rests on coherent values that provide a strong foundation for morally justified decision-making. However, NICE's formal approach also contains several instances of dissonance which undermine coherence and prevent NRE from being fully established. Dissonance arises primarily from four sources: i) NICE's specification of the principle of cost-effectiveness; ii) its approach to prioritising the needs of particular groups; iii) its conception of reasonableness in the context of uncertainty, and iv) its concern for innovation as an independent value.

CONCLUSION

At the time of analysis, the level of coherence across NICE policy provides reason to question the extent to which its formal approach to technology appraisal is morally justified. Some thoughts are offered on why, given these findings, NICE has been able to maintain its legitimacy as a healthcare priority-setter and on what could be done to enhance coherence.

Using automated text classification to explore uncertainty in NICE appraisals for drugs for rare diseases

OBJECTIVE

This study examined the application, feasibility, and validity of supervised learning models for text classification in appraisals for rare disease treatments (RDTs) in relation to uncertainty, and analyzed differences between appraisals based on the classification results.

METHODS

We analyzed appraisals for RDTs (n = 94) published by the National Institute for Health and Care Excellence (NICE) between January 2011 and May 2023. We used Naïve Bayes, Lasso, and Support Vector Machine models in a binary text classification task (classifying paragraphs as either referencing uncertainty in the evidence base or not). To illustrate the results, we tested hypotheses in relation to the appraisal guidance, advanced therapy medicinal product (ATMP) status, disease area, and age group.

RESULTS

The best performing (Lasso) model achieved 83.6 percent classification accuracy (sensitivity = 74.4 percent, specificity = 92.6 percent). Paragraphs classified as referencing uncertainty were significantly more likely to arise in highly specialized technology (HST) appraisals compared to appraisals from the technology appraisal (TA) guidance (adjusted odds ratio = 1.44, 95 percent CI 1.09, 1.90, p = 0.004). There was no significant association between paragraphs classified as referencing uncertainty and appraisals for ATMPs, non-oncology RDTs, and RDTs indicated for children only or adults and children. These results were robust to the threshold value used for classifying paragraphs but were sensitive to the choice of classification model.

CONCLUSION

Using supervised learning models for text classification in NICE appraisals for RDTs is feasible, but the results of downstream analyses may be sensitive to the choice of classification model.