Audit of data redaction practices in NICE technology appraisals from 1999 to 2019

An empirical ethics study of the coherence of NICE technology appraisal policy and its implications for moral justification

BACKGROUND

As the UK's main healthcare priority-setter, the National Institute for Health and Care Excellence (NICE) has good reason to want to demonstrate that its decisions are morally justified. In doing so, it has tended to rely on the moral plausibility of its principle of cost-effectiveness and the assertion that it has adopted a fair procedure. But neither approach provides wholly satisfactory grounds for morally defending NICE's decisions. In this study we adopt a complementary approach, based on the proposition that a priority-setter's claim to moral justification can be assessed, in part, based on the coherence of its approach and that the reliability of any such claim is undermined by the presence of dissonance within its moral system. This study is the first to empirically assess the coherence of NICE's formal approach and in doing so to generate evidence-based conclusions about the extent to which this approach is morally justified.

METHODS

The study is grounded in the theory, methods and standards of empirical bioethics. Twenty NICE policy documents were coded to identify and classify the normative commitments contained within NICE technology appraisal policy as of 31 December 2021. Coherence was systematically assessed by attempting to bring these commitments into narrow reflective equilibrium (NRE) and by identifying sources of dissonance.

FINDINGS

Much of NICE policy rests on coherent values that provide a strong foundation for morally justified decision-making. However, NICE's formal approach also contains several instances of dissonance which undermine coherence and prevent NRE from being fully established. Dissonance arises primarily from four sources: i) NICE's specification of the principle of cost-effectiveness; ii) its approach to prioritising the needs of particular groups; iii) its conception of reasonableness in the context of uncertainty, and iv) its concern for innovation as an independent value.

CONCLUSION

At the time of analysis, the level of coherence across NICE policy provides reason to question the extent to which its formal approach to technology appraisal is morally justified. Some thoughts are offered on why, given these findings, NICE has been able to maintain its legitimacy as a healthcare priority-setter and on what could be done to enhance coherence.

Projecting complete redaction of clinical trial protocols (RAPTURE): redacted cross sectional study

OBJECTIVES

To characterise redactions in clinical trials and estimate a time when all protocols are fully removed (RAPTURE).

DESIGN

Redacted cross sectional study.

SETTING

Published phase 3 randomised controlled trials from 1 January 2010 to ██████████████.

PARTICIPANTS

New England Journal of Medicine, ██████████, and Journal of the American Medical Association.

MAIN OUTCOME MEASURES

█████ ████████ ██████████████ ██████ ██████████ ████████ ████████ ██████████ ███████████ ████████████ ████████████ ████████████████████████ ██████████████████ RESULTS: ████████████████████ met the inclusion criteria, with 268 (56.7%) research protocols available and accessible. The rate of redactions in protocols has increased from 0 in 2010 to 60.8% in 2021 (P<0.001). The degree of data redaction has also increased, with the average cumulative redactions among industry funded trials rising from 0 in 2010 to 3.5 pages in 2021 (P<0.001). Modelling predicts that RAPTURE is expected to occur between 2073 and 2136. Redactions featured predominantly in ████████ sponsored trials and mostly occurred in the statistical design.

CONCLUSIONS

This study highlights the rise in protocol redactions and predicts that, ██████████████████████████████████████████ will be entirely redacted between 2073 and 2136. A legitimate rationale for the redactions could ███ be found. A multipronged strategy against protocol redactions is required to maintain the integrity of science.

AVAILABILITY

This paper is partially redacted, but for the sake of ███████████, a version without any redactions can be found in the supplementary material.

Whether and How Disutilities of Adverse Events were Used in the Economic Evaluation of Drug Therapy for Cancer Treatment

BACKGROUND

The disutilities of adverse events (AEs) are important inputs for cost-utility analysis (CUA), reflecting the impacts of AEs on health outcomes. Health technology assessment institutions and scholars have proposed recommendations for applying disutility values in economic evaluations.

OBJECTIVES

This study aimed to identify the current use of disutilities of AEs as model parameters in the CUA of cancer drug therapy and to compare the discrepancies between the use of disutilities and published recommendations.

METHODS

A systematic search was conducted on the PubMed, Web of Science, and Cochrane Library databases, as well as the official websites of the National Institute for Health and Care Research (NIHR), the Institute for Clinical and Economic Review (ICER), the Institute for Quality and Efficiency in Health Care (IQWiG), the Canadian Agency for Drugs and Technologies in Health (CADTH), and the Centre for Reviews and Dissemination (CRD) for CUAs of drug therapy for cancer published in English from January 2019 to April 2022. Information about the use of disutilities of AEs (whether and how disutilities were used, or why they were not used) in selected studies was extracted and compared with published recommendations. Descriptive analyses were used to summarize the results.

RESULTS

A total of 467 CUAs were included, 54% (254/467) of which included disutilities of AEs in their model. The proportion that included these disutilities increased from 2019 to 2021, ranging from 47% (51/107) to 61% (116/190). Only 6% (15/254) of the CUAs using disutilities of AEs considered all five recommendations about the justification for inclusion and exclusion, description of values and sources, grades of AEs, calculation, and uncertainty analyses. Only 15% (72/467) provided a clear justification for inclusion and exclusion of disutilities of AEs, and 7% (17/254) did not provide values or sources. In total, 69% (175/254) of the analyses focused on AEs of grade 3 or greater, and 11% (28/254) applied utility decrements for grades 1 and 2. Disutilities of AEs were generally calculated using the incidence rates, which were clearly stated in 49% (65/132) of the analyses. Uncertainty analyses were conducted in 84% (214/254) of the CUAs.

CONCLUSIONS

The current use of disutilities of AEs in CUAs shows some discrepancies with recommendations proposed in the literature. One is that detailed information about the use of disutilities of AEs was not reported and the other is that essential methods to analyze the impact of AEs on quality-adjusted life-years were not thoroughly conducted. Therefore, it is suggested that researchers should attach importance to the impact of AEs on health-related quality of life. Furthermore, an application process was developed for the disutilities of AEs to remind and guide researchers to correctly use the disutilities of AEs as parameters in the decision-analytic model.