Composite outcome measurement in clinical research: the triumph of illusion over reality?

Prosthesis usability experience is associated with extent of upper limb prosthesis adoption: A Structural Equation Modeling (SEM) analysis

Factors associated with upper limb prosthesis adoption are not well understood. In this study, we explored how prosthesis usability experience relates to the extent of prosthesis adoption through the development of a structural equation model (SEM). First, items related to prosthesis usability were developed and refined using cognitive testing and pilot testing and employed in a survey of 402 prosthesis users (mean age 61.7 (sd 14.4), 77.1% Veterans). The SEM examined two unidimensional latent constructs: Prosthesis Usability Experience and Prosthesis Adoption-and each had multiple measured indicators. SEMs tested direct as well as moderating and mediating effects between the latent constructs and covariates related to demographics and prosthesis type. SEM found a significant positive association between Prosthesis Usability Experience and Extent of Prosthesis Adoption. Several covariates had direct effects on prosthesis adoption: 1) Extent of Prosthesis Adoption was lower for those with transhumeral and shoulder amputation, and higher for those with bilateral amputation, compared to the reference group with unilateral transradial amputation and 2) Myoelectric multiple degree of freedom (multi-DOF) prosthesis use was associated with lower Extent of Prosthesis Adoption, compared to body-powered prosthesis use. Myoelectric multi-DOF use also modified the effect of Prosthesis Usability Experience on Extent of Prosthesis Adoption. For those with bilateral ULA, the strength of the relationship between Prosthesis Usability Experience and Extent of Prosthesis Adoption was reduced. Findings suggest that in order to increase prosthesis adoption, prosthetics developers and rehabilitation providers should focus on implementing strategies to improve prosthesis usability experience. New Prosthesis Usability Experience measures could be used to identify persons at greater risk for poor prosthesis adoption and target interventions to increase prosthesis use.

Prognostic Value of Coronary CT Angiography-derived Fractional Flow Reserve on 3-year Outcomes in Patients with Stable Angina

Background The prognostic value of coronary CT angiography (CTA)-derived fractional flow reserve (FFR) beyond 1-year outcomes and in patients with high levels of coronary artery calcium (CAC) is uncertain. Purpose To assess the prognostic value of coronary CTA-derived FFR test results on 3-year clinical outcomes in patients with coronary stenosis and among a subgroup of patients with high levels of CAC. Materials and Methods This study represents a 3-year follow-up of patients with new-onset stable angina pectoris who were consecutively enrolled in the Assessing Diagnostic Value of Noninvasive CT-FFR in Coronary Care, known as ADVANCE (ClinicalTrials.gov: NCT02499679) registry, between December 2015 and October 2017 at three Danish sites. A high CAC was defined as an Agatston score of at least 400. A lesion-specific coronary CTA-derived FFR value of 2 cm with distal-to-stenosis value at or below 0.80 represented an abnormal test result. The primary end point was a composite of all-cause death and nonfatal spontaneous myocardial infarction. Event rates were estimated using the one-sample binomial model, and relative risk was compared between participants stratified by results of coronary CTA-derived FFR. Results This study included 900 participants: 523 participants with normal results (mean age, 64 years ± 9.6 [SD]; 318 male participants) and 377 with abnormal results from coronary CTA-derived FFR (mean age, 65 years ± 9.6; 264 male participants). The primary end point occurred in 11 of 523 (2.1%) and 25 of 377 (6.6%) participants with normal and abnormal coronary CTA-derived FFR results, respectively (relative risk, 3.1; 95% CI: 1.6, 6.3; P < .001). In participants with high CAC, the primary end point occurred in four of 182 (2.2%) and 19 of 212 (9.0%) participants with normal and abnormal coronary CTA-derived FFR results, respectively (relative risk, 4.1; 95% CI: 1.4, 11.8; P = .001). Conclusion In individuals with stable angina, a normal coronary CTA-derived FFR test result identified participants with a low 3-year risk of all-cause death or nonfatal spontaneous myocardial infarction, both in the overall cohort and in participants with high CAC scores. Clinical trial registration no. NCT02499679 Published under a CC BY 4.0 license. Supplemental material is available for this article. See also the editorial by Sinitsyn in this issue.

An Update on Outcomes for COPD Pharmacological Trials: A COPD Investigators Report - Reassessment of the 2008 American Thoracic Society/European Respiratory Society Statement on Outcomes for COPD Pharmacological Trials

Background: In 2008, a dedicated American Thoracic Society/European Respiratory Society task force published a paper on the possible use and limitations of clinical outcomes and biomarkers to evaluate the impact of pharmacological therapy in patients with chronic obstructive pulmonary disease. Since then, our scientific understanding of chronic obstructive pulmonary disease has increased considerably; there has been a progressive shift from a one-size-fits-all diagnostic and therapeutic approach to a personalized approach; and many new treatments currently in development will require new endpoints to evaluate their efficacy adequately. Objectives: The emergence of several new relevant outcome measures motivated the authors to review advances in the field and highlight the need to update the content of the original report. Methods: The authors separately created search strategies for the literature, primarily based on their opinions and assessments supported by carefully chosen references. No centralized examination of the literature or uniform criteria for including or excluding evidence were used. Measurements and Main Results: Endpoints, outcomes, and biomarkers have been revisited. The limitations of some of those reported in the American Thoracic Society/European Respiratory Society task force document have been highlighted. In addition, new tools that may be useful, especially in evaluating personalized therapy, have been described. Conclusions: Because the "label-free" treatable traits approach is becoming an important step toward precision medicine, future clinical trials should focus on highly prevalent treatable traits, and this will influence the choice of outcomes and markers to be considered. The use of the new tools, particularly combination endpoints, could help better identify the right patients to be treated with the new drugs.

Making Sense of Composite Endpoints in Clinical Research

Multiple drugs currently used in clinical practice have been approved by regulatory agencies based on studies that utilize composite endpoints. Composite endpoints are appealing because they reduce sample size requirements, follow-up periods, and costs. However, interpreting composite endpoints can be challenging, and their misuse is not uncommon. Incorrect interpretation of composite outcomes can lead to misleading conclusions that impact patient care. To correctly interpret composite outcomes, several important questions should be considered. Are the individual components of the composite outcome equally important to patients? Did the more and less important endpoints occur with similar frequency? Do the component endpoints exhibit similar relative risk reductions? If these questions receive affirmative answers, the use and interpretation of the composite endpoint would be appropriate. However, if any component of the composite endpoint fails to satisfy the aforementioned criteria, interpretation can become difficult, necessitating additional steps. Regulatory agencies acknowledge these challenges and have specific considerations when approving drugs based on studies employing composite endpoints. In conclusion, composite endpoints are valuable tools for evaluating the efficacy and net clinical benefit of interventions; however, cautious interpretation is advised.

Multimorbidity and adverse longitudinal outcomes among patients attending chronic outpatient medical care in Bahir Dar, Northwest Ethiopia

Background

Multimorbidity is becoming more prevalent in low-and middle-income countries (LMICs). However, the evidence base on the burden and its longitudinal outcomes are limited. This study aimed to determine the longitudinal outcomes of patients with multimorbidity among a sample of individuals attending chronic outpatient non communicable diseases (NCDs) care in Bahir Dar, northwest Ethiopia.

Methods

A facility-based longitudinal study was conducted among 1,123 participants aged 40+ attending care for single NCD (n = 491) or multimorbidity (n = 633). Data were collected both at baseline and after 1 year through standardized interviews and record reviews. Data were analyzed using Stata V.16. Descriptive statistics and longitudinal panel data analyzes were run to describe independent variables and identify factors predicting outcomes. Statistical significance was considered at p-value <0.05.

Results

The magnitude of multimorbidity has increased from 54.8% at baseline to 56.8% at 1 year. Four percent (n = 44) of patients were diagnosed with one or more NCDs and those having multimorbidity at baseline were more likely than those without multimorbidity to develop new NCDs. In addition, 106 (9.4%) and 22 (2%) individuals, respectively were hospitalized and died during the follow up period. In this study, about one-third of the participants had higher quality of life (QoL), and those having higher high activation status were more likely to be in the higher versus the combined moderate and lower QoL [AOR1 = 2.35, 95%CI: (1.93, 2.87)] and in the combined higher and moderate versus lower level of QoL [AOR2 = 1.53, 95%CI: (1.25, 1.88)].

Conclusion

Developing new NCDs is a frequent occurrence and the prevalence of multimorbidity is high. Living with multimorbidity was associated with poor progress, hospitalization and mortality. Patients having a higher activation level were more likely than those with low activation to have better QoL. If health systems are to meet the needs of the people with chronic conditions and multimorbidity, it is essential to understand diseases trajectories and of impact of multimorbidity on QoL, and determinants and individual capacities, and to increase their activation levels for better health improve outcomes through education and activation.

A composite measure for patient-reported outcomes in orthopedic care: design principles and validity checks

BACKGROUND

The complex, multidimensional nature of healthcare quality makes provider and treatment decisions based on quality difficult. Patient-reported outcome (PRO) measures can enhance patient centricity and involvement. The proliferation of PRO measures, however, requires a simplification to improve comprehensibility. Composite measures can simplify complex data without sacrificing the underlying information.

OBJECTIVE AND METHODS

We propose a five-step development approach to combine different PRO into one composite measure (PRO-CM): (i) theoretical framework and metric selection, (ii) initial data analysis, (iii) rescaling, (iv) weighting and aggregation, and (v) sensitivity and uncertainty analysis. We evaluate different rescaling, weighting, and aggregation methods by utilizing data of 3145 hip and 2605 knee replacement patients, to identify the most advantageous development approach for a PRO-CM that reflects quality variations from a patient perspective.

RESULTS

The comparison of different methods within steps (iii) and (iv) reveals the following methods as most advantageous: (iii) rescaling via z-score standardization and (iv) applying differential weights and additive aggregation. The resulting PRO-CM is most sensitive to variations in physical health. Changing weighting schemes impacts the PRO-CM most directly, while it proves more robust towards different rescaling and aggregation approaches.

CONCLUSION

Combining multiple PRO provides a holistic picture of patients' health improvement. The PRO-CM can enhance patient understanding and simplify reporting and monitoring of PRO. However, the development methodology of a PRO-CM needs to be justified and transparent to ensure that it is comprehensible and replicable. This is essential to address the well-known problems associated with composites, such as misinterpretation and lack of trust.

Multimorbidity and health-related quality of life among patients attending chronic outpatient medical care in Bahir Dar, Northwest Ethiopia: The application of partial proportional odds model

BACKGROUND

Multimorbidity, the presence of two or more chronic non-communicable diseases (NCDs) in a given person affects all aspects of people's lives. Poor quality of life (QoL) is one of the major consequences of living with multimorbidity. Although healthcare should support multimorbid individuals to achieve a better quality of life, little is known about the effect of multimorbidity on the QoL of patients living with chronic conditions. This study aimed to determine the influence of multimorbidity on QoL among clients attending chronic outpatient medical care in Bahir Dar, Northwest Ethiopia.

METHODOLOGY

A multi-centered facility-based study was conducted among 1440 participants aged 40+ years. Two complementary methods were employed to collect sociodemographic and disease related data. We used the short form (SF-12 V2) instrument to measure quality of life (QoL). The data were analyzed by STATA V.16, and a multivariate partial proportional odds model was fitted to identify covariates associated with quality of life. Statistical significance was considered at p-value <0.05.

PRINCIPAL FINDINGS

Multimorbidity was identified in 54.8% (95% CI = 52.2%-57.4%) of the sample. A significant proportion (33.5%) of the study participants had poor QoL and a quarter (25.8%) of them had moderate QoL. Advanced age, obesity and living with multimorbidity were the factors associated with poor QoL. Conversely, perceived social support and satisfaction with care were the variables positively associated with better QoL.

CONCLUSION

The magnitude of multimorbidity in this study was high and individuals living with multimorbidity had a relatively poorer QoL than those without multimorbidity. Care of people with chronic multiple conditions has to be oriented to the realities of multimorbidity burden and its implication on QoL. It is also imperative to replicate the methods we employed to measure and analyze QoL data in this study for facilitating comparison and further development of the approaches.

Research Question, Objectives, and Endpoints in Clinical and Oncological Research: A Comprehensive Review

Clinical research is a systematic process of conducting research work to find solutions for human health-related problems. It is applied to understand the disease process and assist in the diagnosis, treatment, and prevention. Currently, we are experiencing global unrest caused by the coronavirus disease (COVID-19) pandemic. The novel severe acute respiratory syndrome coronavirus (SARS-CoV-2) has been responsible for the deaths of more than 50 million people worldwide. Also, it has resulted in severe morbidity among the affected population. The cause of such a huge amount of influence on human health by the pandemic was the unavailability of drugs and therapeutic interventions to treat and manage the disease. Cancer is a disease condition wherein the normal cell function is deranged, and the cells multiply in an uncontrolled manner. Based on recent reports by the World Health Organization (WHO), cancer is the second leading cause of death globally. Moreover, the rates of cancers have shown an increasing trend in the past decade. Therefore, it is essential to improve the understanding concerning clinical research to address the health concerns of humans. In this review, we comprehensively discuss critical aspects of clinical research that include the research question, research objectives, patient-reported outcome measures (PROMs), intention-to-treat and per-protocol analysis, and endpoints in clinical and oncological research.

Nothing to Cheer About: Endorsing Imaginary Economic Evaluations and Value Claims with CHEERS 22

One of the more unfortunate features of health technology assessment is the tenacity with which leaders in the field and organizations such as the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) and the Institute for Clinical and Economic Review (ICER) cling to an evaluation framework that fails to meet the standards of normal science. Believers subscribe to a meme that is clearly non-science (metaphysics and pseudoscience) and one that should have been discarded over 30 years ago. Certainly, subscribing to an impossible belief is not unusual; indeed it may make the belief that much stronger. Yet the meme is non-sustainable; it is also pointless as the economic evaluation claims are non-evaluable. There is no acknowledgement of the standards of normal science or the limitations imposed by the axioms of fundamental measurement. The purpose of this commentary is to make the case that the recent release of the Consolidated Health Economic Evaluation Reporting Standards 2022 (CHEERS 22) checklist is misleading; CHEERS 22 fails to address the manifest deficiencies in the approach to economic evaluations endorsed by ISPOR and ICER. Instead, it continues to promote economic evaluations in healthcare that invent evidence and non-empirically evaluable value claims. Given the widespread publicity that has accompanied the release of CHEERS 22, the purpose of this commentary is to detail the deficiencies in CHEERS 22 and propose an alternative framework for economic evaluation in health care to meet the information needs of formulary committees. This means abandoning the standards for economic evaluations that have dominated health technology assessment for 30 years, notably the key role assigned to the mathematically impossible quality adjusted life year (QALY). The proposed new start recommends single attribute evaluable value claims that meet ratio or interval measurement standards and are supported by evaluation protocols.

Composite outcomes at OMERACT: Multi-outcome domains and composite outcome domains

The OMERACT Technical Advisory Group recognises that working groups during the process of creating a core outcome set may identify an outcome domain that would be best represented as a composite that encapsulates these component outcome domains by bringing them together into a single outcome. A multi-outcome domain (MOD) is a within-patient combination of component outcomes, and an individual patient's evaluation depends on the observation of all of the components in that patient with a single overall rating determined according to a specified rule; which is often applicable when we consider a disease activity score. A composite outcome domain (COD) consists of a number of component outcomes and is defined as the occurrence in a patient of one, some or all of these specified components; which is often applicable when we consider the risk of adverse events or remission criteria. We review the general benefits, challenges, reporting and interpretation of using MODs and CODs. The development of the MOD or COD instrument for an OMERACT core outcome measurement set is considered through four distinct steps: choosing relevant outcome domains; finding high quality instruments for each of these outcome domains; weighting the outcome domain instruments in the MOD/COD instrument; and putting MOD/COD instrument through the OMERACT Filter. Guidance and training are in preparation for working groups who will be completing the OMERACT Instrument Selection Algorithm (OFISA). As for other initiatives in OMERACT, we will seek feedback from OMERACT working groups who complete the development of their MOD/COD, which will then be incorporated into the refinement of the guidance and training.

Evaluation of the Unidimensional Fatigue Impact Scale (U-FIS) in Crohn's Disease: The Importance of Local Item Dependency

BACKGROUND AND PURPOSE

The Unidimensional Fatigue Impact Scale (U-FIS) was developed for use in a multiple sclerosis population. The aim was to determine whether the U-FIS is a valid tool for measuring the impact of fatigue in Crohn's disease (CD).

METHOD

CD patients completed the U-FIS as part of a validation study of the Crohn's Life Impact Questionnaire (CLIQ). Data were analyzed according to Rasch measurement theory (RMT).

RESULTS

Two hundred sixty-one completed U-FIS questionnaires were available for analysis. After rescoring the items to resolve disordered thresholds, all 22 items showed acceptable RMT fit. However, there was considerable local item dependency (LID).

CONCLUSION

The U-FIS did not provide unidimensional measurement in a sample of CD patients due to high levels of LID. Combining the three FIS outcomes into a single measure was not justified.

Psychometric properties and diagnostic accuracy of the short form of the geriatric anxiety scale (GAS-10)

BACKGROUND

Anxious symptoms have a negative impact on different aspects of the elderly's quality of life, ranging from the adoption of unhealthy lifestyle behaviours to an increased functional impairment and a greater physical disability. Different brief assessment instruments have been developed as efficacy measures of geriatric anxiety in order to overcome psychometric weaknesses of its long form. Among these, the 10-item Geriatric Anxiety Scale (GAS-10) showed strong psychometric properties in community-dwelling samples. However, its diagnostic accuracy is still unexplored, as well as its discriminative power in clinical samples.

METHODS

In the present study, we explored the psychometric performance of the GAS-10 in the elderly through Item Response Theory in a sample of 1200 Italian community-dwelling middle-aged and elderly adults (53.8% males, mean age = 65.21 ± 9.19 years). Concurrent validity, as well as diagnostic accuracy, was examined in a non-clinical sample (N = 229; 46.72% males) and clinical sample composed of 35 elderly outpatients (74.28% females) with Generalized Anxiety Disorder (GAD).

RESULTS

The GAS-10 displayed good internal construct validity, with unidimensional structure and no local dependency, good accuracy, and no signs of Differential Item Functioning (DIF) or measurement bias due to gender, but negligible due to the age. Differences in concurrent validity and diagnostic accuracy among the long form version of the GAS and the GAS-10 were not found significant. The GAS-10 may be more useful than the longer versions in many clinical and research applications, when time constraints or fatigue are issues.

CONCLUSION

Using the ROC curve, the GAS-10 showed good discriminant validity in categorizing outpatients with GAD disorder, and high anxiety symptoms as measured by the GAS-SF cut-off. The stable cut-off point provided could enhance the clinical usefulness of the GAS-10, which seems to be a promising valid and reliable tool for maximize diagnostic accuracy of geriatric anxiety symptoms.

Peter Rabbit is a Badger in Disguise: Deconstructing the Belief System of the Institute for Clinical and Economic Review in Health Technology Assessment

The Institute for Clinical and Economic Review (ICER), a Boston-based consulting group, has seen itself as the lead organization in the US for evaluating pharmaceuticals and, at product launch, making recommendations for pricing and access. Previous commentaries in Innovations in Pharmacy have made the case that the ICER analytical framework is nonsensical. It abandons the standards of normal science in favor of inventing evidence through unsupported assertions regarding measurement properties and lifetime assumption driven simulations. It has been labeled pseudoscience. Yet ICER persists in its belief that all preference scales have ratio properties. ICER believes it can disregard these standards, notably in respect of the axioms of fundamental evidence, and continue its technology assessment activities. Challenging a belief system is not undertaken lightly, although in the case of ICER the belief system is built on such shaky foundations that the effort seems almost superfluous. This deeply held belief, shared apparently by the majority of health economists according to ICER, that all preference scores have ratio properties with a true zero, is easily overturned: if it has ratio properties how is it that preferences scores have been known for over 30 years to recognize health states worse than death? In other words, they can have negative preferences. Recognizing this manifest contradiction is important because it brings into relief the wider belief system of the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) who share the same mythical certainties. A key issue is one of cultural relativity: can we accept with equanimity the parallel existence of two belief systems in health technology assessment when one is clearly nonsense? The answer proposed here is clearly no; although unfortunately the blowback by ICER and ISPOR will ensure the survival at least in the near term of their unfortunate meme.

Medicaid Formulary Decisions and the Institute for Clinical and Economic Review: Abandoning Pseudoscience in Imaginary Pharmaceutical Pricing Claims

Medicaid formulary committees and other gatekeepers face a difficult task. On the one hand they can utilize technical expertise in evaluating the real world evidence for clinical, quality of life and resource utilization claims for competing products while on the other hand they may be asked to assess claims built by simulation models for pricing and product access. A common option has been to take modeled claims from third parties such as the Institute for Clinical and Economic Review (ICER) at face value without challenging the model structure, its assumptions and its incremental cost-per-QALY claims set against competing products or the existing standard of care. Unfortunately, from the available evidence, it seems clear that many formulary assessment groups, last but not least those for whom the ICER modeling claims are targeted, have little if any appreciation of the limitations of ICER modeling. There are two substantive issues: (i) a failure to appreciate the limitations imposed by the standards of normal science for credible, empirically evaluable and replicable product claims and (ii) an understanding of limitations imposed by the axioms of fundamental measurement. In the latter case, a failure to recognize that the quality adjusted life year (QALY) is an impossible mathematical construct (hence the I-QALY). To these limitations should be added the potential for constructing competing imaginary claims. Surprisingly, ICER has provided the ideal opportunity to construct competing claims with the launch in late 2020 of the ICER Analytics cloud platform. Formulary committees and other health decision makers should be aware that claims based on the ICER Analytics platform together with competing lifetime modelled claims all fail the standards of normal science. Factoring these into formulary decisions is not only misguided but may have unintended consequences for pricing and access that may disadvantage significantly patients and caregivers. We have spent too much time debating the merits or otherwise of the I-QALY for targeted patient groups with the parties failing to recognize that the focus on simulated cost-per-I-QALY value assessments is a mathematical folly; I-QALY claims are a chimera. The I-QALY, at long last, should be abandoned together with modelled lifetime simulations. Medicaid formulary decision makes should rethink the required evidence base for formulary decisions and negotiations. Care should be taken to revisit previous negotiations where ICER recommendations have been utilized to support pricing and access.

Let a Thousand Models Bloom: ICER Analytics Opens the Floodgates to Cloud Pseudoscience

It has been noted on numerous occasions that modeled claims for cost-effectiveness, if driven by assumption for the lifetime of a hypothetical patient population, can be easily 'gamed' to create a required claim. These marketing exercises to support product entry are all too common in the literature. The institute for Clinical and Economic Review (ICER) in its launch of the ICER Analytics platform has provided a framework to support precisely these activities. Following the mainstream methodology in health technology assessment, the ICER Analytics platform facilitates the creation of approximate information to support formulary decisions. This is an odd development because it undercuts ICERs belief that it is the key arbiter in health technology assessment in the US, setting the stage for pricing and access recommendations. With the release of the ICER Analytics platform, others can now customize the 'backbone' ICER model in a disease area (i.e., change assumptions) to develop alternative and competing value assessments and 'fair' price claims. The problem is, of course, that without a reference point, there is no basis for comparing modeled claims other than through challenging assumptions. Indeed, ICER has made this easy by reducing barriers to lifetime model building so that manufacturers and others can create competing (and confusing) claims within, literally, a few minutes. ICER will then become one of a multitude of competing voices for the attention of formulary committees and other health decision makers; letting a thousand imaginary models bloom where no model can be judged on the basis of credible, empirically evaluable and replicable product claims.

To Dream the Impossible Dream: The Commitment by the Institute for Clinical and Economic Review to Rewrite the Axioms of Fundamental Measurement for Hemophilia A and Bladder Cancer Value Claims

Understandably, after 30 years of ignoring the axioms of fundamental measurement, advocates of creating approximate information through the construction of lifetime cost-per-QALY worlds are somewhat unnerved by the realization that their methodology is incompatible with those axioms. This is made all the more unnerving when it is pointed out that this incompatibility was pointed out over 30 years ago, following the formalization of those axioms almost 80 years ago. Why this was overlooked is a mystery. The result was a commitment to the application of ordinal utility and other patient reported outcome measures to support claims for response to competing therapies; most egregiously, the advocacy of cost-per-QALY lifetime models and willingness to pay thresholds to support recommendations for pricing and access to pharmaceutical products and devices. Although this incompatibility has been pointed out in respect of simulation modeling, to groups such as the Institute for Clinical and Economic Review (ICER) they press on, producing evidence reports and recommendations for emerging products that fail the standards of normal science. While these are an analytical dead end, ICER has nowhere else to go. This is their business model; to admit otherwise would mean withdrawing their many evidence reports and admit they were wrong. ICER has rejected this; rather it has decided, together with its academic consultants, to challenge the axioms of fundamental measurement, to produce a parallel measurement universe that can sustain QALYs and the imaginary simulation lifetime models. The purpose here is to make clear that ICER is manifestly wrong and that there is no way it can maintain its credibility in pursuing this path. This is achieved by a deconstruction of the arguments put forward by ICER to defend its new vision of the axioms of fundamental measurement, a vision which provides a case study in the distinction between justified belief and opinion. Fortunately, we have the framework for a new paradigm in value assessment; a paradigm that recognizes the standards of normal science and rejects belief in an alternative reality consistent with fundamental measurement axioms.

Value Assessment, Real World Evidence and Fundamental Measurement: Version 3.0 of the Minnesota Formulary Submission Guidelines

This latest version of the Minnesota guidelines is intended to reassert the application of the standards of normal science in formulary submissions for new and existing pharmaceutical products and devices. This represents a paradigm shift from the existing value assessment standards which are focused on imaginary or I-QALY modeling of lifetime claims. The proposed new paradigm rejects this as pseudoscience; a failure to recognize the standards of normal science, in particular a failure to recognize the constraints of fundamental measurement. As a result, current health technology assessment is dominated by value assessments that create claims that are neither credible, nor empirically evaluable or replicable. The fatal flaw is the failure to recognize that QALYS are an impossible mathematical construct (hence the term I-QALY). The proposed paradigm recognizes that if there are claims for product value then, regardless of whether the claim is for clinical impact, quality of life or resource utilization, all claims must be empirically evaluable. If not, then they should be rejected. The Minnesota guidelines propose a new evidence based approach to formulary assessment, together with ongoing disease area and therapeutic class reviews. The focus is on claims that are specific to target patient populations that are claims for specific attributes and are consistent with the axioms of fundamental measurement. Manufacturers are asked to support claims assessment through protocols detailing the evidence base for claims assessment, the timelines for those assessments and the process by which claims assessments are reported back to formulary committees. Value assessment leads naturally to value contracting, revisiting provisional prices as new information is discovered and delivered to the formulary committee.

The Impossible QALY and the Denial of Fundamental Measurement: Rejecting the University of Washington Value Assessment of Targeted Immune Modulators (TIMS) in Ulcerative Colitis for the Institute for Clinical and Economic Review (ICER)

All too often organizations embrace standards for health technology assessment that fail to meet the standards of normal science. A continuing puzzle is why the axioms of fundamental measurement are ignored by researchers such as the University of Washington Model Group in constructing lifetime cost-per-QALY claims. The University of Washington Model Group is not alone; it is an accepted article of faith that multiattribute utility scales can be manipulated as if they had ratio scale properties, which they do not. This commitment to pseudoscientific claims, embracing intelligent design rather than natural selection, is endorsed by professional groups such as ISPOR as well as by self-appointed arbiters of value assessment such as ICER. Perhaps the answer is peer pressure rather than ignorance of the axioms of fundamental measurement. More to the point, if you have been an advocate of imaginary simulations a Damascene epiphany creates both psychological and professional challenges. After all, if cost-per-QALY constructs are rejected, then it is difficult to see what options there are for those attempting to model cost-effectiveness claims. If it is just ignorance of the axioms of fundamental measurement then a reasonable question is why these axioms, readily available on any number of internet sites, are ignored in health technology assessment programs. The purpose of this commentary is to review the ICER September 11th 2020 evidence report in ulcerative colitis, with particular reference to ICER's responses to questions raised in the public comment period on the measurement properties (or their absence) for utility scales; in this context the EQ-5D instruments. The critique pointed out that the utility scores had ordinal properties. ICER, without proof, disputed this statement asserting that health economists believed (or assumed) they were ratio scales. This is nonsensical. ICER has two options: first, to continue to believe that the EQ-5D instruments had ratio properties or second, to acknowledge that they indeed only had ordinal properties, rejecting their many modeled claims for pricing and access. Not surprisingly, the possibility of a Damascene epiphany was rejected. ICER maintained its assertion that health economists, presumably all of them, believe or possibly just assume for analytical convenience that the EQ-5D-3L and similar measures are in fact on a ratio scale. This introduces a new concept in fundamental measurement: a ratio scale without a true zero but with negative values. ICER is quite prepared to admit that negative I-QALYs are possible and their lifetime cost-per-incremental I-QALY modelling can yield negative I-QALYs.

The Great I-QALY Disaster

The QALY is an impossible construct; it defies common sense. It fails completely once we consider the axioms of fundamental measurement. Utilities as ordinal scales cannot be used to create QALYS. The QALY should never have been introduced to support the value assessment of pharmaceutical products and devices. The result is 30 years of QALY based assessments of pharmaceutical products and devices which are conceptually and technically wrong. They are a charade and will have contributed mistakenly to thousands of formulary decisions. In the search for a common metric to evaluate cost-effectiveness the impossibility of a QALY was overlooked. The result is a disaster, unfolding over decades. Our next steps must be to abandon the QALY paradigm and look ahead to a new value assessment framework.