Why it is hard to use PROMs and PREMs in routine health and care

Patient-reported outcome measures (PROMs) and patient-reported experience measures (PREMs) show the results of healthcare activities as rated by patients and others. Patients or their proxies record feedback using questionnaires. These can enhance quality for all and tailored care for individuals. This paper describes obstacles that inhibit widespread use of PROMs and PREMs and some potential solutions.Implementation is a prerequisite for any innovation to succeed. Health and care services are complex and people need to be engaged at every level. Most people are cautious about proven innovations such as PROMs and PREMs but champions and leaders can help them engage. The NASSS framework (reasons for Non-adoption, Abandonment and failure to Scale up, Spread or Sustain digital health innovations) helps indicate that implementation is complex why it may be resisted.The Plan-Do-Study-Act (PDSA) approach aids implementation and helps ensure that everyone knows who should do what, when, where, how and why. Noise is an under-appreciated problem, especially when tracking patients over time such as before and after treatment. Interoperability of PROMs and PREMs with electronic health records should use Fast Health Interoperability Resources and internationally accepted coding schemes such as SNOMED CT and LOINC.Most projects need multiple measures to meet the needs of everyone involved. Measure selection should focus on their relevance, ease of use, and response rates.If these problems are avoided or mitigated, PROMs and PREMs can help deliver better patient outcomes, patient experience, staff satisfaction and health equity.

First Dark Matter Search Results from the LUX-ZEPLIN (LZ) Experiment

The LUX-ZEPLIN experiment is a dark matter detector centered on a dual-phase xenon time projection chamber operating at the Sanford Underground Research Facility in Lead, South Dakota, USA. This Letter reports results from LUX-ZEPLIN's first search for weakly interacting massive particles (WIMPs) with an exposure of 60 live days using a fiducial mass of 5.5 t. A profile-likelihood ratio analysis shows the data to be consistent with a background-only hypothesis, setting new limits on spin-independent WIMP-nucleon, spin-dependent WIMP-neutron, and spin-dependent WIMP-proton cross sections for WIMP masses above 9  GeV/c^{2}. The most stringent limit is set for spin-independent scattering at 36  GeV/c^{2}, rejecting cross sections above 9.2×10^{-48}  cm at the 90% confidence level.

Routine measurement of patient experience

Patient experience is a key pillar of healthcare quality. We describe a framework of three short generic measures covering Patient Experience, Result Satisfaction and Service Integration. The Result Satisfaction measure is described for the first time.These measures capture twelve aspects of patient experience covering the relationship between patients and clinicians (Patient Experience), the immediate results of the consultation or treatment as perceived by patients (Result Satisfaction) and collaboration between different healthcare services and silos (Service Integration). Each measure has four items.These measures are compared with three national measures: the Friends and Family Test and the General Practice Patient Survey used in England, and HCAHPS used in US hospitals. The expected benefits of national measures are not being achieved and we need to think again about how best to tailor health services to meet patients' expectations.The three measures described (Patient Experience, Result Satisfaction and Service Integration) are generic, short and have low reading ages. They share common forms and scoring schemes, which mean that they can be used individually or in combination at all levels of a healthcare provider.

Mitochondrial Transfusion Improves Mitochondrial Function Through Up-regulation of Mitochondrial Complex II Protein Subunit SDHB in the Hippocampus of Aged Mice

The mitochondrial theory of aging is characterized by mitochondrial electron transport chain dysfunction. As a hallmark of aging, an increasing number of investigations have attempted to improve mitochondrial function in both aging and age-related disease. Emerging from these attempts, methods involving mitochondrial isolation, transfusion, and transplantation have taken center stage. In particular, mitochondrial transfusion refers to the administration of mitochondria from healthy tissue into the bloodstream or into tissues affected by injury, disease, or aging. In this study, methods of mitochondrial isolation and transfusion were developed and utilized. First, we found a significant decrease (p < 0.05) in the expression of mitochondrial complex proteins (I-V) in aged (12 months old) mouse brain tissue (C57BL/6 mice) in comparison to healthy young brain tissue (1 month old). To investigate whether healthy young mitochondria taken from the liver could improve mitochondrial function in older animals, we intravenously injected mitochondria isolated from young C57BL/6 mice into aged mice from the same strain. This study, for the first time, demonstrates that mitochondrial transfusion significantly (p < 0.05) improves mitochondrial function via the up-regulation of the mitochondrial complex II protein subunit SDHB in the hippocampus of aged mice. This result has identified a role for mitochondrial complex II in the aging process. Therefore, mitochondrial complex II could serve as a putative target for therapeutic interventions against aging. However, more importantly, methods of mitochondrial transfusion should be further tested to treat a variety of human diseases or disorders and to slow down or reverse processes of aging.

The realities and expectations of community involvement in COVID-19 research: a Consumer Reference Group perspective

BACKGROUND

Older adults have been disproportionately impacted by the COVID-19 pandemic. COVID-19 restrictions such as stay at home orders and physical distancing measures have been implemented to reduce older adults' risk of infection, however, such measures can have negative effects on older adults' mental health and social wellbeing. In 2020, the research team received funding as part of an Australian COVID-19 research grants program to investigate how services can better meet the mental health and social support needs of older adults during COVID-19. A Consumer Reference Group (CRG) was established to provide a community perspective on all research activities.

MAIN BODY

The CRG comprised of eight older adults aged 65 years and older living in Western Australia. Two members of the CRG were involved in the initial grant proposal, and one member worked for a not-for-profit organisation that provides support and advocacy for older adults. The CRGs role was to provide consumer and community perspectives on the research design, advise on study materials, facilitate links between consumers, the community, and researchers, and advocate on behalf of consumers and the community. The CRG was encouraged to reflect on the research project, their contributions, and the outcomes obtained. In this commentary, we document the CRGs contributions to the project, and record their reflections, including what went well, what were some challenges, the realities of conducting research during COVID-19, and lessons learnt.

CONCLUSION

The CRG were active participants in the research process. They shared their perspectives and made important contributions to the project. Through collaboration with the CRG, we were able to reach four key messages, underpinned by consumers lived experiences, that were used to co-develop knowledge translation products. These were disseminated to service providers and older adults.

Readiness for five digital technologies in general practice: perceptions of staff in one part of southern England

Background

Our aim was to understand how digital readiness within general practice varies between different technologies and to identify how demographic, workplace and external factors affect this. The technologies considered include electronic patient records, telehealth (text messaging and video consultations), patient online access, patient clinical apps and wearables, and social media.

Method

A digital readiness survey tool was developed and used in one area of southern England during Spring 2020. Semistructured qualitative interviews were also carried out with some practice staff and digital technology company representatives.

Results

GPs, nurses and non-clinical staff submitted 287 responses from 27 general practices (out of 33 invited).

Staff digital readiness differs significantly between technologies. The mean perceived digital competency scores on 0–100 scale (high is good) were electronic patient records (75.7), telehealth (64.2), patient online access (65.8), patient clinical apps and wearables (50.8), and social media (51.2).

Younger general practice staff, those in post for 5 or less years are more digitally competent and confident than older staff. This applies to both clinical and non-clinical staff. Older patient population, rurality and smaller practice size are associated with lower digital readiness. Readiness to use digital technology may have improved since the start of the COVID-19 pandemic but barriers remain in poor IT and mobile infrastructure, software usability and interoperability, and concerns about information governance.

Conclusions

Improving digital readiness in general practice is complex and multifactorial. Issues may be alleviated by using dedicated digital implementation teams and closer collaboration between stakeholders (GPs and their staff, patients, funders, technology companies and government).

Antimicrobial use in agriculture: critical review of the factors influencing behaviour

Antimicrobial resistance (AMR) is a global health emergency affecting humans and animals, diminishing the effectiveness of medication used to treat illness. The agri-food sector has attracted increased attention for imprudent antimicrobial use (AMU) and its contribution to AMR. Thus, ascertaining farmers' and veterinarians' behaviours surrounding AMU is essential to address imprudent AMU and generate behaviour change within the agri-food sector. Therefore, the aim of this critical review is to investigate, assess and collate the current body of evidence to identify psychosocial factors including knowledge, understanding, perceptions, attitudes and behaviours surrounding AMU. Database searches were limited to articles utilizing qualitative and quantitative methodologies, available in English with no restriction on publication year. Of the 1156 articles identified, 103 were retained for this review. Findings on the psychosocial aspects were thematically analysed. Five key themes emerged from the data: (i) knowledge and awareness of antimicrobials; (ii) attitudes towards antimicrobials; (iii) influential relationships; (iv) resources; and (v) factors influencing AMU. Results indicated that to overcome barriers experienced by key stakeholders, a carefully considered, evidence-based approach, incorporating behaviour change theory, is required when designing intricate interventions/strategies, in order to elicit successful and sustained AMU behaviour change.

Development of social contact and loneliness measures with validation in social prescribing

Aims

This paper describes two patient-reported measures of social contact and loneliness, which are closely related concepts. The first measure (R-Outcomes Social Contact measure) was developed from scratch, based on customer needs and literature review. It covers emotional and social aspects using positive terms. The second measure (R-Outcomes Loneliness measure) is adapted from the GSS Loneliness Harmonised Standard. Both measures are patient-reported outcome measures, based on patients’ own perception of how they feel.

Method

This development started in 2016 in response to customers’ requests to measure social contact/loneliness for patients in social prescribing projects.

Both measures are compared with three other loneliness measures (the GSS Loneliness Harmonised Standard, De Jong Gierveld and Campaign to End Loneliness). Both measures are short (36 and 21 words, respectively). Mean improvement is reported as a positive number on a 0–100 scale (where high is good).

We tested the psychometric performance and construct validity of the R-Outcomes Social Contact measure using secondary analysis of anonymised data collected before and after social prescribing interventions in one part of Southern England.

Results

In the validation study, 728 responses, collected during 2019–2020, were analysed. 90% were over 70 years old and 62% women. Cronbach’s α=0.76, which suggests that it is appropriate to use a single summary score. Mean Social Contact scores before and after social prescribing intervention were 59.9 (before) and 66.7 (after, p<0.001).

Exploratory factor analysis shows that measures for social contact, health status, health confidence, patient experience, personal well-being, medication adherence and social determinants of health are correlated but distinct factors. Construct validation shows that the results are consistent with nine hypotheses, based on the loneliness literature.

Conclusion

The R-Outcomes Social Contact measure has good psychometric and construct validation results in a population referred to social prescribing. It is complementary to other R-Outcomes measures.

Understanding farmers' and veterinarians' behavior in relation to antimicrobial use and resistance in dairy cattle: A systematic review

To tackle antimicrobial resistance, it is vital that farmers' and veterinarians' antimicrobial use behaviors and attitudes toward resistance are understood so that we can identify how beliefs and motives influence practices. Current literature details qualitative and quantitative research that explores the knowledge, attitudes and perceptions of dairy farmers and veterinarians with respect to antimicrobial resistance and antimicrobial practices, and the reported findings are varied and conflicting. Our objective was to conduct a systematic review to assess the evidence and knowledge gaps in the published literature. We identified articles via database searches of Embase, Medline, PubMed, Scopus, and Web of Science; we limited findings to published articles available in English with no publication year restrictions. Article screening was conducted at 3 levels: title, abstract, and full text. Of the 349 articles identified, 35 were retained for systematic review. Transparency of reporting was assessed for each study using the Consolidated Criteria for Reporting Qualitative Research (COREQ) framework. Quality was assessed using the Critical Appraisal Skills Programme qualitative checklist. Findings relating to dairy farmers' and veterinarians' knowledge, attitudes, and perceptions on antimicrobial resistance and practices were thematically analyzed. The comprehensiveness of reporting was variable: studies reported 5 to 26 of the 32 COREQ checklist items. Five key themes emerged from the data: knowledge and awareness of antimicrobial resistance; factors influencing farmer and veterinarian decision-making; perceived barriers and facilitators to reduced antimicrobial use; perceived responsibility for antimicrobial resistance; and the role of the farmer and veterinarian relationship in reducing antimicrobial use. Awareness of prudent antimicrobial use was not uniform between studies. Many factors influence farmers' and veterinarians' decisions to use antimicrobials, including animal welfare and available resources. The farmer-veterinarian relationship is a potential barrier or facilitator of reduced antimicrobial use, depending on the perceived relationship dynamic. Encouraging collaboration between farmers and veterinarians could lead to shared responsibility for reducing antimicrobial use. This review provided a coherent picture of what is currently known and identified gaps in the current knowledge to inform future behavioral intervention research. Increased knowledge, skill development, resources, engagement, and further research to address the gaps we identified are the main recommendations to effectively overcome barriers, elicit appropriate behavior change, and achieve reduced antimicrobial use in dairy cattle.

A standard set of person-centred outcomes for diabetes mellitus: results of an international and unified approach

AIMS

To select a core list of standard outcomes for diabetes to be routinely applied internationally, including patient-reported outcomes.

METHODS

We conducted a structured systematic review of outcome measures, focusing on adults with either type 1 or type 2 diabetes. This process was followed by a consensus-driven modified Delphi panel, including a multidisciplinary group of academics, health professionals and people with diabetes. External feedback to validate the set of outcome measures was sought from people with diabetes and health professionals.

RESULTS

The panel identified an essential set of clinical outcomes related to diabetes control, acute events, chronic complications, health service utilisation, and survival that can be measured using routine administrative data and/or clinical records. Three instruments were recommended for annual measurement of patient-reported outcome measures: the WHO Well-Being Index for psychological well-being; the depression module of the Patient Health Questionnaire for depression; and the Problem Areas in Diabetes scale for diabetes distress. A range of factors related to demographic, diagnostic profile, lifestyle, social support and treatment of diabetes were also identified for case-mix adjustment.

CONCLUSIONS

We recommend the standard set identified in this study for use in routine practice to monitor, benchmark and improve diabetes care. The inclusion of patient-reported outcomes enables people living with diabetes to report directly on their condition in a structured way.

Comparison of staff and resident health status ratings in care homes

BACKGROUND

Many care home residents cannot self-report their own health status. Previous studies have shown differences between staff and resident ratings. In 2012, we collected 10 168 pairs of health status ratings using the howRu health status measure. This paper examines differences between staff and resident ratings.

METHOD

HowRu is a short generic person-reported outcome measure with four items: pain or discomfort (discomfort), feeling low or worried (distress), limited in what you can do (disability) and require help from others (dependence). A summary score (howRu score) is also calculated. Mean scores are shown on a 0-100 scale. High scores are better than low scores. Differences between resident and staff reports (bias) were analysed at the item and summary level by comparing distributions, analysing correlations and a modification of the Bland-Altman method.

RESULTS AND CONCLUSIONS

Distributions are similar superficially but differ statistically. Spearman correlations are between 0.55 and 0.67. For items, more than 92.9% of paired responses are within one class; for the howRu summary score, 66% are within one class. Mean differences (resident score minus staff score) on 0-100 scale are pain and discomfort (-1.11), distress (0.67), discomfort (1.56), dependence (3.92) and howRu summary score (1.26). The variation is not the same for different severities. At higher levels of pain and discomfort, staff rated their discomfort and distress as better than residents. On the other hand, staff rated disability and dependence as worse than did residents. This probably reflects differences in perspectives. Red amber green (RAG) thresholds of 10 and 5 points are suggested for monitoring changes in care home mean scores.

Measure what we want: a taxonomy of short generic person-reported outcome and experience measures (PROMs and PREMs)

INTRODUCTION

Health and care systems are complex and multifaceted, but most person-reported outcome and experience measures (PROMs and PREMs) address just one aspect. Multiple aspects need measuring to understand how what we do impacts patients, staff and services, and how these are affected by external factors. This needs survey tools that measure what people want, are valid, sensitive, quick and easy to use, and suitable for people with multiple conditions.

METHODS

We have developed a coherent family of short generic PROMs and PREMs that can be used in combination in a pick-and-mix way. Each measure has evolved iteratively over several years, based on literature review, user inputs and field testing. Each has has a common format with four items with four response options and is designed for digital data collection with standardised analytics and data visualisation tools. We focused on brevity and low reading age.

RESULTS

The results are presented in tabular format and as a taxonomy. The taxonomy is categorised by respondent type (patient or staff) and measure type. PROMs have subdomains: quality of life, individual care and community; PREMs have subdomains: service provided, provider culture and innovation. We show 22 patient-reported measures and 17 staff-reported measures. Previously published measures have been validated. Others are described for the first time.

DISCUSSION AND CONCLUSIONS

This family of measures is broad in scope but is not claimed to be comprehensive. Measures share a common look and feel, which enables common methods of data collection, reporting and data visualisation. They are used in service evaluation, quality improvement and as key performance indicators. The taxonomy helps to organise the whole, explain what each measure does and identify gaps and overlaps.

A Taxonomy for Survey Measures Used to Evaluate Complex Digital Health Innovations

Evaluators need to measure whether innovations help patients and staff, but have lacked the tools needed to do this as part of routine care. We provide a taxonomy for the classification of survey measures, which can be used together on a pick and mix basis. These are described in the context of the evaluation of digital health innovations.

Health status of care home residents: practicality and construct validity of data collection by staff at scale

Background

Medical diagnoses and assessed need for care are the prerequisites for planning and delivery of care to residents of care homes. Assessing the effectiveness of care is difficult. The aim of this study was to test the practicality and construct validity of the howRu health status measure using secondary analysis of a large data set.

Method

The data came from a Bupa Care Homes Census in 2012, which covered 24 506 residents in 395 homes internationally (UK, Australia and New Zealand). Staff completed optical mark readable forms about each resident using a short generic health status measure, howRu. Response rates were used to assess practicality and expected relationships between health status and independent variables were used to assess the construct validity.

Results and discussion

19,438 forms were returned (79.3%) in 360 care homes (91.1%); complete health status data were recorded for 18 617 residents (95.8% of those returned). Missing values for any health status items mostly came from a small number of homes. The relationships between howRu and independent variables support construct validity. Factor analysis suggests three latent variables (discomfort, distress and disability/dependence).

Conclusions

HowRu proved easy to use and practical at scale. The howRu health status measure shows good construct validity.

Development and initial testing of a Health Confidence Score (HCS)

Introduction

Patients need to feel confident about looking after their own health. This is needed to improve patient outcomes and clinical support. With few suitable tools available to measure self-care health confidence, we developed and validated a short, generic survey instrument for use in evaluation and quality improvement.

Methods

The Health Confidence Score (HCS) was developed through literature review, patient and expert focus groups and discussions. This paper reports an initial survey (n = 1031, study 1) which identified some issues and a further face-to-face survey (n = 378, study 2) to test the construct and concurrent validity of the final version. Scores were correlated against the My Health Confidence (MHC) rating scale, howRu (health status measure) and relevant demographics.

Results

The HCS is short (50 words) with good readability (reading age 8). It has four items covering health knowledge, capability to self-manage, access to help and shared decision-making; each has four response options (strongly agree, agree, neutral disagree). Items are reported independently and as a summary score.The mean summary score was 76.7 (SD 20.4) on 0-100 scale. Cronbach's alpha = 0.82. Exploratory factor analysis suggested that the four items relate to a single dimension. Correlation of the HCS summary score with MHC was high (Spearman r = 0.76). It was also associated with health status (Spearman r = 0.49), negatively with number of medications taken (r=-0.29) and age (r=-0.22) and not with ethnicity, having children or education level.

Conclusions

The HCS is short, easy to use, with good psychometric properties and construct validity. Each item is meaningful independently and the summary score gives an overall picture of health confidence.

Monitoring work well-being, job confidence and care provided by care home staff using a self-report survey

Background and method

In care homes, staff well-being, job confidence and opinion of the care provided to residents are central to morale and care quality. In this study, care home staff in the East Midlands region of England completed self-reported outcome and experience surveys in two rounds. Mean scores for each home are shown using a scale from 0 (all chose lowest option) to 100 (all chose highest option). High scores are good.

Results

In round 1, 332 staff in 15 homes submitted responses; in round 2, 207 staff in 9 homes. Mean scores in round 1 and round 2 were similar, although those of some homes scores differed significantly, cancelling each other out. Overall, Work Wellbeing mean score was 83 (care home range 48-97), with worthwhileness (92) the highest ranked item and anxiety at work (78) the lowest. Job Confidence mean score was 84 (range 59-94), with able to manage the work (86) highest and involvement in decisions that affect staff (79) lowest. Care Provided mean score was 86 (range 59-97), with treat people kindly (91) highest and well organised (80) lowest. Homes rated as outstanding by the Care Quality Commission had higher scores on average than those rated good, which were higher than those rated as needing improvement.

Conclusions

This study has demonstrated the practicality of measuring staff views of their Work Wellbeing, Job Confidence and Care Provided in care homes. Rather than wait for adverse quality outcomes to be detected, this approach offers a way to track staff morale and declared capability over time.

Personal Wellbeing Score (PWS)-a short version of ONS4: development and validation in social prescribing

Aims

Our aim was to develop a short generic measure of subjective well-being for routine use in patient-centred care and healthcare quality improvement alongside other patient-reported outcome and experience measures.

Methods

The Personal Wellbeing Score (PWS) is based on the Office of National Statistics (ONS) four subjective well-being questions (ONS4) and thresholds. PWS is short, easy to use and has the same look and feel as other measures in the same family of measures. Word length and reading age were compared with eight other measures.

Anonymous data sets from five social prescribing projects were analysed. Internal structure was examined using distributions, intra-item correlations, Cronbach’s α and exploratory factor analysis. Construct validity was assessed based on hypothesised associations with health status, health confidence, patient experience, age, gender and number of medications taken. Scores on referral and after referral were used to assess responsiveness.

Results

Differences between PWS and ONS4 include brevity (42 vs 114 words), reading age (9 vs 12 years), response options (4 vs 11), positive wording throughout and a summary score. 1299 responses (60% female, average age 81 years) from people referred to social prescribing services were analysed; missing values were less than 2%. PWS showed good internal reliability (Cronbach’s α=0.90). Exploratory factor analysis suggested that all PWS items relate to a single dimension. PWS summary scores correlate positively with health confidence (r=0.60), health status (r=0.58), patient experience (r=0.30) and age group (r=0.24). PWS is responsive to social prescribing intervention.

Conclusions

The PWS is a short variant of ONS4. It is easy to use with good psychometric properties, suitable for routine use in quality improvement and health services research.

Digital innovation evaluation: user perceptions of innovation readiness, digital confidence, innovation adoption, user experience and behaviour change

BACKGROUND

Innovation spread is a key policy objective for health systems world-wide, but adoption success varies enormously. We have developed a set of short generic user-reported measures to help understand how and why healthcare innovations spread. This work builds on the literature and on practical experience in developing and using patient-reported outcome measures.

MEASURES

The Innovation Readiness Score measures user perceptions of how much they are open to and up-to-date with new ideas, and whether their organisations are receptive to and capable of innovation. It is based on Rogers' classification of innovativeness (innovator, early adopter, early majority, etc).The Digital Confidence Score rates users' digital literacy and confidence to use digital products, with dimensions of familiarity, social pressure, support and digital self-efficacy.The Innovation Adoption Score rates the adoption process in terms of coherence and reflective thought before, during and after implementation. It is based on Normalisation Process Theory.The User Satisfaction measure assesses a digital product in terms of usefulness, ease of use, support and satisfaction.The Behaviour Change measure covers user perceptions of their capability, opportunity and motivation to change behaviour, based on the COM-B model.These measures have been mapped onto Greenhalgh's NASSS Framework (non-adoption, abandonment and challenges to scale-up, spread and sustainability of health and care technologies).

CONCLUSION

These tools measure different aspects of digital health innovations and may help predict the success of innovation dissemination, diffusion and spread programmes.

The Load Model: an alternative to QALY

BACKGROUND

QALYs are widely used in health economic evaluation, but remain controversial, largely because they do not reflect how many people behave in practice. This paper presents a new conceptual model (Load Model) and illustrates it in comparison with the QALY model.

METHODS

Load is the average annual weight attributed to morbidity and mortality over a defined period, using weightings based on preference judgements. Morbidity Load is attributed to states of illness, according to their perceived severity. When people are in full health, Load is zero (no morbidity). Death is treated as an event with negative consequences, incurred in the year following death. Deaths may be weighted equally, with a fixed negative weight such as -100, or differ according to the context of death. After death, Load is zero. In a worked example, we use the standard gamble method to obtain a weighting for an illness state, for both Load and QALY models. A judge is indifferent between certainty of 1.5 years' illness followed by death, or a 50/50 chance of 1.5 years' full health or 1-year illness, each followed by death. The weightings calculated are applied to a hypothetical life, 72 years in full health followed by 3 years with illness then death, using both models. Three other hypothetical outcomes are also compared.

RESULTS

For an example life, the relative size of the morbidity component compared with the mortality component is much higher in the Load model than in the QALY model. When comparing alternative outcomes, there are also substantial differences between the two models.

CONCLUSIONS

In the Load model the weight of morbidity, relative to mortality, is very different from that in the QALY model. Given the role of the QALYs in economic evaluation, the implications of an alternative, which generates very different results, warrant further exploration.