Lean systems approaches to health technology assessment: a patient-focused alternative to cost-effectiveness analysis

The development of the concept of return-on-investment from large-scale quality improvement programmes in healthcare: an integrative systematic literature review

BACKGROUND

Return on Investment (ROI) is increasingly being used to evaluate financial benefits from healthcare Quality Improvement (QI). ROI is traditionally used to evaluate investment performance in the commercial field. Little is known about ROI in healthcare. The aim of this systematic review was to analyse and develop ROI as a concept and develop a ROI conceptual framework for large-scale healthcare QI programmes.

METHODS

We searched Medline, Embase, Global health, PsycInfo, EconLit, NHS EED, Web of Science, Google Scholar using ROI or returns-on-investment concepts (e.g., cost-benefit, cost-effectiveness, value). We combined this terms with healthcare and QI. Included articles discussed at least three organisational QI benefits, including financial or patient benefits. We synthesised the different ways in which ROI or return-on-investment concepts were used and discussed by the QI literature; first the economically focused, then the non-economically focused QI literature. We then integrated these literatures to summarise their combined views.

RESULTS

We retrieved 10 428 articles. One hundred and two (102) articles were selected for full text screening. Of these 34 were excluded and 68 included. The included articles were QI economic, effectiveness, process, and impact evaluations as well as reports and conceptual literature. Fifteen of 68 articles were directly focused on QI programme economic outcomes. Of these, only four focused on ROI. ROI related concepts in this group included cost-effectiveness, cost-benefit, ROI, cost-saving, cost-reduction, and cost-avoidance. The remaining articles mainly mentioned efficiency, productivity, value, or benefits. Financial outcomes were not the main goal of QI programmes. We found that the ROI concept in healthcare QI aligned with the concepts of value and benefit, both monetary and non-monetary.

CONCLUSION

Our analysis of the reviewed literature indicates that ROI in QI is conceptualised as value or benefit as demonstrated through a combination of significant outcomes for one or more stakeholders in healthcare organisations. As such, organisations at different developmental stages can deduce benefits that are relevant and legitimate as per their contextual needs.

TRIAL REGISTRATION

Review registration: PROSPERO; CRD42021236948.

Engaging Patients and Caregivers in an Early Health Economic Evaluation: Discerning Treatment Value Based on Lived Experience

BACKGROUND

Traditionally, economic evaluations have engaged clinicians and policymakers; however, patients and their caregivers have insight that can ensure that the economic evaluation process appropriately reflects disease consequences and adequately addresses their priorities related to treatment.

OBJECTIVE

We aimed to identify patient priorities to inform an early economic evaluation of chimeric antigen receptor T-cell therapy for adults with relapsed or refractory B-cell acute lymphoblastic leukemia.

METHODS

We conducted two online group discussions of four participants each, involving patients with experience of hematological cancer and a caregiver. We used an adapted version of the nominal group technique, a consensus-building discussion approach, to generate focused qualitative data.

RESULTS

Patients and a caregiver acknowledged both the costs directly related to clinical care, such as the out-of-pocket cost of drugs, and the indirect treatment costs, such as the cost of transport, accommodation, and food. The emotional and physical toll of treatment and the influence of treatment on employment and education were additional costs emphasized by participants. Treatment benefits prioritized by participants included the efficacy of treatment, manageable side effects, improved quality of life, accessibility of treatment, and short treatment duration.

CONCLUSIONS

Engaging patients and caregivers in an early economic evaluation could help identify additional costs and benefits of therapies that are not typically recognized in economic evaluations but have the potential to increase the commercial viability of novel therapies. This research also demonstrates how patients and caregivers can be engaged at different levels in the development of early economic evaluation models.

Identifying and understanding benefits associated with return-on-investment from large-scale healthcare Quality Improvement programmes: an integrative systematic literature review

BACKGROUND

We previously developed a Quality Improvement (QI) Return-on-Investment (ROI) conceptual framework for large-scale healthcare QI programmes. We defined ROI as any monetary or non-monetary value or benefit derived from QI. We called the framework the QI-ROI conceptual framework. The current study describes the different categories of benefits covered by this framework and explores the relationships between these benefits.

METHODS

We searched Medline, Embase, Global health, PsycInfo, EconLit, NHS EED, Web of Science, Google Scholar, organisational journals, and citations, using ROI or returns-on-investment concepts (e.g., cost-benefit, cost-effectiveness, value) combined with healthcare and QI. Our analysis was informed by Complexity Theory in view of the complexity of large QI programmes. We used Framework analysis to analyse the data using a preliminary ROI conceptual framework that was based on organisational obligations towards its stakeholders. Included articles discussed at least three organisational benefits towards these obligations, with at least one financial or patient benefit. We synthesized the different QI benefits discussed.

RESULTS

We retrieved 10 428 articles. One hundred and two (102) articles were selected for full text screening. Of these 34 were excluded and 68 included. Included articles were QI economic, effectiveness, process, and impact evaluations as well as conceptual literature. Based on these literatures, we reviewed and updated our QI-ROI conceptual framework from our first study. Our QI-ROI conceptual framework consists of four categories: 1) organisational performance, 2) organisational development, 3) external outcomes, and 4) unintended outcomes (positive and negative). We found that QI benefits are interlinked, and that ROI in large-scale QI is not merely an end-outcome; there are earlier benefits that matter to organisations that contribute to overall ROI. Organisations also found positive aspects of negative unintended consequences, such as learning from failed QI.

DISCUSSION AND CONCLUSION

Our analysis indicated that the QI-ROI conceptual framework is made-up of multi-faceted and interconnected benefits from large-scale QI programmes. One or more of these may be desirable depending on each organisation's goals and objectives, as well as stage of development. As such, it is possible for organisations to deduce incremental benefits or returns-on-investments throughout a programme lifecycle that are relevant and legitimate.

Early assessment of innovation in a healthcare setting

OBJECTIVES

Early assessment can assist in allocating resources for innovation effectively and produce the most beneficial technology for an institution. The aim of the present study was to identify methods and discuss the analytical approaches applied for the early assessment of innovation in a healthcare setting.

METHODS

Knowledge synthesis based on a structured search (using the MEDLINE, Embase, and Cochrane databases) and thematic analysis was conducted. An analytical framework based on the stage of innovation (developmental, introduction, or early diffusion) was applied to assess whether methods vary according to stage. Themes (type of innovation, study, analysis, study design, method, and main target audience) were then decided among the authors. Identified methods and analysis were discussed according to the innovation stage.

RESULTS

A total of 1,064 articles matched the search strategy. Overall, thirty-nine articles matched the inclusion criteria. The use of methods has a tendency to change according to the stage of innovation. Stakeholder analysis was a prominent method in the innovation stages and particularly in the developmental stage, as the introduction and early diffusion stage has more availability of data and may apply more complex methods. Barriers to the identified methods were also discussed as all of the innovation stages suffered from lack of data and substantial uncertainty.

CONCLUSIONS

Although this review has identified applicable approaches for early assessment in different innovation stages, research is required regarding the value of the available data and methods and tools to enhance interactions between different parties at different stages of innovation.

Patient preferences for the treatment of type 2 diabetes: a scoping review

BACKGROUND

As more studies report on patient preferences, techniques are needed to identify, assess and, eventually, synthesize results from a diverse set of methodologies. Data on patient preferences are valuable to decision makers in a variety of ways. Preferences for outcomes can be used to inform decision and cost-effectiveness models, while preferences for treatments can inform patient-centered outcomes research (PCOR) and patient-centered care.

OBJECTIVES

This project sought to identify and assess the literature reporting on the treatment preferences of adult patients with type 2 diabetes. In addition to cataloging the preference elicitation methods used, we developed and assessed a novel quality assessment checklist for preference-based studies.

DATA SOURCES

PubMed, EMBASE, CINAHL, and EconLit databases were searched to identify studies examining patient preferences for medications for type 2 diabetes studies published since inception of each database.

STUDY ELIGIBILITY CRITERIA, PARTICIPANTS, AND INTERVENTIONS

The review protocol specified inclusion of studies reporting diabetes-treatment preferences among adults with type 2 diabetes, using a range of preference measurement methods. Studies were excluded if participants were not patients with type 2 diabetes and if treatments were not pharmacological therapies targeting glycemic control, or if no primary preference information was collected. Two investigators independently reviewed titles, abstracts, and articles sequentially to select studies for data abstraction based on the inclusion and exclusion criteria. Disagreements were resolved by consensus.

STUDY APPRAISAL AND SYNTHESIS METHODS

Data on study country, year, number of respondents, preference elicitation method, number of attributes, subgroup analyses, and funding source were abstracted into standardized tables. A novel checklist (PREFS) was used to assess the data quality and validity across different types of preference studies by assessing the following: purpose of the study; respondent sampling; explanation of preference assessment methods; findings reported for total sample; and significance testing. Each item was scored, and an aggregate score was then calculated (ranging from 0 to 5).

RESULTS

Of the 2,100 unique citations, 61 met the inclusion criteria. The studies used conjoint analysis (n = 10), time trade-off (n = 6), standard gamble (n = 2), contingent valuation (n = 1), other stated preference methods (n = 39), and revealed preferences (n = 5). Sample sizes ranged from 27 to 14,033, with an average of 562 respondents, and two-thirds included a subgroup analysis. Most studies were conducted in one country, predominantly the USA (n = 27), UK (n = 14), Canada (n = 10), and Germany (n = 7), while 14 were conducted in multiple (2-18) countries across two or more countries. There was an increase in the annual rate of studies published over time from the time of the first publication in 1985 (p = < 0.01). Most (n = 52) studies were funded by pharmaceutical or device companies, with government, academic, association, and hospital sources also funding studies. One study met all five of the PREFS criteria and 12 met four; yet four studies met none of the criteria. The average was 27.

LIMITATIONS

Currently, preferences reviews are limited by the mixed quality in the reporting of studies, the publication bias inherent in the literature, a lack of guidelines to conduct various methods, and the difficulty of synthesizing results from different studies. Our study is also limited by its focus on English language articles.

CONCLUSIONS AND IMPLICATIONS OF KEY FINDINGS

This study provides the first systematic evaluation of the methods used in the broad existing body of research into patient preferences for type 2 diabetes medications and can serve as a primary source of information for decision makers. Future work is necessary to assess the utility of the results of reviews of preference information and to develop best-practice guidelines for the reporting of, and methods of conducting, preference studies and systematic reviews of such studies.

REGISTRATION

This systematic review was registered with PROSPERO (registration number CRD42012002285).

Things are Looking up Since We Started Listening to Patients: Trends in the Application of Conjoint Analysis in Health 1982-2007

Clinical and healthcare decision makers have repeatedly endorsed patient-centered care as a goal of the health system. However, traditional methods of evaluation reinforce societal views, and research focusing on views of patients is often referred to as 'soft science.' Conjoint analysis presents a scientifically rigorous research tool that can be used to understand patient preferences and inform decision making. This paper documents applications of conjoint analysis in medicine and systematically reviews this literature in order to identify publication trends and the range of topics to which conjoint analysis has been applied. In addition, we document important methodological aspects such as sample size, experimental design, and method of analysis.Publications were identified through a MEDLINE search using multiple search terms for identification. We classified each article into one of three categories: clinical applications (n = 122); methodological contributions (n = 56); and health system applications (n = 47). Articles that did not use or adequately discuss conjoint analysis methods (n = 164) were discarded. We identified a near exponential increase in the application of conjoint analyses over the last 10 years of the study period (1997-2007). Over this period, the proportion of applications on clinical topics increased from 40% of articles published in MEDLINE from 1998 to 2002, to 64% of articles published from 2003 to 2007 (p = 0.002).The average sample size among articles focusing on health system applications (n = 556) was significantly higher than clinical applications (n = 277) [p = 0.001], although this 2-fold difference was primarily due to a number of outliers reporting sample sizes in the thousands. The vast majority of papers claimed to use orthogonal factorial designs, although over a quarter of papers did not report their design properties. In terms of types of analysis, logistic regression was favored among clinical applications (28%), while probit was most commonly used among health systems applications (38%). However, 25% of clinical applications and 33% of health systems articles failed to report what regression methods were used. We used the International Classification of Diseases - version 9 (ICD-9) coding system to categorize clinical applications, with approximately 26% of publications focusing on neoplasm. Program planning and evaluation applications accounted for 22% of the health system articles.While interest in conjoint analysis in health is likely to continue, better guidelines for conducting and reporting conjoint analyses are needed.

Patient-based health technology assessment: a vision of the future

BACKGROUND

In conjunction with other important movements in contemporary medicine, including evidence-based medicine (EBM), health technology assessment (HTA) has promoted a culture of critical evaluation. Despite this impact, institutional and methodological challenges are associated with HTA. For example, only in recent years has HTA attempted an open dialogue with patients; however, this is normally done by giving them a "seat" at the HTA decision-making table, rather than by more scientific means.

OBJECTIVES

The aim of this study was to develop a working definition of patient-based HTA, to identify the current barriers to adopting a patient-based model, and to formulate a vision of how a patient-based HTA could be used to promote patient empowerment and patient-centered care.

RESULTS

In the ideal setting, a patient-based HTA would promote patient knowledge by providing access to information and promoting an informed dialogue between patients and their healthcare professionals. To implement a patient-based HTA, the focus must turn to the patient's issues and incorporate each patient's unique perspective and preferences. Processes must change to increase patient participation in all levels of HTA and aim to promote empowered patients who can make informed decisions.

CONCLUSIONS

Present-day HTA is broad and has numerous stakeholders, with none so important as the patient. By asking patient-oriented questions in HTA and better involving patients throughout the entire process, we can easily promote patient empowerment, and as such make patients more capable to play a more active role in healthcare decision making.