IMPORTANCE OF CONTEXTUAL DATA IN PRODUCING HEALTH TECHNOLOGY ASSESSMENT RECOMMENDATIONS: A CASE STUDY

Production and use of rapid responses during the COVID-19 pandemic in Quebec (Canada): perspectives from evidence synthesis producers and decision makers

BACKGROUND

The COVID-19 pandemic has required evidence to be made available more rapidly than usual, in order to meet the needs of decision makers in a timely manner. These exceptional circumstances have caused significant challenges for organizations and teams responsible for evidence synthesis. They had to adapt to provide rapid responses to support decision-making. This study aimed to document (1) the challenges and adaptations made to produce rapid responses during the pandemic, (2) their perceived usefulness, reported use and factors influencing their use and (3) the methodological adaptations made to produce rapid responses.

METHODS

A qualitative study was conducted in 2021 with eight organizations in the health and social services system in Quebec (Canada), including three institutes with a provincial mandate. Data collection included focus groups (n = 9 groups in 8 organizations with 64 participants), interviews with decision makers (n = 12), and a document analysis of COVID-19 rapid responses (n = 128). A thematic analysis of qualitative data (objectives 1 and 2) and a descriptive analysis of documents (objective 3) were conducted.

RESULTS

The results highlight the teams and organizations' agility to deal with the many challenges encountered during the pandemic (e.g., increased their workloads, adoption of new technological tools or work processes, improved collaboration, development of scientific monitoring, adaptation of evidence synthesis methodologies and products). The challenge of balancing rigor and speed was reported by teams and organizations. When available at the right time, rapid responses have been reported as a useful tool for informing or justifying decisions in a context of uncertainty. Several factors that may influence their use were identified (e.g., clearly identify needs, interactions with producers, perceived rigor and credibility, precise and feasible recommendations). Certain trends in the methodological approaches used to speed up the evidence synthesis process were identified.

CONCLUSIONS

This study documented rapid responses producers' experiences during the COVID-19 pandemic in Quebec, and decision makers who requested, consulted, or used these products. Potential areas of improvements are identified such as reinforce coordination, improve communication loops, clarify guidelines or methodological benchmarks, and enhance utility of rapid response products for decision makers.

Evolving Use of Health Technology Assessment in Medical Device Procurement-Global Systematic Review: An ISPOR Special Interest Group Report

OBJECTIVES

To review the current academic evidence describing how data from health technology assessments (HTAs) informs procurement decisions for medical devices.

METHODS

A systematic literature review was performed to identify relevant studies and criteria used in medical device purchasing or procurement decisions. Included articles were screened for relevancy and risk of bias. The included studies were summarized qualitatively.

RESULTS

A total of 292 studies were screened, of which 11 matched the inclusion criteria. Included studies' geographies and HTA maturity varied. Some studies described hospital-level HTA processes, whereas others focused on national-level recommendations. Criteria for procurement decisions included standard HTA factors, such as efficacy, cost, cost-effectiveness, and budget impact; broader issues were also noted, including impact on the organization, ethical aspects, staff workload, and volume. There was little consideration of device-specific characteristics, such as life cycle, learning curve, or incremental technical innovation. Few decisions referred to HTA reports as part of the procurement decision; similarly, few HTA reports included a procurement perspective to help guide the procurement bodies.

CONCLUSIONS

There is minimal evidence that notes HTA influencing medical device procurement. Procurement bodies and hospitals may not be incentivized to publish their work and transparency could be improved; further research would better describe the link between HTA and procurement. Such research would enable the HTA agencies to meaningfully assess devices to target procurement bodies and allow device sponsors to prioritize evidence. This could limit redundancy, improve evidence, and ultimately promote savings to healthcare systems and expand access.

Agreement between single plane and biplane derived angiographic fractional flow reserve in patients with intermediate coronary artery stenosis

Fractional flow reserve (FFR) is often used to evaluate the physiological severity of intermediate coronary stenoses, but less-invasive assessment methods are desirable. We evaluated the feasibility of angiographic FFR (angioFFR) calculated from two projections acquired simultaneously by a biplane C-arm system and angioFFR calculated from two projections acquired independently by one plane of the same biplane C-arm system. AngioFFR was validated against FFR in terms of detection of hemodynamically relevant coronary artery stenoses. Twenty-two Patients who underwent angiography and FFR for coronary artery disease were included. We used a non-commercial prototype to calculate biplane angioFFR for 22 vessels (19 LAD, 1 LCx, 2 RCA) and single plane angioFFR for 17 of the same 22 vessels. FFR < 0.8 was measured in 8 vessels. The Pearson correlation coefficients with FFR were 0.55 for single plane angioFFR and 0.61 for biplane angioFFR and the diagnostic accuracies were 88% (95% CI 73-100%) for single plane angioFFR and 86% (95% CI 72-100%) for biplane angioFFR. Bland-Altman plots revealed that compared with FFR, the limits of agreement for single plane angioFFR were  - 0.07 to 0.19 (mean difference 0.06, p = 0.002) and the limits of agreement for biplane FFR were  - 0.09 to 0.15 (mean difference 0.03, p = 0.03). In conclusion, angioFFR calculated from single or biplane acquisitions by a biplane C-arm is feasible and may evolve to a tool for less invasive imaging-based assessment of myocardial ischemia.

Challenges for developing health-care knowledge in the digital age

Purpose

Health-care knowledge is dispersed among different departments in a health care organization, which makes it difficult at times to provide quality care services to patients. Therefore, this study aims to identify the main challenges in adopting health information technology (HIT).

Design/methodology/approach

This study surveyed 148 stakeholders in 4 key categories [patients, health-care providers, United Arab Emirates (UAE) citizens and foresight experts] to identify the challenges they face in adopting health care technologies. Responses were analyzed using exploratory (EFA) and confirmatory factor analysis (CFA).

Findings

EFA revealed four key latent factors predicting resistance to HIT adoption, namely, organizational strategy (ORGS); technical barriers; readiness for big data and the internet of things (IoT); and orientation (ORI). ORGS accounted for the greatest amount of variance. CFA indicated that readiness for big data and the IoT was only moderately correlated with HIT adoption, but the other three factors were strongly correlated. Specific items relating to cost, the effectiveness and usability of the technology and the organization were strongly correlated with HIT adoption. These results indicate that, in addition to financial considerations, effective HIT adoption requires ensuring that technologies will be easy to implement to ensure their long-term use.

Research limitations/implications

The results indicate that readiness for big data and the IoT-related infrastructure poses a challenge to HIT adoption in the UAE context. Respondents believed that the infrastructure of big data can be helpful in more efficiently storing and sharing health-care information. On the technological side, respondents felt that they may experience a steep learning curve. Regarding ORI, stakeholders expected many more such initiatives from health-care providers to make it more knowledge-specific and proactive.

Practical implications

This study has implications for knowledge management in the health -care sector for information technologies. The HIT can help firms in creating a knowledge eco-system, which is not possible in a dispersed knowledge environment. The utilization of the knowledge base that emerged from the practices and data can help the health care sector to set new standards of information flow and other clinical services such as monitoring the self-health condition. The HIT can further influence the actions of the pharmaceutical and medical device industry.

Originality/value

This paper highlights the challenges in HIT adoption and the most prominent factors. The conceptual model was empirically tested after the collection of primary data from the UAE using stakeholder theory.

Patients, users, caregivers, and citizens' involvement in local health technology assessment unit in Quebec: a survey

OBJECTIVES

Increasing emphasis is given on involving patients in health technology assessment (HTA). While this is mainly done at the level of regional and national HTA agencies, this tendency is also emerging in local HTA units. In this study, we provide the results of a survey conducted in local HTA units in the province of Quebec, Canada. The aim of the survey was to provide an overview of local HTA unit practices to involve patients, users, caregivers, and citizens in their process, their interest in doing so, and their information needs for this.

METHODS

The survey was conducted in 2017 with a response rate of eleven units over a possibility of twelve.

RESULTS

Three units out of eleven (27.3 percent) never involved patients or members of the public in their processes and all indicated that they will involve them in the next few years. The three most important needs for support identified in the HTA units were in: recruiting and selecting patients; integrating experiential knowledge; and knowing and implementing the best methods and practices for partnership.

CONCLUSION

Patient involvement in local HTA units is quickly evolving and that is why they urgently need tools to involve more effectively patients and members of the public in their process.

Artificial Intelligence and Health Technology Assessment: Anticipating a New Level of Complexity

Artificial intelligence (AI) is seen as a strategic lever to improve access, quality, and efficiency of care and services and to build learning and value-based health systems. Many studies have examined the technical performance of AI within an experimental context. These studies provide limited insights into the issues that its use in a real-world context of care and services raises. To help decision makers address these issues in a systemic and holistic manner, this viewpoint paper relies on the health technology assessment core model to contrast the expectations of the health sector toward the use of AI with the risks that should be mitigated for its responsible deployment. The analysis adopts the perspective of payers (ie, health system organizations and agencies) because of their central role in regulating, financing, and reimbursing novel technologies. This paper suggests that AI-based systems should be seen as a health system transformation lever, rather than a discrete set of technological devices. Their use could bring significant changes and impacts at several levels: technological, clinical, human and cognitive (patient and clinician), professional and organizational, economic, legal, and ethical. The assessment of AI's value proposition should thus go beyond technical performance and cost logic by performing a holistic analysis of its value in a real-world context of care and services. To guide AI development, generate knowledge, and draw lessons that can be translated into action, the right political, regulatory, organizational, clinical, and technological conditions for innovation should be created as a first step.

Peruvian Valuation of the EQ-5D-5L: A Direct Comparison of Time Trade-Off and Discrete Choice Experiments

OBJECTIVES

(1) To produce Peruvian general population EQ-5D-5L value sets on a quality-adjusted life-year scale, (2) to investigate the feasibility of a "Lite" protocol less reliant on the composite time trade-off (cTTO), and (3) to compare cTTO and discrete choice experiment (DCE) value sets.

METHODS

A random sample of adults (N = 1000) in Lima, Arequipa, and Iquitos did a home interview; 300 were randomly selected to complete 11 cTTOs first. All respondents completed a DCE, including 10 latent-scale pairs (A/B) with 5 EQ-5D-5L attributes, and 12 matched pairs (A/B and B/C) with 5 EQ-5D-5L and one lifespan attributes. We estimated a cTTO heteroscedastic tobit (N = 300) model and 3 DCE Zermelo-Bradley-Terry models (N = 300, 700, and 1000).

RESULTS

Each model produced a consistent value set (20 positive incremental parameters). Nevertheless, their lowest quality-adjusted life-year values differed greatly (cTTO: -1.076 [N = 300]; DCE: -0.984 [300], 0.048 [700], -0.213 [1000]). Compared with the cTTO, the DCE (N = 300) produced different parameters (Pearson's correlation = 0.541), fewer insignificant parameters (0 vs 8), and fewer values less than 0 (26% vs 44%). Compared with the DCE (N = 300), the DCE (N = 700) produced higher values but similar parameters (Pearson's correlation = 0.800).

CONCLUSIONS

Besides producing EQ-5D-5L value sets for Peru, the results casts doubt about the feasibility of a Lite protocol like the one in this study. Additionally, fundamental differences between cTTO and DCE-without the existence of a gold standard-need further clarification. The choice between the two rational value sets produced in the current study is a matter of judgment and may have substantial policy implications.

The Lean and Agile Multi-dimensional Process (LAMP) - a new framework for rapid and iterative evidence generation to support health-care technology design and development

Introduction: Health technology assessments (HTA) are tools for policymaking and resource allocation. Early HTAs are increasingly used in design and development of new technologies. Conducting early HTAs is challenging, due to a lack of evidence and significant uncertainties in the technology and the market. A multi-disciplinary approach is considered essential. However, an operational framework that can enable the integration of multi-dimensional evidence into commercialization remains lacking.Areas covered: We developed the Lean and Agile Multi-dimensional Process (LAMP), an early HTA framework, for embedding commercial decision-making in structured evidence generation activities, divided into phases. Diverse evidence in unmet needs, user acceptance, cost-effectiveness, and market competitiveness are being generated in increasing depth. This supports the emergence of design and value propositions that align technology capabilities and clinical and user needs.Expert opinion: We have been applying LAMP to working with medical device and diagnostic industry in the UK. The framework can be adapted to suit different technologies, decision needs, time scales, and resources. LAMP offers a practical solution to the multi-disciplinary approach. Methodologists drive the process by performing evidence generation and synthesis as and by enabling interactions between manufacturers, designers, clinicians, and other key stakeholders.

A discrete choice experiment on preferences of patients with low back pain about non-surgical treatments: identification, refinement and selection of attributes and levels

Objectives: Hospital-based health technology assessment (HB-HTA) needs to consider all relevant data to help decision making, including patients' preferences. In this study, we comprehensively describe the process of identification, refinement and selection of attributes and levels for a discrete choice experiment (DCE). Methods: A mixed-methods design was used to identify attributes and levels explaining low back pain (LBP) patients' choice for a non-surgical treatment. This design combined a systematic literature review with a patients' focus group, one-on-one interactions with experts and patients, and discussions with stakeholder committee members. Following the patient's focus group, preference exercises were conducted. A consensus about the attributes and levels was researched during discussions with committee members. Results: The literature review yielded 40 attributes to consider in patients' treatment choice. During the focus group, one additional attribute emerged. The preference exercises allowed selecting eight attributes for the DCE. These eight attributes and their levels were discussed and validated by the committee members who helped reframe two levels in one of the attributes and delete one attribute. The final seven attributes were: treatment modality, pain reduction, onset of treatment efficacy, duration of efficacy, difficulty in daily living activities, sleep problem, and knowledge about their body and pain. Conclusion: This study is one of the few to comprehensively describe the selection process of attributes and levels for a DCE. This may help ensure transparency and judge the quality of the decision-making process. In the context of a HB-HTA unit, this strengthens the legitimacy to perform a DCE to better inform decision makers in a patient-centered care approach.

Bi-plane and single plane angiography: a study to compare contrast usage and radiation doses for adult cardiac patients in diagnostic studies

OBJECTIVE:

This study compares the performance of bi-plane coronary angiography against single plane angiography in terms of the volume of contrast used (ml) and the total dose-area product (DAP) (μGym²) to the patient measured directly via flat panel detectors.

METHODS:

A total of 5176 adult diagnostic cardiac angiograms from a hospital in Brisbane, Australia were retrospectively studied. Patients with aortograms, iliac or femoral artery imaging, and stenting or graft interventions were excluded. Student's t-tests were used to compare means, and confounding variables were compared using multivariate regression. This quantified the effects of bi-plane system use holding constant other factors (e.g.) body mass index (BMI), age, room, sex, number of digital acquisitions and fluoro time.

RESULTS:

Bi-plane imaging had an average difference in mean contrast use of -15.1 ml [15.5% 95% confidence interval (CI) (-13.2, -17.0) p<0.001], multivariate regression demonstrated a -27.0 ml reduction in contrast use [28% 95% CI (-29.0, -24.83) p<0.0001] when the significant effects of fluoro time, number of digital acquisitions, BMI and sex were held constant. Bi-plane imaging had an average difference in mean DAP of + 887.1 μGym² [23% 95% CI (+1110.7, +663.4) p < 0.001], whilst multivariate regression found a +628.3 Gym² increase in DAP [16% 95% CI (+467.5, +789.3) p<0.001] when the significant effects of fluoro time, number of digital acquisitions, BMI and sex were held constant.

CONCLUSION:

These results demonstrate that bi-plane imaging uses less contrast media than single-plane imaging for coronary angiography at the expense of more radiation. Bi-plane imaging may be preferable in patients with renal impairment, however single plane imaging may be preferable in those without renal impairment.

ADVANCES IN KNOWLEDGE:

This is a large cohort and statistically comprehensive study comparing bi-plane and single plane coronary angiography. Other studies 4, 5, 6, 12 have used Student's t-tests to measure the difference between means, however this provides no causative information on the differences found. This study provides a view of the causative impact of bi-plane usage on DAP and contrast use via multivariate regression modelling.

HEALTH TECHNOLOGY ASSESSMENT UNIT PROCESSES FOR THE VALIDATION OF AN INFORMATION TOOL TO INVOLVE PATIENTS IN THE SAFETY OF THEIR CARE

Introduction:

Patients and families play an important role in preventing adverse events. The quality council at our hospital produced a communication tool in considering the main causes of adverse events and requested the health technology assessment (HTA) unit to validate it.

Objectives:

Assess the validity of the content of a tablemat sticker as an information tool for hospitalized patients.

Methods:

A qualitative validation was first performed with individual interviews and focus groups to evaluate the understanding of the content. The tool was modified and as a second step, a survey was conducted on patients and their families from a surgical care unit to validate their understanding and relevance of the content.

Results:

From the survey, patients and families found the tablemat attractive and stimulating (97 percent). It encouraged them to communicate with staff about the safety of their care (84 percent). They understood well the objective (79 percent) and text (90 percent), but less for the pictograms (30 percent to 62 percent). The communication and recommendations to avoid falling were good and 99 percent were wearing the medical identification. However, it was not clear that these indicators represented the real concerns of the patients and healthcare staff because no user evaluation was done when developing the tool.

Conclusions:

The tool was well understood, but some improvements are needed considering that pictograms were not always well understood and so need careful consideration from patient perspective. The HTA unit recommended conducting an unbiased survey to assess the concerns of patients and professionals to identify the most relevant indicators.