Co-construction of health technology assessment recommendations with patients: An example with cardiac defibrillator replacement

Reflections on co-creating a model for the value assessment of artificial intelligence technologies

AIMS

A multidisciplinary group of experts and patients developed the Model for ASsessing the value of Artificial Intelligence (MAS-AI) to ensure an evidence-based and patient-centered approach to introducing artificial intelligence technologies in healthcare. In this article, we share our experiences with meaningfully involving a patient in co-creating a research project concerning complex and technically advanced topics.

METHODS

The co-creation was evaluated by means of initial reflections from the research team before the project started, in a continuous logbook, and through semi-structured interviews with patients and two researchers before and after the active co-creation phase of the project.

RESULTS

There were initial doubts about the feasibility of including patients in this type of project. Co-creation ensured relevance to patients, a holistic research approach and the debate of ethical considerations. Due to one patient dropping out, it is important to foresee and support the experienced challenges of time and energy spent by the patient in future projects. Having a multidisciplinary team helped the collaboration. A mutual reflective evaluation provided insights into the process which we would otherwise have missed.

CONCLUSIONS

We found it possible to create complex and data-intense research projects with patients. Including patients benefitted the project and gave researchers new perspectives on their own research. Mutual reflection throughout the project is key to maximise learning for all parties involved.

Assessing medical devices: a qualitative study from the validate perspective

OBJECTIVES

Our objective was to explore procedures and methods used at health technology assessment (HTA) agencies for assessing medical devices and the underlying views of HTA practitioners about appropriate methodology to identify challenges in adopting new methodologies for assessing devices. We focused on the role of normative commitments of HTA practitioners in the adoption of new methods.

METHODS

An online survey, including questions on procedures, scoping, and assessments of medical devices, was sent to members of the International Network of Agencies for Health Technology Assessment. Interviews were conducted with survey respondents and HTA practitioners involved in assessments of transcatheter aortic valve implantation to gain an in-depth understanding of choices made and views about assessing medical devices. Survey and interview questions were inspired by the "values in doing assessments of health technologies" approach towards HTA, which states that HTA addresses value-laden questions and information.

RESULTS

The current practice of assessing medical devices at HTA agencies is predominantly based on procedures, methods, and epistemological principles developed for assessments of drugs. Both practical factors (available time, demands of decision-makers, existing legal frameworks, and HTA guidelines), as well as commitments of HTA practitioners to principles of evidence-based medicine, make the adoption of a new methodology difficult.

CONCLUSIONS

There is a broad recognition that assessments of medical devices may need changes in HTA methodology. In order to realize this, the HTA community may require both a discussion on the role, responsibility, and goals of HTA, and resulting changes in institutional context to adopt new methodologies.

How to mobilise users' experiential knowledge in the evaluation of advanced technologies and practices in Quebec? The example of the permanent users' and relatives' panel

INTRODUCTION

With the purpose of supporting scientific professionals and helping them to better integrate the expertise of users in their work, a users' and relatives' panel (URP) was set up at the National Institute for Excellence in Health and Social Services in Quebec (INESSS), Canada for the social services and mental health directorate. URPs are advisory structures that mobilise the experiential knowledge of people affected by various issues.

OBJECTIVES

The objective of this study is to assess from a diverse stakeholders' perceptions: (1) the experience of developing and implementing the URP within the context of an Agencies for Health Technology Assessment and Assessment of Social Services (AHTAASS), (2) the contribution of such a URP, (3) the challenges encountered and (4) the perspectives of improvement for the following years.

METHODOLOGY

We conducted a qualitative descriptive evaluation study. Nineteen interviews were conducted: six with URP members and 13 with staff representatives. The documents related to the creation of the panel, the URP minutes summarising the discussions and the reports published during that period were collected and analysed. Following a preliminary round of data analysis, a debriefing meeting was conducted with a few participants to validate the results.

RESULTS

The panel was set up as part of the INESSS' desire to better integrate experiential knowledge into its recommendations. Twelve projects were presented to the panel on various themes. The URP enabled health professionals to consider dimensions they had not identified, to better integrate the experiential data collected from users into their work and to develop recommendations that made more sense to users. Panel members and INESSS professionals learned to work together, moving the working methods from consultation to collaboration and even coconstruction. Based on the panel's significant contribution, the INESSS decided to maintain it and to strengthen its place in its system to better integrate the experiential knowledge of users into its work.

CONCLUSION

This research illustrates how AHTAASS can set up a URP composed exclusively of users, and how it can contribute and be evaluated. It shows that URPs are structures that value the sharing of experiential knowledge of its members, humanise decision-making and give meaning to the work done by scientific professionals.

PATIENT OR PUBLIC CONTRIBUTION

One patient-researcher has contributed to the preparation and writing of this manuscript.

Roles, outcomes, and enablers within research partnerships: A rapid review of the literature on patient and public involvement and engagement in health research

BACKGROUND

Recent studies mention a need to investigate partnership roles and dynamics within patient and public involvement and engagement (PPIE) in health research, and how impact and outcomes are achieved. Many labels exist to describe involvement processes, but it is unknown whether the label has implications on partnerships and outcomes. This rapid review investigates how roles between patients, relatives and researchers in a broad variety of PPIE activities in health research are described in peer reviewed papers and explores what enables these partnerships.

METHODS

Rapid review of articles published between 2012 and February 2022 describing, evaluating, or reflecting on experiences of PPIE in health research. All research disciplines and research areas were eligible. Four databases (Medline, Embase, PsychInfo and CINAHL) were searched between November 2021 and February 2022. We followed PRISMA guidelines and extracted descriptive factors: year, origin, research area and discipline, study focus, framework used and co-authorship. On a selection of articles, we performed a narrative analysis of partnership roles using Smits et al.'s. Involvement Matrix. Lastly, we performed a meta synthesis of reported enablers and outcomes of the partnerships. Patients and Relatives (PRs) have been involved in the whole rapid review process and are co-authors of this article.

RESULTS

Seventy articles from various research disciplines and areas were included. Forty articles were selected for a narrative analysis of the role description of PRs and researchers, and a meta synthesis of enablers and outcomes. Most articles described researchers as decision-makers throughout the research cycle. PRs most often were partners when they were included as co-authors; they were mostly partners in the design, analysis, write-up, and dissemination stages. Enablers of partnerships included: PR training, personality of PRs and communication skills, trust, remuneration and time.

CONCLUSIONS

Researchers' decision-making roles gives them control of where and when to include PRs in their projects. Co-authorship is a way of acknowledging patients' contributions which may lead to legitimation of their knowledge and the partnership. Authors describe common enablers, which can help future partnership formation.

Patient participation in priority setting: Co-existing participant roles

Reflecting a 'participatory turn' in healthcare, a variety of activities have been adopted in many countries to harness the views of patients, relatives and service users. While celebrated as a way of empowering patients and increasing the legitimacy of decisions that impact on patient care, critics contend that practices of patient participation often fall short of the ideals they purport to implement. In this article, we investigate how patients' participation in medical priority setting corresponds with the regulative ideals of deliberation and how the practices of participation influence the ability of patients to make their voices heard. Building on document analysis and semi-structured interviews with 12 patient representatives and four scientific officers in the Danish Medicines Council, the analysis demonstrates that conflicting notions of valid knowledge constituted a main challenge for patient participation. The study contributes to the literature on patient participation through a conceptualization of four co-existing participant roles: 1) compliant keepers of experiential knowledge, 2) lay experts investing in evidence production, 3) knowledge translators engaged in alliance building, and 4) demonstrators promoting public contestation. We suggest that a main challenge for PP initiatives is to take into account this variation in patients' engagement.

Patient and public involvement in health technology assessment: update of a systematic review of international experiences

OBJECTIVE

To summarize current evidence on patient and public involvement (PPI) in health technology assessment (HTA) in order to synthesize the barriers and facilitators, and to propose a framework to assess its impact.

METHODS

We conducted an update of a systematic review published in 2011 considering the recent scientific literature (qualitative, quantitative, and mixed-methods studies). We searched papers published between March 2009 (end of the initial search) and December 2019 in five databases using specific search strategies. We identified other publications through citation tracking and contacting authors of previous related studies. Reviewers independently selected relevant studies based on prespecified inclusion and exclusion criteria. We extracted information using a pre-established grid.

RESULTS

We identified a total of 7872 publications from the main search strategy. Ultimately, thirty-one distinct new studies met the inclusion criteria, whereas seventeen studies were included in the previous systematic review. PPI is realized through two main strategies: (i) patients and public members participate directly in decision-making processes (participation) and (ii) patients or public perspectives are solicited to inform decisions (consultation or indirect participation). This review synthesizes the barriers and facilitators to PPI in HTA, and a framework to assess its impact is proposed.

CONCLUSION

The number of studies on patients or public involvement in HTA has dramatically increased in recent years. Findings from this updated systematic review show that PPI is done mostly through consultation and that direct involvement is less frequent. Several barriers to PPI in HTA exist, notably the lack of information to patients and public about HTA and the lack of guidance and policies to support PPI in HTA.

Evaluation of the impact of patient input in health technology assessments at NICE

OBJECTIVE

Accounts of patient experiences are increasingly used in health technology assessment (HTA) processes. However, we know little about their impact on the decision-making process. This study aims to assess the level and the type of impact of patient input to highly specialised technologies (HSTs) and interventional procedures (IPs) guidance at the National Institute for Health and Care Excellence (NICE).

METHODS

A questionnaire was developed to capture quantitative and qualitative data on the amount and type of impact of patient input into NICE HTAs. It was completed by committee members of the guidance-producing programs after a discussion of the considered topics. The data were analyzed by topic and overall, for each program, and compared across programs.

RESULTS

Patient input was assessed on ten pieces of HST guidance published between January 2015 and November 2019, and on twenty-six pieces of IP guidance scoped between February 2016 and October 2018. A total of 96 responses were collected for HST and 440 for IP. The level of impact of patient input was higher for HST than for IP. For HST, no respondents stated that it had no impact, whereas in IP, 35 percent of respondents did. The most common types of impact found for HST and IP were that it helped interpret the other evidence and that it provided new evidence.

CONCLUSIONS

The impact of patient input is not necessarily explicit in changing recommendations, but it provides context, reassurance, and new information to the committee for the decision-making process in HTAs.

Patients and public are important stakeholders in health technology assessment but the level of involvement is low - a call to action

BACKGROUND

Health technology assessment (HTA) agencies have an important role in the evaluation and approval of new technologies. They determine their value within a health system so to promote equitable, quality care with available healthcare resources. Many HTA agencies have some mechanism for involving patients in their processes, but there is great variability and an absence of comprehensive, robust practices for involvement. The accelerating pace of medical innovation creates a need to improve the depth and breadth of patient involvement in the HTA process.

MAIN BODY

In this 'Call to action', we present ideas from three HTA expert commentaries calling for collaborative learning and to share innovative ideas for changes in HTA. We also draw on examples of HTA agencies creatively pursuing this goal. We propose a 'Call to action' for HTA stakeholders to undertake serious dialogue with patient advocates aimed at creating shared goals. HTA agencies can use these goals to ensure meaningful patient involvement at every step of the HTA process. Five elements are explored. In 'Recognizing the value of shared purpose', we highlight examples of HTA agencies that have patients working in partnership with medical practitioners and HTA staff. Results include improved processes that instil confidence. 'Committing to patient involvement as part of HTA culture' highlights several initiatives aimed at changes in HTA organisational culture to be more inclusive of patients. In 'Aligning patient and HTA goals' we cite work in Belgium and New Zealand which places a greater emphasis on quality of life rather than life expectancy and cost-effectiveness. By 'Integrating patient involvement at every step of the HTA process' patients can make vital contributions at every stage of the HTA process. We provide two examples of where HTA agencies have successfully involved patients early in the process in order to broaden the scope of evaluations. 'Developing a common language and working together' can support transformative dialogue through 'unified language'.

CONCLUSION

The authors of this commentary ask that agencies and stakeholders involved in HTA take up this call to work together for visionary and transformative elevation of the voice of patients in HTA worldwide.

Developing recommendations for the diagnosis and treatment of Lyme disease: the role of the patient's perspective in a controversial environment

CONTEXT

The Ministry of Health in Québec requested the National Institute of Excellence in Health and Social Services to produce clinical and implementation recommendations for the prophylaxis, diagnosis, and treatment of Lyme disease.

OBJECTIVES

(i) Describe the process of trialing different modalities of patient engagement as a means to integrate a diversity of patient perspectives and (ii) Describe the learning process of INESSS regarding the integration of the patient perspective.

METHODOLOGY

All documents were analyzed, and a survey with all advisory committee members and semi-structured interviews with stakeholders were conducted. Each interview was transcribed verbatim and imported into QDA miner software for the purposes of analysis. Data analysis was carried out concurrently with data collection to allow for an iterative approach between data collection and analysis.

RESULTS

Five methods to integrate the perspectives of patients were used: (i) interviews with patients, (ii) inclusion of patient partners within the advisory committee, (iii) literature review, (iv) focus groups with one patient association, and (v) feedback from patient associations on recommendations intended for decision makers and other targeted stakeholders. The patient partners influenced decisions by sharing their experiential knowledge. The patient interviews and the literature review added an in-depth perspective on the disease and experience with the healthcare system. The patient association members shared their perspectives and helped disseminate the recommendation to sustain a practice change.

CONCLUSION

The combination of methods to collect and integrate patients' knowledge and patient associations' perspectives helped develop a comprehensive understanding of a controversial object of evaluation.

Patient and caregiver engagement in the Patient-Centered Outcomes Research Institute (PCORI) Health Care Horizon Scanning System (HCHSS) process

OBJECTIVE

The Patient-Centered Outcomes Research Institute (PCORI) horizon scanning system is an early warning system for healthcare interventions in development that could disrupt standard care. We report preliminary findings from the patient engagement process.

METHODS

The system involves broadly scanning many resources to identify and monitor interventions up to 3 years before anticipated entry into U.S. health care. Topic profiles are written on included interventions with late-phase trial data and circulated with a structured review form for stakeholder comment to determine disruption potential. Stakeholders include patients and caregivers recruited from credible community sources. They view an orientation video, comment on topic profiles, and take a survey about their experience.

RESULTS

As of March 2020, 312 monitored topics (some of which were archived) were derived from 3,500 information leads; 121 met the criteria for topic profile development and stakeholder comment. We invited fifty-four patients and caregivers to participate; thirty-nine reviewed at least one report. Their perspectives informed analyst nominations for fourteen topics in two 2019 High Potential Disruption Reports. Thirty-four patient stakeholders completed the user-experience survey. Most agreed (68 percent) or somewhat agreed (26 percent) that they were confident they could provide useful comments. Ninety-four percent would recommend others to participate.

CONCLUSIONS

The system has successfully engaged patients and caregivers, who contributed unique and important perspectives that informed the selection of topics deemed to have high potential to disrupt clinical care. Most participants would recommend others to participate in this process. More research is needed to inform optimal patient and caregiver stakeholder recruitment and engagement methods and reduce barriers to participation.

Patients, users, caregivers, and citizens' involvement in local health technology assessment unit in Quebec: a survey

OBJECTIVES

Increasing emphasis is given on involving patients in health technology assessment (HTA). While this is mainly done at the level of regional and national HTA agencies, this tendency is also emerging in local HTA units. In this study, we provide the results of a survey conducted in local HTA units in the province of Quebec, Canada. The aim of the survey was to provide an overview of local HTA unit practices to involve patients, users, caregivers, and citizens in their process, their interest in doing so, and their information needs for this.

METHODS

The survey was conducted in 2017 with a response rate of eleven units over a possibility of twelve.

RESULTS

Three units out of eleven (27.3 percent) never involved patients or members of the public in their processes and all indicated that they will involve them in the next few years. The three most important needs for support identified in the HTA units were in: recruiting and selecting patients; integrating experiential knowledge; and knowing and implementing the best methods and practices for partnership.

CONCLUSION

Patient involvement in local HTA units is quickly evolving and that is why they urgently need tools to involve more effectively patients and members of the public in their process.

Artificial Intelligence and Health Technology Assessment: Anticipating a New Level of Complexity

Artificial intelligence (AI) is seen as a strategic lever to improve access, quality, and efficiency of care and services and to build learning and value-based health systems. Many studies have examined the technical performance of AI within an experimental context. These studies provide limited insights into the issues that its use in a real-world context of care and services raises. To help decision makers address these issues in a systemic and holistic manner, this viewpoint paper relies on the health technology assessment core model to contrast the expectations of the health sector toward the use of AI with the risks that should be mitigated for its responsible deployment. The analysis adopts the perspective of payers (ie, health system organizations and agencies) because of their central role in regulating, financing, and reimbursing novel technologies. This paper suggests that AI-based systems should be seen as a health system transformation lever, rather than a discrete set of technological devices. Their use could bring significant changes and impacts at several levels: technological, clinical, human and cognitive (patient and clinician), professional and organizational, economic, legal, and ethical. The assessment of AI's value proposition should thus go beyond technical performance and cost logic by performing a holistic analysis of its value in a real-world context of care and services. To guide AI development, generate knowledge, and draw lessons that can be translated into action, the right political, regulatory, organizational, clinical, and technological conditions for innovation should be created as a first step.

Co-construction of health technology assessment recommendations with patients: An example with cardiac defibrillator replacement

CONTEXT

The National Institute of Excellence in Health and Social Services (INESSS), which functions as the Québec health technology assessment (HTA) agency, tested a new way to engage patients along with health-care professionals in the co-construction of recommendations regarding implantable cardioverter-defibrillator replacement.

OBJECTIVE

The objective of this article was to describe the process of co-construction of recommendations and to propose methods of building best practices for patient involvement (PI) in HTA.

DESIGN

Throughout the process, documents were collected and participant observations were made. Individual interviews were conducted with patients, health-care professionals and the INESSS scientific team, from January to March 2018.

RESULTS

Three committees were established: an expert patient committee to reflect on patient experience literature; an expert health professional committee to reflect on medical literature; and a co-construction committee through which both patients and health-care professionals contributed to develop the recommendations. The expert patients validated and contextualized a literature review produced by the scientific team. This allowed the scientists to consider aspects related to the patient experience and to integrate the feedback from patients into HTA recommendations. The most important factor contributing to a positive PI experience was the structured methodology for selecting patient participants, and a key factor that inhibited the process was a lack of training in PI on the part of the scientific team.

CONCLUSIONS

This experience demonstrates that it is possible to co-construct recommendations, even for technically complex HTA subjects, through a more democratic process than usual which led to more patient-focused guidance.