Review of early assessment models of innovative medical technologies

Artificial intelligence in healthcare: why not apply the medico-legal method starting with the Collingridge dilemma?

Technology has greatly influenced and radically changed human life, from communication to creativity and from productivity to entertainment. The authors, starting from considerations concerning the implementation of new technologies with a strong impact on people's everyday lives, take up Collingridge's dilemma and relate it to the application of AI in healthcare. Collingridge's dilemma is an ethical and epistemological problem concerning the relationship between technology and society which involves two approaches. The proactive approach and socio-technological experimentation taken into account in the dilemma are discussed, the former taking health technology assessment (HTA) processes as a reference and the latter the AI studies conducted so far. As a possible prevention of the critical issues raised, the use of the medico-legal method is proposed, which classically lies between the prevention of possible adverse events and the reconstruction of how these occurred.The authors believe that this methodology, adopted as a European guideline in the medico-legal field for the assessment of medical liability, can be adapted to AI applied to the healthcare scenario and used for the assessment of liability issues. The topic deserves further investigation and will certainly be taken into consideration as a possible key to future scenarios.

Frameworks for Health Technology Assessment at an Early Stage of Product Development: A Review and Roadmap to Guide Applications

OBJECTIVES

Early health technology assessment (eHTA) can be used to evaluate and optimize a medical product's value proposition and to inform go/no-go decisions by using health economic modeling, literature scanning, and stakeholder preference studies at an early stage of development. eHTA frameworks offer high-level guidance on conducting this complex, iterative, and multidisciplinary process. The objective of this study was to review and summarize existing eHTA frameworks, understood as systematic approaches to guide early evidence generation and decision making.

METHODS

Using a rapid review methodology, we identified all relevant studies published in English, French, and Spanish from PubMed/MEDLINE and Embase until February 2022. We only included frameworks relevant to the preclinical and early clinical (phase I) stages of medical product development.

RESULTS

From 737 reviewed abstracts, 53 publications describing 46 frameworks were selected for inclusion and classified into categories based on their scope: (1) criteria frameworks, which provide an overview of eHTA; (2) process frameworks, which offer stepwise guidance for conducting eHTA, including preferred methods; and (3) methods frameworks, which provide detailed descriptions of specific eHTA methods. Most of the frameworks did not specify their target users or the specific stage of technology development.

CONCLUSIONS

Despite some variability and gaps found across existing frameworks, the structure provided by this review helps inform eHTA applications. Remaining challenges are the frameworks' limited accessibility to users without a background in health economics, poor distinctions being made among early lifecycle stages and technology types, and the inconsistent terminology used to describe eHTA in different contexts.

Headroom Analysis for Early Economic Evaluation: A Systematic Review

OBJECTIVES

The headroom analysis is an early economic evaluation that quantifies the highest price at which an intervention may still be cost effective. Currently, there is no comprehensive review on how it is applied. This study investigated the application of the headroom analysis, specifically (1) how the headroom analysis is framed (2) the analytical approach and sources of evidence used, and (3) how expert judgement is used and reported.

METHODS

A systematic search was conducted in PubMed, Embase, Web of Science, EconLit, and Google Scholar on 28 April 2022. Studies were eligible if they reported an application of the headroom analysis. Data were presented in tabular form and summarised descriptively.

RESULTS

We identified 42 relevant papers. The headroom analysis was applied to medicines (29%), diagnostic or screening tests (29%), procedures, programmes and systems (21%), medical devices (19%), and a combined test and device (2%). All studies used model-based analyses, with 40% using simple models and 60% using more comprehensive models. Thirty-three percent of the studies assumed perfect effectiveness of the health technology, while 67% adopted realistic assumptions. Ten percent of the studies calculated an effectiveness-seeking headroom instead of a cost-seeking headroom. Expert judgement was used in 71% of the studies; 23 studies (55%) used expert opinion, 6 studies (14%) used expert elicitation, and 1 study (2%) used both.

CONCLUSIONS

Because the application of the headroom analysis varies considerably, we recommend its appropriate use and clear reporting of analytical approaches, level of evidence available, and the use of expert judgement.

Methods for Early Assessment of the Societal Value of Health Technologies: A Scoping Review and Proposal for Classification

OBJECTIVES

Early assessments of health technologies help to better align and integrate their development and assessment. Such assessments can take many forms and serve different purposes, hampering users in their selection of the most appropriate method for a specific goal. The aim of this scoping review was to structure the large set of methods according to their specific goal.

METHODS

A scoping review was conducted using PubMed and reference lists of retrieved articles, to identify review studies with a methodological focus. From the included reviews, all individual methods were listed. Based on additional literature and examples, we extracted the specific goal of each method. All goals were clustered to derive a set of subclasses and methods were grouped into these subclasses.

RESULTS

Of the 404 screened, 5 reviews were included, and 1 was added when searching reference lists. The reviews described 56 methods, of which 43 (77%) were included and classified as methods to (1) explore the nature and magnitude of the problem, (2) estimate the nature and magnitude of the expected (societal) value, (3) identify conditions for the potential value to materialize, and (4) help develop and design the type of research that is needed.

CONCLUSIONS

The wide range of methods for exploring the societal value of health technologies at an early stage of development can be subdivided into a limited number of classes, distinguishing methods according to their specific objective. This facilitates selection of appropriate methods, depending on the specific needs and aims.

Responsible innovation in health and health system sustainability: Insights from health innovators' views and practices

Although healthcare managers make increasingly difficult decisions about health innovations, the way they may interact with innovators to foster health system sustainability remains underexplored. Drawing on the Responsible Innovation in Health (RIH) framework, this paper analyses interviews (n=37) with Canadian and Brazilian innovators to identify: how they operationalize inclusive design processes; what influences the responsiveness of their innovation to system-level challenges; and how they consider the level and intensity of care required by their innovation. Our qualitative findings indicate that innovators seek to: 1) engage stakeholders at an early ideation stage through context-specific methods combining both formal and informal strategies; 2) address specific system-level benefits but often struggle with the positioning of their solution within the health system; and 3) mitigate staff shortages in specialized care, increase general practitioners’ capacity or patients and informal caregivers’ autonomy. These findings provide empirical insights on how healthcare managers can promote and organize collaborative processes that harness innovation towards more sustainable health systems. By adopting a RIH-oriented managerial role, they can set in place more inclusive design processes, articulate key system-level challenges, and help innovators adjust the level and intensity of care required by their innovation.

Innovators' views on involving users and patients in surgical device development: a qualitative interview study

OBJECTIVES

Involving end-users and patients in the development of surgical devices, even when patients are not end-users, is deemed important in policy and in academia since it could improve strategic choices in research and development (R&D). Nonetheless, research into innovators' views on end-user and patient involvement is rare. This study explores what end-users and patients are being involved by innovators during development, what methods for involvement are being used and what topics are being discussed with these end-users and patients.

DESIGN

A qualitative study featuring semi-structured interviews with innovators of surgical devices. Interviews were recorded and a thematic analysis was performed on verbatim transcripts.

PARTICIPANTS

15 interviews were conducted with 19 innovators of 14 surgical devices.

SETTING

Innovation practices of surgical devices in the Netherlands and Belgium.

RESULTS

End-users were engaged in R&D with formal methods and in unsystematic ways. These users all work in the clinical domain, for example, as surgeons or nurses. The innovators engaged users to analyse problems for which a device could be a solution, define functionalities, make design choices, analyse usability, ensure safety and improve aesthetics. Patients were rarely involved. Innovators stated that patients are not considered to be end-users, that physicians can represent patient interests and that involving patients is unethical as false expectations could be raised.

CONCLUSION

Innovators involve end-users with methods and unsystematic ways in the development of surgical devices. Despite governmental calls for patient involvement in the development of medical devices and surgical devices, innovators do not generally involve patients.

On the integration of early health technology assessment in the innovation process: reflections from five stakeholders

Early health technology assessment (HTA), which includes all methods used to inform industry and other stakeholders about the potential value of new medical products in development, including methods to quantify and manage uncertainty, has seen many applications in recent years. However, it is still unclear how such early value assessments can be integrated into the technology innovation process. This commentary contributes to the discussion on the purposes early HTA can serve. Similarities and differences in the perspectives of five stakeholders (i.e., the hospital, the patient, the assessor, the medical device industry, and the policy maker) on the purpose, value, and potential challenges of early HTA are described. All five stakeholders agreed that integrating early HTA in the innovation process has the possibility to shape and refine an innovation, and inform research and development decisions. The early assessment, using a variety of methodologies, can provide insights that are relevant for all stakeholders but several challenges, for example, feasibility and responsibility, need to be addressed before early HTA can become standard practice. For early evaluations to be successful, all relevant stakeholders including patients need to be involved. Also, nimble, flexible assessment methods are needed that fit the dynamics of medical technology. Best practices should be shared to optimize both the innovation process and the methods to perform an early value assessment.

Transforming Disciplinary Traditions Comment on "Problems and Promises of Health Technologies: The Role of Early Health Economic Modeling"

Grutters et al show that economic assessments can inform the development of new health technologies at an early stage. This is an important contribution to health services and policy research, which implies a "shift away" from the more traditional forms of academic health economic modeling. Because transforming established disciplinary traditions is both valuable and demanding, we invite scholars to further the discussion on how the value of health innovations should be appraised in view of today’s societal challenges.

The Lean and Agile Multi-dimensional Process (LAMP) - a new framework for rapid and iterative evidence generation to support health-care technology design and development

Introduction: Health technology assessments (HTA) are tools for policymaking and resource allocation. Early HTAs are increasingly used in design and development of new technologies. Conducting early HTAs is challenging, due to a lack of evidence and significant uncertainties in the technology and the market. A multi-disciplinary approach is considered essential. However, an operational framework that can enable the integration of multi-dimensional evidence into commercialization remains lacking.Areas covered: We developed the Lean and Agile Multi-dimensional Process (LAMP), an early HTA framework, for embedding commercial decision-making in structured evidence generation activities, divided into phases. Diverse evidence in unmet needs, user acceptance, cost-effectiveness, and market competitiveness are being generated in increasing depth. This supports the emergence of design and value propositions that align technology capabilities and clinical and user needs.Expert opinion: We have been applying LAMP to working with medical device and diagnostic industry in the UK. The framework can be adapted to suit different technologies, decision needs, time scales, and resources. LAMP offers a practical solution to the multi-disciplinary approach. Methodologists drive the process by performing evidence generation and synthesis as and by enabling interactions between manufacturers, designers, clinicians, and other key stakeholders.

Early assessment of innovation in a healthcare setting

OBJECTIVES

Early assessment can assist in allocating resources for innovation effectively and produce the most beneficial technology for an institution. The aim of the present study was to identify methods and discuss the analytical approaches applied for the early assessment of innovation in a healthcare setting.

METHODS

Knowledge synthesis based on a structured search (using the MEDLINE, Embase, and Cochrane databases) and thematic analysis was conducted. An analytical framework based on the stage of innovation (developmental, introduction, or early diffusion) was applied to assess whether methods vary according to stage. Themes (type of innovation, study, analysis, study design, method, and main target audience) were then decided among the authors. Identified methods and analysis were discussed according to the innovation stage.

RESULTS

A total of 1,064 articles matched the search strategy. Overall, thirty-nine articles matched the inclusion criteria. The use of methods has a tendency to change according to the stage of innovation. Stakeholder analysis was a prominent method in the innovation stages and particularly in the developmental stage, as the introduction and early diffusion stage has more availability of data and may apply more complex methods. Barriers to the identified methods were also discussed as all of the innovation stages suffered from lack of data and substantial uncertainty.

CONCLUSIONS

Although this review has identified applicable approaches for early assessment in different innovation stages, research is required regarding the value of the available data and methods and tools to enhance interactions between different parties at different stages of innovation.

A qualitative exploration of early assessment of innovative medical technologies

Background

Hospitals increasingly make decisions about early development of and investment in innovative medical technologies (IMTs), but at present often without an early assessment of their potential to ensure selection of the most promising candidates for further development. This paper explores how early assessment is carried out in different health organisations and then discusses relevant learning points for hospitals.

Methods

A qualitative study design with a structured interview guide covering four themes was used. Content analyses of interview notes were performed covering four predetermined themes: context, basis for decision-making, process and structure, and perceptions. A fifth theme, handling cognitive bias, was identified during data analysis.

Results

A total of 11 organisations participated; eight from the private health industry and three public hospitals. The interviews identified four areas in which early assessment is performed in similar manner across the studied organisations and four areas where differences exist between public hospitals and private organisations. Public hospitals indicate a lower degree of formalised early assessment and less satisfaction with how early assessment is performed, compared to private organisations. Based on the above findings, two learning points may carry promise for hospitals. First, having dedicated prioritising committees for IMTs making stop/go decisions. This committee is separate from the IMT development processes and involved staff. Secondly, the committee should base decisions on a transparent early assessment decision-support tool, which include a broad set of domains, is iterative, describes uncertainty, and minimise cognitive biases.

Conclusions

Similarities and differences in the way early assessment is done in different health organisations were identified. These findings suggest promising learning points for the development of an early assessment model for hospitals.

Electronic supplementary material

The online version of this article (10.1186/s12913-018-3647-z) contains supplementary material, which is available to authorized users.