Outcome selection, measurement and reporting for new surgical procedures and devices: a systematic review of IDEAL/IDEAL-D studies to inform development of a core outcome set

Development of a core data set for describing, measuring and reporting the learning curve in studies of novel invasive procedures: study protocol

INTRODUCTION

The introduction of novel surgical techniques and procedures remains poorly regulated and standardised. Although the learning curve associated with invasive procedures is a critical part of innovation, it is currently inconsistently defined, measured and reported. This study aims to develop a core data set that can be applied in all studies describing or measuring the learning curve in novel invasive procedures.

METHODS

A core data set will be developed using methods adapted from the Core Outcome Measures in Effectiveness Trials initiative. The study will involve three phases: (1) Identification of a comprehensive list of data items through (a) an umbrella review of existing systematic reviews on the learning curve in surgery and (b) qualitative interviews with key stakeholders. (2) Key stakeholders (eg, clinical innovators, clinicians, patients, methodologists, statisticians, journal editors and governance representatives) will complete a Delphi survey to score the importance of each data item, generating a shortened list. (3) Consensus meeting(s) with stakeholders to discuss and agree on the final core data set.

ETHICS AND DISSEMINATION

The study is approved by an Institutional Ethics Committee at the University of Bristol (ref: 111362). Participants will complete written informed consent to participate. Dissemination strategies include scientific meeting presentations, peer-reviewed journal publications, patient engagement events, use of social media platforms, workshops and other events.

Examining the application of the IDEAL framework in the reporting and evaluation of innovative invasive procedures: secondary qualitative analysis of a systematic review

OBJECTIVES

The development of new surgical procedures is fundamental to advancing patient care. The Idea, Developments, Exploration, Assessment and Long-term (IDEAL) framework describes study designs for stages of innovation. It can be difficult to apply due to challenges in defining and identifying innovative procedures. This study examined how the IDEAL framework is operationalised in real-world settings; specifically, the types of innovations evaluated using the framework and how authors justify their choice of IDEAL study design.

DESIGN

Secondary qualitative analysis of a systematic review.

DATA SOURCES

Citation searches (Web of Science and Scopus) identified studies following the IDEAL framework and citing any of the ten key IDEAL/IDEAL_D papers.

ELIGIBILITY CRITERIA

Studies of invasive procedures/devices of any design citing any of the ten key IDEAL/IDEAL_D papers.

DATA EXTRACTION AND SYNTHESIS

All relevant text was extracted. Three frameworks were developed, namely: (1) type of innovation under evaluation; (2) terminology used to describe stage of innovation and (3) reported rationale for IDEAL stage.

RESULTS

48 articles were included. 19/48 described entirely new procedures, including those used for the first time in a different clinical context (n=15/48), reported as IDEAL stage 2a (n=8, 53%). Terminology describing stage of innovation was varied, inconsistent and ambiguous and was not defined. Authors justified their choice of IDEAL study design based on limitations in published evidence (n=36) and unknown feasibility and safety (n=32) outcomes.

CONCLUSION

Identifying stage of innovation is crucial to inform appropriate study design and governance decisions. Authors' rationale for choice of IDEAL stage related to the existing evidence base or lack of sufficient outcome data for procedures. Stage of innovation was poorly defined with inconsistent descriptions. Further work is needed to develop methods to identify innovation to inform practical application of the IDEAL framework. Defining the concept of innovation in terms of uncertainty, risk and degree of evidence may help to inform decision-making.

Design and Conduct of Randomized Clinical Trials Evaluating Surgical Innovations in Ophthalmology: A Systematic Review

PURPOSE

Surgical innovations are necessary to improve patient care. After an initial exploratory phase, novel surgical technique should be compared with alternative options or standard care in randomized controlled trials (RCTs). However, surgical RCTs have unique methodological challenges. Our study sought to investigate key aspects of the design, conduct, and reporting of RCTs of novel surgeries.

DESIGN

Systematic review.

METHODS

The protocol was prospectively registered in PROSPERO (CRD42021253297). RCTs evaluating novel surgeries for cataract, vitreoretinal, glaucoma, and corneal diseases were included. Medline, EMBASE, Cochrane Library, and Clinicaltrials.gov were searched. The search period was January 1, 2016, to June 16, 2021.

RESULTS

A total of 52 ophthalmic surgery RCTs were identified in the fields of glaucoma (n = 12), vitreoretinal surgery (n = 5), cataract (n = 19), and cornea (n = 16). A description defining the surgeon's experience or level of expertise was reported in 30 RCTs (57%) and was presented in both control and intervention groups in 11 (21%). Specification of the number of cases performed in the particular surgical innovation being assessed prior to the trial was reported in 10 RCTs (19%) and an evaluation of quality of the surgical intervention in 7 (13%). Prospective trial registration was recorded in 12 RCTs (23%) and retrospective registration in 13 (25%); and there was no registration record in the remaining 28 (53%) studies.

CONCLUSIONS

Important aspects of the study design such as the surgical learning curve, surgeon's previous experience, quality assurance, and trial registration details were often missing in novel ophthalmic surgical procedures. The Idea, Development, Exploration, Assessment, Long-term follow-up (IDEAL) framework aims to improve the quality of study design.

Development of a conceptual framework for reporting modifications in surgical innovation: scoping review

BACKGROUND

Innovative surgical procedures and devices are often modified throughout their development and introduction into clinical practice. A systematic approach to reporting modifications may support shared learning and foster safe and transparent innovation. Definitions of 'modifications', and how they are conceptualized and classified so they can be reported and shared effectively, however, are lacking. This study aimed to explore and summarize existing definitions, perceptions, classifications and views on modification reporting to develop a conceptual framework for understanding and reporting modifications.

METHODS

A scoping review was conducted in accordance with PRISMA Extension for Scoping Reviews (PRISMA-ScR) guidelines. Targeted searches and two database searches were performed to identify relevant opinion pieces and review articles. Included were articles relating to modifications to surgical procedures/devices. Data regarding definitions, perceptions and classifications of modifications, and views on modification reporting were extracted verbatim. Thematic analysis was undertaken to identify themes, which informed development of the conceptual framework.

RESULTS

Forty-nine articles were included. Eight articles included systems for classifying modifications, but no articles reported an explicit definition of modifications. Some 13 themes relating to perception of modifications were identified. The derived conceptual framework comprises three overarching components: baseline data about modifications, details about modifications and impact/consequences of modifications.

CONCLUSION

A conceptual framework for understanding and reporting modifications that occur during surgical innovation has been developed. This is a first necessary step to support consistent and transparent reporting of modifications, to facilitate shared learning and incremental innovation of surgical procedures/devices. Testing and operationalization is now needed to realize the value of this framework.

A Core Outcome Set for Seamless, Standardized Evaluation of Innovative Surgical Procedures and Devices (COHESIVE): A Patient and Professional Stakeholder Consensus Study

OBJECTIVE

To develop a COS, an agreed minimum set of outcomes to measure and report in all studies evaluating the introduction and evaluation of novel surgical techniques.

SUMMARY OF BACKGROUND DATA

Agreement on the key outcomes to measure and report for safe and efficient surgical innovation is lacking, hindering transparency and risking patient harm.

METHODS

(I) Generation of a list of outcome domains from published innovation-specific literature, policy/regulatory body documents, and surgeon interviews; (II) Prioritization of identified outcome domains using an international, multi-stakeholder Delphi survey; (III) Consensus meeting to agree the final COS. Participants were international stakeholders, including patients/public, surgeons, device manufacturers, regulators, trialists, methodologists, and journal editors.

RESULTS

A total of 7972 verbatim outcomes were identified, categorized into 32 domains, and formatted into survey items/questions. Four hundred ten international participants (220 professionals, 190 patients/public) completed at least one round 1 survey item, of which 153 (69.5%) professionals and 116 (61.1%) patients completed at least one round 2 item. Twelve outcomes were scored "consensus in" ("very important" by ≥70% of patients and professionals) and 20 "no consensus." A consensus meeting, involvingcontext: modifications, unexpected disadvantages, device problems, technical procedure completion success, patients' experience relating to the procedure being innovative, surgeons'/operators' experience. Other domains relate to intended benefits, whether the overall desired effect was achieved and expected disadvantages.

CONCLUSIONS

The COS is recommended for use in all studies before definitive randomized controlled trial evaluation to promote safe, transparent, and efficient surgical innovation.

A standardized measurement instrument was recommended for evaluating operator experience in complex healthcare interventions

OBJECTIVE

During development of complex surgical innovations, modifications occur to optimize safety and efficacy. Operators' experiences (how professionals feel undertaking the innovation) drive this process but comprehensive overviews of measures of this concept are lacking. This study identified and appraised measures to assess operators' experience of surgical innovation.

STUDY DESIGN AND SETTING

There were three phases: (1) Literature reviews identified measures of operators' experience and concepts measured were extracted and grouped into domains. (2) Quality appraisal was conducted to assess content validity of identified instruments and was supported by COnsensus-based Standards for the selection of health Measurement Instruments methodology. Self-reported measurement instruments that had underdone formal development were eligible. Content validity was assessed using COnsensus-based Standards for the selection of health Measurement Instruments criteria for good content validity (rated sufficient/insufficient/indeterminate/inconsistent), informed by standards for measurement development and domains identified in phase 1. (3) Instruments determined suitable and of sufficient quality underwent supplemental appraisal in interviews with international multidisciplinary professionals and a focus group.

RESULTS

Literature reviews identified 16 measurement instruments from 243 studies. Most assessed 'psychological' experiences and 'usability'. No instrument was specifically validated for innovative surgery. Three instruments were rated 'sufficient' (Surgery Task Load Index [SURG-TLX]) or 'indeterminate' (Spielberger State-Trait Anxiety Inventory, Imperial Stress Assessment Tool). Twenty professionals were interviewed (seven female; 15 specialties; six countries) and the focus group included 10 participants (four professionals, six researchers). The SURG-TLX was considered the most relevant, comprehensive, and comprehensible instrument.

CONCLUSION

The SURG-TLX is preliminarily recommended to measure operators' experiences of innovation. Further work exploring its role and impact on surgical innovation is required.

Innovation in Neurosurgery: Lessons Learned, Obstacles, and Potential Funding Sources

Innovation is central to neurosurgery and has dramatically increased over the last twenty years. Although the specialty innovates as a whole, only 3-4.7% of practicing neurosurgeons hold patents. Various roadblocks to innovation impede this process such as lack of understanding, increasing regulatory complexity, and lack of funding. Newly emerging technologies allow us to understand how to innovate and how to learn from other medical specialties. By further understanding the process of innovation, and the funding that supports it, Neurosurgery can continue to hold innovation as one of its's central tenets.

Shared Learning Utilizing Digital Methods in Surgery to Enhance Transparency in Surgical Innovation: Protocol for a Scoping Review

BACKGROUND

Surgical innovation can lead to important improvements in patient outcomes. Currently, information and knowledge about novel procedures and devices are disseminated informally and in an unstandardized way (eg, through social media). This can lead to ineffective and inefficient knowledge sharing among surgeons, which can result in the harmful repetition of mistakes and delay in the uptake of promising innovation. Improvements are needed in the way that learning in surgical innovation is shared through the development of novel, real-time methods, informed by a contemporary and comprehensive investigation of existing methods.

OBJECTIVE

The aim of this scoping review is to explore the application of existing digital methods for training/education and feedback to surgeons in the context of performing invasive surgical procedures. This work will (1) summarize existing methods for shared learning in surgery and how they are characterized and operationalized, (2) examine the impact of their application, and (3) explore their benefits and barriers to implementation. The findings of this scoping review will inform the development of novel, real-time methods to optimize shared learning in surgical innovation.

METHODS

This study will adhere to the recommended guidelines for conducting scoping reviews. A total of 6 different searches will be conducted within multiple sources (2 electronic databases, journals, social media, gray literature, commercial websites, and snowball searches) to comprehensively identify relevant articles and data. Searches will be limited to articles published in the English language within the last 5 years. Wherever possible, a 2-stage study selection process will be followed whereby the eligibility of articles will be assessed through the title, abstract, and full-text screening independently by 2 reviewers. Inclusion criteria will be articles providing data on (1) fully qualified theater staff involved in performing invasive procedures, (2) one or more methods for shared learning (ie, digital means for training/education and feedback), and (3) qualitative or quantitative evaluations of this method. Data will be extracted (10% double data extraction by an independent reviewer) into a piloted proforma and analyzed using descriptive statistics, narrative summaries, and principles of thematic analysis.

RESULTS

The study commenced in October 2021 and is planned to be completed in 2023. To date, systematic searches were applied to 2 electronic databases (MEDLINE and Web of Science) and returned a total of 10,093 records. The results of this scoping review will be published as open access in a peer-reviewed journal.

CONCLUSIONS

This scoping review of methods for shared learning in surgery is, to our knowledge, the most comprehensive and up-to-date investigation that maps current information on this topic. Ultimately, efficient and effective sharing of information and knowledge of novel procedures and devices has the potential to optimize the evaluation of early-phase surgical research and reduce harmful innovation.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/37544.

Identification of outcomes to inform the development of a core outcome set for surgical innovation: a targeted review of case studies of novel surgical devices

OBJECTIVE

Outcome selection and reporting in studies of novel surgical procedures and devices lacks standardisation, hindering safe and effective evaluation. A core outcome set (COS) to measure and report in all studies of surgical innovation is needed. We explored outcomes in a specific sample of innovative surgical device case studies to identify outcome domains specifically relevant to innovation to inform the development of a COS.

DESIGN

A targeted review of 11 purposive selected case studies of innovative surgical devices.

METHODS

Electronic database searches in PubMed (July 2018) identified publications reporting the introduction and evaluation of each device. Outcomes were extracted and categorised into domains until no new domains were conceptualised. Outcomes specifically relevant to evaluating innovation were further scrutinised.

RESULTS

112 relevant publications were identified, and 5926 outcomes extracted. Heterogeneity in study type, outcome selection and reporting was observed across surgical devices. Categorisation of outcomes was performed for 2689 (45.4%) outcomes into five broad outcome domains. Outcomes considered key to the evaluation of innovation (n=66; 2.5%) were further categorised as surgeon/operator experience (n=40; 1.5%), unanticipated events (n=15, 0.6%) and modifications (n=11; 0.4%).

CONCLUSION

Outcome domains unique to evaluating innovative surgical devices have been identified. Findings have been combined with multiple other data sources relevant to the evaluation of surgical innovation to inform the development of a COS to measure and report in all studies evaluating novel surgical procedures/devices.

Surgical innovation in LMICs-The perspective from India

Surgical Innovations are central to surgical progress, and have led to exponential growth in various fields of Surgery. Surgical Innovations in Lower and Middle Income Countries are the result of creativity of frontline health workers in search of simple, safe and ethical solutions for their unique challenges. The key lies in: 'simplifying the idea/technique/device' to find patients' needs-driven low-cost innovative surgical solutions; which can be used on a wider scale to achieve health equity for underserved populations. Local surgeons understand the difficulties and nuances of various problems and can provide local-evidence-based customized solutions for their patients' health problems. We developed a Surgical Innovation Ecosystem allowing us to see difficulties as opportunities, learn from everyone and conduct research on what is 'important' rather than what is 'interesting'. Barriers to Surgical Innovations in Lower and Middle Income Countries are well known; however, a roadmap to overcome these barriers is now available. The right balance has to be found between encouraging creativity and innovation while maintaining ethical awareness and responsibility to patients. Introduction and adoption of Surgical Innovations are governed by evidence-based principles and have to undergo a rigorous and scientific evaluation. Science of Surgical Innovations has finally come of age and is getting its due recognition and the pioneering innovators are receiving the much needed appreciation and support.

Using qualitative research methods to understand how surgical procedures and devices are introduced into NHS hospitals: the Lotus study protocol

INTRODUCTION

The development of innovative invasive procedures and devices are essential to improving outcomes in healthcare. However, how these are introduced into practice has not been studied in detail. The Lotus study will follow a wide range of 'case studies' of new procedures and/or devices being introduced into NHS trusts to explore what information is communicated to patients, how procedures are modified over time and how outcomes are selected and reported.

METHODS AND ANALYSIS

This qualitative study will use ethnographic approaches to investigate how new invasive procedures and/or devices are introduced. Consultations in which the innovation is discussed will be audio-recorded to understand information provision practice. To understand if and how procedures evolve, they will be video recorded and non-participant observations will be conducted. Post-operative interviews will be conducted with the innovating team and patients who are eligible for the intervention. Audio-recordings will be audio-recorded, transcribed verbatim and analysed thematically using constant comparison techniques. Video-recordings will be reviewed to deconstruct procedures into key components and document how the procedure evolves. Comparisons will be made between the different data sources.

ETHICS AND DISSEMINATION

The study protocol has Health Research Authority (HRA) and Health and Care Research Wales approval (Ref 18/SW/0277). Results will be disseminated at appropriate conferences and will be published in peer-reviewed journals. The findings of this study will provide a better understanding of how innovative invasive procedures and/or devices are introduced into practice.

Reporting Modifications in Surgical Innovation: A Systematic Scoping Review Protocol

INTRODUCTION

Innovation in surgery drives improvements to patient care. New surgical procedures and devices typically undergo a series of modifications as they are developed and refined during their introduction into clinical practice. These changes should ideally be reported and shared between surgeon-innovators to promote efficient, safe and transparent innovation. Currently, agreement on how modifications should be defined, conceptualised and classified, so they can be reported and shared efficiently and transparently, is lacking. The aim of this review is to examine and summarise existing literature on definitions, perceptions and classifications of modifications to surgical procedures/devices, including views on how to measure and report them. The findings will inform future work to standardise reporting and sharing of modifications in surgical innovation.

MATERIALS AND METHODS

A systematic scoping review will be conducted adhering to PRISMA-ScR guidelines. Included articles will focus on review articles and opinion pieces relevant to modifications to new surgical procedures or devices introduced to clinical practice. Methods to identify relevant literature will include systematic searches in MEDLINE (Ovid version), targeted internet searches (Google Scholar) and snowball searches. A two-stage screening process (titles/abstracts/keywords and full-texts) will use specified exclusion/inclusion criteria to identify eligible articles. Data on how modifications are i) defined, ii) perceived, and iii) classified, and iv) views on how modifications should be measured and reported, will be extracted verbatim. Inductive thematic analysis will be applied to extracted data where appropriate. Results will be presented as a narrative summary including descriptive characteristics of included articles. Findings will inform a preliminary conceptual framework to facilitate the systematic reporting and sharing of modifications to novel procedures and devices.

HIGHLIGHTS

This work will generate an in-depth understanding of how modifications are currently defined, perceived and classified, and views on how they may be reported, in the context of surgical innovation.Rigorous and comprehensive search methods will be applied to identify a wide range of diverse data sources for inclusion in the review.A summary of existing relevant literature on modifications is a necessary step to inform development of a framework for transparent, real-time reporting and sharing of modifications in future studies of innovative invasive procedures/devices.