Novel ways to explore surgical interventions in randomised controlled trials: applying case study methodology in the operating theatre

Surgical trial design for incorporating the effects of learning: what is the current methodological guidance, and is it sufficient?

BACKGROUND

Surgical interventions are complex. Key elements of this complexity are the surgeon and their learning curve. They pose methodological challenges in the design, analysis and interpretation of surgical RCTs. We identify, summarise, and critically examine current guidance about how to incorporate learning curves in the design and analysis of RCTs in surgery.

EXAMINING CURRENT GUIDANCE

Current guidance presumes that randomisation must be between levels of just one treatment component, and that the evaluation of comparative effectiveness will be made via the average treatment effect (ATE). It considers how learning effects affect the ATE, and suggests solutions which seek to define the target population such that the ATE is a meaningful quantity to guide practice. We argue that these are solutions to a flawed formulation of the problem, and are inadequate for policymaking in this setting.

REFORMULATING THE PROBLEM

The premise that surgical RCTs are limited to single-component comparisons, evaluated via the ATE, has skewed the methodological discussion. Forcing a multi-component intervention, such as surgery, into the framework of the conventional RCT design ignores its factorial nature. We briefly discuss the multiphase optimisation strategy (MOST), which for a Stage 3 trial would endorse a factorial design. This would provide a wealth of information to inform nuanced policy but would likely be infeasible in this setting. We discuss in more depth the benefits of targeting the ATE conditional on operating surgeon experience (CATE). The value of estimating the CATE for exploring learning effects has been previously recognised, but with discussion limited to analysis methods only. The robustness and precision of such analyses can be ensured via the trial design, and we argue that trial designs targeting CATE represent a clear gap in current guidance.

CONCLUSION

Trial designs that facilitate robust, precise estimation of the CATE would allow for more nuanced policymaking, leading to patient benefit. No such designs are currently forthcoming. Further research in trial design to facilitate the estimation of the CATE is needed.

Association between Surgical Technical Skills and Clinical Outcomes: A Systematic Literature Review and Meta-Analysis

Background

A systematic literature review and meta-analysis was conducted to assess the association between intraoperative surgical skill and clinical outcomes.

Methods

Peer-reviewed, original research articles published through August 31, 2021 were identified from PubMed and Embase. From the 1,513 potential articles, seven met eligibility requirements, reporting on 151 surgeons and 17,932 procedures. All included retrospective assessment of operative videos. Associations between surgical skill and outcomes were assessed by pooling odds ratios (OR) using random-effects models with the inverse variance method. Eligible studies included pancreaticoduodenectomy, gastric bypass, laparoscopic gastrectomy, prostatectomy, colorectal, and hemicolectomy procedures.

Results

Meta-analytic pooling identified significant associations between the highest vs. lowest quartile of surgical skill and reoperation (OR: 0.44; 95% confidence interval [CI]: 0.23, 0.83), hemorrhage (OR: 0.66; 95% CI, 0.65, 0.68), obstruction (OR: 0.33; 95% CI, 0.30, 0.35), and any medical complication (OR: 0.23, 95% CI, 0.19, 0.27). Nonsignificant inverse associations were noted between skill and readmission, emergency department visit, mortality, leak, infection, venous thromboembolism, and cardiac and pulmonary complications.

Conclusions

Overall, surgeon technical skill appears to predict clinical outcomes. However, there are surprisingly few articles that evaluate this association. The authors recommend a thoughtful approach for the development of a comprehensive surgical quality infrastructure that could significantly reduce the challenges identified by this study.

What should be included in case report forms? Development and application of novel methods to inform surgical study design: a mixed methods case study in parastomal hernia prevention

OBJECTIVES

To describe the development and application of methods to optimise the design of case report forms (CRFs) for clinical studies evaluating surgical procedures, illustrated with an example of abdominal stoma formation.

DESIGN

(1) Literature reviews, to identify reported variations in surgical components of stoma formation, were supplemented by (2) intraoperative qualitative research (observations, videos and interviews), to identify unreported variations used in practice to generate (3) a 'long list' of items, which were rationalised using (4) consensus methods, providing a pragmatic list of CRF items to be captured in the Cohort study to Investigate the Prevention of parastomal HERnias (CIPHER) study.

SETTING

Two secondary care surgical centres in England.

PARTICIPANTS

Patients undergoing stoma formation, surgeons undertaking stoma formation and stoma nurses.

OUTCOME MEASURES

Successful identification of key CRF items to be captured in the CIPHER study.

RESULTS

59 data items relating to stoma formation were identified and categorised within six themes: (1) surgical approach to stoma formation; (2) trephine formation; (3) reinforcing the stoma trephine with mesh; (4) use of the stoma as a specimen extraction site; (5) closure of other wounds during the procedure; and (6) spouting the stoma.

CONCLUSIONS

This study used multimodal data collection to understand and capture the technical variations in stoma formation and design bespoke CRFs for a multicentre cohort study. The CIPHER study will use the CRFs to examine associations between the technical variations in stoma formation and risks of developing a parastomal hernia.

TRIAL REGISTRATION NUMBER

ISRCTN17573805.

Novel device for assisted vaginal birth: using integrated qualitative case study methodology to optimise Odon Device use within a feasibility study in a maternity unit in the Southwest of England

OBJECTIVE

When novel devices are used 'in human' for the first time, their optimal use is uncertain because clinicians only have experience from preclinical studies. This study aimed to investigate factors that might optimise use of the Odon Device for assisted vaginal birth.

DESIGN

We undertook qualitative case studies within the ASSIST Study, a feasibility study of the Odon Device. Each 'case' was defined as one use of the device and included at least one of the following: observation of the attempted assisted birth, and an interview with the obstetrician, midwife or woman. Data collection and thematic analysis ran iteratively and in parallel.

SETTING

Tertiary referral National Health Service maternity unit in the Southwest of England.

PARTICIPANTS

Women requiring a clinically indicated assisted vaginal birth.

INTERVENTION

The Odon Device, an innovative device for assisted vaginal birth.

PRIMARY AND SECONDARY OUTCOME MEASURES

Determining the optimal device technique, device design and defining clinical parameters for use.

RESULTS

Thirty-nine cases involving an attempted Odon-assisted birth were included in this study, of which 19 resulted in a successful birth with the device. Factors that improved use included optimisation of device technique, device design and clinical parameters for use. Technique adaptations included: applying the device during, rather than between, contractions; having a flexible approach to the application angle; and deflating the air cuff sooner than originally proposed. Three design modifications were proposed involving the deflation button and sleeve. Although use of the device was found to be appropriate in all fetal positions, it was considered contraindicated when the fetal station was at the ischial spines.

CONCLUSIONS

Case study methodology facilitated the acquisition of rapid insights into device function in clinical practice, providing key insights regarding use, design and key clinical parameters for success. This methodology should be considered whenever innovative devices are introduced into clinical practice.

TRIAL REGISTRATION NUMBER

ISRCTN10203171.

Women's experiences of the Odon Device to assist vaginal birth and participation in intrapartum research: a qualitative study in a maternity unit in the Southwest of England

OBJECTIVE

To investigate women's experiences of having a birth assisted by the Odon Device (an innovative device for assisted vaginal birth) and participation in intrapartum research.

DESIGN

Qualitative semistructured interviews and observations undertaken in the context of case study work embedded in the ASSIST feasibility study.

SETTING

A tertiary referral National Health Service (NHS) maternity unit in the Southwest of England, between 8 October 2018 and 26 January 2019.

PARTICIPANTS

Eight women, four operators and 11 midwives participated with eight observations of the assisted vaginal birth, eight interviews with women in the postnatal period, 39 interviews/reflections with operators and 19 interviews with midwives. Women in the case study research were recruited from participants in the main ASSIST Study.

INTERVENTION

The Odon Device, an innovative device for assisted vaginal birth.

RESULTS

Thirty-nine case studies were undertaken. Triangulation of data sources (participant observation, interviews with women, operators and midwives) enabled the exploration of women's experiences of the Odon Device and recruitment in the intrapartum trial. Experiences were overwhelmingly positive. Women were motivated to take part by a wish for a kinder birth, and because they perceived both the recruitment and research processes (including observation) to be highly acceptable, regardless of whether the Odon-assisted birth was successful or not.

CONCLUSIONS

Interviews and observations from multiple stakeholders enabled insight into women's experiences of an innovative device for assisted vaginal birth. Applying these qualitative methods more broadly may illuminate perspectives of key stakeholders in future intrapartum intervention research and beyond.

TRIAL REGISTRATION NUMBER

ISRCTN10203171; ASSIST Study registration; https://doi.org/10.1186/ISRCTN10203171.

Using qualitative research methods to understand how surgical procedures and devices are introduced into NHS hospitals: the Lotus study protocol

INTRODUCTION

The development of innovative invasive procedures and devices are essential to improving outcomes in healthcare. However, how these are introduced into practice has not been studied in detail. The Lotus study will follow a wide range of 'case studies' of new procedures and/or devices being introduced into NHS trusts to explore what information is communicated to patients, how procedures are modified over time and how outcomes are selected and reported.

METHODS AND ANALYSIS

This qualitative study will use ethnographic approaches to investigate how new invasive procedures and/or devices are introduced. Consultations in which the innovation is discussed will be audio-recorded to understand information provision practice. To understand if and how procedures evolve, they will be video recorded and non-participant observations will be conducted. Post-operative interviews will be conducted with the innovating team and patients who are eligible for the intervention. Audio-recordings will be audio-recorded, transcribed verbatim and analysed thematically using constant comparison techniques. Video-recordings will be reviewed to deconstruct procedures into key components and document how the procedure evolves. Comparisons will be made between the different data sources.

ETHICS AND DISSEMINATION

The study protocol has Health Research Authority (HRA) and Health and Care Research Wales approval (Ref 18/SW/0277). Results will be disseminated at appropriate conferences and will be published in peer-reviewed journals. The findings of this study will provide a better understanding of how innovative invasive procedures and/or devices are introduced into practice.

Evaluation of the completeness of intervention reporting in Cochrane surgical systematic reviews using the TIDieR-SR checklist: a cross-sectional study

INTRODUCTION

Complete reporting of systematic reviews of interventions is essential to the interpretation of research findings and the reproducibility of research results. The Template for Intervention Description and Replication (TIDieR) checklist-and the version specific to systematic reviews (TIDieR-SR)-was created to provide authors and researchers an evidence-based guide for reporting trial and systematic review interventions. In this study, we apply TIDieR-SR to Cochrane systematic reviews of surgical interventions.

METHODS

We searched the Cochrane Database for relevant systematic reviews. Two investigators applied inclusion/exclusion criteria to all titles/abstracts and full texts. These same investigators extracted all data in duplicate while masked to the other's data. The primary outcome was adherence to TIDieR-SR items.

RESULTS

Two hundred and thirty-eight systematic reviews were included. Overall, included SRs adhered to a median of 6 (IQR 5-7) out of eight TIDieR-SR items. The item with the lowest adherence was item 7 (share intervention materials, 1/238 (0.4%).

DISCUSSION

Our results are encouraging, but the generalisability of our findings is compromised by the inclusion of only Cochrane systematic reviews. Future reporting of intervention materials is likely to improve the application of effective surgical interventions in the clinical practice.

Protocol for an open label: phase I trial within a cohort of foetal cell transplants in people with Huntington's disease

Huntington's disease is a progressive neurodegenerative disorder characterized by motor, cognitive and psychiatric symptoms. Currently, no disease-modifying therapies are available to slow or halt disease progression. Huntington's disease is characterized by relatively focal and specific loss of striatal medium spiny neurons, which makes it suitable for cell-replacement therapy, a process involving the transplantation of donor cells to replace those lost due to disease. TRIal DEsigns for delivery of Novel Therapies in neurodegeneration is a phase I Trial Within a Cohort designed to assess safety and feasibility of transplanting human foetal striatal cells into the striatum of people with Huntington's disease. A minimum of 18 participants will be enrolled in the study cohort, and up to five eligible participants will be randomly selected to undergo transplantation of 12-22 million foetal cells in a dose escalation paradigm. Independent reviewers will assess safety outcomes (lack of significant infection, bleeding or new neurological deficit) 4 weeks after surgery, and ongoing safety will be established before conducting each subsequent surgery. All participants will undergo detailed clinical and functional assessment at baseline (6 and 12 months). Surgery will be performed 1 month after baseline, and transplant participants will undergo regular clinical follow-up for at least 12 months. Evaluation of trial processes will also be undertaken. Transplant participants and their carers will be interviewed ∼1 month before and after surgery. Interviews will also be conducted with non-transplanted participants and healthcare staff delivering the intervention and involved in the clinical care of participants. Evaluation of clinical and functional efficacy outcomes and intervention costs will be carried out to explore plausible trial designs for subsequent randomized controlled trials aimed at evaluating efficacy and cost-effectiveness of cell-replacement therapy. TRIal DEsigns for delivery of Novel Therapies in neurodegeneration will enable the assessment of the safety, feasibility, acceptability and cost of foetal cell transplants in people with Huntington's disease. The data collected will inform trial designs for complex intra-cranial interventions in a range of neurodegenerative conditions and facilitate the development of stable surgical pipelines for delivery of future stem cell trials. Trial Registration: ISRCTN52651778.

Special issues raised by evolving areas of clinical research

Each study presents its own set of ethical considerations. Certain kinds of ethical issues are inherent in particular areas of clinical research, regardless of specific ethical questions associated with a specific study. In this chapter, some of the most common special areas of clinical research are presented, highlighting the ethical issues most frequently associated with each.

Protocol for developing quality assurance measures to use in surgical trials: an example from the ROMIO study

INTRODUCTION

Randomised controlled trials (RCTs) in surgery are frequently criticised because surgeon expertise and standards of surgery are not considered or accounted for during study design. This is particularly true in pragmatic trials (which typically involve multiple centres and surgeons and are based in 'real world' settings), compared with explanatory trials (which are smaller and more tightly controlled).

OBJECTIVE

This protocol describes a process to develop and test quality assurance (QA) measures for use within a predominantly pragmatic surgical RCT comparing minimally invasive and open techniques for oesophageal cancer (the NIHR ROMIO study). It builds on methods initiated in the ROMIO pilot RCT.

METHODS AND ANALYSIS

We have identified three distinct types of QA measure: (i) entry criteria for surgeons, through assessment of operative videos, (ii) standardisation of operative techniques (by establishing minimum key procedural phases) and (iii) monitoring of surgeons during the trial, using intraoperative photography to document key procedural phases and standardising the pathological assessment of specimens. The QA measures will be adapted from the pilot study and tested iteratively, and the video and photo assessment tools will be tested for reliability and validity.

ETHICS AND DISSEMINATION

Ethics approval was obtained (NRES Committee South West-Frenchay, 25 April 2016, ref: 16/SW/0098). Results of the QA development study will be submitted for publication in a peer-reviewed journal.

TRIAL REGISTRATION NUMBER

ISRCTN59036820, ISRCTN10386621.

Exploring qualitative methods reported in registered trials and their yields (EQUITY): systematic review

Background

The value of qualitative methods within trials is widely recognised, but their full potential is not being realised. There are also issues with the visibility, recognition and reporting of qualitative methods in trials. To identify potential improvements in qualitative research within trials, we need to study trials that have included qualitative methods. We aimed to explore the frequency of reporting qualitative methods in registered trials, the types of trials using qualitative methods and where in the world these trials were conducted.

Methods

We included registries if they were searchable using keywords and held summaries of trials rather than listing reports or publications. We searched the included registries from the first available record in 1999 to the end of 2016 for the term ‘qualitative’. We included trials only if we could confirm that they used qualitative methods through documented use of qualitative data collection and analysis in the registry summary. We analysed registered trials reporting the use of qualitative methods by: year registered, the country responsible for overseeing governance of the trial and the type of trial intervention (categorised as surgical, medical device, behavioural, drug or other).

Results

We included three registries: ClinicalTrials.gov, the International Standard Randomised Controlled Trial Number Registry (ISRCTN) and the World Health Organisation International Clinical Trials Registry Platform (WHO ICTRP). A total of 615,311 trials appear in these three registries from 1999 until the end of 2016. Numbers differed across registries with the WHO ICTRP the largest (366,753 trials), ClinicalTrials.gov the second largest (233,277) and ISRCTN the smallest (15,301). Of these registered trials, we confirmed that 1492 (0.24%) reported using qualitative methods. The ISRCTN contributed the highest percentage of trials reported as using qualitative methods (3.4%); in contrast, ClinicalTrials.gov reported 0.3% and WHO ICTRP reported 0.03%. The number and percentage of trials reported to use qualitative methods increased over time from 0 (0.0%) in 1999 to 285 (0.38%) in 2016. Trials reported as using qualitative methods originated from 52 countries across the world. Most were in Western higher-income countries: 38% in the United Kingdom and 28% in the United States. Most registered trials reported as using qualitative methods evaluated behavioural (39%) or other interventions with many fewer trials evaluating drugs (5%), medical devices (5%) or surgical interventions (4%).

Conclusion

The reported use of qualitative methods in registered trials has increased over time and worldwide. They are reportedly more frequent in high-income countries and in trials of behavioural and other interventions. Trialists and other stakeholders need to recognise the benefits of using qualitative methods in surgical, device and drug trials, and trials conducted in poorer countries. Moreover, they should seriously consider using qualitative methods in these trials.

Recommendations for Randomised Trials in Surgical Oncology

Trials of surgical procedures in the treatment of malignant disease face a unique set of challenges. This review aimed to describe recommendations for the design, delivery and reporting of randomised trials in surgical oncology. A literature search was carried out without date limits to identify articles related to trial methodology research in surgery and surgical oncology. A narrative review was framed around two open National Institute of Health Research portfolio trials in colon and rectal cancer: the STAR-TREC trial (ISRCTN14240288) and the ROCCS trial (ISRCTN46330337). Twelve specific challenges were highlighted: standardisation of technique; pilot and feasibility studies; balancing treatments; the recruitment pathway; outcome measures; patient and public representation; trainee-led networks; randomisation; novel techniques and training; learning curves; blinding; follow-up. Evidence-based recommendations were made for the future design and conduct of surgical oncology trials. Better understanding of the challenges facing trials in the surgical treatment of cancer will accelerate high-quality evaluation and rapid adoption of innovation for the benefit of patient care.

Delivering successful randomized controlled trials in surgery: Methods to optimize collaboration and study design

Randomized controlled trials in surgery are notoriously difficult to design and conduct due to numerous methodological and cultural challenges. Over the last 5 years, several UK-based surgical trial-related initiatives have been funded to address these issues. These include the development of Surgical Trials Centers and Surgical Specialty Leads (individual surgeons responsible for championing randomized controlled trials in their specialist fields), both funded by the Royal College of Surgeons of England; networks of research-active surgeons in training; and investment in methodological research relating to surgical randomized controlled trials (to address issues such as recruitment, blinding, and the selection and standardization of interventions). This article discusses these initiatives more in detail and provides exemplar cases to illustrate how the methodological challenges have been tackled. The initiatives have surpassed expectations, resulting in a renaissance in surgical research throughout the United Kingdom, such that the number of patients entering surgical randomized controlled trials has doubled.

Standardizing and monitoring the delivery of surgical interventions in randomized clinical trials

Background

The complexity of surgical interventions has major implications for the design of RCTs. Trials need to consider how and whether to standardize interventions so that, if successful, they can be implemented in practice. Although guidance exists for standardizing non‐pharmaceutical interventions in RCTs, their application to surgery is unclear. This study reports new methods for standardizing the delivery of surgical interventions in RCTs.

Methods

Descriptions of 160 surgical interventions in existing trial reports and protocols were identified. Initially, ten reports were scrutinized in detail using a modified framework approach for the analysis of qualitative data, which informed the development of a preliminary typology. The typology was amended with iterative sequential application to all interventions. Further testing was undertaken within ongoing multicentre RCTs.

Results

The typology has three parts. Initially, the overall technical purpose of the intervention is described (exploration, resection and/or reconstruction) in order to establish its constituent components and steps. This detailed description of the intervention is then used to establish whether and how each component and step should be standardized, and the standards documented within the trial protocol. Finally, the typology provides a framework for monitoring the agreed intervention standards during the RCT. Pilot testing within ongoing RCTs enabled standardization of the interventions to be agreed, and case report forms developed to capture deviations from these standards.

Conclusion

The typology provides a framework for use during trial design to standardize the delivery of surgical interventions and document these details within protocols. Application of this typology to future RCTs may clarify details of the interventions under evaluation and help successful interventions to be implemented.

Design a perfect study