Inherent problems with randomized clinical trials with observational/no treatment arms

Vascular Registries Contributing to VASCUNET Collaborative Abdominal Aortic Aneurysm Outcome Projects: A Scoping Review

OBJECTIVE

Vascular surgery registries report on procedures and outcomes to promote patient safety and drive quality improvement. International registries have contributed significantly to the VASCUNET collaborative abdominal aortic aneurysm (AAA) outcome projects. This scoping review aimed to outline the national registries in vascular surgery that currently participate in the VASCUNET collaborative AAA projects.

METHODS

A scoping review of all published VASCUNET AAA studies and validation reports between 1997 and 2024 was undertaken. A survey was conducted among representatives of the international vascular registries contributing to VASCUNET collaborative AAA projects.

RESULTS

Currently, vascular registries from 10 countries (Australia, Denmark, Finland, Hungary, Iceland, New Zealand, Norway, Sweden, Switzerland, and the UK) contribute to the current VASCUNET collaborative AAA project, of which eight have national coverage. In the past, three countries (Germany, Malta, and Italy) have participated in previous VASCUNET AAA projects, and a further three countries (Serbia, Greece, and Portugal) have planned participation in future projects. External validity is high for all current registries, with most reporting rates of > 90%. The majority have internal validation processes to assess data accuracy. VASCUNET mediated validation has also been performed by the consortium for five countries to date (Hungary, Sweden, Denmark, Malta, and Switzerland), for which a high degree of external and internal validity was identified. Most registries have established mechanisms for data linkage with national administrative datasets or insurance claims datasets and contribute to quality improvement through regular reporting to participating centres.

CONCLUSION

National vascular registries from nations participating in the VASCUNET collaborative AAA projects are largely comprehensive, with high case ascertainment rates and good quality data with internal quality assurance. This provides a template for new registries wishing to join the VASCUNET collaboration and a benchmark for future research.

Strategies to Improve Recruitment to a De-escalation Trial: A Mixed-Methods Study of the OPTIMA Prelim Trial in Early Breast Cancer

AIMS

De-escalation trials are challenging and sometimes may fail due to poor recruitment. The OPTIMA Prelim randomised controlled trial (ISRCTN42400492) randomised patients with early stage breast cancer to chemotherapy versus 'test-directed' chemotherapy, with a possible outcome of no chemotherapy, which could confer less treatment relative to routine practice. Despite encountering challenges, OPTIMA Prelim reached its recruitment target ahead of schedule. This study reports the root causes of recruitment challenges and the strategies used to successfully overcome them.

MATERIALS AND METHODS

A mixed-methods recruitment intervention (QuinteT Recruitment Intervention) was used to investigate the recruitment difficulties and feedback findings to inform interventions and optimise ongoing recruitment. Quantitative site-level recruitment data, audio-recorded recruitment appointments (n = 46), qualitative interviews (n = 22) with trialists/recruiting staff (oncologists/nurses) and patient-facing documentation were analysed using descriptive, thematic and conversation analyses. Findings were triangulated to inform a 'plan of action' to optimise recruitment.

RESULTS

Despite best intentions, oncologists' routine practices complicated recruitment. Discomfort about deviating from the usual practice of recommending chemotherapy according to tumour clinicopathological features meant that not all eligible patients were approached. Audio-recorded recruitment appointments revealed how routine practices undermined recruitment. A tendency to justify chemotherapy provision before presenting the randomised controlled trial and subtly indicating that chemotherapy would be more/less beneficial undermined equipoise and made it difficult for patients to engage with OPTIMA Prelim. To tackle these challenges, individual and group recruiter feedback focussed on communication issues and vignettes of eligible patients were discussed to address discomforts around approaching patients. 'Tips' documents concerning structuring discussions and conveying equipoise were disseminated across sites, together with revisions to the Patient Information Sheet.

CONCLUSIONS

This is the first study illuminating the tension between oncologists' routine practices and recruitment to de-escalation trials. Although time and resources are required, these challenges can be addressed through specific feedback and training as the trial is underway.

Why I use both prospective randomized trials and registry data when choosing the personalized treatment of an AAA patient

Randomized controlled trials (RCTs) have been the core of level 1 data in medical and surgical science for at least the last three decades. However, frequently patient selection is very narrow, anatomic criteria do not match real-world experience, and much of the work is done in selected academic centers. We use RCTs to help explain the rational for intervention and then rely on longitudinal registries and single center data to give the patients a real-world expectation concerning outcomes and complications in our hands.

[Identifying barriers to screening for abdominal aortic aneurysm in general practice: Qualitative study of 14 general practitioners in Paris]

INTRODUCTION

Abdominal aortic aneurysm (AAA) is a silent pathology with often fatal consequences in case of rupture. AAA screening, recommended in France and many other countries, has shown its effectiveness in reducing specific mortality. However, AAA screening rate remains insufficient.

OBJECTIVE

To identify barriers to AAA screening in general practice.

MATERIAL AND METHOD

Qualitative study carried out during 2016 among general practitioners based in Paris.

RESULTS

Fourteen physicians were included. Most of the barriers were related to the physician: unawareness about AAA and screening recommendations, considering AAA as a secondary question not discussed with the patient, abdominal aorta not included in cardiovascular assessment, no search for a familial history of AAA, AAA considered a question for the specialist, lack of time, lack of training, numerous screenings to propose, oversight. Some barriers are related to the patient: unawareness of the pathology and family history of AAA, refusal, questioning the pertinence of the doctor's comments, failure to respect the care pathway. Others are related to AAA: source of anxiety, low prevalence, rarity of complications. The remaining barriers are related to screening: cost-benefit and risk-benefit ratios, sonographer unavailability, constraint for the patient, overmedicalization.

CONCLUSION

Information and training of general practitioners about AAA must be strengthened in order to optimize AAA screening and reduce specific mortality.

Induced Bias Due to Crossover Within Randomized Controlled Trials in Surgical Oncology: A Meta-regression Analysis of Minimally Invasive versus Open Surgery for the Treatment of Gastrointestinal Cancer

BACKGROUND

Randomized controlled trials (RCTs) inform clinical practice and have provided the evidence base for introducing minimally invasive surgery (MIS) in surgical oncology. Crossover (unplanned intraoperative conversion of MIS to open surgery) may affect clinical outcomes and the effect size generated from RCTs with homogenization of randomized groups.

OBJECTIVES

Our aims were to identify modifiable factors associated with crossover and assess the impact of crossover on clinical endpoints.

METHODS

A systematic review was performed to identify all RCTs comparing MIS with open surgery for gastrointestinal cancer (1990-2017). Meta-regression analysis was performed to analyze factors associated with crossover and the influence of crossover on endpoints, including 30-day mortality, anastomotic leak rate, and early complications.

RESULTS

Forty RCTs were included, reporting on 11,625 patients from 320 centers. Crossover was shown to affect one in eight patients (mean 12.6%, range 0-45%) and increased with American Society of Anesthesiologists score (β = + 0.895; p = 0.050). Pretrial surgeon volume (β = - 2.344; p = 0.037), composite RCT quality score (β = - 7.594; p = 0.014), and site of tumor (β = - 12.031; p = 0.021, favoring lower over upper gastrointestinal tumors) showed an inverse relationship with crossover. Importantly, multivariate weighted linear regression revealed a statistically significant positive correlation between crossover and 30-day mortality (β = + 0.125; p = 0.033), anastomotic leak rate (β = + 0.550; p = 0.004), and early complications (β = + 1.255; p = 0.001), based on intention-to-treat analysis.

CONCLUSIONS

Crossover in trials was associated with an increase in 30-day mortality, anastomotic leak rate, and early complications within the MIS group based on intention-to-treat analysis, although our analysis did not assess causation. Credentialing surgeons by procedural volume and excluding high comorbidity patients from initial trials are important in minimizing crossover and optimizing RCT validity.

OPTIMA prelim: a randomised feasibility study of personalised care in the treatment of women with early breast cancer

BACKGROUND

There is uncertainty about the chemotherapy sensitivity of some oestrogen receptor (ER)-positive, human epidermal growth factor receptor 2 (HER2)-negative breast cancers. Multiparameter assays that measure the expression of several tumour genes simultaneously have been developed to guide the use of adjuvant chemotherapy for this breast cancer subtype. The assays provide prognostic information and have been claimed to predict chemotherapy sensitivity. There is a dearth of prospective validation studies. The Optimal Personalised Treatment of early breast cancer usIng Multiparameter Analysis preliminary study (OPTIMA prelim) is the feasibility phase of a randomised controlled trial (RCT) designed to validate the use of multiparameter assay directed chemotherapy decisions in the NHS.

OBJECTIVES

OPTIMA prelim was designed to establish the acceptability to patients and clinicians of randomisation to test-driven treatment assignment compared with usual care and to select an assay for study in the main RCT.

DESIGN

Partially blinded RCT with adaptive design.

SETTING

Thirty-five UK hospitals.

PARTICIPANTS

Patients aged ≥ 40 years with surgically treated ER-positive HER2-negative primary breast cancer and with 1-9 involved axillary nodes, or, if node negative, a tumour at least 30 mm in diameter.

INTERVENTIONS

Randomisation between two treatment options. Option 1 was standard care consisting of chemotherapy followed by endocrine therapy. In option 2, an Oncotype DX(®) test (Genomic Health Inc., Redwood City, CA, USA) performed on the resected tumour was used to assign patients either to standard care [if 'recurrence score' (RS) was > 25] or to endocrine therapy alone (if RS was ≤ 25). Patients allocated chemotherapy were blind to their randomisation.

MAIN OUTCOME MEASURES

The pre-specified success criteria were recruitment of 300 patients in no longer than 2 years and, for the final 150 patients, (1) an acceptance rate of at least 40%; (2) recruitment taking no longer than 6 months; and (3) chemotherapy starting within 6 weeks of consent in at least 85% of patients.

RESULTS

Between September 2012 and 3 June 2014, 350 patients consented to join OPTIMA prelim and 313 were randomised; the final 150 patients were recruited in 6 months, of whom 92% assigned chemotherapy started treatment within 6 weeks. The acceptance rate for the 750 patients invited to participate was 47%. Twelve out of the 325 patients with data (3.7%, 95% confidence interval 1.7% to 5.8%) were deemed ineligible on central review of receptor status. Interviews with researchers and recordings of potential participant consultations made as part of the integral qualitative recruitment study provided insights into recruitment barriers and led to interventions designed to improve recruitment. Patient information was changed as the result of feedback from three patient focus groups. Additional multiparameter analysis was performed on 302 tumour samples. Although Oncotype DX, MammaPrint(®)/BluePrint(®) (Agendia Inc., Irvine, CA, USA), Prosigna(®) (NanoString Technologies Inc., Seattle, WA, USA), IHC4, IHC4 automated quantitative immunofluorescence (AQUA(®)) [NexCourse BreastTM (Genoptix Inc. Carlsbad, CA, USA)] and MammaTyper(®) (BioNTech Diagnostics GmbH, Mainz, Germany) categorised comparable numbers of tumours into low- or high-risk groups and/or equivalent molecular subtypes, there was only moderate agreement between tests at an individual tumour level (kappa ranges 0.33-0.60 and 0.39-0.55 for tests providing risks and subtypes, respectively). Health economics modelling showed the value of information to the NHS from further research into multiparameter testing is high irrespective of the test evaluated. Prosigna is currently the highest priority for further study.

CONCLUSIONS

OPTIMA prelim has achieved its aims of demonstrating that a large UK clinical trial of multiparameter assay-based selection of chemotherapy in hormone-sensitive early breast cancer is feasible. The economic analysis shows that a trial would be economically worthwhile for the NHS. Based on the outcome of the OPTIMA prelim, a large-scale RCT to evaluate the clinical effectiveness and cost-effectiveness of multiparameter assay-directed chemotherapy decisions in hormone-sensitive HER2-negative early breast would be appropriate to take place in the NHS.

TRIAL REGISTRATION

Current Controlled Trials ISRCTN42400492.

FUNDING

This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 20, No. 10. See the NIHR Journals Library website for further project information. The Government of Ontario funded research at the Ontario Institute for Cancer Research. Robert C Stein received additional support from the NIHR University College London Hospitals Biomedical Research Centre.

Comparing alternative design options for chronic disease prevention interventions

BACKGROUND

While the randomized clinical trial is considered to provide the highest level of evidence in clinical medicine, its superiority to other study designs in the context of prevention studies is debated. The purpose of this review was (i) to gather evidence about challenges facing both randomized controlled trials and observational designs for the conduct of population-based chronic disease prevention interventions and (ii) to consider the suitability of recently proposed hybrid designs for population-based prevention intervention studies.

METHODS

Rapid review methods were employed for this study. Articles published within 2007-2012, were included if they: (i) discussed challenges or benefits related to any intervention study design, (ii) compared randomized controlled trials (RCT) and observational designs or (iii) introduced a new study design potentially applicable to population-based interventions. After initial screening, papers retained for inclusion were subjected to content analysis and synthesis.

RESULTS

A total of 35 included articles were reviewed and used for synthesis. Both RCTs and observational studies are subject to multiple challenges, the main being external and internal validity for RCTs and observational designs, respectively. Four new hybrid designs identified.

CONCLUSION

Although any high quality design can produce high level of evidence, multiple challenges with prevention intervention RCTs or observational studies identified. New hybrid designs that carry benefits of randomized and observational methods may be the road ahead for to assess the effects of population-based interventions.

Intended versus inferred care after PET performed for initial staging in the National Oncologic PET Registry

The National Oncologic PET Registry (NOPR) collected data on intended management before and after PET in cancer patients. We have previously reported that PET was associated with a change in intended management of about one third of patients and was consistent across cancer types. It is uncertain if intended management plans reflect the actual care these patients received. One approach to assess actual care received is using administrative claims to categorize the type and timing of clinical services.

METHODS

NOPR data from 2006 to 2008 were linked to Medicare claims for consenting patients aged 65 y or older undergoing initial-staging PET scanning for bladder, ovarian, pancreatic, small cell lung, or stomach cancers. We determined the 60-d agreement between claims-inferred care and NOPR treatment plans.

RESULTS

Patients (n = 4,661) were assessed, and 30%-52% had metastatic disease. Planned treatments were about two-thirds monotherapy, of which 46% was systemic therapy only, and one-third combinations. Claims paid by 60 d confirmed the NOPR plan of any systemic therapy, radiotherapy, or surgery in 79.3%, 64.7%, and 63.6%, respectively. Single-mode plans were much more often confirmed: systemic therapy in more than 85% of patients with ovarian, pancreatic, and small cell lung cancers and surgery in more than 73% of those with bladder, pancreatic, and stomach cancers. Intended combination treatments had claims for both in only 28% of patients receiving surgery-based combinations and in 55% receiving chemoradiotherapy. About 90% of patients with NOPR-planned systemic therapy had evaluation or management claims from a medical oncologist. An age of less than 75 y was associated more often with confirmation of chemotherapy, less often for radiotherapy but not with confirmation of surgery. Performance status or comorbidity did not explain confirmation rates within action categories, but confirmation rates were higher if the referrer specialized in the planned treatment.

CONCLUSION

Claims confirmations of NOPR intent for initial staging were widely variable but were higher than previously reported for restaging PET, suggesting that measuring change in intended management is a reasonable method for assessing the impact diagnostic tests have on actual care.

Influence and critique of the PIVOTAL and the EVAR 2 Trials

Management of a condition that has potentially life-threatening consequences may not lend itself effectively to the scrutiny of a randomized clinical trial when an observation or no treatment option is offered as part of the trial. This type of trial often experiences a significant rate of crossover of subjects from no treatment to treatment, and when results are analyzed on an intent-to-treat basis, they may fail to resolve the issue under study. These trials are frequently used as Level 1 medical evidence and the potential impact on clinical decision-making and reimbursement can be quite significant and long-lasting. The authors observed this phenomenon during participation in the Positive Impact of Endovascular Options for Treating Aneurysms Early (PIVOTAL) trial and have observed it in an analysis of the Endovascular Aneurysm Repair 2 (EVAR 2) trial. Possible solutions to mitigate the high crossover effect are offered for consideration. Some clinical conditions dealing with potentially life-threatening problems probably do not lend themselves to be studied in randomized prospective clinical trials containing an observation or no treatment arm.

Current Endovascular Management of Abdominal Aortic Aneurysm

Fatal rupture of abdominal aortic aneurysm (AAA) remains a feared complication. Development of vascular surgery techniques over 50 years ago has fulfilled the promise of preventing rupture, but the significant morbidity associated with open repair causes physicians and their older patients pause. With the advent of less invasive endovascular techniques and devices, patients now have another viable treatment option. We review some of the important trials as well as discuss developments in the continually evolving field of endovascular repair of AAAs.