Ethical Problems of Observational Studies and Big Data Compared to Randomized Trials

The temptation to use prospective observational studies (POS) instead of conducting difficult trials (RCTs) has always existed, but with the advent of powerful computers and large databases, it can become almost irresistible. We examine the potential consequences, were this to occur, by comparing two hypothetical studies of a new treatment: one RCT, and one POS. The POS inevitably submits more patients to inferior research methodology. In RCTs, patients are clearly informed of the research context, and 1:1 randomized allocation between experimental and validated treatment balances risks for each patient. In POS, for each patient, the risks of receiving inferior treatment are impossible to estimate. The research context and the uncertainty are down-played, and patients and clinicians are at risk of becoming passive research subjects in studies performed from an outsider's view, which potentially has extraneous objectives, and is conducted without their explicit, autonomous, and voluntary involvement and consent.

Endovascular Parent Vessel Occlusion Versus Flow Diversion in the Treatment of Large and Giant Aneurysms: A Randomized Comparison

OBJECTIVE

Parent vessel occlusion (PVO) is a time-honored treatment for unclippable or uncoilable intracranial aneurysms. Flow diversion (FD) is a recent endovascular alternative that can occlude the aneurysm and spare the parent blood vessel. Our aim was to compare outcomes of FD with endovascular PVO.

METHODS

This is a prespecified treatment subgroup analysis of the Flow diversion in Intracranial Aneurysms trial (FIAT). FIAT was an investigator-led parallel-group all-inclusive pragmatic randomized trial. For each patient, clinicians had to prespecify an alternative management option to FD before stratified randomization. We report all patients for whom PVO was selected as the best alternative treatment to FD. The primary outcome was a composite of core-lab determined angiographic occlusion or near-occlusion at 3-12 months combined with an independent clinical outcome (mRS<3). Primary analyses were intent-to-treat. There was no blinding.

RESULTS

There were 45 patients (16.2% of the 278 FIAT patients randomized between 2011 and 2020 in 3 centers): 22 were randomly allocated to FD and 23 to PVO. Aneurysms were mainly large or giant (mean 22 mm) anterior circulation (mainly carotid) aneurysms. A poor primary outcome was reached in 11/22 FD (50.0%) compared to 9/23 PVO patients (39.1%) (RR: 1.28, 95% CI [0.66-2.47]; P = 0.466). Morbidity (mRS >2) at 1 year occurred in 4/22 FD and 6/23 PVO patients. Angiographic results and serious adverse events were similar.

CONCLUSIONS

The comparison between PVO and FD was inconclusive. More randomized trials are needed to better determine the role of FD in large aneurysms eligible for PVO.

The Management of Persistent Distal Occlusions after Mechanical Thrombectomy and Thrombolysis: An Inter- and Intrarater Agreement Study

BACKGROUND AND PURPOSE

The best management of patients with persistent distal occlusion after mechanical thrombectomy with or without IV thrombolysis remains unknown. We sought to evaluate the variability and agreement in decision-making for persistent distal occlusions.

MATERIALS AND METHODS

A portfolio of 60 cases was sent to clinicians with varying backgrounds and experience. Responders were asked whether they considered conservative management or rescue therapy (stent retriever, aspiration, or intra-arterial thrombolytics) a treatment option as well as their willingness to enroll patients in a randomized trial. Agreement was assessed using κ statistics.

RESULTS

The electronic survey was answered by 31 physicians (8 vascular neurologists and 23 interventional neuroradiologists). Decisions for rescue therapies were more frequent (n = 1116/1860, 60%) than for conservative management (n = 744/1860, 40%; P < .001). Interrater agreement regarding the final management decision was "slight" (κ = 0.12; 95% CI, 0.09-0.14) and did not improve when subgroups of clinicians were studied according to background, experience, and specialty or when cases were grouped according to the level of occlusion. On delayed re-questioning, 23 of 29 respondents (79.3%) disagreed with themselves on at least 20% of cases. Respondents were willing to offer trial participation in 1295 of 1860 (69.6%) cases.

CONCLUSIONS

Individuals did not agree regarding the best management of patients with persistent distal occlusion after mechanical thrombectomy and IV thrombolysis. There is sufficient uncertainty to justify a dedicated randomized trial.

Nonaneurysmal perimesencephalic subarachnoid hemorrhage on noncontrast head CT: An accuracy, inter-rater, and intra-rater reliability study

BACKGROUND AND PURPOSE

To evaluate the reliability and accuracy of nonaneurysmal perimesencephalic subarachnoid hemorrhage (NAPSAH) on Noncontrast Head CT (NCCT) between numerous raters.

MATERIALS AND METHODS

45 NCCT of adult patients with SAH who also had a catheter angiography (CA) were independently evaluated by 48 diverse raters; 45 raters performed a second assessment one month later. For each case, raters were asked: 1) whether they judged the bleeding pattern to be perimesencephalic; 2) whether there was blood anterior to brainstem; 3) complete filling of the anterior interhemispheric fissure (AIF); 4) extension to the lateral part of the sylvian fissure (LSF); 5) frank intraventricular hemorrhage; 6) whether in the hypothetical presence of a negative CT angiogram they would still recommend CA. An automatic NAPSAH diagnosis was also generated by combining responses to questions 2-5. Reliability was estimated using Gwet's AC1 (κG), and the relationship between the NCCT diagnosis of NAPSAH and the recommendation to perform CA using Cramer's V test. Multi-rater accuracy of NCCT in predicting negative CA was explored.

RESULTS

Inter-rater reliability for the presence of NAPSAH was moderate (κG = 0.58; 95%CI: 0.47, 0.69), but improved to substantial when automatically generated (κG = 0.70; 95%CI: 0.59, 0.81). The most reliable criteria were the absence of AIF filling (κG = 0.79) and extension to LSF (κG = 0.79). Mean intra-rater reliability was substantial (κG = 0.65). NAPSAH weakly correlated with CA decision (V = 0.50). Mean sensitivity and specificity were 58% (95%CI: 44%, 71%) and 83 % (95%CI: 72 %, 94%), respectively.

CONCLUSION

NAPSAH remains a diagnosis of exclusion. The NCCT diagnosis was moderately reliable and its impact on clinical decisions modest.

Understanding the role of randomization in clinical research and practice

BACKGROUND

The scientific role randomization plays in clinical research is universally recognized, but poorly understood. In stark contrast, the ethical role randomization plays in the proper care of patients in the presence of uncertainty has been almost completely ignored.

METHODS

We review the introduction of randomization in the design of experiments, its first use in Britain, and its essential role in analysis of statistical results. We also review Thomas Chalmers' argument from 1975 that showed the ethical role randomization can play in the care of patients. We discuss how Chalmers' vision can be generalized to all contexts of clinical uncertainty.

DISCUSSION

Randomization is not only essential to the validity of statistical tests, it is also the best way to learn from experience. Although Chalmers' admonition to 'Randomize the first patient' pertained to the use of innovations, the notion that randomized allocation can be done in the best interest of patients is generalizable to all medical or surgical interventions that have yet to be proven beneficial, opening the perspective that care research can be integrated into practice in the best medical interest of patients.

CONCLUSION

Randomized allocation plays crucial scientific and ethical roles both in research and practice. It is the most efficient way to learn from experience. Prior to this gain in knowledge, it is the way to optimize care in the presence of uncertainty.

Understanding categories or subgroups and a common clinical reasoning error: the example of aneurysm size and neck width

BACKGROUND

Individualized clinical decisions are often made by considering some patient or lesion characteristics that are thought to have an impact on the efficacy or safety of treatment. For example, aneurysm size and neck width have often been determinants of treatment choices in neurovascular practice.

METHODS

We review observational and randomized data on the influence of aneurysm or neck size on angiographic results of coiling, stent-assisted coiling, or surgical clipping. New RCT data are used to demonstrate the shortcomings of managing patients using clinical judgment regarding patient or lesion characteristics. We discuss why clinical decisions should not be based on comparisons of different patients treated by the same treatment. Clinical decision making requires a comparison between the same patients treated with different treatments in a randomized trial.

RESULTS

The results of endovascular treatment of large or wide-necked aneurysms are always inferior to those of small or narrow-necked aneurysms, in observational as well as in randomized studies. However, this fact alone is not sufficient to infer that patients with small aneurysms should be coiled, while those with large aneurysms should be managed with stenting or surgical clipping. The purported superiority of clipping for large aneurysms could not be demonstrated in recent RCTs (while surgery was found superior for small aneurysms). Similarly, the superiority of stent-assisted coiling for wide-necked aneurysms was not shown in another recent RCT.

CONCLUSION

Clinical experience and observational studies alone can mislead practice. Proper clinical decisions for individuals requires randomized evidence.

Understanding crossovers and potential ways to mitigate the problem: Lessons from influential trials on lumbar microdiscectomy

BACKGROUND

Lumbar microdiscectomy is the most frequent surgical intervention used in the treatment of sciatica from herniated lumbar discs. Many discectomy trials have been plagued with an excessive number of crossovers that have rendered results inconclusive.

METHODS

We review the design and results of influential lumbar microdiscectomy trials. We also discuss the various strategies that have been used to decrease the number of crossovers or to mitigate the effects of crossovers on analyses.

RESULTS

Randomized trials on lumbar discectomy were affected by crossover rates of 8% to 42%. Various strategies that have been used to decrease that number or to mitigate the effects on results include: patient selection, blinding (placebo-controlled trials), an immediate access to surgery for the surgical group (but limited access to surgery for the conservative group), shortening the follow-up period necessary to reach the primary outcome measure, postponing crossovers to surgery after determination of the primary outcome, and modifying the primary outcome measure to include treatment failures. Crossovers should be anticipated and compensated for by increasing the number of participants.

CONCLUSION

Non-adherence to randomly allocated management options can deprive trials of the statistical power needed to inform clinical care. Crossovers and ways to mitigate related problems should be anticipated at the time of trial design.

Understanding how the research question impacts trial design: Examples from discectomy trials

BACKGROUND

Formulating a pertinent research question is of the utmost importance in clinical research. An ill-conceived question may lead to an erroneous trial design, which may adversely affect the care of patients and provide uninformative or even misleading results.

METHODS

We review the research question of a randomized trial on the timing of lumbar discectomy. We compare the resulting design with other trials, real or hypothetical, that would have been more appropriate.

RESULTS

The RCT we examine randomly allocated patients to early or delayed surgery to answer a theoretical question of the effect of time on the efficacy of surgery. The trial was interpreted to have shown that early surgery was associated with better clinical and functional outcomes as compared to delayed surgery. This conclusion is clinically misleading. Valid comparisons between groups should be performed on intent-to-treat analyses and at the same time points after randomization (and not at a fixed follow-up period after surgery). The clinically pertinent comparison is not between the theoretical efficacy of surgery performed at various times, but between surgery and conservative management in patients presenting at various times. Better-designed trials on the clinical benefits of lumbar discectomy, including the treatment of chronic sciatica, have been published.

CONCLUSION

Theoretical research questions inspired from observational data can lead to erroneous trial design. Prospective randomized trials impact practice immediately: they are unique occasions to address clinical problems and optimize care under uncertainty in real time. However, they require the research question to be formulated with great care.

The small trial problem

BACKGROUND

Many randomized trials that aim to assess new or commonly used medical or surgical interventions have been so small that the validity of conclusions becomes questionable.

METHODS

We illustrate the small trial problem using the power calculation of five Cochrane-reviewed studies that compared vertebroplasty versus placebo interventions. We discuss some of the reasons why the statistical admonition not to dichotomize continuous variables may not apply to the calculation of the number of patients required for trials to be meaningful.

RESULTS

Placebo-controlled vertebroplasty trials planned to recruit between 23 and 71 patients per group. Four of five studies used the standardized mean difference of a continuous pain variable (centimeters on the visual analog scale (VAS)) to plan implausibly small trials. What is needed is not a mean effect at the population level but a measure of efficacy at the patient level. Clinical practice concerns the care of individual patients that vary in many more respects than the variation around the mean of a single selected variable. The inference from trial to practice concerns the frequency of success of the experimental intervention performed one patient at a time. A comparison of the proportions of patients reaching a certain threshold is a more meaningful method that appropriately requires larger trials.

CONCLUSION

Most placebo-controlled vertebroplasty trials used comparisons of means of a continuous variable and were consequently very small. Randomized trials should instead be large enough to account for the diversity of future patients and practices. They should offer an evaluation of a clinically meaningful number of interventions performed in various contexts. Implications of this principle are not specific to placebo-controlled surgical trials. Trials designed to inform practice require a per-patient comparison of outcomes and the size of the trial should be planned accordingly.

A Pragmatic Randomized Trial Comparing Surgical Clipping and Endovascular Treatment of Unruptured Intracranial Aneurysms

BACKGROUND AND PURPOSE

Surgical clipping and endovascular treatment are commonly used in patients with unruptured intracranial aneurysms. We compared the safety and efficacy of the 2 treatments in a randomized trial.

MATERIALS AND METHODS

Clipping or endovascular treatments were randomly allocated to patients with one or more 3- to 25-mm unruptured intracranial aneurysms judged treatable both ways by participating physicians. The study hypothesized that clipping would decrease the incidence of treatment failure from 13% to 4%, a composite primary outcome defined as failure of aneurysm occlusion, intracranial hemorrhage during follow-up, or residual aneurysms at 1 year, as adjudicated by a core lab. Safety outcomes included new neurologic deficits following treatment, hospitalization of >5 days, and overall morbidity and mortality (mRS > 2) at 1 year. There was no blinding.

RESULTS

Two hundred ninety-one patients were enrolled from 2010 to 2020 in 7 centers. The 1-year primary outcome, ascertainable in 290/291 (99%) patients, was reached in 13/142 (9%; 95% CI, 5%-15%) patients allocated to surgery and in 28/148 (19%; 95% CI, 13%-26%) patients allocated to endovascular treatments (relative risk: 2.07; 95% CI, 1.12-3.83; P = .021). Morbidity and mortality (mRS >2) at 1 year occurred in 3/143 and 3/148 (2%; 95% CI, 1%-6%) patients allocated to surgery and endovascular treatments, respectively. Neurologic deficits (32/143, 22%; 95% CI, 16%-30% versus 19/148, 12%; 95% CI, 8%-19%; relative risk: 1.74; 95% CI, 1.04-2.92; P = .04) and hospitalizations beyond 5 days (69/143, 48%; 95% CI, 40%-56% versus 12/148, 8%; 95% CI, 5%-14%; relative risk: 0.18; 95% CI, 0.11-0.31; P < .001) were more frequent after surgery.

CONCLUSIONS

Surgical clipping is more effective than endovascular treatment of unruptured intracranial aneurysms in terms of the frequency of the primary outcome of treatment failure. Results were mainly driven by angiographic results at 1 year.

Stent-Assisted Coiling in the Treatment of Unruptured Intracranial Aneurysms: A Randomized Clinical Trial

BACKGROUND AND PURPOSE

Stent-assisted coiling may improve angiographic results of endovascular treatment of unruptured intracranial aneurysms compared with coiling alone, but this has never been shown in a randomized trial.

MATERIALS AND METHODS

The Stenting in the Treatment of Aneurysm Trial was an investigator-led, parallel, randomized (1:1) trial conducted in 4 university hospitals. Patients with intracranial aneurysms at risk of recurrence, defined as large aneurysms (≥10 mm), postcoiling recurrent aneurysms, or small aneurysms with a wide neck (≥4 mm), were randomly allocated to stent-assisted coiling or coiling alone. The composite primary efficacy outcome was "treatment failure," defined as initial failure to treat the aneurysm; aneurysm rupture or retreatment during follow-up; death or dependency (mRS > 2); or an angiographic residual aneurysm adjudicated by an independent core laboratory at 12 months. The primary hypothesis (revised for slow accrual) was that stent-assisted coiling would decrease treatment failures from 33% to 15%, requiring 200 patients. Primary analyses were intent to treat.

RESULTS

Of 205 patients recruited between 2011 and 2021, ninety-four were allocated to stent-assisted coiling and 111 to coiling alone. The primary outcome, ascertainable in 203 patients, was reached in 28/93 patients allocated to stent-assisted coiling (30.1%; 95% CI, 21.2%-40.6%) compared with 30/110 (27.3%; 95% CI, 19.4%-36.7%) allocated to coiling alone (relative risk = 1.10; 95% CI, 0.7-1.7; P  = .66). Poor clinical outcomes (mRS >2) occurred in 8/94 patients allocated to stent-assisted coiling (8.5%; 95% CI, 4.0%-16.6%) compared with 6/111 (5.4%; 95% CI, 2.2%-11.9%) allocated to coiling alone (relative risk = 1.6; 95% CI, 0.6%-4.4%; P  = .38).

CONCLUSIONS

The STAT trial did not show stent-assisted coiling to be superior to coiling alone for wide-neck, large, or recurrent unruptured aneurysms.

Understanding the difference between theory and practice: The extracranial-intracranial bypass trials in prevention of ischemic stroke

BACKGROUND

Patients with atherosclerotic carotid or middle cerebral artery occlusions suffer ischemic events that might theoretically be preventable with a surgical extracranial-intracranial bypass, but theory by itself does not justify surgical interventions.

METHODS

We review landmark randomized trials on EC-IC bypass surgery for the treatment of ischemic stroke in patients with atherosclerotic stenoses or occlusions.

RESULTS

The initial EC-IC bypass trial from 1985 did not show any clinical benefit from surgery. The carotid occlusion surgery study (COSS) performed more than 20 years later included only patients highly selected to potentially benefit from bypass by using modern perfusion studies. While EC-IC bypasses were successfully created and they did improve cerebral perfusion, the COSS study also failed to show any clinical benefit to the participating patients.

CONCLUSION

Neurosurgical interventions must not only work in theory; they must improve patient outcomes in real practice.

Research participants may not recall their participation but have a better understanding of alternative management options than patients in routine care

BACKGROUND

Patient understanding of care interventions, of the clinical uncertainty, and of their participation in clinical research is often poor. We hypothesized that compared to routine care, patients would better understand the prevailing uncertainty when they participated in research.

METHODS

A questionnaire was administered to patients at the time they attended a follow-up neurovascular clinic 4 to 52 weeks after a care episode where they did or did not participate in a clinical trial. Patients were not reminded whether they had previously participated in a clinical trial. Questions concerned their understanding of the risks/benefits of interventions, the availability of alternative options, whether their personal opinion was taken into consideration, the reason for the final decision, their confidence at having received the best management, and whether they had been research participants.

RESULTS

Between June 2019 and June 2020, 167 patients were recruited; 71 had truly been research participants, while 96 had not. A greater proportion of research patients were aware of the existence of management alternatives (65% versus 44%; P=0.008). Patients of both groups believed their personal opinion counted in the final decision (76% versus 70%), and patients were equally confident that they had received the best management (94%). Research patients believed they had participated in research 46% of the time, compared to 12% of routine care patients (P=0.003).

CONCLUSION

Many patients do not recall that they participated in a clinical trial, but they have a better understanding of the clinical uncertainty and of the availability of alternative management options.

Understanding how the primary endpoint impacts the interpretation of trial results: The Japanese Adult Moyamoya bypass trial

BACKGROUND

The meaning of a clinical trial depends to a large extent on the choice of the primary outcome measure, which can be explanatory or pragmatic.

METHODS

We review the Japanese Adult Moyamoya (JAM) trial, that compared surgical extracranial to intracranial (EC-IC) bypass and medical management of hemorrhagic moyamoya disease. We also review some principles which guide the selection of the primary trial endpoint.

DISCUSSION

The main component of the primary outcome measure in JAM was rebleeding, a surrogate outcome that allowed investigators to demonstrate that surgical bypass had causal efficacy. However, the number of patients with a poor outcome, defined as those with a modified Rankin score (mRS)>2, would have been a more pragmatic choice. Unfortunately, the trial was too small to show that patients benefited from surgery.

CONCLUSION

The JAM trial showed that EC-IC bypass can decrease rebleeding in moyamoya patients, but whether patients have better outcomes with surgery remains uncertain. Hard pragmatic clinical primary outcome measures are necessary to guide surgical care.

Understanding explanatory and pragmatic trials: Examples from randomized controlled trials on vertebroplasty

OBJECTIVES

To better understand the explanatory-pragmatic distinction in the design and interpretation of randomized controlled trials (RCTs).

METHODS

We review the explanatory-pragmatic distinction in clinical trial design. We use the PRECIS-2 tool to evaluate the trial design of selected RCTs on percutaneous vertebroplasty for osteoporotic vertebral compression fractures. We discuss difficulties in the selection of criteria and in the construction of PRECIS diagrams. We also examine how inconsistency in the selection of various items of trial design can cause confusion in the interpretation of results.

RESULTS

The selection of criteria and the scoring of multiple PRECIS domains were subjective and thus debatable. The pragmascope patterns of various vertebroplasty trials were heterogeneous. Many trials had both pragmatic and explanatory components. Some placebo-controlled trial goals seem to have been explanatory, but their design actually included enough pragmatic items such that the meaning of negative trial results remains ambiguous.

CONCLUSION

The results of a trial cannot be interpreted without understanding the various design choices made along the explanatory-pragmatic spectrum.

Understanding power in randomized trials: The example of vertebroplasty

OBJECTIVES

Randomized trials (RCTs) should include a sufficient number of patients to reduce the risk that the observed outcome is a result of chance rather than from a truly different treatment effect. Trials must be even larger to claim an absence of treatment effect in a placebo-controlled trial. To estimate the size of the trial and maximize power, it is often suggested to use a comparison between the means of a continuous variable.

METHODS

We examine the RCTs that have compared vertebroplasty and placebo for patients with osteoporotic fractures. Most trials compared the means of a continuous pain score to yield implausibly small trials, as small as 24 patients per group.

RESULTS

The minimally significant difference between groups has no precise clinical meaning for patients when it is based on a comparison of means of pain scores. A comparison of the proportions of patients reaching a per-patient outcome measure of treatment success is much more pertinent if the trial is to inform the care of future patients. The resulting trials will admittedly need to be larger, but they will be much less likely to fall prey to the 'evidence of absence' fallacy. Furthermore, trial size should also take into consideration harder clinical outcome measures, such as death and disability.

CONCLUSION

When the goal of a trial is to inform outcome-based medical care, comparing the proportions of patients reaching a clinically pertinent outcome is more appropriate than comparing the means of a continuous variable.

Understanding the research-care demarcation and why it must be revised

BACKGROUND

A clean-cut separation between research and care was artificially created at the time of the Belmont report more than 40 years ago. The demarcation was initially controversial but eventually was implemented for political reasons. We examine why it must be revised.

METHODS

We review historical research scandals as well as the theoretical basis for the Belmont demarcation. We then discuss consequences on medical practice and propose an alternative.

DISCUSSION

Most research scandals involved abusing human beings supposedly for the sake of science. Belmont commissioners were aware the research/care problem was double-headed. While research subjects should be protected from abuse in the research context, patients need to be protected from unvalidated medical and surgical interventions in the care context. For political reasons the Commission recommended the regulation of research but to leave medical practice untouched. Thus the Commission had to distinguish research from care. The notion of 'generalizable knowledge' was introduced to define and regulate research, but the inadvertent result was that by trying to protect research subjects, the regulation has not only failed to protect all other patients, but also encouraged the widespread practice of unvalidated interventions within the care context. The notion of validated care should be re-introduced into a proper analysis of the care-research demarcation, for care research is an integral ingredient of a good medical practice.

CONCLUSION

The research-care demarcation should be revised to leave room for the validated/unvalidated care distinction. Care research, essential to guide medical practice, should be facilitated at all levels.

Understanding the choice of control group: A systematic review of vertebroplasty trials for osteoporotic vertebral compression fractures

OBJECTIVES

To better understand the choice of the comparator intervention in the design of clinical trials and its impact on the meaning of results we review randomized trials on vertebroplasty.

METHODS

We conducted a systematic and narrative review of all randomized trials on vertebroplasty. Trials are categorized according to the comparator intervention (non-surgical management, placebo/sham vertebroplasty, and kyphoplasty).

RESULTS

All trials were too small to show a difference in objective clinical outcomes, and 20 of 23 RCTs used mean pain scores to compare interventions. Most trials comparing vertebroplasty with non-surgical management concluded that vertebroplasty was superior. Trials comparing kyphoplasty with vertebroplasty showed similar results for both interventions. However, 4 of 5 trials comparing vertebroplasty with placebo surgery failed to show a significant difference between groups.

CONCLUSION

The clinical results of an intervention cannot be interpreted without a comparison that involves a control group. The choice of comparator intervention can change the meaning of the trial. A large pragmatic trial, using hard clinical outcomes such as morbidity and mortality as a primary outcome measure, would be needed to assess the potential clinical benefits of vertebroplasty.

Endovascular treatment of brain arteriovenous malformations: clinical outcomes of patients included in the registry of a pragmatic randomized trial

OBJECTIVE

The role of endovascular treatment in the management of patients with brain arteriovenous malformations (AVMs) remains uncertain. AVM embolization can be offered as stand-alone curative therapy or prior to surgery or stereotactic radiosurgery (SRS) (pre-embolization). The Treatment of Brain AVMs Study (TOBAS) is an all-inclusive pragmatic study that comprises two randomized trials and multiple registries.

METHODS

Results from the TOBAS curative and pre-embolization registries are reported. The primary outcome for this report is death or dependency (modified Rankin Scale [mRS] score > 2) at last follow-up. Secondary outcomes include angiographic results, perioperative serious adverse events (SAEs), and permanent treatment-related complications leading to an mRS score > 2.

RESULTS

From June 2014 to May 2021, 1010 patients were recruited in TOBAS. Embolization was chosen as the primary curative treatment for 116 patients and pre-embolization prior to surgery or SRS for 92 patients. Clinical and angiographic outcomes were available in 106 (91%) of 116 and 77 (84%) of 92 patients, respectively. In the curative embolization registry, 70% of AVMs were ruptured, and 62% were low-grade AVMs (Spetzler-Martin grade I or II), while the pre-embolization registry had 70% ruptured AVMs and 58% low-grade AVMs. The primary outcome of death or disability (mRS score > 2) occurred in 15 (14%, 95% CI 8%–22%) of the 106 patients in the curative embolization registry (4 [12%, 95% CI 5%–28%] of 32 unruptured AVMs and 11 [15%, 95% CI 8%–25%] of 74 ruptured AVMs) and 9 (12%, 95% CI 6%–21%) of the 77 patients in the pre-embolization registry (4 [17%, 95% CI 7%–37%] of 23 unruptured AVMs and 5 [9%, 95% CI 4%–20%] of 54 ruptured AVMs) at 2 years. Embolization alone was confirmed to occlude the AVM in 32 (30%, 95% CI 21%–40%) of the 106 curative attempts and in 9 (12%, 95% CI 6%–21%) of 77 patients in the pre-embolization registry. SAEs occurred in 28 of the 106 attempted curative patients (26%, 95% CI 18%–35%, including 21 new symptomatic hemorrhages [20%, 95% CI 13%–29%]). Five of the new hemorrhages were in previously unruptured AVMs (n = 32; 16%, 95% CI 5%–33%). Of the 77 pre-embolization patients, 18 had SAEs (23%, 95% CI 15%–34%), including 12 new symptomatic hemorrhages [16%, 95% CI 9%–26%]). Three of the hemorrhages were in previously unruptured AVMs (3/23; 13%, 95% CI 3%–34%).

CONCLUSIONS

Embolization as a curative treatment for brain AVMs was often incomplete. Hemorrhagic complications were frequent, even when the specified intent was pre-embolization before surgery or SRS. Because the role of endovascular treatment remains uncertain, it should preferably, when possible, be offered in the context of a randomized trial.

Surgical or Endovascular Treatment of MCA Aneurysms: An Agreement Study

BACKGROUND AND PURPOSE

MCA aneurysms are still commonly clipped surgically despite the recent development of a number of endovascular tools and techniques. We measured clinical uncertainty by studying the reliability of decisions made for patients with middle cerebral artery (MCA) aneurysms.

MATERIALS AND METHODS

A portfolio of 60 MCA aneurysms was presented to surgical and endovascular specialists who were asked whether they considered surgery or endovascular treatment to be an option, whether they would consider recruitment of the patient in a randomized trial, and whether they would provide their final management recommendation. Agreement was studied using κ statistics. Intrarater reliability was assessed with the same, permuted portfolio of cases of MCA aneurysm sent to the same specialists 1 month later.

RESULTS

Surgical management was the preferred option for neurosurgeons (n = 844/1320; [64%] responses/22 raters), while endovascular treatment was more commonly chosen by interventional neuroradiologists (1149/1500 [76.6%] responses/25 raters). Interrater agreement was only "slight" for all cases and all judges (κ = 0.094; 95% CI, 0.068-0.130). Agreement was no better within specialties or with more experience. On delayed requestioning, 11 of 35 raters (31%) disagreed with themselves on at least 20% of cases. Surgical management and endovascular treatment were always judged to be a treatment option, for all patients. Trial participation was offered to patients 65% of the time.

CONCLUSIONS

Individual clinicians did not agree regarding the best management of patients with MCA aneurysms. A randomized trial comparing endovascular with surgical management of patients with MCA aneurysms is in order.

Surgical treatment of brain arteriovenous malformations: clinical outcomes of patients included in the registry of a pragmatic randomized trial

OBJECTIVE

The Treatment of Brain Arteriovenous Malformations Study (TOBAS) is a pragmatic study that includes 2 randomized trials and registries of treated or conservatively managed patients. The authors report the results of the surgical registry.

METHODS

TOBAS patients are managed according to an algorithm that combines clinical judgment and randomized allocation. For patients considered for curative treatment, clinicians selected from surgery, endovascular therapy, or radiation therapy as the primary curative method, and whether observation was a reasonable alternative. When surgery was selected and observation was deemed unreasonable, the patient was not included in the randomized controlled trial but placed in the surgical registry. The primary outcome of the trial was mRS score > 2 at 10 years (at last follow-up for the current report). Secondary outcomes include angiographic results, perioperative serious adverse events, and permanent treatment-related complications leading to mRS score > 2.

RESULTS

From June 2014 to May 2021, 1010 patients were recruited at 30 TOBAS centers. Surgery was selected for 229/512 patients (44%) considered for curative treatment; 77 (34%) were included in the surgery versus observation randomized trial and 152 (66%) were placed in the surgical registry. Surgical registry patients had 124/152 (82%) ruptured and 28/152 (18%) unruptured arteriovenous malformations (AVMs), with the majority categorized as low-grade Spetzler-Martin grade I-II AVM (118/152 [78%]). Thirteen patients were excluded, leaving 139 patients for analysis. Embolization was performed prior to surgery in 78/139 (56%) patients. Surgical angiographic cure was obtained in 123/139 all-grade (89%, 95% CI 82%-93%) and 105/110 low-grade (95%, 95% CI 90%-98%) AVM patients. At the mean follow-up of 18.1 months, 16 patients (12%, 95% CI 7%-18%) had reached the primary safety outcome of mRS score > 2, including 11/16 who had a baseline mRS score ≥ 3 due to previous AVM rupture. Serious adverse events occurred in 29 patients (21%, 95% CI 15%-28%). Permanent treatment-related complications leading to mRS score > 2 occurred in 6/139 patients (4%, 95% CI 2%-9%), 5 (83%) of whom had complications due to preoperative embolization.

CONCLUSIONS

The surgical treatment of brain AVMs in the TOBAS registry was curative in 88% of patients. The participation of more patients, surgeons, and centers in randomized trials is needed to definitively establish the role of surgery in the treatment of unruptured brain AVMs. Clinical trial registration no.: NCT02098252 (ClinicalTrials.gov).

Flow Diversion in the Treatment of Intracranial Aneurysms: A Pragmatic Randomized Care Trial

BACKGROUND AND PURPOSE

Flow diversion is a recent endovascular treatment for intracranial aneurysms. We compared the safety and efficacy of flow diversion with the alternative standard management options.

MATERIALS AND METHODS

A parallel group, prerandomized, controlled, open-label pragmatic trial was conducted in 3 Canadian centers. The trial included all patients considered for flow diversion. A Web-based platform 1:1 randomly allocated patients to flow diversion or 1 of 4 alternative standard management options (coiling with/without stent placement, parent vessel occlusion, surgical clipping, or observation) as prespecified by clinical judgment. Patients ineligible for alternative standard management options were treated with flow diversion in a registry. The primary safety outcome was death or dependency (mRS > 2) at 3 months. The composite primary efficacy outcome included the core lab-determined angiographic presence of a residual aneurysm, aneurysm rupture, progressive mass effect during follow-up, or death or dependency (mRS > 2) at 3-12 months.

RESULTS

Between May 2011 and November 2020, three hundred twenty-three patients were recruited: Two hundred seventy-eight patients (86%) had treatment randomly allocated (139 to flow diversion and 139 to alternative standard management options), and 45 (14%) received flow diversion in the registry. Patients in the randomized trial frequently had unruptured (83%), large (52% ≥10 mm) carotid (64%) aneurysms. Death or dependency at 3 months occurred in 16/138 patients who underwent flow diversion and 12/137 patients receiving alternative standard management options (relative risk, 1.33; 95% CI, 0.65-2.69; P = .439). A poor primary efficacy outcome was found in 30.9% (43/139) with flow diversion and 45.6% (62/136) of patients receiving alternative standard management options, with an absolute risk difference of 14.7% (95% CI, 3.3%-26.0%; relative risk, 0.68; 95% CI, 0.50-0.92; P = .014).

CONCLUSIONS

For patients with mostly unruptured, large, anterior circulation (carotid) aneurysms, flow diversion was more effective than the alternative standard management option in terms of angiographic outcome.

Understanding the problems with recruitment in surgical randomized trials: A lesson from landmark trials on temporal lobe epilepsy

BACKGROUND

Surgical randomized trials are difficult to accomplish. One major problem is recruitment of a sufficient number of patients to address the clinical problem.

METHODS

We review the various ways patient recruitment in surgical RCTs can be promoted. We examine two landmark trials on the surgical treatment of temporal lobe epilepsy (TLE), one that was successful, and one which did not attain the target number of participants.

DISCUSSION

Both designs of the Canadian and American trials of surgery for TLE included a benefit to participants: the Canadian trial gave a chance to have immediate access to investigation and treatment, as compared to a 1 year delay (considered 'standard care' in that center), while the American trial offered free surgical management to both arms. Patients were recruited and treatments randomly allocated prior to knowing for certain whether they were surgical candidates or not. This design choice may have helped circumvent the 'equipoise problem'. The Canadian trial offered participation to drug-resistant patients that were already routinely referred to surgical centers, while the success of the American trial which limited recruitment to the early period of drug resistance was dependent on a change of practice of referring clinicians which did not materialize.

CONCLUSION

The surgical treatment of drug-resistant temporal lobe epilepsy has been validated using RCT methods. Ways to promote participation in surgical trials should be further investigated.

Understanding burden of proof and equipoise in the design of pragmatic clinical trials: An example from a trial on brain arteriovenous malformations

BACKGROUND AND PURPOSE

The burden of proof principle is rarely discussed and poorly understood, but central to the proper design of pragmatic clinical trials. A better understanding of the principle could play an important role in the re-introduction of scientific methods within practice and in revising fundamental problems with the current research-care separation.

METHODS

We analyze the design of the ARUBA trial on the management of unruptured brain arteriovenous malformations. We also review how the concept of clinical equipoise was introduced to address a misconceived problem of research ethics.

RESULTS

The ARUBA trial hypothesis in favour of conservative management of brain arteriovenous malformations failed to take into account the fact that the burden of proof was on surgery, endovascular treatment or radiation therapy. Thus, results remained inconclusive and other trials are needed. The equipoise notion fails to take into account that the burden of proof is on unvalidated medical or surgical interventions, if we want to provide outcome-based medical care that patients can trust.

CONCLUSION

The burden of proof principle is essential to properly design pragmatic trials. This principle also explains why in certain circumstances optimal care is a randomized care trial.

Understanding the reliability of trial outcome measures: The example of angiographic results of surgical or endovascular treatments of aneurysms

BACKGROUND

The reliability of outcome measures is of central importance in clinical research. Studies of reliability remain rare in the neurovascular field.

METHODS

A narrative review of the history (1997-2021) of reporting angiographic results of the surgical or endovascular treatments of aneurysms serves to illustrate the importance of precisely defining outcome measures. We also review how the reliability of an angiographic classification system was studied.

DISCUSSION

Outcome measures are commonly used without precise definitions. When definitions or classification systems exist, they are rarely verified for their reliability. Twenty-five years following its introduction, a classification of angiographic results of aneurysm treatments is still being studied and modified.

CONCLUSION

The reliability of outcome measures should be made a research priority if we are to practice outcome-based medical or surgical care.

Recruitment in a pragmatic randomized trial on the management of unruptured intracranial aneurysms

OBJECTIVES

The Comprehensive Aneurysm Management (CAM) study is a pragmatic trial designed to manage UIA patients within a care research framework.

METHOD

CAM is an all-inclusive study. Management options are allocated according to an algorithm combining pre-randomization and clinical judgment. Eligible patients are offered 1:1 randomized allocation of intervention versus conservative management and 1:1 randomization allocation of surgical versus endovascular treatment. Ineligible patients are registered. The primary outcome is survival without dependency (mRS<3) at 10 years. All UIA patients at one center are reported.

RESULTS

Between February 2020 and July 2021, 403 UIA patients were recruited: 179 (44%) in one of the RCTs and 224 (56%) in one of the registries. Conservative management was recommended for 205/403 patients (51%); of 198 (49%) patients considered for curative treatment, 159 (80%) were randomly allocated conservative (n=81) or curative treatment (n=78). These patients were younger and had larger aneurysms than those in the observation registry (P = .004). In 39/198 patients (20%), conservative management was not considered reasonable (17 patients were recommended endovascular, 2 surgery, and 20 the RCT comparing endovascular with surgical treatment). In total, 70 patients were recruited in the RCT comparing surgery and endovascular treatment. After informed discussion at time of consent, 141/159 patients (89%) agreed with the randomly allocated management plan, while 11% crossed-over to the alternative management option.

CONCLUSION

CAM was successfully integrated into routine practice. Meaningful conclusions can be obtained if multiple centers actively participate in the trial.

Middle Cerebral Artery Aneurysm Trial (MCAAT): A randomized care trial comparing surgical and endovascular management of MCA aneurysm patients

BACKGROUND

Whether the best management of middle cerebral artery (MCA) aneurysm patients is surgical or endovascular remains uncertain, with little evidence to guide decision-making. A randomized care trial offering MCA aneurysm patients a 50% chance of surgical and a 50% chance of endovascular management may optimize outcomes in the presence of uncertainty.

METHODS

The Middle Cerebral Artery Aneurysm Trial (MCAAT) is an investigator-initiated, multi-center, parallel group, prospective, 1:1 randomized controlled clinical trial. All adult patients with MCA aneurysms, ruptured or unruptured, amenable to surgical and endovascular treatment can be included. The composite primary outcome is 'Treatment Success': i) occlusion or exclusion of the aneurysm using the allocated treatment modality; ii) no intracranial hemorrhage during follow-up; iii) no retreatment of the target aneurysm during follow-up, iv) no residual aneurysm on angiographic follow-up and v) independence (mRS <3) at 1 year. The trial tests two versions of the same hypothesis (one for ruptured and one for unruptured MCA aneurysm patients): Surgical management will lead to a 15% absolute increase in the proportion of patients reaching Treatment Success from 55% to 70% (ruptured) or from 75% to 90% (unruptured aneurysm patients) compared to endovascular treatment (any method). In this pragmatic trial, outcome evaluations are by treating physicians, except for 1 year angiographic results which will be core lab assessed. The trial will be monitored by an independent data safety monitoring committee to assure safety of participants. MCAAT is registered at clinicaltrials.gov: NCT05161377.

CONCLUSION

Patients with MCA aneurysms can be optimally managed within a care trial protocol.

Repeat Resection in Recurrent Glioblastoma (3rGBM) Trial: a randomized care trial

BACKGROUND

The prognosis for patients with recurrent glioblastoma (GBM) is dismal, and the question of repeat surgery at time of recurrence is common. Re-operation in the management of these patients remains controversial, as there is no randomized evidence of benefit. An all-inclusive pragmatic care trial is needed to evaluate the role of repeat resection.

METHODS

3rGBM is a multicenter, pragmatic, prospective, parallel-group randomized care trial, with 1:1 allocation to repeat resection or standard care with no repeat resection. To test the hypothesis that repeat resection can improve overall survival by at least 3 months (from 6 to 9 months), 250 adult patients with prior resection of pathology-proven glioblastoma for whom the attending surgeon believes repeat resection may improve quality survival will be enrolled. A surrogate measure of quality of life, the number of days outside of hospital/nursing/palliative care facility, will also be compared. Centers are invited to participate without financial compensation and without contracts. Clinicians may apply to local authorities to approve an investigator-led in-house trial, using a common protocol, web-based randomization platform, and simple standardized case report forms.

DISCUSSION

The 3rGBM trial is a modern transparent care research framework with no additional risks, tests, or visits other than what patients would encounter in normal care. The burden of proof remains on repeat surgical management of recurrent GBM, because this management has yet to be shown beneficial. The trial is designed to help patients and surgeons manage the uncertainty regarding optimal care.

CLINICAL TRIAL REGISTRATION

http://www.clinicaltrials.gov. Unique identifier: NCT04838782.

Noninvasive Angiographic Results of Clipped or Coiled Intracranial Aneurysms: An Inter- and Intraobserver Reliability Study

BACKGROUND AND PURPOSE

Noninvasive angiography is commonly used to assess the outcome of surgical or endovascular treatment of intracranial aneurysms in clinical series or randomized trials. We sought to assess whether a standardized 3-grade classification system could be reliably used to compare the CTA and MRA results of both treatments.

MATERIALS AND METHODS

An electronic portfolio composed of CTAs of 30 clipped and MRAs of 30 coiled aneurysms was independently evaluated by 24 raters of diverse experience and training backgrounds. Twenty raters performed a second evaluation 1 month later. Raters were asked which angiographic grade and management decision (retreatment; close or long-term follow-up) would be most appropriate for each case. Agreement was analyzed using the Krippendorff α (α_K) statistic, and the relationship between angiographic grade and clinical management choice, using the Fisher exact and Cramer V tests.

RESULTS

Interrater agreement was substantial (α_K = 0.63; 95% CI, 0.55-0.70); results were slightly better for MRA results of coiling (α_K = 0.69; 95% CI, 0.56-0.76) than for CTA results of clipping (α_K = 0.58; 95% CI, 0.44-0.69). Intrarater agreement was substantial to almost perfect. Interrater agreement regarding clinical management was moderate for both clipped (α_K = 0.49; 95% CI, 0.32-0.61) and coiled subgroups (α_K = 0.47; 95% CI, 0.34-0.54). The choice of clinical management was strongly associated with the size of the residuum (mean Cramer V = 0.77 [SD, 0.14]), but complete occlusions (grade 1) were followed more closely after coiling than after clipping (P = .01).

CONCLUSIONS

A standardized 3-grade scale was found to be a reliable and clinically meaningful tool to compare the results of clipping and coiling of aneurysms using CTA or MRA.

Douglas Altman's 2009 Grand Lecture: Can we trust our literature?

Recent studies of the medical literature have revealed numerous and serious problems. Errors in the design, methods and interpretation of studies can frequently be identified. A huge hidden problem is publication bias, the tendency for positive articles to be published, while negative articles are either not written or submitted. This can systematically lead to an overestimation of the value of treatments, of diagnostic or prognostic studies. Even more worrisome is selective reporting: only a subset of a wide array of tested hypotheses are presented (the ones that turned out to be positive with significance testing). This is particularly true for secondary endpoints and subgroup findings, but even the primary endpoints of trials have been modified when publications are compared to protocols. The peer-review process is fallible. Even if it were strengthened, reviewers cannot examine what is not reported. Hence many problems can only be mitigated with better reporting. Numerous initiatives have proposed guidelines to promote transparent reporting, but progress is slow.

Clinical Uncertainty and Equipoise in the Management of Recurrent Glioblastoma

BACKGROUND

A significant proportion of glioblastoma (GBM) patients are considered for repeat resection, but evidence regarding best management remains elusive. Our aim was to measure the degree of clinical uncertainty regarding reoperation for patients with recurrent GBM.

METHODS

We first performed a systematic review of agreement studies examining the question of repeat resection for recurrent GBM. An electronic portfolio of 37 pathologically confirmed recurrent GBM patients including pertinent magnetic resonance images and clinical information was assembled. To measure clinical uncertainty, 26 neurosurgeons from various countries, training backgrounds, and years' experience were asked to select best management (repeat surgery, other nonsurgical management, or conservative), confidence in recommended management, and whether they would include the patient in a randomized trial comparing surgery with nonsurgical options. Agreement was evaluated using κ statistics.

RESULTS

The literature review did not reveal previous agreement studies examining the question. In our study, agreement regarding best management of recurrent GBM was slight, even when management options were dichotomized (repeat surgery vs. other options; κ=0.198 [95% confidence interval: 0.133-0.276]). Country of practice, years' experience, and training background did not change results. Disagreement and clinical uncertainty were more pronounced within clinicians with (κ=0.167 [0.055-0.314]) than clinicians without neuro-oncology fellowship training (κ=0.601 [0.556-0.646]). A majority (51%) of responders were willing to include the patient in a randomized trial comparing repeat surgery with nonsurgical alternatives in 26/37 (69%) of cases.

CONCLUSION

There is sufficient uncertainty and equipoise regarding the question of reoperation for patients with recurrent glioblastoma to support the need for a randomized controlled trial.

NEWTON-2 Cisternal (Nimodipine Microparticles to Enhance Recovery While Reducing Toxicity After Subarachnoid Hemorrhage): A Phase 2, Multicenter, Randomized, Open-Label Safety Study of Intracisternal EG-1962 in Aneurysmal Subarachnoid Hemorrhage

BACKGROUND

A sustained release microparticle formulation of nimodipine (EG-1962) was developed for treatment of patients with aneurysmal subarachnoid hemorrhage (aSAH).

OBJECTIVE

To assess safety, tolerability, and pharmacokinetics of intracisternal EG-1962 in an open-label, randomized, phase 2 study of up to 12 subjects.

METHODS

Subjects were World Federation of Neurological Surgeons grades 1 to 2, modified Fisher grades 2 to 4, and underwent aneurysm clipping within 48 h of aSAH. EG-1962, containing 600 mg nimodipine, was administered into the basal cisterns. Outcome on the extended Glasgow Outcome Scale (eGOS), pharmacokinetics, delayed cerebral ischemia and infarction, rescue therapy, and safety were evaluated.

RESULTS

The study was halted when a phase 3 study of intraventricular EG-1962 stopped because that study was unlikely to meet its primary endpoint. Six subjects were randomized (5 EG-1962 and 1 oral nimodipine). After 90-d follow-up, favorable outcome on the eGOS occurred in 1 of 5 EG-1962 and in the single oral nimodipine patient. Four EG-1962 and the oral nimodipine subject had angiographic vasospasm. One EG-1962 subject had delayed cerebral ischemia, and all subjects with angiographic vasospasm received rescue therapy except 1 EG-1962 patient. One subject treated with EG-1962 developed right internal carotid and middle cerebral artery narrowing 5 mo after placement of EG-1962, leading to occlusion and cerebral infarction. Pharmacokinetics showed similar plasma concentrations of nimodipine in both groups.

CONCLUSION

Angiographic vasospasm and unfavorable clinical outcome still occurred after placement of EG-1962. Internal carotid artery narrowing and occlusion after placement of EG-1962 in the basal cisterns has not been reported.