Understanding the research-care demarcation and why it must be revised

BACKGROUND

A clean-cut separation between research and care was artificially created at the time of the Belmont report more than 40 years ago. The demarcation was initially controversial but eventually was implemented for political reasons. We examine why it must be revised.

METHODS

We review historical research scandals as well as the theoretical basis for the Belmont demarcation. We then discuss consequences on medical practice and propose an alternative.

DISCUSSION

Most research scandals involved abusing human beings supposedly for the sake of science. Belmont commissioners were aware the research/care problem was double-headed. While research subjects should be protected from abuse in the research context, patients need to be protected from unvalidated medical and surgical interventions in the care context. For political reasons the Commission recommended the regulation of research but to leave medical practice untouched. Thus the Commission had to distinguish research from care. The notion of 'generalizable knowledge' was introduced to define and regulate research, but the inadvertent result was that by trying to protect research subjects, the regulation has not only failed to protect all other patients, but also encouraged the widespread practice of unvalidated interventions within the care context. The notion of validated care should be re-introduced into a proper analysis of the care-research demarcation, for care research is an integral ingredient of a good medical practice.

CONCLUSION

The research-care demarcation should be revised to leave room for the validated/unvalidated care distinction. Care research, essential to guide medical practice, should be facilitated at all levels.

Understanding the choice of control group: A systematic review of vertebroplasty trials for osteoporotic vertebral compression fractures

OBJECTIVES

To better understand the choice of the comparator intervention in the design of clinical trials and its impact on the meaning of results we review randomized trials on vertebroplasty.

METHODS

We conducted a systematic and narrative review of all randomized trials on vertebroplasty. Trials are categorized according to the comparator intervention (non-surgical management, placebo/sham vertebroplasty, and kyphoplasty).

RESULTS

All trials were too small to show a difference in objective clinical outcomes, and 20 of 23 RCTs used mean pain scores to compare interventions. Most trials comparing vertebroplasty with non-surgical management concluded that vertebroplasty was superior. Trials comparing kyphoplasty with vertebroplasty showed similar results for both interventions. However, 4 of 5 trials comparing vertebroplasty with placebo surgery failed to show a significant difference between groups.

CONCLUSION

The clinical results of an intervention cannot be interpreted without a comparison that involves a control group. The choice of comparator intervention can change the meaning of the trial. A large pragmatic trial, using hard clinical outcomes such as morbidity and mortality as a primary outcome measure, would be needed to assess the potential clinical benefits of vertebroplasty.

Endovascular treatment of brain arteriovenous malformations: clinical outcomes of patients included in the registry of a pragmatic randomized trial

OBJECTIVE

The role of endovascular treatment in the management of patients with brain arteriovenous malformations (AVMs) remains uncertain. AVM embolization can be offered as stand-alone curative therapy or prior to surgery or stereotactic radiosurgery (SRS) (pre-embolization). The Treatment of Brain AVMs Study (TOBAS) is an all-inclusive pragmatic study that comprises two randomized trials and multiple registries.

METHODS

Results from the TOBAS curative and pre-embolization registries are reported. The primary outcome for this report is death or dependency (modified Rankin Scale [mRS] score > 2) at last follow-up. Secondary outcomes include angiographic results, perioperative serious adverse events (SAEs), and permanent treatment-related complications leading to an mRS score > 2.

RESULTS

From June 2014 to May 2021, 1010 patients were recruited in TOBAS. Embolization was chosen as the primary curative treatment for 116 patients and pre-embolization prior to surgery or SRS for 92 patients. Clinical and angiographic outcomes were available in 106 (91%) of 116 and 77 (84%) of 92 patients, respectively. In the curative embolization registry, 70% of AVMs were ruptured, and 62% were low-grade AVMs (Spetzler-Martin grade I or II), while the pre-embolization registry had 70% ruptured AVMs and 58% low-grade AVMs. The primary outcome of death or disability (mRS score > 2) occurred in 15 (14%, 95% CI 8%–22%) of the 106 patients in the curative embolization registry (4 [12%, 95% CI 5%–28%] of 32 unruptured AVMs and 11 [15%, 95% CI 8%–25%] of 74 ruptured AVMs) and 9 (12%, 95% CI 6%–21%) of the 77 patients in the pre-embolization registry (4 [17%, 95% CI 7%–37%] of 23 unruptured AVMs and 5 [9%, 95% CI 4%–20%] of 54 ruptured AVMs) at 2 years. Embolization alone was confirmed to occlude the AVM in 32 (30%, 95% CI 21%–40%) of the 106 curative attempts and in 9 (12%, 95% CI 6%–21%) of 77 patients in the pre-embolization registry. SAEs occurred in 28 of the 106 attempted curative patients (26%, 95% CI 18%–35%, including 21 new symptomatic hemorrhages [20%, 95% CI 13%–29%]). Five of the new hemorrhages were in previously unruptured AVMs (n = 32; 16%, 95% CI 5%–33%). Of the 77 pre-embolization patients, 18 had SAEs (23%, 95% CI 15%–34%), including 12 new symptomatic hemorrhages [16%, 95% CI 9%–26%]). Three of the hemorrhages were in previously unruptured AVMs (3/23; 13%, 95% CI 3%–34%).

CONCLUSIONS

Embolization as a curative treatment for brain AVMs was often incomplete. Hemorrhagic complications were frequent, even when the specified intent was pre-embolization before surgery or SRS. Because the role of endovascular treatment remains uncertain, it should preferably, when possible, be offered in the context of a randomized trial.

Surgical or Endovascular Treatment of MCA Aneurysms: An Agreement Study

BACKGROUND AND PURPOSE

MCA aneurysms are still commonly clipped surgically despite the recent development of a number of endovascular tools and techniques. We measured clinical uncertainty by studying the reliability of decisions made for patients with middle cerebral artery (MCA) aneurysms.

MATERIALS AND METHODS

A portfolio of 60 MCA aneurysms was presented to surgical and endovascular specialists who were asked whether they considered surgery or endovascular treatment to be an option, whether they would consider recruitment of the patient in a randomized trial, and whether they would provide their final management recommendation. Agreement was studied using κ statistics. Intrarater reliability was assessed with the same, permuted portfolio of cases of MCA aneurysm sent to the same specialists 1 month later.

RESULTS

Surgical management was the preferred option for neurosurgeons (n = 844/1320; [64%] responses/22 raters), while endovascular treatment was more commonly chosen by interventional neuroradiologists (1149/1500 [76.6%] responses/25 raters). Interrater agreement was only "slight" for all cases and all judges (κ = 0.094; 95% CI, 0.068-0.130). Agreement was no better within specialties or with more experience. On delayed requestioning, 11 of 35 raters (31%) disagreed with themselves on at least 20% of cases. Surgical management and endovascular treatment were always judged to be a treatment option, for all patients. Trial participation was offered to patients 65% of the time.

CONCLUSIONS

Individual clinicians did not agree regarding the best management of patients with MCA aneurysms. A randomized trial comparing endovascular with surgical management of patients with MCA aneurysms is in order.

Surgical treatment of brain arteriovenous malformations: clinical outcomes of patients included in the registry of a pragmatic randomized trial

OBJECTIVE

The Treatment of Brain Arteriovenous Malformations Study (TOBAS) is a pragmatic study that includes 2 randomized trials and registries of treated or conservatively managed patients. The authors report the results of the surgical registry.

METHODS

TOBAS patients are managed according to an algorithm that combines clinical judgment and randomized allocation. For patients considered for curative treatment, clinicians selected from surgery, endovascular therapy, or radiation therapy as the primary curative method, and whether observation was a reasonable alternative. When surgery was selected and observation was deemed unreasonable, the patient was not included in the randomized controlled trial but placed in the surgical registry. The primary outcome of the trial was mRS score > 2 at 10 years (at last follow-up for the current report). Secondary outcomes include angiographic results, perioperative serious adverse events, and permanent treatment-related complications leading to mRS score > 2.

RESULTS

From June 2014 to May 2021, 1010 patients were recruited at 30 TOBAS centers. Surgery was selected for 229/512 patients (44%) considered for curative treatment; 77 (34%) were included in the surgery versus observation randomized trial and 152 (66%) were placed in the surgical registry. Surgical registry patients had 124/152 (82%) ruptured and 28/152 (18%) unruptured arteriovenous malformations (AVMs), with the majority categorized as low-grade Spetzler-Martin grade I-II AVM (118/152 [78%]). Thirteen patients were excluded, leaving 139 patients for analysis. Embolization was performed prior to surgery in 78/139 (56%) patients. Surgical angiographic cure was obtained in 123/139 all-grade (89%, 95% CI 82%-93%) and 105/110 low-grade (95%, 95% CI 90%-98%) AVM patients. At the mean follow-up of 18.1 months, 16 patients (12%, 95% CI 7%-18%) had reached the primary safety outcome of mRS score > 2, including 11/16 who had a baseline mRS score ≥ 3 due to previous AVM rupture. Serious adverse events occurred in 29 patients (21%, 95% CI 15%-28%). Permanent treatment-related complications leading to mRS score > 2 occurred in 6/139 patients (4%, 95% CI 2%-9%), 5 (83%) of whom had complications due to preoperative embolization.

CONCLUSIONS

The surgical treatment of brain AVMs in the TOBAS registry was curative in 88% of patients. The participation of more patients, surgeons, and centers in randomized trials is needed to definitively establish the role of surgery in the treatment of unruptured brain AVMs. Clinical trial registration no.: NCT02098252 (ClinicalTrials.gov).

Flow Diversion in the Treatment of Intracranial Aneurysms: A Pragmatic Randomized Care Trial

BACKGROUND AND PURPOSE

Flow diversion is a recent endovascular treatment for intracranial aneurysms. We compared the safety and efficacy of flow diversion with the alternative standard management options.

MATERIALS AND METHODS

A parallel group, prerandomized, controlled, open-label pragmatic trial was conducted in 3 Canadian centers. The trial included all patients considered for flow diversion. A Web-based platform 1:1 randomly allocated patients to flow diversion or 1 of 4 alternative standard management options (coiling with/without stent placement, parent vessel occlusion, surgical clipping, or observation) as prespecified by clinical judgment. Patients ineligible for alternative standard management options were treated with flow diversion in a registry. The primary safety outcome was death or dependency (mRS > 2) at 3 months. The composite primary efficacy outcome included the core lab-determined angiographic presence of a residual aneurysm, aneurysm rupture, progressive mass effect during follow-up, or death or dependency (mRS > 2) at 3-12 months.

RESULTS

Between May 2011 and November 2020, three hundred twenty-three patients were recruited: Two hundred seventy-eight patients (86%) had treatment randomly allocated (139 to flow diversion and 139 to alternative standard management options), and 45 (14%) received flow diversion in the registry. Patients in the randomized trial frequently had unruptured (83%), large (52% ≥10 mm) carotid (64%) aneurysms. Death or dependency at 3 months occurred in 16/138 patients who underwent flow diversion and 12/137 patients receiving alternative standard management options (relative risk, 1.33; 95% CI, 0.65-2.69; P = .439). A poor primary efficacy outcome was found in 30.9% (43/139) with flow diversion and 45.6% (62/136) of patients receiving alternative standard management options, with an absolute risk difference of 14.7% (95% CI, 3.3%-26.0%; relative risk, 0.68; 95% CI, 0.50-0.92; P = .014).

CONCLUSIONS

For patients with mostly unruptured, large, anterior circulation (carotid) aneurysms, flow diversion was more effective than the alternative standard management option in terms of angiographic outcome.

Understanding the problems with recruitment in surgical randomized trials: A lesson from landmark trials on temporal lobe epilepsy

BACKGROUND

Surgical randomized trials are difficult to accomplish. One major problem is recruitment of a sufficient number of patients to address the clinical problem.

METHODS

We review the various ways patient recruitment in surgical RCTs can be promoted. We examine two landmark trials on the surgical treatment of temporal lobe epilepsy (TLE), one that was successful, and one which did not attain the target number of participants.

DISCUSSION

Both designs of the Canadian and American trials of surgery for TLE included a benefit to participants: the Canadian trial gave a chance to have immediate access to investigation and treatment, as compared to a 1 year delay (considered 'standard care' in that center), while the American trial offered free surgical management to both arms. Patients were recruited and treatments randomly allocated prior to knowing for certain whether they were surgical candidates or not. This design choice may have helped circumvent the 'equipoise problem'. The Canadian trial offered participation to drug-resistant patients that were already routinely referred to surgical centers, while the success of the American trial which limited recruitment to the early period of drug resistance was dependent on a change of practice of referring clinicians which did not materialize.

CONCLUSION

The surgical treatment of drug-resistant temporal lobe epilepsy has been validated using RCT methods. Ways to promote participation in surgical trials should be further investigated.

Understanding burden of proof and equipoise in the design of pragmatic clinical trials: An example from a trial on brain arteriovenous malformations

BACKGROUND AND PURPOSE

The burden of proof principle is rarely discussed and poorly understood, but central to the proper design of pragmatic clinical trials. A better understanding of the principle could play an important role in the re-introduction of scientific methods within practice and in revising fundamental problems with the current research-care separation.

METHODS

We analyze the design of the ARUBA trial on the management of unruptured brain arteriovenous malformations. We also review how the concept of clinical equipoise was introduced to address a misconceived problem of research ethics.

RESULTS

The ARUBA trial hypothesis in favour of conservative management of brain arteriovenous malformations failed to take into account the fact that the burden of proof was on surgery, endovascular treatment or radiation therapy. Thus, results remained inconclusive and other trials are needed. The equipoise notion fails to take into account that the burden of proof is on unvalidated medical or surgical interventions, if we want to provide outcome-based medical care that patients can trust.

CONCLUSION

The burden of proof principle is essential to properly design pragmatic trials. This principle also explains why in certain circumstances optimal care is a randomized care trial.

Understanding the reliability of trial outcome measures: The example of angiographic results of surgical or endovascular treatments of aneurysms

BACKGROUND

The reliability of outcome measures is of central importance in clinical research. Studies of reliability remain rare in the neurovascular field.

METHODS

A narrative review of the history (1997-2021) of reporting angiographic results of the surgical or endovascular treatments of aneurysms serves to illustrate the importance of precisely defining outcome measures. We also review how the reliability of an angiographic classification system was studied.

DISCUSSION

Outcome measures are commonly used without precise definitions. When definitions or classification systems exist, they are rarely verified for their reliability. Twenty-five years following its introduction, a classification of angiographic results of aneurysm treatments is still being studied and modified.

CONCLUSION

The reliability of outcome measures should be made a research priority if we are to practice outcome-based medical or surgical care.

Recruitment in a pragmatic randomized trial on the management of unruptured intracranial aneurysms

OBJECTIVES

The Comprehensive Aneurysm Management (CAM) study is a pragmatic trial designed to manage UIA patients within a care research framework.

METHOD

CAM is an all-inclusive study. Management options are allocated according to an algorithm combining pre-randomization and clinical judgment. Eligible patients are offered 1:1 randomized allocation of intervention versus conservative management and 1:1 randomization allocation of surgical versus endovascular treatment. Ineligible patients are registered. The primary outcome is survival without dependency (mRS<3) at 10 years. All UIA patients at one center are reported.

RESULTS

Between February 2020 and July 2021, 403 UIA patients were recruited: 179 (44%) in one of the RCTs and 224 (56%) in one of the registries. Conservative management was recommended for 205/403 patients (51%); of 198 (49%) patients considered for curative treatment, 159 (80%) were randomly allocated conservative (n=81) or curative treatment (n=78). These patients were younger and had larger aneurysms than those in the observation registry (P = .004). In 39/198 patients (20%), conservative management was not considered reasonable (17 patients were recommended endovascular, 2 surgery, and 20 the RCT comparing endovascular with surgical treatment). In total, 70 patients were recruited in the RCT comparing surgery and endovascular treatment. After informed discussion at time of consent, 141/159 patients (89%) agreed with the randomly allocated management plan, while 11% crossed-over to the alternative management option.

CONCLUSION

CAM was successfully integrated into routine practice. Meaningful conclusions can be obtained if multiple centers actively participate in the trial.

Middle Cerebral Artery Aneurysm Trial (MCAAT): A randomized care trial comparing surgical and endovascular management of MCA aneurysm patients

BACKGROUND

Whether the best management of middle cerebral artery (MCA) aneurysm patients is surgical or endovascular remains uncertain, with little evidence to guide decision-making. A randomized care trial offering MCA aneurysm patients a 50% chance of surgical and a 50% chance of endovascular management may optimize outcomes in the presence of uncertainty.

METHODS

The Middle Cerebral Artery Aneurysm Trial (MCAAT) is an investigator-initiated, multi-center, parallel group, prospective, 1:1 randomized controlled clinical trial. All adult patients with MCA aneurysms, ruptured or unruptured, amenable to surgical and endovascular treatment can be included. The composite primary outcome is 'Treatment Success': i) occlusion or exclusion of the aneurysm using the allocated treatment modality; ii) no intracranial hemorrhage during follow-up; iii) no retreatment of the target aneurysm during follow-up, iv) no residual aneurysm on angiographic follow-up and v) independence (mRS <3) at 1 year. The trial tests two versions of the same hypothesis (one for ruptured and one for unruptured MCA aneurysm patients): Surgical management will lead to a 15% absolute increase in the proportion of patients reaching Treatment Success from 55% to 70% (ruptured) or from 75% to 90% (unruptured aneurysm patients) compared to endovascular treatment (any method). In this pragmatic trial, outcome evaluations are by treating physicians, except for 1 year angiographic results which will be core lab assessed. The trial will be monitored by an independent data safety monitoring committee to assure safety of participants. MCAAT is registered at clinicaltrials.gov: NCT05161377.

CONCLUSION

Patients with MCA aneurysms can be optimally managed within a care trial protocol.

Repeat Resection in Recurrent Glioblastoma (3rGBM) Trial: a randomized care trial

BACKGROUND

The prognosis for patients with recurrent glioblastoma (GBM) is dismal, and the question of repeat surgery at time of recurrence is common. Re-operation in the management of these patients remains controversial, as there is no randomized evidence of benefit. An all-inclusive pragmatic care trial is needed to evaluate the role of repeat resection.

METHODS

3rGBM is a multicenter, pragmatic, prospective, parallel-group randomized care trial, with 1:1 allocation to repeat resection or standard care with no repeat resection. To test the hypothesis that repeat resection can improve overall survival by at least 3 months (from 6 to 9 months), 250 adult patients with prior resection of pathology-proven glioblastoma for whom the attending surgeon believes repeat resection may improve quality survival will be enrolled. A surrogate measure of quality of life, the number of days outside of hospital/nursing/palliative care facility, will also be compared. Centers are invited to participate without financial compensation and without contracts. Clinicians may apply to local authorities to approve an investigator-led in-house trial, using a common protocol, web-based randomization platform, and simple standardized case report forms.

DISCUSSION

The 3rGBM trial is a modern transparent care research framework with no additional risks, tests, or visits other than what patients would encounter in normal care. The burden of proof remains on repeat surgical management of recurrent GBM, because this management has yet to be shown beneficial. The trial is designed to help patients and surgeons manage the uncertainty regarding optimal care.

CLINICAL TRIAL REGISTRATION

http://www.clinicaltrials.gov. Unique identifier: NCT04838782.

Noninvasive Angiographic Results of Clipped or Coiled Intracranial Aneurysms: An Inter- and Intraobserver Reliability Study

BACKGROUND AND PURPOSE

Noninvasive angiography is commonly used to assess the outcome of surgical or endovascular treatment of intracranial aneurysms in clinical series or randomized trials. We sought to assess whether a standardized 3-grade classification system could be reliably used to compare the CTA and MRA results of both treatments.

MATERIALS AND METHODS

An electronic portfolio composed of CTAs of 30 clipped and MRAs of 30 coiled aneurysms was independently evaluated by 24 raters of diverse experience and training backgrounds. Twenty raters performed a second evaluation 1 month later. Raters were asked which angiographic grade and management decision (retreatment; close or long-term follow-up) would be most appropriate for each case. Agreement was analyzed using the Krippendorff α (α_K) statistic, and the relationship between angiographic grade and clinical management choice, using the Fisher exact and Cramer V tests.

RESULTS

Interrater agreement was substantial (α_K = 0.63; 95% CI, 0.55-0.70); results were slightly better for MRA results of coiling (α_K = 0.69; 95% CI, 0.56-0.76) than for CTA results of clipping (α_K = 0.58; 95% CI, 0.44-0.69). Intrarater agreement was substantial to almost perfect. Interrater agreement regarding clinical management was moderate for both clipped (α_K = 0.49; 95% CI, 0.32-0.61) and coiled subgroups (α_K = 0.47; 95% CI, 0.34-0.54). The choice of clinical management was strongly associated with the size of the residuum (mean Cramer V = 0.77 [SD, 0.14]), but complete occlusions (grade 1) were followed more closely after coiling than after clipping (P = .01).

CONCLUSIONS

A standardized 3-grade scale was found to be a reliable and clinically meaningful tool to compare the results of clipping and coiling of aneurysms using CTA or MRA.

Douglas Altman's 2009 Grand Lecture: Can we trust our literature?

Recent studies of the medical literature have revealed numerous and serious problems. Errors in the design, methods and interpretation of studies can frequently be identified. A huge hidden problem is publication bias, the tendency for positive articles to be published, while negative articles are either not written or submitted. This can systematically lead to an overestimation of the value of treatments, of diagnostic or prognostic studies. Even more worrisome is selective reporting: only a subset of a wide array of tested hypotheses are presented (the ones that turned out to be positive with significance testing). This is particularly true for secondary endpoints and subgroup findings, but even the primary endpoints of trials have been modified when publications are compared to protocols. The peer-review process is fallible. Even if it were strengthened, reviewers cannot examine what is not reported. Hence many problems can only be mitigated with better reporting. Numerous initiatives have proposed guidelines to promote transparent reporting, but progress is slow.

Clinical Uncertainty and Equipoise in the Management of Recurrent Glioblastoma

BACKGROUND

A significant proportion of glioblastoma (GBM) patients are considered for repeat resection, but evidence regarding best management remains elusive. Our aim was to measure the degree of clinical uncertainty regarding reoperation for patients with recurrent GBM.

METHODS

We first performed a systematic review of agreement studies examining the question of repeat resection for recurrent GBM. An electronic portfolio of 37 pathologically confirmed recurrent GBM patients including pertinent magnetic resonance images and clinical information was assembled. To measure clinical uncertainty, 26 neurosurgeons from various countries, training backgrounds, and years' experience were asked to select best management (repeat surgery, other nonsurgical management, or conservative), confidence in recommended management, and whether they would include the patient in a randomized trial comparing surgery with nonsurgical options. Agreement was evaluated using κ statistics.

RESULTS

The literature review did not reveal previous agreement studies examining the question. In our study, agreement regarding best management of recurrent GBM was slight, even when management options were dichotomized (repeat surgery vs. other options; κ=0.198 [95% confidence interval: 0.133-0.276]). Country of practice, years' experience, and training background did not change results. Disagreement and clinical uncertainty were more pronounced within clinicians with (κ=0.167 [0.055-0.314]) than clinicians without neuro-oncology fellowship training (κ=0.601 [0.556-0.646]). A majority (51%) of responders were willing to include the patient in a randomized trial comparing repeat surgery with nonsurgical alternatives in 26/37 (69%) of cases.

CONCLUSION

There is sufficient uncertainty and equipoise regarding the question of reoperation for patients with recurrent glioblastoma to support the need for a randomized controlled trial.