Measuring clinical uncertainty as a preliminary step to randomized controlled trials

The Management of Persistent Distal Occlusions after Mechanical Thrombectomy and Thrombolysis: An Inter- and Intrarater Agreement Study

BACKGROUND AND PURPOSE

The best management of patients with persistent distal occlusion after mechanical thrombectomy with or without IV thrombolysis remains unknown. We sought to evaluate the variability and agreement in decision-making for persistent distal occlusions.

MATERIALS AND METHODS

A portfolio of 60 cases was sent to clinicians with varying backgrounds and experience. Responders were asked whether they considered conservative management or rescue therapy (stent retriever, aspiration, or intra-arterial thrombolytics) a treatment option as well as their willingness to enroll patients in a randomized trial. Agreement was assessed using κ statistics.

RESULTS

The electronic survey was answered by 31 physicians (8 vascular neurologists and 23 interventional neuroradiologists). Decisions for rescue therapies were more frequent (n = 1116/1860, 60%) than for conservative management (n = 744/1860, 40%; P < .001). Interrater agreement regarding the final management decision was "slight" (κ = 0.12; 95% CI, 0.09-0.14) and did not improve when subgroups of clinicians were studied according to background, experience, and specialty or when cases were grouped according to the level of occlusion. On delayed re-questioning, 23 of 29 respondents (79.3%) disagreed with themselves on at least 20% of cases. Respondents were willing to offer trial participation in 1295 of 1860 (69.6%) cases.

CONCLUSIONS

Individuals did not agree regarding the best management of patients with persistent distal occlusion after mechanical thrombectomy and IV thrombolysis. There is sufficient uncertainty to justify a dedicated randomized trial.

Surgical or Endovascular Treatment of MCA Aneurysms: An Agreement Study

BACKGROUND AND PURPOSE

MCA aneurysms are still commonly clipped surgically despite the recent development of a number of endovascular tools and techniques. We measured clinical uncertainty by studying the reliability of decisions made for patients with middle cerebral artery (MCA) aneurysms.

MATERIALS AND METHODS

A portfolio of 60 MCA aneurysms was presented to surgical and endovascular specialists who were asked whether they considered surgery or endovascular treatment to be an option, whether they would consider recruitment of the patient in a randomized trial, and whether they would provide their final management recommendation. Agreement was studied using κ statistics. Intrarater reliability was assessed with the same, permuted portfolio of cases of MCA aneurysm sent to the same specialists 1 month later.

RESULTS

Surgical management was the preferred option for neurosurgeons (n = 844/1320; [64%] responses/22 raters), while endovascular treatment was more commonly chosen by interventional neuroradiologists (1149/1500 [76.6%] responses/25 raters). Interrater agreement was only "slight" for all cases and all judges (κ = 0.094; 95% CI, 0.068-0.130). Agreement was no better within specialties or with more experience. On delayed requestioning, 11 of 35 raters (31%) disagreed with themselves on at least 20% of cases. Surgical management and endovascular treatment were always judged to be a treatment option, for all patients. Trial participation was offered to patients 65% of the time.

CONCLUSIONS

Individual clinicians did not agree regarding the best management of patients with MCA aneurysms. A randomized trial comparing endovascular with surgical management of patients with MCA aneurysms is in order.

Clinical Uncertainty and Equipoise in the Management of Recurrent Glioblastoma

BACKGROUND

A significant proportion of glioblastoma (GBM) patients are considered for repeat resection, but evidence regarding best management remains elusive. Our aim was to measure the degree of clinical uncertainty regarding reoperation for patients with recurrent GBM.

METHODS

We first performed a systematic review of agreement studies examining the question of repeat resection for recurrent GBM. An electronic portfolio of 37 pathologically confirmed recurrent GBM patients including pertinent magnetic resonance images and clinical information was assembled. To measure clinical uncertainty, 26 neurosurgeons from various countries, training backgrounds, and years' experience were asked to select best management (repeat surgery, other nonsurgical management, or conservative), confidence in recommended management, and whether they would include the patient in a randomized trial comparing surgery with nonsurgical options. Agreement was evaluated using κ statistics.

RESULTS

The literature review did not reveal previous agreement studies examining the question. In our study, agreement regarding best management of recurrent GBM was slight, even when management options were dichotomized (repeat surgery vs. other options; κ=0.198 [95% confidence interval: 0.133-0.276]). Country of practice, years' experience, and training background did not change results. Disagreement and clinical uncertainty were more pronounced within clinicians with (κ=0.167 [0.055-0.314]) than clinicians without neuro-oncology fellowship training (κ=0.601 [0.556-0.646]). A majority (51%) of responders were willing to include the patient in a randomized trial comparing repeat surgery with nonsurgical alternatives in 26/37 (69%) of cases.

CONCLUSION

There is sufficient uncertainty and equipoise regarding the question of reoperation for patients with recurrent glioblastoma to support the need for a randomized controlled trial.

Practicing outcome-based medical care using pragmatic care trials

The current separation between medical research and care is an obstacle to essential aspects of good medical practice: the verification that care interventions actually deliver the good outcomes they promise, and the use of scientific methods to optimize care under uncertainty. Pragmatic care trials have been designed to address these problems. Care trials are all-inclusive randomized trials integrated into care. Every item of trial design is selected in the best medical interest of participating patients. Care trials can eventually show what constitutes good medical practice based on patient outcomes. In the meantime, care trials give clinicians and patients the scientific methods necessary for optimization of medical care when no one really knows what to do.We report the progress of 9 randomized care trials that were used to guide the endovascular or surgical management of 1212 patients with acute stroke, intracranial aneurysms, and arteriovenous malformations in a single center in an elective or acute care context. Care trials were used to address long-standing dilemmas regarding rival medical, surgical, or endovascular management options or to offer innovative instead of standard treatments. The trial methodology, by replacing unrepeatable treatment decisions by 1:1 randomized allocation whenever reliable knowledge was not available, had an immediate impact, transforming unverifiable dogmatic medical practice into verifiable outcome-based medical care. We believe the approach is applicable to all medical or surgical domains, but widespread adoption may require the revision of many currently prevalent views regarding the role of research in clinical practice.