Use of the IDEAL framework in the urological literature: where are we in 2018?

Examining the application of the IDEAL framework in the reporting and evaluation of innovative invasive procedures: secondary qualitative analysis of a systematic review

OBJECTIVES

The development of new surgical procedures is fundamental to advancing patient care. The Idea, Developments, Exploration, Assessment and Long-term (IDEAL) framework describes study designs for stages of innovation. It can be difficult to apply due to challenges in defining and identifying innovative procedures. This study examined how the IDEAL framework is operationalised in real-world settings; specifically, the types of innovations evaluated using the framework and how authors justify their choice of IDEAL study design.

DESIGN

Secondary qualitative analysis of a systematic review.

DATA SOURCES

Citation searches (Web of Science and Scopus) identified studies following the IDEAL framework and citing any of the ten key IDEAL/IDEAL_D papers.

ELIGIBILITY CRITERIA

Studies of invasive procedures/devices of any design citing any of the ten key IDEAL/IDEAL_D papers.

DATA EXTRACTION AND SYNTHESIS

All relevant text was extracted. Three frameworks were developed, namely: (1) type of innovation under evaluation; (2) terminology used to describe stage of innovation and (3) reported rationale for IDEAL stage.

RESULTS

48 articles were included. 19/48 described entirely new procedures, including those used for the first time in a different clinical context (n=15/48), reported as IDEAL stage 2a (n=8, 53%). Terminology describing stage of innovation was varied, inconsistent and ambiguous and was not defined. Authors justified their choice of IDEAL study design based on limitations in published evidence (n=36) and unknown feasibility and safety (n=32) outcomes.

CONCLUSION

Identifying stage of innovation is crucial to inform appropriate study design and governance decisions. Authors' rationale for choice of IDEAL stage related to the existing evidence base or lack of sufficient outcome data for procedures. Stage of innovation was poorly defined with inconsistent descriptions. Further work is needed to develop methods to identify innovation to inform practical application of the IDEAL framework. Defining the concept of innovation in terms of uncertainty, risk and degree of evidence may help to inform decision-making.

The Evolution of Retzius-Sparing Robotic-Assisted Laparoscopic Prostatectomy: An IDEAL Perspective

PURPOSE

The Retzius-sparing (RS) approach represents an important surgical innovation in how robotic-assisted laparoscopic prostatectomy (RALP) is performed.The aim of this study was to examine to what extent its development has followed the Idea, Development, Evolution, Assessment and Long-term study (IDEAL) recommendations.

MATERIAL AND METHODS

We conducted a comprehensive literature search for studies up to the 18th of March 2021. Abstracted data-points included authorship, year of publication, study design, reported endpoints, and length-of follow-up. We mapped each study to the five IDEAL stages of surgical innovation using published criteria.

RESULTS

Of 415 references, 118 were included in our analysis. Five academic centers authored over 50% of all study reports, with the groups from Seoul (24; 20.3%), Milan (15; 12.7%) and Ninjang (10; 8.5%) being the main contributors. Approximately 40% of studies (50/118) were reported as full-text publications. Most of the reports mapped to retrospective studies (97/118; 82.2%) with approximately one-third (31/97; 32.0%) reporting the use of prospectively collected data. Cumulatively, 17,974 RS-RALP were reported on. Of those, 13,929 were unique cases. Approximately 23% of cases were reported in multiple publications (4,045/17,974). We mapped two, 12, and three studies to the Idea, Assessment and Long-term study stages, respectively, and zero to the Development and Evaluation stages.

CONCLUSIONS

Few reported studies followed the IDEAL stages for surgical innovation; none addressed the stages of Development and Evaluation. Future systematic, prospectively planned assessments would be helpful to refine the approach and address issues related to the surgical learning curve.

Solo-surgeon pure laparoscopic donor nephrectomy using passive camera holder: IDEAL stage 2a study

BACKGROUND

Solo-surgery can be defined as a practice of a surgeon operating alone using a camera holder, without other surgical members except for a scrub nurse. This study was designed to evaluate the feasibility and safety of solo-surgeon pure laparoscopic donor nephrectomy.

METHODS

The study protocol was approved by the Institutional Review Board of Asan Medical Center, Seoul, Korea. The brief study protocol was registered on the Clinical Research Information Service site of the Korea Centers for Disease Control and Prevention. Candidates fulfilling all inclusion and exclusion criteria were enrolled in the clinical trial and underwent solo-surgeon pure laparoscopic donor nephrectomy. The feasibility was assessed by the proportion of subjects who could undergo solo-surgeon pure laparoscopic donor nephrectomy without difficulty. The perioperative complications were identified to assess the safety of solo-surgeon pure laparoscopic donor nephrectomy.

RESULTS

Of the 47 potential candidates from November 2018 to August 2019, 40 were enrolled in the clinical trial and seven excluded due to declining participation. The feasibility of solo-surgeon pure laparoscopic donor nephrectomy was 100%, without an occasion of any difficulty requiring conversion to the human assisted pure laparoscopic donor nephrectomy. Fourteen intraoperative complications occurred in 10 patients. The most common intraoperative complication was spleen injury. Two of three cases classified as the Satava classification grade II were due to the incomplete stapling of endoscopic stapler. Seventy-eight postoperative complications occurred in 34 patients. The most common postoperative complication was nausea/vomiting and followed by aspartate aminotransferase/alanine aminotransferase elevation. Most postoperative complication was independent of the solo-surgery itself.

CONCLUSIONS

Solo-surgeon pure laparoscopic donor nephrectomy using passive camera holder is technically feasible. In terms of safety, it is necessary to adjust the scope of surgery performed alone. Trial Registration CRIS, KCT0003458. Registered 30/01/2019, Retrospectively registered, https://cris.nih.go.kr/cris/search/detailSearch.do/15868 .

Minimally invasive treatments for benign prostatic hyperplasia: a Cochrane network meta-analysis

OBJECTIVE

To assess the comparative effectiveness and ranking of minimally invasive treatments (MITs) for lower urinary tract symptoms (LUTS) in men with benign prostatic hyperplasia (BPH).

MATERIALS AND METHODS

We searched multiple databases up to 24 February 2021. We included randomized controlled trials assessing the following treatments: convective radiofrequency water vapour thermal therapy (WVTT; or Rezūm); prostatic arterial embolization (PAE); prostatic urethral lift (PUL; or Urolift); temporary implantable nitinol device (TIND); and transurethral microwave thermotherapy (TUMT) compared to transurethral resection of the prostate (TURP) or sham surgery. We performed a frequentist network meta-analysis.

RESULTS

We included 27 trials involving 3017 men. The overall certainty of the evidence of most outcomes according to GRADE was low to very low. Compared to TURP, we found that PUL and PAE may result in little to no difference in urological symptoms, while WVTT, TUMT and TIND may result in worse urological symptoms. MITs may result in little to no difference in quality of life, compared to TURP. MITs may result in a large reduction in major adverse events compared to TURP. We were uncertain about the effects of PAE and PUL on retreatment compared to TURP, however, TUMT may result in higher retreatment rates. We were very uncertain of the effects of MITs on erectile function and ejaculatory function. Among MITs, PUL and PAE had the highest likelihood of being the most efficacious for urinary symptoms and quality of life, TUMT for major adverse events, WVTT and TIND for erectile function and PUL for ejaculatory function. Excluding WVTT and TIND, for which there were only studies with short-term (3-month) follow-up, PUL had the highest likelihood of being the most efficacious for retreatment.

CONCLUSIONS

Minimally invasive treatments may result in similar or worse effects concerning urinary symptoms and quality of life compared to TURP at short-term follow-up.

Minimally invasive treatments for lower urinary tract symptoms in men with benign prostatic hyperplasia: a network meta-analysis

BACKGROUND

A variety of minimally invasive treatments are available as an alternative to transurethral resection of the prostate (TURP) for management of lower urinary tract symptoms (LUTS) in men with benign prostatic hyperplasia (BPH). However, it is unclear which treatments provide better results.

OBJECTIVES

Our primary objective was to assess the comparative effectiveness of minimally invasive treatments for lower urinary tract symptoms in men with BPH through a network meta-analysis. Our secondary objective was to obtain an estimate of relative ranking of these minimally invasive treatments, according to their effects.

SEARCH METHODS

We performed a comprehensive search of multiple databases (CENTRAL, MEDLINE, Embase, Scopus, Web of Science and LILACS), trials registries, other sources of grey literature, and conference proceedings, up to 24 February 2021. We had no restrictions on language of publication or publication status.

SELECTION CRITERIA

We included parallel-group randomized controlled trials assessing the effects of the following minimally invasive treatments, compared to TURP or sham treatment, on men with moderate to severe LUTS due to BPH: convective radiofrequency water vapor therapy (CRFWVT); prostatic arterial embolization (PAE); prostatic urethral lift (PUL); temporary implantable nitinol device (TIND); and transurethral microwave thermotherapy (TUMT).

DATA COLLECTION AND ANALYSIS

Two review authors independently screened the literature, extracted data, and assessed risk of bias. We performed statistical analyses using a random-effects model for pair-wise comparisons and a frequentist network meta-analysis for combined estimates. We interpreted them according to Cochrane methods. We planned subgroup analyses by age, prostate volume, and severity of baseline symptoms. We used risk ratios (RRs) with 95% confidence intervals (CIs) to express dichotomous data and mean differences (MDs) with 95% CIs to express continuous data. We used the GRADE approach to rate the certainty of evidence.

MAIN RESULTS

We included 27 trials involving 3017 men, mostly over age 50, with severe LUTS due to BPH. The overall certainty of evidence was low to very low due to concerns regarding bias, imprecision, inconsistency (heterogeneity), and incoherence. Based on the network meta-analysis, results for our main outcomes were as follows. Urologic symptoms (19 studies, 1847 participants): PUL and PAE may result in little to no difference in urologic symptoms scores (MD of International Prostate Symptoms Score [IPSS]) compared to TURP (3 to 12 months; MD range 0 to 35; higher scores indicate worse symptoms; PUL: 1.47, 95% CI -4.00 to 6.93; PAE: 1.55, 95% CI -1.23 to 4.33; low-certainty evidence). CRFWVT, TUMT, and TIND may result in worse urologic symptoms scores compared to TURP at short-term follow-up, but the CIs include little to no difference (CRFWVT: 3.6, 95% CI -4.25 to 11.46; TUMT: 3.98, 95% CI 0.85 to 7.10; TIND: 7.5, 95% CI -0.68 to 15.69; low-certainty evidence). Quality of life (QoL) (13 studies, 1459 participants): All interventions may result in little to no difference in the QoL scores, compared to TURP (3 to 12 months; MD of IPSS-QoL score; MD range 0 to 6; higher scores indicate worse symptoms; PUL: 0.06, 95% CI -1.17 to 1.30; PAE: 0.09, 95% CI -0.57 to 0.75; CRFWVT: 0.37, 95% CI -1.45 to 2.20; TUMT: 0.65, 95% CI -0.48 to 1.78; TIND: 0.87, 95% CI -1.04 to 2.79; low-certainty evidence). Major adverse events (15 studies, 1573 participants): TUMT probably results in a large reduction of major adverse events compared to TURP (RR 0.20, 95% CI 0.09 to 0.43; moderate-certainty evidence). PUL, CRFWVT, TIND and PAE may also result in a large reduction in major adverse events, but CIs include substantial benefits and harms at three months to 36 months; PUL: RR 0.30, 95% CI 0.04 to 2.22; CRFWVT: RR 0.37, 95% CI 0.01 to 18.62; TIND: RR 0.52, 95% CI 0.01 to 24.46; PAE: RR 0.65, 95% CI 0.25 to 1.68; low-certainty evidence). Retreatment (10 studies, 799 participants): We are uncertain about the effects of PAE and PUL on retreatment compared to TURP (12 to 60 months; PUL: RR 2.39, 95% CI 0.51 to 11.1; PAE: RR 4.39, 95% CI 1.25 to 15.44; very low-certainty evidence). TUMT may result in higher retreatment rates (RR 9.71, 95% CI 2.35 to 40.13; low-certainty evidence). Erectile function (six studies, 640 participants): We are very uncertain of the effects of minimally invasive treatments on erectile function (MD of International Index of Erectile Function [IIEF-5]; range 5 to 25; higher scores indicates better function; CRFWVT: 6.49, 95% CI -8.13 to 21.12; TIND: 5.19, 95% CI -9.36 to 19.74; PUL: 3.00, 95% CI -5.45 to 11.44; PAE: -0.03, 95% CI -6.38, 6.32; very low-certainty evidence). Ejaculatory dysfunction (eight studies, 461 participants): We are uncertain of the effects of PUL, PAE and TUMT on ejaculatory dysfunction compared to TURP (3 to 12 months; PUL: RR 0.05, 95 % CI 0.00 to 1.06; PAE: RR 0.35, 95% CI 0.13 to 0.92; TUMT: RR 0.34, 95% CI 0.17 to 0.68; low-certainty evidence). TURP is the reference treatment with the highest likelihood of being the most efficacious for urinary symptoms, QoL and retreatment, but the least favorable in terms of major adverse events, erectile function and ejaculatory function. Among minimally invasive procedures, PUL and PAE have the highest likelihood of being the most efficacious for urinary symptoms and QoL, TUMT for major adverse events, PUL for retreatment, CRFWVT and TIND for erectile function and PUL for ejaculatory function.

AUTHORS' CONCLUSIONS

Minimally invasive treatments may result in similar or worse effects concerning urinary symptoms and QoL compared to TURP at short-term follow-up. They may result in fewer major adverse events, especially in the case of PUL and PAE; resulting in better rankings for symptoms scores. PUL may result in fewer retreatments compared to other interventions, especially TUMT, which had the highest retreatment rates at long-term follow-up. We are very uncertain about the effects of these interventions on erectile function. There was limited long-term data, especially for CRFWVT and TIND. Future high-quality studies with more extended follow-up, comparing different, active treatment modalities, and adequately reporting critical outcomes relevant to patients, including those related to sexual function, could provide more information on the relative effectiveness of these interventions.

Newer Minimally Invasive Treatment Modalities to Treat Lower Urinary Tract Symptoms Attributed to Benign Prostatic Hyperplasia

Context

Several newer device-based procedures have recently become available for treating men with lower urinary tract symptoms attributed to benign prostatic hyperplasia, but their effectiveness remains uncertain.

Objective

To assess the longer-term comparative effectiveness (defined as >12 mo of follow-up) of the newer treatment modalities prostatic urethral lift (PUL), transurethral prostate convective radiofrequency water vapor (Rezūm), Aquablation, and prostatic arterial embolization (PAE).

Evidence acquisition

Ovid Medline, the Cochrane Central Register of Controlled Trials (CENTRAL), and the Agency for Healthcare Research and Quality databases were searched through September 30, 2019; hand searches of references of relevant studies were also performed. Eligible studies were randomized controlled trials (RCTs) published in English language. We excluded observational studies.

Evidence synthesis

One RCT (n = 91) found that patients undergoing PUL may be less likely to respond (risk ratio [RR] 0.8; 95% confidence interval [CI] 0.7-1.0; low certainty of evidence [CoE]) and have a higher mean International Prostate Symptom Score (IPSS; mean difference 6.1; 95% CI 2.2-10.0; low CoE) than those undergoing transurethral resection of the prostate (TURP). Among patients undergoing PAE, one small RCT (n = 30) reported similar IPSS response rates (RR 0.9; 95% CI 0.7-1.1; low CoE) and one trial (n = 107) found similar mean IPSS (-0.7; 95% CI -1.3 to 2.7; moderate CoE) scores to those among patients undergoing TURP. A single study on Aquablation reported 12 mo of follow-up only, and a single 3-mo trial compared Rezūm with sham treatment.

Conclusions

The current best evidence underlying these newer therapies is limited to few trials (PUL and PAE), short-term follow-up of 12 mo (Aquablation and Rezūm), or sham comparison only (Rezūm).

Patient summary

Evidence for four of the newer surgical treatments for men with an enlarged prostate is limited to few small trials with short-term follow-up; only one trial compared a new treatment modality with sham surgery.

The IDEAL Reporting Guidelines: A Delphi Consensus Statement Stage Specific Recommendations for Reporting the Evaluation of Surgical Innovation

OBJECTIVE

The aim of this study was to define reporting standards for IDEAL format studies.

BACKGROUND

The IDEAL Framework and Recommendations establish an integrated pathway for evaluation of new surgical techniques and complex therapeutic technologies. However guidance on implementation has been incomplete, and incorrect use is commonly seen. We describe the consensus development of reporting guidelines for the IDEAL stages, and plans for their dissemination and evaluation.

METHODS

Using the EQUATOR Network recommendations, participants with knowledge of IDEAL were surveyed to determine which IDEAL stages needed reporting guidelines. Draft checklists for stages 1, 2a, 2b, and 4 were subsequently developed by 3 researchers (N.B., A.H., P.M.), and revised through a 2-round Delphi consensus process. A final consensus teleconference resolved outstanding disagreements and clarified wording for checklist items.

RESULTS

Sixty-one participants completed the initial survey, a clear majority indicating that new reporting guidelines were needed for IDEAL Stage 1 (69.5%), Stage 2a (78%), Stage 2b (74.6%), and Stage 4 (66%). A proposed set of checklists was modified by survey participants in 2 online Delphi rounds (n = 54 and n = 47, respectively), resulting in a penultimate checklist for each stage. Fourteen expert working group members finalized the checklist items and successfully resolved any outstanding areas without agreement on a consensus call.

CONCLUSIONS

Participants familiar with IDEAL called for reporting guidelines for studies in all IDEAL stages except stage 3. The checklists developed have the potential to improve standards of reporting and thereby advance the quality of research on surgery and complex interventions and technologies, but require further evaluation in use.

The IDEAL framework in neurosurgery: a bibliometric analysis

BACKGROUND

The Idea, Development, Exploration, Assessment and Long-term study (IDEAL) framework was created to provide a structured way for assessing and evaluating novel surgical techniques and devices.

OBJECTIVES

The aim of this paper was to investigate the utilization of the IDEAL framework within neurosurgery, and to identify factors influencing implementation.

METHODS

A bibliometric analysis of the 7 key IDEAL papers on Scopus, PubMed, Embase, Web of Science, and Google Scholar databases (2009-2019) was performed. A second journal-specific search then identified additional papers citing the IDEAL framework. Publications identified were screened by two independent reviewers to select neurosurgery-specific articles.

RESULTS

The citation search identified 1336 articles. The journal search identified another 16 articles. Following deduplication and review, 51 relevant articles remained; 14 primary papers (27%) and 37 secondary papers (73%). Of the primary papers, 5 (36%) papers applied the IDEAL framework to their research correctly; two were aligned to the pre-IDEAL stage, one to the Idea and Development stages, and two to the Exploration stage. Of the secondary papers, 21 (57%) explicitly discussed the IDEAL framework. Eighteen (86%) of these were supportive of implementing the framework, while one was not, and two were neutral.

CONCLUSION

The adoption of the IDEAL framework in neurosurgery has been slow, particularly for early-stage neurosurgical techniques and inventions. However, the largely positive reviews in secondary literature suggest potential for increased use that may be achieved with education and publicity.