1
|
Li KD, Venishetty N, Fernandez AM, Hakam N, Ghaffar U, Gupta S, Patel HV, Breyer BN. Fragility of overactive bladder medication clinical trials: A systematic review. Neurourol Urodyn 2024; 43:1523-1533. [PMID: 38594889 DOI: 10.1002/nau.25468] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/26/2024] [Accepted: 04/01/2024] [Indexed: 04/11/2024]
Abstract
PURPOSE Overactive bladder (OAB) syndrome significantly impairs quality of life, often necessitating pharmacological interventions with associated risks. The fragility of OAB trial outcomes, as measured by the fragility index (FI: smallest number of event changes to reverse statistical significance) and quotient (FQ: FI divided by total sample size expressed as a percentage), is critical yet unstudied. MATERIALS AND METHODS We conducted a systematic search for randomized controlled trials on OAB medications published between January 2000 and August 2023. Inclusion criteria were trials with two parallel arms reporting binary outcomes related to OAB medications. We extracted trial details, outcomes, and statistical tests employed. We calculated FI and FQ, analyzing associations with trial characteristics through linear regression. RESULTS We included 57 trials with a median sample size of 211 participants and a 12% median lost to follow-up. Most studies investigated anticholinergics (37/57, 65%). The median FI/FQ was 5/3.5%. Larger trials were less fragile (median FI 8; FQ 1.0%) compared to medium (FI: 4; FQ 2.5%) and small trials (FI: 4; FQ 8.3%). Double-blinded studies exhibited higher FQs (median 2.9%) than unblinded trials (6.7%). Primary and secondary outcomes had higher FIs (median 5 and 6, respectively) than adverse events (FI: 4). Each increase in 10 participants was associated with a +0.19 increase in FI (p < 0.001). CONCLUSIONS A change in outcome for a median of five participants, or 3.5% of the total sample size, could reverse the direction of statistical significance in OAB trials. Studies with larger sample sizes and efficacy outcomes from blinded trials were less fragile.
Collapse
Affiliation(s)
- Kevin D Li
- Department of Urology, University of California San Francisco, San Francisco, California, USA
- Department of Epidemiology and Biostatistics, University of California San Francisco, San Francisco, California, USA
| | - Nikit Venishetty
- Paul L. Foster School of Medicine, Texas Tech University Health Sciences Center, El Paso, Texas, USA
| | - Adrian M Fernandez
- Department of Urology, University of California San Francisco, San Francisco, California, USA
| | - Nizar Hakam
- Department of Urology, University of California San Francisco, San Francisco, California, USA
| | - Umar Ghaffar
- Department of Urology, University of California San Francisco, San Francisco, California, USA
| | - Shiv Gupta
- Department of Urology, University of California San Francisco, San Francisco, California, USA
| | - Hiren V Patel
- Department of Urology, University of California San Francisco, San Francisco, California, USA
| | - Benjamin N Breyer
- Department of Urology, University of California San Francisco, San Francisco, California, USA
- Department of Epidemiology and Biostatistics, University of California San Francisco, San Francisco, California, USA
| |
Collapse
|
2
|
Suresh NV, Go BC, Fritz CG, Harris J, Ahluwalia V, Xu K, Lu J, Rajasekaran K. The fragility index: how robust are the outcomes of head and neck cancer randomised, controlled trials? J Laryngol Otol 2024; 138:451-456. [PMID: 37795709 PMCID: PMC10950446 DOI: 10.1017/s0022215123001755] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/01/2022] [Revised: 08/12/2023] [Accepted: 08/29/2023] [Indexed: 10/06/2023]
Abstract
BACKGROUND The fragility index represents the minimum number of patients required to convert an outcome from statistically significant to insignificant. This report assesses the fragility index of head and neck cancer randomised, controlled trials. METHODS Studies were extracted from PubMed/Medline, Scopus, Embase and Cochrane databases. RESULTS Overall, 123 randomised, controlled trials were included. The sample size and fragility index medians (interquartile ranges) were 103 (56-213) and 2 (0-5), respectively. The fragility index exceeded the number of patients lost to follow up in 42.3 per cent (n = 52) of studies. A higher fragility index correlated with higher sample size (r = 0.514, p < 0.001), number of events (r = 0.449, p < 0.001) and statistical significance via p-value (r = -0.367, p < 0.001). CONCLUSION Head and neck cancer randomised, controlled trials demonstrated low fragility index values, in which statistically significant results could be nullified by altering the outcomes of just two patients, on average. Future head and neck oncology randomised, controlled trials should report the fragility index in order to provide insight into statistical robustness.
Collapse
Affiliation(s)
- Neeraj V Suresh
- Department of Otorhinolaryngology – Head and Neck Surgery, University of Pennsylvania, Philadelphia, PA, USA
- Department of Otolaryngology – Head and Neck Surgery, Yale University, New Haven, CT, USA
| | - Beatrice C Go
- Department of Otorhinolaryngology – Head and Neck Surgery, University of Pennsylvania, Philadelphia, PA, USA
| | - Christian G Fritz
- Department of Otorhinolaryngology – Head and Neck Surgery, University of Pennsylvania, Philadelphia, PA, USA
| | - Jacob Harris
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Vinayak Ahluwalia
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Katherine Xu
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Joseph Lu
- Sidney Kimmel Medical College at Thomas Jefferson University, Philadelphia, PA, USA
| | - Karthik Rajasekaran
- Department of Otorhinolaryngology – Head and Neck Surgery, University of Pennsylvania, Philadelphia, PA, USA
- Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, PA, USA
| |
Collapse
|
3
|
Hayes J, Zuercher M, Gai N, Chowdhury AR, Aoyama K. The Fragility Index of randomized controlled trials in pediatric anesthesiology. Can J Anaesth 2023; 70:1449-1460. [PMID: 37286747 DOI: 10.1007/s12630-023-02513-3] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2022] [Revised: 01/16/2023] [Accepted: 01/23/2023] [Indexed: 06/09/2023] Open
Abstract
PURPOSE The P value is a widely used measure of statistical importance but has many drawbacks and limitations, one being that it does not reflect the robustness of the results of a clinical trial. The Fragility Index (FI) was developed as a measure of how many outcome events would need to change to nonevents to render a significant P value nonsignificant (P ≥ 0.05). The FI of trials from other medical specialties is typically < 5. We aimed to determine the FI of pediatric anesthesiology randomized controlled trials (RCT) and to test for association with various characteristics of the included trials. METHODS We conducted a comprehensive systematic search of high-impact anesthesia, surgical, and medical journals from the last 25 years for trials comparing an intervention between two groups with a statistically significant P value (< 0.05) for a dichotomous outcome. We also compared FI values for variables that reflect the quality and importance of a trial. RESULTS The median [interquartile range] FI was 3 [1-7] and correlated positively with the number of participants (rS = 0.41; P < 0.001) and events (rS = 0.42; P < 0.001), and negatively with the P value (rPB = -0.36; P < 0.001). Other measures of trial quality and impact or importance were not strongly associated with the FI. CONCLUSIONS The FI of published trials in pediatric anesthesiology is similarly low as in other medical specialties. Larger trials with more events and P values ≤ 0.01 were associated with a higher FI.
Collapse
Affiliation(s)
- Jason Hayes
- Department of Anesthesia and Pain Medicine, The Hospital for Sick Children (SickKids), 555 University Avenue, Toronto, ON, M5G 1X8, Canada.
- Department of Anesthesiology and Pain Medicine, University of Toronto, Toronto, ON, Canada.
| | - Mael Zuercher
- Department of Anesthesia and Pain Medicine, The Hospital for Sick Children (SickKids), 555 University Avenue, Toronto, ON, M5G 1X8, Canada
| | - Nan Gai
- Department of Anesthesia and Pain Medicine, The Hospital for Sick Children (SickKids), 555 University Avenue, Toronto, ON, M5G 1X8, Canada
- Department of Anesthesiology and Pain Medicine, University of Toronto, Toronto, ON, Canada
| | - Apala Roy Chowdhury
- Department of Anesthesia and Pain Medicine, The Hospital for Sick Children (SickKids), 555 University Avenue, Toronto, ON, M5G 1X8, Canada
| | - Kazuyoshi Aoyama
- Department of Anesthesia and Pain Medicine, The Hospital for Sick Children (SickKids), 555 University Avenue, Toronto, ON, M5G 1X8, Canada
- Department of Anesthesiology and Pain Medicine, University of Toronto, Toronto, ON, Canada
- Program in Child Health Evaluative Sciences, SickKids Research Institute, Toronto, ON, Canada
| |
Collapse
|
4
|
Morris SC, Gowd AK, Agarwalla A, Phipatanakul WP, Amin NH, Liu JN. Fragility of statistically significant findings from randomized clinical trials of surgical treatment of humeral shaft fractures: A systematic review. World J Orthop 2022; 13:825-836. [PMID: 36189338 PMCID: PMC9516622 DOI: 10.5312/wjo.v13.i9.825] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/26/2021] [Revised: 02/28/2022] [Accepted: 08/17/2022] [Indexed: 02/06/2023] Open
Abstract
BACKGROUND Despite recent meta-analyses of randomized controlled trials (RCTs), there remains no consensus regarding the preferred surgical treatment for humeral shaft fractures. The fragility index (FI) is an emerging tool used to evaluate the robustness of RCTs by quantifying the number of participants in a study group that would need to switch outcomes in order to reverse the study conclusions.
AIM To investigate the fragility index of randomized control trials assessing outcomes of operative fixation in proximal humerus fractures.
METHODS We completed a systematic review of RCTs evaluating the surgical treatment of humeral shaft fractures. Inclusion criteria included: articles published in English; patients randomized and allotted in 1:1 ratio to 2 parallel arms; and dichotomous outcome variables. The FI was calculated for total complications, each complication individually, and secondary surgeries using the Fisher exact test, as previously published.
RESULTS Fifteen RCTs were included in the analysis comparing open reduction plate osteosynthesis with dynamic compression plate or locking compression plate, intramedullary nail, and minimally invasive plate osteosynthesis. The median FI was 0 for all parameters analyzed. Regarding individual outcomes, the FI was 0 for 81/91 (89%) of outcomes. The FI exceeded the number lost to follow up in only 2/91 (2%) outcomes.
CONCLUSION The FI shows that data from RCTs regarding operative treatment of humeral shaft fractures are fragile and does not demonstrate superiority of any particular surgical technique.
Collapse
Affiliation(s)
- Stephen Craig Morris
- Department of Orthopaedic Surgery, Loma Linda University, Loma Linda, CA 92354, United States
| | - Anirudh K Gowd
- Department of Orthopaedic Surgery, Wake Forest University Baptist Medical Center, Winston-Salem, NC 27157, United States
| | - Avinesh Agarwalla
- Department of Orthopaedic Surgery, Westchester Medical Center, Valhalla, NY 10595, United States
| | - Wesley P Phipatanakul
- Department of Orthopaedic Surgery, Loma Linda University, Loma Linda, CA 92354, United States
| | - Nirav H Amin
- Department of Orthopaedic Surgery, Premier Orthopaedic and Trauma Specialists, Pomona, CA 91767, United States
| | - Joseph N Liu
- Department of Orthopedic Surgery, USC Epstein Family Center for Sports Medicine, Los Angeles, CA 90089, United States
| |
Collapse
|
5
|
Li B, Kong I, McGrath M, Farrokhyar F, Braga LH. Evaluating the literature on preoperative androgen stimulation for hypospadias repair using the fragility index - can we trust observational studies? J Pediatr Urol 2021; 17:661-669. [PMID: 34518122 DOI: 10.1016/j.jpurol.2021.07.027] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/20/2021] [Revised: 07/24/2021] [Accepted: 07/27/2021] [Indexed: 10/20/2022]
Abstract
BACKGROUND Preoperative androgen stimulation (PAS) is typically used in hypospadias repair for patients with a proximal meatus or small glans size. Hypospadias PAS literature suffer from small sample sizes and lack of power to claim robust conclusions. Small changes in the number of events may completely change the statistical significance, making the conclusions drawn unreliable. Fragility index (FI) is the number of additional events needed to occur in either the control or experimental group to turn a statistically significant result to a non-significant result. The objective of the report was to assess the quality of available literature revolving around PAS use in hypospadias repair and its effects on post-operative complication rates using FI. METHODS A comprehensive search of MEDLINE, EMBASE, and grey literature (ESPU and SPU abstracts) was conducted to identify RCTs and observational studies investigating the effect of PAS on complications post-hypospadias repair between 1990 and 2020. The FI was calculated for each study. Postoperative complications were defined as: fistula, stricture/stenosis, diverticula, and dehiscence. The odds ratio (OR), 95% confidence intervals (CI), corresponding p-values was calculated for each study. A random effects mixed model was implemented to combine the ORs for each study design. RESULTS Fourteen studies qualified for inclusion, of which nine were observational studies and five were RCTs (Figure 1). The median sample size was 110 patients (IQR 69-171). The summary ORs for observational studies was 1.74 (95% CI: 1.10 to 2.74; p = 0.020) and for RCTs was 0.71 (9% CI: 0.34 to 1.47; p = 0.350). The median FI was 0 (IQR 0-2) of the included studies. DISCUSSION PAS use does not appear to significantly affect complication rates shown in RCTs, however, observational studies cumulatively suggested significantly greater odds of complications after PAS. The FI is best used for RCTs with 1-to-1 randomization and binary data. Observational studies are rarely balanced for demographics and comorbidities with unequal sample size between comparable groups. The study was limited by substantial variability in how PAS was delivered to patients, leading to restricted comparability. CONCLUSION Strong conclusions regarding the influence of PAS on hypospadias repair outcomes cannot be properly drawn based on the current literature due to deficits from either a statistical or methodological standpoint. The current PAS literature has shown inconclusive results, calling for well-designed RCTs, involving standardized surgical techniques and PAS protocols, to evaluate the true effect of PAS on complications post-hypospadias repair.
Collapse
Affiliation(s)
- Bruce Li
- Michael G. DeGroote School of Medicine, McMaster University, Hamilton, ON, Canada
| | - Isaac Kong
- Michael G. DeGroote School of Medicine, McMaster University, Hamilton, ON, Canada
| | - Melissa McGrath
- Department of Health Research, Methods, Evidence & Impact, McMaster University, Hamilton, ON, Canada; McMaster Pediatric Surgical Research Collaborative, McMaster University, Hamilton, ON, Canada
| | - Forough Farrokhyar
- Department of Health Research, Methods, Evidence & Impact, McMaster University, Hamilton, ON, Canada
| | - Luis H Braga
- Department of Health Research, Methods, Evidence & Impact, McMaster University, Hamilton, ON, Canada; Division of Urology, McMaster University, Hamilton, ON, Canada; McMaster Pediatric Surgical Research Collaborative, McMaster University, Hamilton, ON, Canada.
| |
Collapse
|
6
|
Ngah VD, Mazingisa AV, Zunza M, Wiysonge CS. A Review of Adherence and Predictors of Adherence to the CONSORT Statement in the Reporting of Tuberculosis Vaccine Trials. Vaccines (Basel) 2020; 8:E770. [PMID: 33339360 PMCID: PMC7766843 DOI: 10.3390/vaccines8040770] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2020] [Revised: 11/18/2020] [Accepted: 11/19/2020] [Indexed: 12/20/2022] Open
Abstract
The statement on Consolidated Standards of Reporting Trials (CONSORT) ensures transparency in the reporting of randomized trials. However, it is unclear if the statement has led to improvement in the quality of reporting of tuberculosis (TB) vaccine trials. We explored the quality of reporting of TB vaccine trials according to the latest version of the CONSORT statement, released in 2010. We searched PubMed and the Cochrane Central Register of Controlled Trials in August 2019. We conducted screening, study selection, and data extraction in duplicate; and resolved differences through discussion. We assessed reporting to be adequate if trials reported at least 75% of the CONSORT 2010 items. We conducted a trend analysis to assess if there was improvement in reporting over time. We also used logistic regression to assess factors associated with adequate reporting. We included 124 trials in the analyses. The mean proportion of adherence was 67.3% (95% confidence interval 64.4% to 70.1%), with only 46 (37%) trials having adequate reporting. There was a significant improvement in the quality of reporting over time (p < 0.0001). Trials published in journals with impact factors between 10 and 20 were more likely to have adequate reporting (odds ratio 9.4; 95% confidence interval 1.30 to 67.8), compared to lower-impact-factor journals. Despite advances over time, the reporting of TB vaccine trials is still inadequate and requires improvement.
Collapse
Affiliation(s)
- Veranyuy D. Ngah
- Division of Epidemiology and Biostatistics, Department of Global Health, Faculty of Medicine and Health Sciences, Stellenbosch University, Francie van Zijl Drive, Tygerberg, 7505 Cape Town, South Africa; (M.Z.); (C.S.W.)
- Cochrane South Africa, South African Medical Research Council, Francie van Zijl Drive, Parow Valley, 7501 Cape Town, South Africa;
| | - Akhona V. Mazingisa
- Cochrane South Africa, South African Medical Research Council, Francie van Zijl Drive, Parow Valley, 7501 Cape Town, South Africa;
| | - Moleen Zunza
- Division of Epidemiology and Biostatistics, Department of Global Health, Faculty of Medicine and Health Sciences, Stellenbosch University, Francie van Zijl Drive, Tygerberg, 7505 Cape Town, South Africa; (M.Z.); (C.S.W.)
| | - Charles S. Wiysonge
- Division of Epidemiology and Biostatistics, Department of Global Health, Faculty of Medicine and Health Sciences, Stellenbosch University, Francie van Zijl Drive, Tygerberg, 7505 Cape Town, South Africa; (M.Z.); (C.S.W.)
- Cochrane South Africa, South African Medical Research Council, Francie van Zijl Drive, Parow Valley, 7501 Cape Town, South Africa;
- School of Public Health and Family Medicine, University of Cape Town, Anzio Road, Observatory, 7925 Cape Town, South Africa
| |
Collapse
|
7
|
Albersheim J, Smith DW, Pariser JJ, Dahm P. The reporting quality of randomized controlled trials and experimental animal studies for urethroplasty. World J Urol 2020; 39:2677-2683. [PMID: 33175208 DOI: 10.1007/s00345-020-03501-8] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2020] [Accepted: 10/14/2020] [Indexed: 12/31/2022] Open
Abstract
OBJECTIVES To assess the reporting quality of randomized controlled trials and experimental animal studies examining urethroplasty in reconstructive urological surgery literature. METHODS We performed a comprehensive literature search to identify all urethroplasty-related RCTs examining humans as well as animal models. We used the Consolidated Standards of Reporting Trials (CONSORT) and the Animals in Research: Reporting in vivo Experiments (ARRIVE) guidelines to assess reporting quality. Two reviewers performed data abstraction independently and in duplicate. We then generated descriptive statistics including CONSORT (0-25) and ARRIVE (0-20) summary scores using the median and interquartile range. RESULTS Twenty studies were ultimately included; 14 randomized controlled trials and 6 experimental animal studies. All studies were two-armed, parallel group studies. Median sample sizes (and interquartile range) of the human and animal studies were 48.5 (31.8-53.8) and 18 (15.3-27.5), respectively. The median CONSORT and ARRIVE scores were 10.0 (8.75-12.63) and 7.97 (6.79-8.64), respectively. Human randomized controlled trials did not consistently report the method of allocation concealment (6/14; 42.9%), blinding (2/14; 14.3%), or discuss the generalizability of the results (6/14; 42.9%). Animal studies infrequently reported why a given animal model was used (1/6; 16.7%), how they were allocated to groups (0/6; 0%) or what the experimental primary and secondary outcomes were (0/6; 0%). CONCLUSIONS Urethroplasty literature is marked by a paucity of both randomized controlled trials and experimental design animal studies. The existing studies are inconsistently reported and are therefore of uncertain methodological quality.
Collapse
Affiliation(s)
- Jacob Albersheim
- Urology Section, Minneapolis VA Health Care System, 112D, One Veterans Drive, Minneapolis, MN, 55417, USA.,Department of Urology, University of Minnesota, Minneapolis, MN, USA
| | - Daniel W Smith
- Urology Section, Minneapolis VA Health Care System, 112D, One Veterans Drive, Minneapolis, MN, 55417, USA.,Department of Urology, University of Minnesota, Minneapolis, MN, USA
| | - Joseph J Pariser
- Department of Urology, University of Minnesota, Minneapolis, MN, USA
| | - Philipp Dahm
- Urology Section, Minneapolis VA Health Care System, 112D, One Veterans Drive, Minneapolis, MN, 55417, USA. .,Department of Urology, University of Minnesota, Minneapolis, MN, USA.
| |
Collapse
|
8
|
Rickard M, Keefe DT, Drysdale E, Erdman L, Hannick JH, Milford K, Santos JD, Mistry N, Koyle MA, Lorenzo AJ. Trends and relevance in the bladder and bowel dysfunction literature: PlumX metrics contrasted with fragility indicators. J Pediatr Urol 2020; 16:477.e1-477.e7. [PMID: 32684443 DOI: 10.1016/j.jpurol.2020.06.015] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/13/2020] [Revised: 05/26/2020] [Accepted: 06/12/2020] [Indexed: 12/31/2022]
Abstract
INTRODUCTION The concepts of fragility index (FI) and fragility quotient (FQ) have been previously described. PlumX metrics encompass online "footprints" of research in addition to traditional citations. Herein we explore PlumX metrics against the quality of BBD literature. OBJECTIVE To explore altmetrics against the quality of bladder and bowel dysfunction (BBD) literature. STUDY DESIGN A literature search was conducted using Pubmed, Medline, Embase for BBD and related terms. A total of 54,045 abstracts were screened, followed by 693 full text reviews and data extraction from 126. Studies were included if they reported on 2 groups being compared, had dichotomous outcomes, and had significant results. RESULTS The median FI score was 4 (0-500) and there were 20 studies which had a FI of 0. The FQ had a median of 0.04 (0-0.32). PlumX usage was 263 ± 540, captures were 45 ± 60 and social media attention was 2 ± 2. Overall, 42% of papers were clinical trials (RCTs). When compared to other study designs, we noted a significant difference in PlumX captures (57 ± 72 RCT vs. 35 ± 47 other; p = 0.03). RCTs had higher usage, social media engagement and citations however, the differences were not significant. H-Index had a significant correlation with FI (p = 0.036), however correlations for PlumX usage and captures, while modestly positive (0.04-0.10) for the FI and FQ, were not significant. A comparison of FI and FQ by topic can be reviewed in the Summary Table. DISCUSSION When considering the FI and FQ robustness indicators of the BBD literature, we found similarities when compared to other studies. It was reported that overall, the hydronephrosis literature was fragile with many studies requiring only a few events to nullify significance, regardless of the study design. Similarly, in a review of pediatric vesicoureteral reflux (VUR) clinical trials, results were also fragile. When comparing fragility measures to altmetric variables we noted that despite the growing popularity of altmetrics, citation counts, and h-indices remain the traditional measures to monitor research consumption. There has been a reported correlation between manuscript citation counts, author h-index, altmetrics measures in several specialties and across many domains of research including medical sciences, arts, and the humanities, however in the present study only weak correlations were noted. CONCLUSION The body of BBD comparative studies is fragile in keeping with other pediatric urology literature populations. Despite fragile results, RCTs generate slightly moreattention as measured by select PlumX metrics. These results suggest the need for including fragility measures in our literature, aiming to focus attention towards more robust articles.
Collapse
Affiliation(s)
- Mandy Rickard
- Division of Urology, Hospital for Sick Children and Department of Surgery, University of Toronto, Ontario, Canada.
| | - Daniel T Keefe
- Division of Urology, Hospital for Sick Children and Department of Surgery, University of Toronto, Ontario, Canada
| | - Erik Drysdale
- Center for Computational Medicine, Hospital for Sick Children, Toronto, Ontario, Canada
| | - Lauren Erdman
- Center for Computational Medicine, Hospital for Sick Children, Toronto, Ontario, Canada
| | - Jessica H Hannick
- Division of Pediatric Urology, UH Rainbow Babies and Children's Hospital, Cleveland, OH, USA
| | - Karen Milford
- Division of Urology, Hospital for Sick Children and Department of Surgery, University of Toronto, Ontario, Canada
| | - Joana Dos Santos
- Division of Urology, Hospital for Sick Children and Department of Surgery, University of Toronto, Ontario, Canada
| | - Niraj Mistry
- Department of Pediatrics, Hospital for Sick Children, Toronto, Ontario, Canada
| | - Martin A Koyle
- Division of Urology, Hospital for Sick Children and Department of Surgery, University of Toronto, Ontario, Canada
| | - Armando J Lorenzo
- Division of Urology, Hospital for Sick Children and Department of Surgery, University of Toronto, Ontario, Canada
| |
Collapse
|
9
|
The fragility of statistically significant findings from randomized controlled trials in pediatric appendicitis: A systematic review. J Pediatr Surg 2020; 55:800-804. [PMID: 32145976 DOI: 10.1016/j.jpedsurg.2020.01.030] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/08/2020] [Accepted: 01/25/2020] [Indexed: 12/22/2022]
Abstract
PURPOSE Randomized controlled trials (RCT) in pediatric appendicitis remain limited, and the robustness of available evidence is unknown. The aim of this study was to determine the fragility of results in pediatric appendicitis RCTs. METHODS A systematic search of Embase and MEDLINE was performed. Eligible studies were two-armed RCTs that included at least one statistically significant dichotomous outcome, had parallel-group allocation, and assessed pediatric patients (0-17) with a primary diagnosis of appendicitis. The Fragility Index (FI) for one statistically significant outcome per trial was calculated using a Fisher's exact test, with statistical significance set at p < 0.05. RESULTS Six studies were identified for inclusion. Studies included a median of 103 patients (interquartile range [IQR] 86-127), with a median of 18 (IQR 4.5-41.25) events for analyzed outcomes. The primary outcome variable was included in analysis for 4(67%) studies. The median FI across studies was 3 (IQR 0.75-4.25), with results ranging from 0 to 5. Results indicate that overall, converting 3 patients from non-events to events in a single trial arm would change the significant dichotomous outcome to nonsignificant. CONCLUSION The fragility of results in RCTs in pediatric appendicitis should be considered before clinical practice is changed. Investigators should consider reporting the FI alongside study results, as p-values alone may be misleading. TYPE OF STUDY Randomized Controlled Trial. LEVEL OF EVIDENCE Level I.
Collapse
|
10
|
Thompson E, Lai A, Morrey L, Borofsky MS, Dahm P. A Longitudinal Assessment of the Reporting Quality of Randomized Controlled Trials for Surgical Interventions to Treat Nephrolithiasis Over 16 Years (2002 to 2017). J Endourol 2020; 34:502-508. [PMID: 31984770 DOI: 10.1089/end.2019.0649] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/23/2022] Open
Abstract
Purpose: Transparently reported, high-quality randomized controlled trials (RCTs) play a critical role in guiding evidence based clinical practice and informing evidence-based guidelines in patients with nephrolithiasis. Prior studies have found reporting quality to be low. We performed this study to assess whether the reporting of RCTs has improved over time. Materials and Methods: This study was governed by an a priori protocol. We performed a systematic literature search for RCTs analyzing nephrolithiasis treatment. Selection of eligible studies and data abstraction were performed by two of three reviewers independently and in duplicate. We developed and pilot tested a data extraction checklist based on the Consolidated Standards of Reporting Trials (CONSORT) criteria on a scale of 0 to 25. Our primary outcome measure was the mean CONSORT score. We performed statistical hypothesis testing to compare scores between 2002-2006, 2007-2011, and 2012-2017. Results: A total of 203 studies (2002-06: 38; 2007-11: 64; 2012-17: 101) met inclusion criteria. The most common procedure types studied were percutaneous nephrolithotomy (35.1%), shockwave lithotripsy (25.4%), and ureteroscopy (22.9%). Asia contributed an increasing proportion of studies (25.6%, 44.6%, and 74.3%, respectively) in these three time periods. The main journals of publication were the Journal of Endourology (23.9%), the Journal of Urology (19.5%), and Urology (8.3%). The mean ± standard error of the CONSORT summary scores was 11.4 ± 0.4, (2002 to 2006), 12.1 ± 0.3, (2007 to 2011), and 13.3 ± 0.4 (p = 0.003) reflecting an increase by 1.92 (95% confidence interval: 0.86 - 2.98). Conclusions: The number of RCTs investigating the use of urologic devices to treat stone disease has substantially increased overtime. There has been a small improvement in reporting quality; however, this remains suboptimal overall. Increased efforts to promote the transparent reporting of RCTs in endourology are warranted.
Collapse
Affiliation(s)
- Ellen Thompson
- Department of Urology, University of Minnesota, Minneapolis, Minnesota, USA.,Urology Section, Minneapolis VA Health Care System, Minneapolis, Minnesota, USA
| | - Andrew Lai
- Department of Urology, University of Illinois, Chicago, Illinois, USA
| | - Luke Morrey
- Department of Urology, University of Illinois, Chicago, Illinois, USA
| | - Michael S Borofsky
- Department of Urology, University of Minnesota, Minneapolis, Minnesota, USA
| | - Philipp Dahm
- Department of Urology, University of Minnesota, Minneapolis, Minnesota, USA.,Urology Section, Minneapolis VA Health Care System, Minneapolis, Minnesota, USA
| |
Collapse
|
11
|
Liampas I, Chlinos A, Siokas V, Brotis A, Dardiotis E. Assessment of the reporting quality of RCTs for novel oral anticoagulants in venous thromboembolic disease based on the CONSORT statement. J Thromb Thrombolysis 2019; 48:542-553. [DOI: 10.1007/s11239-019-01931-9] [Citation(s) in RCA: 17] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/09/2023]
|