1
|
Meade MH, Buchan L, Michael M, Woods B. The Fragility Index: Understanding Its Application in Clinical Research. Clin Spine Surg 2024; 37:337-339. [PMID: 39037066 DOI: 10.1097/bsd.0000000000001668] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/19/2022] [Accepted: 06/28/2024] [Indexed: 07/23/2024]
Abstract
With the vast increase in spinal surgery research and accessibility, critical evaluation of studies is paramount. Historically, P values and confidence intervals have been the gold standard, but more recently, the inclusion of the Fragility Index has brought a more holistic approach. The Fragility Index aims to communicate the robustness of a trial and how tenuous statistical significance may be. It can be used in conjunction with more traditional methods for evaluating research.
Collapse
Affiliation(s)
- Matthew H Meade
- Division of Orthopaedic Surgery, Rowan University, Stratford, NJ
| | - Levi Buchan
- Division of Orthopaedic Surgery, Rowan University, Stratford, NJ
| | - Mark Michael
- Division of Orthopaedic Surgery, Rowan University, Stratford, NJ
| | - Barrett Woods
- The Rothman Institute at Thomas Jefferson University, Division of Orthopaedic Spine Surgery, Philadelphia, PA
| |
Collapse
|
2
|
Omar WM, Khader IRA, Hani SB, ALBashtawy M. The Glasgow Coma Scale and Full Outline of Unresponsiveness score evaluation to predict patient outcomes with neurological illnesses in intensive care units in West Bank: a prospective cross-sectional study. Acute Crit Care 2024; 39:408-419. [PMID: 39266276 PMCID: PMC11392694 DOI: 10.4266/acc.2024.00570] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/01/2024] [Accepted: 07/21/2024] [Indexed: 09/14/2024] Open
Abstract
BACKGROUND Determining the clinical neurological state of the patient is essential for making decisions and forecasting results. The Glasgow Coma Scale and the Full Outline of Unresponsiveness (FOUR) Scale are commonly used tools for measuring behavioral consciousness. This study aims to compare scales among patients with neurological disorders in intensive care units (ICUs) in the West Bank. METHODS A prospective cross-sectional design was employed. All patients admitted to ICUs who met inclusion criteria were involved in this study. Data were collected from from An-Najah National University, Al-Watani, and Rafedia Hospital. Both tools were used to collect data. RESULTS A total of 84 patients were assessed, 69.0% of the patients were male, and the average length of stay was 6.4 days. The mean score on the Glasgow Coma scale was 11.2 on admission 11.6 after 48 hours, and 12.2 on discharge. The mean FOUR Scale score was 12.2 on admission, 12.4 after 48 hours, and 12.5 at discharge. CONCLUSIONS This study indicates that both the Glasgow Coma Scale and the FOUR scale are effective in predicting outcomes for neurologically deteriorated critically ill patients. However, the FOUR scale proved to be more reliable when assessing outcomes in ICU patients.
Collapse
Affiliation(s)
| | | | - Salam Bani Hani
- Department of Nursing, Irbid National University, Irbid, Jordan
| | - Mohammed ALBashtawy
- Department of Community and Mental Health, Princess Salma Faculty of Nursing, Al al-Bayt University, Mafraq, Jordan
| |
Collapse
|
3
|
Muthu S, Ramakrishnan E. Fragility Analysis of Statistically Significant Outcomes of Randomized Control Trials in Spine Surgery: A Systematic Review. Spine (Phila Pa 1976) 2021; 46:198-208. [PMID: 32756285 DOI: 10.1097/brs.0000000000003645] [Citation(s) in RCA: 18] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/05/2023]
Abstract
STUDY DESIGN Systematic review. OBJECTIVES The aim of this study was to assess the robustness of statistically significant outcomes from randomized control trials (RCTs) in spine surgery using Fragility Index (FI) which is a novel metric measuring the number of events upon which statistical significance of the outcome depends. SUMMARY OF BACKGROUND DATA Many trials in Spine surgery were characterized by fewer outcome events along with small sample size. FI helps us identify the robustness of the results from such studies with statistically significant dichotomous outcomes. METHODS We conducted independent and in duplicate, a systematic review of published RCTs in spine surgery from PubMed Central, Embase, and Cochrane Database. RCTs with 1:1 prospective study design and reporting statistically significant dichotomous primary or secondary outcomes were included. FI was calculated for each RCT and its correlation with various factors was analyzed. RESULTS Seventy trials met inclusion criteria with a median sample size of 133 (interquartile range [IQR]: 80-218) and median reported events per trial was 38 (IQR: 13-94). The median FI score was 2 (IQR: 0-5), which means if we switch two patients from nonevent to event, the statistical significance of the outcome is lost. The FI score was less than the number of patients lost to follow-up in 28 of 70 trials. The FI score was found to positively correlated with sample size (r = 0.431, P = 0.001), total number of outcome events (r = 0.305, P = 0.01) while negatively correlated with P value (r = -0.392, P = 0.001). Funding, journal impact-factor, risk of bias domains, and year of publication did not have a significant correlation. CONCLUSION Statistically significant dichotomous outcomes reported in spine surgery RCTs are more often fragile and outcomes of the patients lost to follow-up could have changed the significance of results and hence it needs caution before transcending their results into clinical application. The addition of FI in routine reporting of RCTs would guide readers on the robustness of the statistical significance of outcomes. RCTs with FI ≥5 without any patient lost to follow-up can be considered to have clinically robust results.Level of Evidence: 1.
Collapse
Affiliation(s)
- Sathish Muthu
- Government Hospital, Velayuthampalayam, Karur, Tamil Nadu, India
| | - Eswar Ramakrishnan
- Institute of Orthopaedics and Traumatology, Madras Medical College & Rajiv Gandhi Government General Hospital, Chennai, Tamil Nadu, India
| |
Collapse
|
4
|
Evaniew N, Sharifi B, Waheed Z, Fallah N, Ailon T, Dea N, Paquette S, Charest-Morin R, Street J, Fisher CG, Dvorak MF, Noonan VK, Rivers CS, Kwon BK. The influence of neurological examination timing within hours after acute traumatic spinal cord injuries: an observational study. Spinal Cord 2019; 58:247-254. [DOI: 10.1038/s41393-019-0359-0] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2019] [Revised: 09/09/2019] [Accepted: 09/10/2019] [Indexed: 11/09/2022]
|
5
|
|
6
|
Important considerations in calculating and reporting of sample size in randomized controlled trials. Med J Islam Repub Iran 2017; 31:127. [PMID: 29951427 PMCID: PMC6014761 DOI: 10.14196/mjiri.31.127] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/01/2017] [Indexed: 11/18/2022] Open
Abstract
Background: The calculation of the sample size is one of the most important steps in designing a randomized controlled trial. The
purpose of this study is drawing the attention of researchers to the importance of calculating and reporting the sample size in randomized
controlled trials.
Methods: We reviewed related literature and guidelines and discussed some important issues in sample size calculation and reporting
in randomized controlled trials.
Conclusion: The calculation of the sample size is one of the most important steps in designing a randomized controlled trial. According
to the CONSORT (Consolidated Standards of Reporting Trials) guideline and other standard guidelines for designing and
reporting of RCTs, sample size calculations should be reported and justified in all published RCTs. Because sample size calculations
are prone to bias and because of the high ethical and financial costs related to conducting an RCT, we recommend involving a biostatistician
at the designing stage of the study and to ask for statistical advice for sample size calculations.
Collapse
|
7
|
Wibault J, Öberg B, Dedering Å, Löfgren H, Zsigmond P, Persson L, Andell M, R. Jonsson M, Peolsson A. Neck-Related Physical Function, Self-Efficacy, and Coping Strategies in Patients With Cervical Radiculopathy: A Randomized Clinical Trial of Postoperative Physiotherapy. J Manipulative Physiol Ther 2017; 40:330-339. [DOI: 10.1016/j.jmpt.2017.02.012] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2015] [Revised: 03/10/2016] [Accepted: 03/30/2016] [Indexed: 12/13/2022]
|
8
|
The fragility of statistically significant findings from randomized trials in spine surgery: a systematic survey. Spine J 2015; 15:2188-97. [PMID: 26072464 DOI: 10.1016/j.spinee.2015.06.004] [Citation(s) in RCA: 152] [Impact Index Per Article: 16.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/28/2015] [Revised: 04/27/2015] [Accepted: 06/01/2015] [Indexed: 02/03/2023]
Abstract
BACKGROUND CONTEXT Randomized controlled trials (RCTs) are the most trustworthy source for evaluating treatment effects, but RCTs of spine surgery interventions often produce discordant results. The Fragility Index is a novel metric to inform about the robustness of statistically significant results. PURPOSE The aim was to determine the robustness of statistically significant results from RCTs of spine surgery interventions. STUDY DESIGN/SETTING This was a systematic survey. PATIENT SAMPLE The sample included RCTs of spine surgery interventions. OUTCOME MEASURES The Fragility Index is the minimum number of patients in a trial whose status would have to change from a nonevent to an event to change a statistically significant result to a nonsignificant result. Events refer to the occurrence of any dichotomous outcome, such as successful fusion, incident fracture, adjacent segment degeneration, or achievement of a certain functional score. A small Fragility Index indicates that the statistical significance of a result hinges on only a few events, and a large Fragility Index increases one's confidence in the observed treatment effects. METHODS We systematically reviewed a database for evidence-based orthopedics and identified all the RCTs that reported at least one positive outcome (ie, p<.05). Two reviewers independently assessed eligibility and extracted data. We used the Fisher exact test to compute Fragility Index values and multivariable linear regression to evaluate potential associated factors. RESULTS We identified 40 eligible RCTs with a median sample size of 132 patients (interquartile range [IQR] 79-208) and a median total number of outcome events for the chosen outcome of 31 (IQR 13-63). The median Fragility Index was two (IQR 1-3), which means that adding two events to one of the trial's treatment arms eliminated its statistical significance. The Fragility Index was less than or equal to three events in 75% of the trials, and was less than or equal to the number of patients lost to follow-up in 65% of the trials. Fragility Index values correlated positively with total sample size (r=0.35; p<.05). When adjusted for losses to follow-up and risk of bias, increasing Fragility Index values were associated only with increasingly significant reported p values (p<.01). CONCLUSIONS Statistically significant results in spine surgery RCTs are frequently fragile. The addition of only a small number of outcome events can completely eliminate significance. Surgeons, researchers, and other evidence users should exercise caution when interpreting the findings from RCTs with low Fragility Index values and applying these results to patient care.
Collapse
|
9
|
Dvorak MF, Noonan VK, Fallah N, Fisher CG, Rivers CS, Ahn H, Tsai EC, Linassi AG, Christie SD, Attabib N, Hurlbert RJ, Fourney DR, Johnson MG, Fehlings MG, Drew B, Bailey CS, Paquet J, Parent S, Townson A, Ho C, Craven BC, Gagnon D, Tsui D, Fox R, Mac-Thiong JM, Kwon BK. Minimizing errors in acute traumatic spinal cord injury trials by acknowledging the heterogeneity of spinal cord anatomy and injury severity: an observational Canadian cohort analysis. J Neurotrauma 2014; 31:1540-7. [PMID: 24811484 DOI: 10.1089/neu.2013.3278] [Citation(s) in RCA: 57] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/26/2023] Open
Abstract
Clinical trials of therapies for acute traumatic spinal cord injury (tSCI) have failed to convincingly demonstrate efficacy in improving neurologic function. Failing to acknowledge the heterogeneity of these injuries and under-appreciating the impact of the most important baseline prognostic variables likely contributes to this translational failure. Our hypothesis was that neurological level and severity of initial injury (measured by the American Spinal Injury Association Impairment Scale [AIS]) act jointly and are the major determinants of motor recovery. Our objective was to quantify the influence of these variables when considered together on early motor score recovery following acute tSCI. Eight hundred thirty-six participants from the Rick Hansen Spinal Cord Injury Registry were analyzed for motor score improvement from baseline to follow-up. In AIS A, B, and C patients, cervical and thoracic injuries displayed significantly different motor score recovery. AIS A patients with thoracic (T2-T10) and thoracolumbar (T11-L2) injuries had significantly different motor improvement. High (C1-C4) and low (C5-T1) cervical injuries demonstrated differences in upper extremity motor recovery in AIS B, C, and D. A hypothetical clinical trial example demonstrated the benefits of stratifying on neurological level and severity of injury. Clinically meaningful motor score recovery is predictably related to the neurological level of injury and the severity of the baseline neurological impairment. Stratifying clinical trial cohorts using a joint distribution of these two variables will enhance a study's chance of identifying a true treatment effect and minimize the risk of misattributed treatment effects. Clinical studies should stratify participants based on these factors and record the number of participants and their mean baseline motor scores for each category of this joint distribution as part of the reporting of participant characteristics. Improved clinical trial design is a high priority as new therapies and interventions for tSCI emerge.
Collapse
Affiliation(s)
- Marcel F Dvorak
- 1 Department of Orthopedics, University of British Columbia , Vancouver, British Columbia, Canada
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
Collapse
|
10
|
Giuffrida MA. Type II error and statistical power in reports of small animal clinical trials. J Am Vet Med Assoc 2014; 244:1075-80. [DOI: 10.2460/javma.244.9.1075] [Citation(s) in RCA: 33] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022]
|
11
|
Chan AW, Tetzlaff JM, Gøtzsche PC, Altman DG, Mann H, Berlin JA, Dickersin K, Hróbjartsson A, Schulz KF, Parulekar WR, Krleza-Jeric K, Laupacis A, Moher D. SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials. BMJ 2013; 346:e7586. [PMID: 23303884 PMCID: PMC3541470 DOI: 10.1136/bmj.e7586] [Citation(s) in RCA: 3384] [Impact Index Per Article: 307.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 10/04/2012] [Indexed: 02/06/2023]
Abstract
High quality protocols facilitate proper conduct, reporting, and external review of clinical trials. However, the completeness of trial protocols is often inadequate. To help improve the content and quality of protocols, an international group of stakeholders developed the SPIRIT 2013 Statement (Standard Protocol Items: Recommendations for Interventional Trials). The SPIRIT Statement provides guidance in the form of a checklist of recommended items to include in a clinical trial protocol. This SPIRIT 2013 Explanation and Elaboration paper provides important information to promote full understanding of the checklist recommendations. For each checklist item, we provide a rationale and detailed description; a model example from an actual protocol; and relevant references supporting its importance. We strongly recommend that this explanatory paper be used in conjunction with the SPIRIT Statement. A website of resources is also available (www.spirit-statement.org). The SPIRIT 2013 Explanation and Elaboration paper, together with the Statement, should help with the drafting of trial protocols. Complete documentation of key trial elements can facilitate transparency and protocol review for the benefit of all stakeholders.
Collapse
Affiliation(s)
- An-Wen Chan
- Women's College Research Institute at Women's College Hospital, Department of Medicine, University of Toronto, Toronto, Canada M5G 1N8
| | | | | | | | | | | | | | | | | | | | | | | | | |
Collapse
|
12
|
|
13
|
Hanekom SD, Brooks D, Denehy L, Fagevik-Olsén M, Hardcastle TC, Manie S, Louw Q. Reaching consensus on the physiotherapeutic management of patients following upper abdominal surgery: a pragmatic approach to interpret equivocal evidence. BMC Med Inform Decis Mak 2012; 12:5. [PMID: 22309427 PMCID: PMC3395830 DOI: 10.1186/1472-6947-12-5] [Citation(s) in RCA: 18] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2010] [Accepted: 02/06/2012] [Indexed: 02/07/2023] Open
Abstract
BACKGROUND Postoperative pulmonary complications remain the most significant cause of morbidity following open upper abdominal surgery despite advances in perioperative care. However, due to the poor quality primary research uncertainty surrounding the value of prophylactic physiotherapy intervention in the management of patients following abdominal surgery persists. The Delphi process has been proposed as a pragmatic methodology to guide clinical practice when evidence is equivocal. METHODS The objective was to develop a clinical management algorithm for the post operative management of abdominal surgery patients. Eleven draft algorithm statements extracted from the extant literature by the primary research team were verified and rated by scientist clinicians (n=5) in an electronic three round Delphi process. Algorithm statements which reached a priori defined consensus-semi-interquartile range (SIQR)<0.5-were collated into the algorithm. RESULTS The five panelists allocated to the abdominal surgery Delphi panel were from Australia, Canada, Sweden, and South Africa. The 11 draft algorithm statements were edited and 5 additional statements were formulated. The panel reached consensus on the rating of all statements. Four statements were rated essential. CONCLUSION An expert Delphi panel interpreted the equivocal evidence for the physiotherapeutic management of patients following upper abdominal surgery. Through a process of consensus a clinical management algorithm was formulated. This algorithm can now be used by clinicians to guide clinical practice in this population.
Collapse
Affiliation(s)
- Susan D Hanekom
- Department of Interdisciplinary Health Sciences, Division of Physiotherapy, Faculty of Health Sciences, Stellenbosch University, Francie van Zyl Drive, Tygerberg 7505 South Africa
| | - Dina Brooks
- Department of Physical Therapy 160-500 University Avenue, Toronto, Ontario M5G 1V7 Canada
| | - Linda Denehy
- Department of Physiotherapy, The University of Melbourne, Parkville Melbourne, 3010 Australia
| | - Monika Fagevik-Olsén
- Department of Physical Therapy, Sahlgrenska University Hospital, Gothenburg, 413 45, Sweden
| | - Timothy C Hardcastle
- Trauma Surgery and Trauma ICU, Inkosi Albert Luthuli central Hospital & University of KwaZulu-Natal 800 Bellair Rd Mayville Durban 4058 South Africa
| | - Shamila Manie
- Department of Health and Rehabilitation Sciences, Division of Physiotherapy, University of Cape Town, Old Main Building, Groote Schuur Hospital, Observatory Cape Town 7925 South Africa
| | - Quinette Louw
- Department of Interdisciplinary Health Sciences, Division of Physiotherapy, Faculty of Health Sciences, Stellenbosch University, Francie van Zyl Drive, Tygerberg 7505 South Africa
| |
Collapse
|
14
|
José-Antonio SS, Baabor-Aqueveque M, Silva-Morales F. Philosophy and concepts of modern spine surgery. ACTA NEUROCHIRURGICA. SUPPLEMENT 2011; 108:23-31. [PMID: 21107934 DOI: 10.1007/978-3-211-99370-5_5] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/30/2023]
Abstract
The main goal of improving pain and neurological deficit in the practice of spine surgery is changing for a more ambitious goal, namely to improve the overall quality of life and the future of patients through three major actions (1) preserving the vertebral anatomical structures; (2) preserving the paravertebral anatomical structures; and (3) preserving the functionality of the segment. Thus, three new concepts have emerged (a) minimal surgery; (b) minimal access surgery; and (c) motion preservation surgery. These concepts are covered in a new term, minimally invasive spine surgery (MISS) The term "MISS" is not about one or several particular surgical techniques, but a new way of thinking, a new philosophy. Although the development of minimally invasive spine surgery is recent, its application includes all spine segments and almost all the existing conditions, including deformities.Evidence-based medicine (EBM), a term coined by Alvan Feinstein in the 1960s (Feinstein A (1964) Annals of Internal Medicine 61: 564-579; Feinstein A (1964) Annals of Internal Medicine 61: 757-781; Feinstein A (1964) Annals of Internal Medicine 61: 944-965; Feinstein A (1964) Annals of Internal Medicine 61: 1162-1193.), emphasizes the possibility of combining art and science following the strict application of scientific methods in the treatment of patients (Feinstein A (1964) Annals of Internal Medicine 61: 944-965; Feinstein A (1964) Annals of Internal Medicine 61: 1162-1193.), which may represent the advantages of objectivity and rationality in the use of different treatments (Fig. 11). However, EBM has many obvious defects, especially in spine surgery it is almost impossible to develop double-blind protocols (Andersson G, Bridwell K, Danielsson A, et al (2007) Spine 32: S64-S65.). In most cases, the only evidence one can find in the literature is the lack of evidence (Resnick D (2007) Spine 32:S15-S19.), however, the lack of evidence does not mean its absence. Only then, with a rigorous self-analysis, we may take a clear path towards a new philosophy in spine surgery. Of course, feedback from patients through satisfaction and clinical scales can guide our direction and provide the energy needed to maintain the enthusiasm (Fig. 12).
Collapse
Affiliation(s)
- Soriano-Sánchez José-Antonio
- The American British Cowdray Medical Center, Avenue Carlos Graef Fernández no. 154 Consultorio 106 y 107 Tlaxala Santa Fe, Delegación Cuajimalpa, CP, 05300, México D.F.
| | | | | |
Collapse
|
15
|
P value and the theory of hypothesis testing: an explanation for new researchers. Clin Orthop Relat Res 2010; 468:885-92. [PMID: 19921345 PMCID: PMC2816758 DOI: 10.1007/s11999-009-1164-4] [Citation(s) in RCA: 89] [Impact Index Per Article: 6.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/11/2009] [Accepted: 11/02/2009] [Indexed: 01/31/2023]
Abstract
In the 1920s, Ronald Fisher developed the theory behind the p value and Jerzy Neyman and Egon Pearson developed the theory of hypothesis testing. These distinct theories have provided researchers important quantitative tools to confirm or refute their hypotheses. The p value is the probability to obtain an effect equal to or more extreme than the one observed presuming the null hypothesis of no effect is true; it gives researchers a measure of the strength of evidence against the null hypothesis. As commonly used, investigators will select a threshold p value below which they will reject the null hypothesis. The theory of hypothesis testing allows researchers to reject a null hypothesis in favor of an alternative hypothesis of some effect. As commonly used, investigators choose Type I error (rejecting the null hypothesis when it is true) and Type II error (accepting the null hypothesis when it is false) levels and determine some critical region. If the test statistic falls into that critical region, the null hypothesis is rejected in favor of the alternative hypothesis. Despite similarities between the two, the p value and the theory of hypothesis testing are different theories that often are misunderstood and confused, leading researchers to improper conclusions. Perhaps the most common misconception is to consider the p value as the probability that the null hypothesis is true rather than the probability of obtaining the difference observed, or one that is more extreme, considering the null is true. Another concern is the risk that an important proportion of statistically significant results are falsely significant. Researchers should have a minimum understanding of these two theories so that they are better able to plan, conduct, interpret, and report scientific experiments.
Collapse
|
16
|
The use of confidence intervals in reporting orthopaedic research findings. Clin Orthop Relat Res 2009; 467:3334-9. [PMID: 19333667 PMCID: PMC2772925 DOI: 10.1007/s11999-009-0817-7] [Citation(s) in RCA: 33] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/17/2008] [Accepted: 03/16/2009] [Indexed: 01/31/2023]
Abstract
Conflict between clinical importance and statistical significance is an important problem in medical research. Although clinical importance is best described by asking for the effect size or how much, statistical significance can only suggest whether there is any difference. One way to combine statistical significance and effect sizes is to report confidence intervals. We therefore assessed the reporting of confidence intervals in the orthopaedic literature and factors influencing this frequency. In parallel, we tested the predictive value of statistical significance for effect size. In a random sample of predetermined size, we found one in five orthopaedic articles reported confidence intervals. Participation of an individual trained in research methods increased the odds of doing so fivefold. The use of confidence intervals was independent of impact factor, year of publication, and significance of outcomes. The probability of statistically significant results to predict at least a 10% between-group difference was only 69% (95% confidence interval, 55%-83%), suggesting that a high proportion of statistically significant results do not reflect large treatment effects. Confidence intervals could help avoid such erroneous interpretation by showing the effect size explicitly.
Collapse
|
17
|
Porzsolt F, Pressel H, Maute-Stephan C, Kindervater R, Geldmacher J, Meierkord S, Sigle JM, Eisemann M. Appraisal of health care: from patient value to societal benefit. J Public Health (Oxf) 2009. [DOI: 10.1007/s10389-009-0294-1] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022] Open
|
18
|
Tsang R, Colley L, Lynd LD. Inadequate statistical power to detect clinically significant differences in adverse event rates in randomized controlled trials. J Clin Epidemiol 2009; 62:609-16. [PMID: 19013761 DOI: 10.1016/j.jclinepi.2008.08.005] [Citation(s) in RCA: 98] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/18/2008] [Revised: 08/15/2008] [Accepted: 08/18/2008] [Indexed: 12/29/2022]
Affiliation(s)
- Ruth Tsang
- Vancouver General Hospital, Vancouver, British Columbia, Canada
| | | | | |
Collapse
|
19
|
Biau DJ, Kernéis S, Porcher R. Statistics in brief: the importance of sample size in the planning and interpretation of medical research. Clin Orthop Relat Res 2008; 466:2282-8. [PMID: 18566874 PMCID: PMC2493004 DOI: 10.1007/s11999-008-0346-9] [Citation(s) in RCA: 207] [Impact Index Per Article: 12.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/01/2007] [Accepted: 05/22/2008] [Indexed: 01/31/2023]
Abstract
The increasing volume of research by the medical community often leads to increasing numbers of contradictory findings and conclusions. Although the differences observed may represent true differences, the results also may differ because of sampling variability as all studies are performed on a limited number of specimens or patients. When planning a study reporting differences among groups of patients or describing some variable in a single group, sample size should be considered because it allows the researcher to control for the risk of reporting a false-negative finding (Type II error) or to estimate the precision his or her experiment will yield. Equally important, readers of medical journals should understand sample size because such understanding is essential to interpret the relevance of a finding with regard to their own patients. At the time of planning, the investigator must establish (1) a justifiable level of statistical significance, (2) the chances of detecting a difference of given magnitude between the groups compared, ie, the power, (3) this targeted difference (ie, effect size), and (4) the variability of the data (for quantitative data). We believe correct planning of experiments is an ethical issue of concern to the entire community.
Collapse
Affiliation(s)
- David Jean Biau
- Département de Biostatistique et Informatique Médicale, INSERM-UMR-S 717, AP-HP, Université Paris 7, Hôpital Saint Louis, Paris Cedex 10, 75475 France.
| | | | | |
Collapse
|
20
|
Yuen SY, Pope JE. Learning from past mistakes: assessing trial quality, power and eligibility in non-renal systemic lupus erythematosus randomized controlled trials. Rheumatology (Oxford) 2008; 47:1367-72. [PMID: 18577549 DOI: 10.1093/rheumatology/ken230] [Citation(s) in RCA: 17] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/08/2023] Open
Abstract
OBJECTIVES To evaluate the post hoc study power of randomized controlled trials (RCTs) in the treatment of non-renal SLE and to determine the generalizability of these RCTs using an SLE database. METHODS RCTs in non-renal SLE were identified using PubMed (1975-2007). Inclusion/exclusion criteria, trial quality (5-point scale) and results of each study were recorded. The inclusion/exclusion criteria were compared with an SLE database to determine the proportion of patients from the database who would theoretically be eligible for these trials. For each negative study, we calculated the post hoc study power. We also looked for temporal improvements of trials in the literature and examined if pharmaceutical involvement influenced trial quality. RESULTS Sixty-four articles were included; the mean power of 30 negative studies was 24.6 +/- s.e.m. 3.9% (range 2.5-81.1%). Only one study had a power > 80%. Overall, potential eligibility of SLE patients in the database was 45.1 +/- s.e.m. 3.6%. Only 14 studies (21.9%) were of good quality. Fortunately, RCT quality is improving over time (trials <1995, compared with 1996-2002 and >2003; P < 0.001). Trials with pharmaceutical involvement had a significantly higher number of enrollees and better study quality. CONCLUSIONS Negative RCTs in SLE were mostly underpowered but the generalizability of these trials was high. Determination of study power and the impact of eligibility criteria on generalizability of study results are crucial in the design of clinical trials to ensure applicability to clinical practice.
Collapse
Affiliation(s)
- S Y Yuen
- St Joseph's Health Care London, 268 Grosvenor Street, Box 5777, London, ON N6A 4V2, Canada.
| | | |
Collapse
|
21
|
Post RB, van der Sluis CK, Leferink VJM, Dijkstra PU, ten Duis HJ. Nonoperatively treated type A spinal fractures: mid-term versus long-term functional outcome. INTERNATIONAL ORTHOPAEDICS 2008; 33:1055-60. [PMID: 18548248 PMCID: PMC2898974 DOI: 10.1007/s00264-008-0593-0] [Citation(s) in RCA: 9] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/29/2008] [Revised: 04/22/2008] [Accepted: 04/23/2008] [Indexed: 11/25/2022]
Abstract
This study focuses on the mid-term (four years) and long-term (ten years) functional outcome of patients treated nonoperatively for a type A spinal fracture without primary neurological deficit. Functional outcome was measured using the visual analogue scale spine score (VAS) and the Roland–Morris disability questionnaire (RMDQ). The 50 patients included were on average 41.2 years old at the time of injury. Four years post injury, a mean VAS score of 74.5 and a mean RMDQ score of 4.9 were found. Ten years after the accident, the mean VAS and RMDQ scores were 72.6 and 4.7, respectively (NS). No significant relationships were found between the difference scores of the VAS and RMDQ compared with age, gender, fracture sub-classification, and time between measurements. Three (6%) patients had a poor long-term outcome. None of the patients required surgery for late onset pain or progressive neurological deficit. Functional outcome after a nonoperatively treated type A spinal fracture is good, both four and ten years post injury. For the group as a whole, four years after the fracture a steady state exists in functional outcome, which does not change for ten years at least after the fracture.
Collapse
Affiliation(s)
- R. B. Post
- Centre for Rehabilitation, University Medical Centre Groningen, P.O. Box 30.001, 9700 RB Groningen, The Netherlands
| | - C. K. van der Sluis
- Centre for Rehabilitation, University Medical Centre Groningen, P.O. Box 30.001, 9700 RB Groningen, The Netherlands
- Share Graduate School for Health Care Research, University of Groningen, Groningen, The Netherlands
| | - V. J. M. Leferink
- Department of Surgery, Alysis Health Care Organization, Zevenaar Hospital, Zevenaar, The Netherlands
| | - P. U. Dijkstra
- Centre for Rehabilitation, University Medical Centre Groningen, P.O. Box 30.001, 9700 RB Groningen, The Netherlands
| | - H. J. ten Duis
- Department of Surgery, University Medical Centre Groningen, Groningen, The Netherlands
| |
Collapse
|
22
|
Soucacos PN, Johnson EO, Babis G. Randomised controlled trials in orthopaedic surgery and traumatology: overview of parameters and pitfalls. Injury 2008; 39:636-42. [PMID: 18533154 DOI: 10.1016/j.injury.2008.02.011] [Citation(s) in RCA: 23] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/03/2007] [Accepted: 02/06/2008] [Indexed: 02/02/2023]
Abstract
There is a growing consensus that randomised controlled clinical trial (RCT) provide a secure basis for determining treatment effects. Prospective randomised clinical trials can be a powerful tool in medical science and evidence-based medicine. A well-defined study hypothesis, with a prospectively applied study design, blinded and randomised treatment allocation and assessment, with appropriate control groups can provide strong evidence in support of treatment decisions. However, the recent reviews of the medical literature indicate that the study design itself does not ensure the quality of science or useful and valid scientific data. Thus, regardless of the study design or level of evidence, it remains imperative for the physician and surgeon to critically evaluate a scientific report. Moreover, as randomisation, concealment of treatment allocation and blinding are difficult issues to resolve in orthopaedic surgery, future trials should focus on detailed and correct reporting of outcome measures.
Collapse
Affiliation(s)
- Panayotis N Soucacos
- Department of Orthopaedic Surgery, University of Athens, School of Medicine, K.A.T Accident Hospital, Athens, Greece.
| | | | | |
Collapse
|
23
|
Sung J, Siegel J, Tornetta P, Bhandari M. The orthopaedic trauma literature: an evaluation of statistically significant findings in orthopaedic trauma randomized trials. BMC Musculoskelet Disord 2008; 9:14. [PMID: 18230147 PMCID: PMC2254414 DOI: 10.1186/1471-2474-9-14] [Citation(s) in RCA: 16] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/06/2007] [Accepted: 01/29/2008] [Indexed: 11/10/2022] Open
Abstract
Background Evidence-based medicine posits that health care research is founded upon clinically important differences in patient centered outcomes. Statistically significant differences between two treatments may not necessarily reflect a clinically important difference. We aimed to quantify the sample sizes and magnitude of treatment effects in a review of orthopaedic randomized trials with statistically significant findings. Methods We conducted a comprehensive search (PubMed, Cochrane) for all randomized controlled trials between 1/1/95 to 12/31/04. Eligible studies include those that focused upon orthopaedic trauma. Baseline characteristics and treatment effects were abstracted by two reviewers. Briefly, for continuous outcome measures (ie functional scores), we calculated effect sizes (mean difference/standard deviation). Dichotomous variables (ie infection, nonunion) were summarized as absolute risk differences and relative risk reductions (RRR). Effect sizes >0.80 and RRRs>50% were defined as large effects. Using regression analysis we examined the association between the total number of outcome events and treatment effect (dichotomous outcomes). Results Our search yielded 433 randomized controlled trials (RCTs), of which 76 RCTs with statistically significant findings on 184 outcomes (122 continuous/62 dichotomous outcomes) met study eligibility criteria. The mean effect size across studies with continuous outcome variables was 1.7 (95% confidence interval: 1.43–1.97). For dichotomous outcomes, the mean risk difference was 30% (95%confidence interval:24%–36%) and the mean relative risk reduction was 61% (95% confidence interval: 55%–66%; range: 0%–97%). Fewer numbers of total outcome events in studies was strongly correlated with increasing magnitude of the treatment effect (Pearson's R = -0.70, p < 0.01). When adjusted for sample size, the number of outcome events revealed an independent association with the size of the treatment effect (Odds ratio = 50, 95% confidence interval: 3.0–1000, p = 0.006). Conclusion Our review suggests that statistically significant results in orthopaedic trials have the following implications-1) On average large risk reductions are reported 2) Large treatment effects (>50% relative risk reduction) are correlated with few number of total outcome events. Readers should interpret the results of such small trials with these issues in mind.
Collapse
Affiliation(s)
- Jinsil Sung
- Department of Surgery, McMaster University, 293 Wellington Street N, Suite 110, Hamilton, Ontario, L8L 8E7, Canada.
| | | | | | | |
Collapse
|
24
|
Abstract
STUDY DESIGN : Literature review. OBJECTIVE : To outline the components and application of evidence-based medicine (EBM) with an emphasis on the critical components of conduct and appraisal of clinical research. SUMMARY OF BACKGROUND DATA : "Evidence-based medicine" is now a commonplace phrase representing the hallmark of excellence in clinical practice. EBM integrates a question, thoughtful comprehensive evaluation of the pertinent literature, with clinical experience and patient preference to make optimal patient care decisions. These decisions must be evaluated with objective outcome measures to ensure effectiveness. There have been some misconceptions around the application of EBM and that it is synonymous with randomized controlled trials (RCTs) or based purely on levels of evidence. METHODS : Narrative and review of literature. CONCLUSION : Clinicians must understand the importance of the research question, study design, and outcomes in order to apply the best available research to patient care. Treatment recommendations evolving from critical appraisal are not only based on levels of evidence, but the risk benefit ratio and cost. The true philosophy of EBM, however, is not for research to supplant individual clinical experience and the patient's informed preference, but to integrate them with the best available research. Healthcare professionals and administrators must grasp that EBM is not a RCT. They must realize that the question being asked and the research circumstances dictate the study design. Furthermore, they must not diminish the role of clinical expertise and informed patient preference in EBM.
Collapse
Affiliation(s)
- Charles G Fisher
- Combined Neurosurgical and Orthopaedic Spine Program, Department of Orthopaedic Surgery, University of British Columbia, Vancouver, British Columbia, Canada.
| | | |
Collapse
|
25
|
Cunningham MRA, Warme WJ, Schaad DC, Wolf FM, Leopold SS. Industry-funded positive studies not associated with better design or larger size. Clin Orthop Relat Res 2007; 457:235-41. [PMID: 17195818 DOI: 10.1097/blo.0b013e3180312057] [Citation(s) in RCA: 22] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/31/2023]
Abstract
Previous studies have associated commercial funding with positive outcomes in orthopaedic research. Those reports, however, failed to account for potential confounding variables that can lead to a disproportion of positive outcomes, including sample size, study design, and study quality. We tested the hypothesis that nonscientific factors (funding source, orthopaedic subspecialty, and geographic region of origin) are associated with positive study outcomes, but not the result of differences in study design, study quality, or sample size. All 747 abstracts presented at the 2004 American Academy of Orthopaedic Surgeons annual meeting underwent blinded analysis using previously published criteria. Studies that received commercial funding were more likely to conclude with positive outcomes. Subspecialty and country of origin were not associated with positive outcomes. Commercially funded studies were not more likely than non-funded studies to be well-designed. When control groups were used, those in commercially funded studies were not larger than those used in nonfunded studies. Our data suggest commercial funding was associated with positive outcomes, but we found no evidence to suggest commercially funded studies were better designed or larger than non-funded studies.
Collapse
Affiliation(s)
- Mary R A Cunningham
- Department of Orthopaedics, University of Washington Medical Center, Seattle, WA 98195, USA
| | | | | | | | | |
Collapse
|
26
|
Abstract
STUDY DESIGN An evidence-based review and summary of literature from multiple disciplines involved in spine trauma. OBJECTIVES To outline epidemiologic, clinical, and research issues influencing spine trauma in a longitudinal perspective. In addition, to provide guidance to clinicians and researchers to ensure that philosophies pertaining to the betterment of spine trauma care are understood and supported. SUMMARY OF BACKGROUND DATA Epidemiologic data have provided insight into future demands the elderly patient with spine injury will place on the health care system. Regional trauma programs have emerged with further specialization resulting in regionalized spine trauma care. Evidence-based guidelines have streamlined imaging, and biomaterial advancements have facilitated the stabilization of the spinal column and decompression of the spinal cord. Promising experimental therapies promoting axonal regeneration and neuroprotective agents are beginning clinical trials, generating cautious optimism that effective therapies for spinal cord injuries will emerge. The unsustainable economics of increasing technology and patient expectations will make economic evaluation critical. METHODS Evidence-based review of current literature and expert opinion. CONCLUSIONS Multicenter spine trauma registries with patient-reported outcomes will allow many questions around spine trauma to be answered using the highest levels of evidence. This process in synergy with technical and biologic developments should ensure progress toward optimal care of the spine trauma patient. Future challenges will be to treat the breadth and magnitude of the discoveries within the fiscal restraints of the health care system and ensure its affordability for society.
Collapse
Affiliation(s)
- Charles G Fisher
- Combined Neurosurgical and Orthopaedic Spine Program, Department of Orthopaedics, University of British Columbia, Vancouver Hospital and Health Sciences Centre, Vancouver, British Columbia, Canada.
| | | | | |
Collapse
|
27
|
Thomas KC, Bailey CS, Dvorak MF, Kwon B, Fisher C. Comparison of operative and nonoperative treatment for thoracolumbar burst fractures in patients without neurological deficit: a systematic review. J Neurosurg Spine 2006; 4:351-8. [PMID: 16703901 DOI: 10.3171/spi.2006.4.5.351] [Citation(s) in RCA: 98] [Impact Index Per Article: 5.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
ObjectDespite extensive published research on thoracolumbar burst fractures, controversy still surrounds which is the most appropriate treatment. The objective of this study was to evaluate the scientific literature on operative and nonoperative treatment of patients with thoracolumbar burst fractures and no neurological deficit.MethodsIn their search of the literature, the authors identified all possible relevant studies concerning thoracolumbar burst fracture without neurological deficit. Two independent observers performed study selection, methodological quality assessment, and data extraction in a blinded and objective manner for all papers identified during the search. In a synthesis of the literature, the authors obtained evidence for both operative and nonoperative treatments.ConclusionsThere is a lack of evidence demonstrating the superiority of one approach over the other as measured using generic and disease-specific health-related quality of life scales. There is no scientific evidence linking posttraumatic kyphosis to clinical outcomes. The authors found that there is a strong need for improved clinical research methodology to be applied to this patient population.
Collapse
Affiliation(s)
- Kenneth C Thomas
- Vancouver Hospital and Health Sciences Centre, University of British Columbia, Vancouver, British Columbia, Canada
| | | | | | | | | |
Collapse
|
28
|
Abstract
There are several issues that one has to take into consideration in order to avoid potential pitfalls in the design of orthopaedic studies. This article highlights how to avoid common errors and how to continue the drive towards the unattainable, but laudable, goal of perfection.
Collapse
Affiliation(s)
- Christopher M Bono
- Boston University School of Medicine, Boston University Medical Center, MA 02118, USA.
| | | |
Collapse
|
29
|
Brox JI, Reikerås O, Sørensen R, Riise R, Nygård Ø, Ingebrigtsen T, Keller A, Indahl A, Holm I, Friis A, Koller AK, Eriksen H. Re: Hägg O, Fritzell P. Letter. Spine 2003; 29: 1160-1. Spine (Phila Pa 1976) 2004; 29:2088-9. [PMID: 15371716 DOI: 10.1097/01.brs.0000138564.88529.8f] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/01/2023]
|