Opposing systematic reviews: the effects of two quality rating instruments on evidence regarding t'ai chi and bone mineral density in postmenopausal women

Quantitative Ultrasound Examination (QUS) of the Calcaneus in Long-Term Martial Arts Training on the Example of Long-Time Practitioners of Okinawa Kobudo/Karate Shorin-Ryu

Quantitative research of bone tissue related to physical activity (PA) and sport has a preventive dimension. Increasing the parameters of bone tissue strength, especially reaching the maximum value of peak bone strength in childhood, adolescence, and early adulthood due to practicing sports can contribute to maintaining bone health throughout life. Practicing martial arts (tai chi, traditional karate, judo, and boxing) can effectively improve the quality of bone and reduce the risk of falls and fractures. The study aimed to evaluate the calcaneus bones among Okinawa Kobudo/Karate Shorin-Ryu practitioners using the quantitative ultrasound method as an indicator for assessing bone fracture risk. Forty-four adult martial arts practitioners with a mean age of 36.4 participated in this study. Quantitative ultrasound (QUS) with a portable Bone Ultrasonometer was used in this study. Speed of sound (SOS), broadband ultrasound attenuation (BUA), and the stiffness index (SI) were measured. Subjects were assigned to two groups of black and color belts, according to the advancement in Kobudo/karate practice. The measurements of the SI, BUA, SOS, T-score, and Z-score were significantly higher in subjects from the advanced, long-term practice (black belts) (p < 0.05). The long-term martial arts training in traditional karate and Kobudo significantly impacts the parameters of the calcaneus quantitative ultrasound measurements. Significantly higher bone density was observed among the black belt holders. Long-term practice subjects achieved results far beyond the norm for their age groups. Further studies using non-invasive methods of bone quantification are needed to determine the specific conditions for preventing osteoporosis through physical activity, sports, and martial arts, particularly the duration of the activity, the magnitude of loads, and other related factors.

The Composite Quality Score (CQS) as a trial appraisal tool: inter-rater reliability and rating time

OBJECTIVE

To establish the CQS inter-rater reliability and rating time and to compare both against that of the Jadad scale and Cochrane's Risk of Bias Tool (ROBT).

MATERIAL AND METHODS

Four independent raters rated 45 trial reports. The inter-rater reliability was established by use of the Brennan-Prediger coefficient (BPC). The coefficients were compared using the two-sample z-test. Secondary analysis included comparison of the inter-rater reliability of the randomization component of all tools, as well as of the allocation concealment component of the CQS to that of the ROBT. The mean rating time with standard deviation (SD) for each tool was determined using one-way repeated measures analysis of variance. Post hoc comparisons were made using the Tukey-Kramer adjustment for three pair-wise multiple comparisons.

RESULTS

The inter-rater reliability was significantly higher for the CQS (BPC, 95% CI: 0.95, 0.87-1.00) compared to Jadad (0.70, 0.58-0.82) (adjusted p = 0.0005) and most components of ROBT. The mean (SD) time to complete the CQS (4.0 (1.0) min) did not differ significantly from that of the Jadad scale 4.8 (1.1) min (adjusted p = 0.11), but was significantly shorter compared to that of the ROBT 15.3 (5.9) min (adjusted p < 0.0001).

CONCLUSIONS

The results suggest the CQS to be a very reliable and fast trial appraisal tool. Clinical relevance The higher the inter-rater reliability, the higher the probability that trial results reflect therapeutic truth. The CQS will need to take further bias sources into consideration, in order to increase its utility.

A Critical Review of Grading Systems: Implications for Public Health Policy

Grading instruments are an important part of evidence-based medicine and are used to inform health policy and the development of clinical practice guidelines. They are extensively used in the development of clinical guidelines and the assessment of research publications, having particular impact on health care and policy sectors. The positive effects of using grading instruments are, however, potentially undermined by their misuse and a number of shortcomings. This review found eight key concerns about grading instruments: (1) lack of information on validity and reliability, (2) poor concurrent validity, (3) may not account for external validity, (4) may not be inherently logical, (5) susceptibility to subjectivity, (6) complex systems with inadequate instructions, (7) may be biased toward randomized controlled trial (RCT) studies, and (8) may not adequately address the variety of non-RCTs. This narrative review concludes that there is a need to take into account these criticisms and domain-specific limitations, to enable the use and development of the most appropriate grading instruments. Grading systems need to be matched to both the research question being asked and the type of evidence being used.

The art and science of study identification: a comparative analysis of two systematic reviews

BACKGROUND

Systematic reviews (SRs) form the foundation for guidelines and evidence-based policy in medicine and public health. Although similar systematic reviews may include non-identical sets of studies, and it is recognized that different sets of studies may lead to different conclusions, little work has been published on why SR study cohorts differ.

METHODS

We took advantage of concurrent publication of two SRs on the same topic - prevention of child exposure to tobacco smoke - to understand why study cohorts differed in the two reviews. We identified all studies included in just one review, investigated validity of specified reasons for exclusions, and, using database records, explored reasons for study non-identification. We assessed review methods and discordancy, and attempted to assess whether changes in study cohorts would have changed conclusions.

RESULTS

Sixty-one studies were included in the two reviews. Thirty-five studies were present in just one review; of these, twenty were identified and excluded by the parallel review. Omissions were due to: review scope (9 studies, 26%), outcomes of interest not measured (8 studies, 23%), exclusion of reports with inadequate reporting (6 studies, 17%), mixed or unclear reasons (3 studies, 8%), search strategies concerning filters, tagging, and keywords (3 studies, 8%), search strategies regarding sources (PUBMED not searched) (2 studies, 6%); discordant interpretation of same eligibility criteria (2 studies, 6%), and non-identification due to non-specific study topic (2 studies, 6%). Review conclusions differed, but were likely due to differences in synthesis methods, not differences in study cohorts.

CONCLUSIONS

The process of study identification for SRs is part art and part science. While some differences are due to differences in review scope, outcomes measured, or reporting practices, others are caused by search methods or discrepancies in reviewer interpretations. Different study cohorts may or may not be a cause of differing SR results. Completeness of SR study cohorts could be enhanced by 1 - independent identification of studies by at least two reviewers, as recommended by recent guidelines, 2 - searching PUBMED with free-text keywords in addition to MEDLINE to identify recent studies, and 3 - Using validated search filters.

Hierarchy of evidence and appraisal of limitations (HEAL) grading system

Despite more than 30 years of effort that has been dedicated to the improvement of grading systems for evaluating the quality of research study designs considerable shortcomings continue. These shortcomings include the failure to define key terms, provide a comprehensive list of design flaws, demonstrate the reliability of such grading systems, properly value non-randomized controlled trials, and develop theoretically-derived systems for penalizing and promoting the evidence generated by a study. Consequently, in light of the importance of grading guidelines in evidence-based medicine, steps must be taken to remedy these deficiencies. This article presents two methods--a grading system and a measure of methodological bias--for evaluating the quality of evidence produced by an efficacy study.

The effect of qigong on depressive and anxiety symptoms: a systematic review and meta-analysis of randomized controlled trials

Objective. To evaluate clinical trial evidence of the effectiveness of qigong exercise on depressive and anxiety symptoms. Methods. Thirteen databases were searched from their respective inception through December 2012. Relevant randomized controlled trials (RCTs) were included. Effects of qigong across trials were pooled. Standardized mean differences (SMDs) were calculated for the pooled effects. Heterogeneity was assessed using the I² test. Study quality was evaluated using the Wayne Checklist. Results. Twelve RCTs met the inclusion criteria. The results of meta-analyses suggested a beneficial effect of qigong exercise on depressive symptoms when compared to waiting-list controls or usual care only (SMD = −0.75; 95% CI, −1.44 to −0.06), group newspaper reading (SMD = −1.24; 95% CI, −1.64 to −0.84), and walking or conventional exercise (SMD = −0.52; 95% CI, −0.85 to −0.19), which might be comparable to that of cognitive-behavioral therapy (P = 0.54). Available evidence did not suggest a beneficial effect of qigong exercise on anxiety symptoms. Conclusion. Qigong may be potentially beneficial for management of depressive symptoms, but the results should be interpreted with caution due to the limited number of RCTs and associated methodological weaknesses. Further rigorously designed RCTs are warranted.

Hibiscus sabdariffa L. in the treatment of hypertension and hyperlipidemia: a comprehensive review of animal and human studies

The effectiveness of Hibiscus sabdariffa L. (HS) in the treatment of risk factors associated with cardiovascular disease is assessed in this review by taking a comprehensive approach to interpreting the randomized clinical trial (RCT) results in the context of the available ethnomedical, phytochemical, pharmacological, and safety and toxicity information. HS decoctions and infusions of calyxes, and on occasion leaves, are used in at least 10 countries worldwide in the treatment of hypertension and hyperlipidemia with no reported adverse events or side effects. HS extracts have a low degree of toxicity with a LD50 ranging from 2,000 to over 5,000mg/kg/day. There is no evidence of hepatic or renal toxicity as the result of HS extract consumption, except for possible adverse hepatic effects at high doses. There is evidence that HS acts as a diuretic, however in most cases the extract did not significantly influence electrolyte levels. Animal studies have consistently shown that consumption of HS extract reduces blood pressure in a dose dependent manner. In RCTs, the daily consumption of a tea or extract produced from HS calyxes significantly lowered systolic blood pressure (SBP) and diastolic blood pressure (DBP) in adults with pre to moderate essential hypertension and type 2 diabetes. In addition, HS tea was as effective at lowering blood pressure as the commonly used blood pressure medication Captropril, but less effective than Lisinopril. Total cholesterol, low-density lipoprotein cholesterol (LDL-C), and triglycerides were lowered in the majority of normolipidemic, hyperlipidemic, and diabetic animal models, whereas high-density lipoprotein cholesterol (HDL-C) was generally not affected by the consumption of HS extract. Over half of the RCTs showed that daily consumption of HS tea or extracts had favorable influence on lipid profiles including reduced total cholesterol, LDL-C, triglycerides, as well as increased HDL-C. Anthocyanins found in abundance in HS calyxes are generally considered the phytochemicals responsible for the antihypertensive and hypocholesterolemic effects, however evidence has also been provided for the role of polyphenols and hibiscus acid. A number of potential mechanisms have been proposed to explain the hypotensive and anticholesterol effects, but the most common explanation is the antioxidant effects of the anthocyanins inhibition of LDL-C oxidation, which impedes atherosclerosis, an important cardiovascular risk factor. This comprehensive body of evidence suggests that extracts of HS are promising as a treatment of hypertension and hyperlipidemia, however more high quality animal and human studies informed by actual therapeutic practices are needed to provide recommendations for use that have the potential for widespread public health benefit.

An Application of a New Evidence Grading System to Research on the Chronic Care Model

Methodological quality undergirds all evidence-based medicine because without strong evidence supporting or refuting the efficacy of an intervention, the movement toward basing medical decisions and practice on scientific evidence is not sustainable. Recently, the consensus that had existed regarding the hierarchy of evidence produced by a study design was challenged on the basis that existing guidelines failed to properly define key terms, weight the merits of certain non-randomized controlled trials, and employ a comprehensive list of study design limitations to render evaluative conclusions, to name a few of the challenges. The present study introduces a new grading system that overcomes, or at the very least greatly diminishes, these challenges. This new method is applied to the literature on the Chronic Care Model and the results are then compared to several of the most popular grading guidelines currently in use. These results revealed substantial differences between the guidelines in accordance with previous research that challenged existing methods. Furthermore, the present study lends support to the proposed grading guideline although further research into its validity and reliability is needed.