Lessons from overviews of cardiovascular trials

Animal-Assisted and Pet-Robot Interventions for Ameliorating Behavioral and Psychological Symptoms of Dementia: A Systematic Review and Meta-Analysis

Patients with dementia suffer from psychological symptoms such as depression, agitation, and aggression. One purpose of dementia intervention is to manage patients’ inappropriate behaviors and psychological symptoms while taking into consideration their quality of life (QOL). Animal-assisted intervention (AAI) and pet-robot intervention (PRI) are effective intervention strategies for older people with cognitive impairment and dementia. In addition, AAI and PRI have been shown to have positive effects on behavioral and psychological symptoms of dementia (BPSD). However, studies into the association between AAI/PRI and BPSD have elicited inconsistent results. Thus, we performed a meta-analysis to investigate this association. We analyzed nine randomized controlled trials on AAI and PRI for dementia patients published between January 2000 and August 2019 and evaluated the impact of AAI/PRI on agitation, depression, and QOL. We found that AAI and PRI significantly reduce depression in patients with dementia. Subsequent studies should investigate the impact of AAI and PRI on the physical ability and cognitive function of dementia patients and conduct a follow-up to investigate their effects on the rate of progression and reduction of symptoms of dementia. Our research will help with neuropsychological and environmental intervention to delay or improve the development and progression of BPSD.

The Oxford Implementation Index: a new tool for incorporating implementation data into systematic reviews and meta-analyses

OBJECTIVES

This article presents a new tool that helps systematic reviewers to extract and compare implementation data across primary trials. Currently, systematic review guidance does not provide guidelines for the identification and extraction of data related to the implementation of the underlying interventions.

STUDY DESIGN AND SETTING

A team of systematic reviewers used a multistaged consensus development approach to develop this tool. First, a systematic literature search on the implementation and synthesis of clinical trial evidence was performed. The team then met in a series of subcommittees to develop an initial draft index. Drafts were presented at several research conferences and circulated to methodological experts in various health-related disciplines for feedback. The team systematically recorded, discussed, and incorporated all feedback into further revisions. A penultimate draft was discussed at the 2010 Cochrane-Campbell Collaboration Colloquium to finalize its content.

RESULTS

The Oxford Implementation Index provides a checklist of implementation data to extract from primary trials. Checklist items are organized into four domains: intervention design, actual delivery by trial practitioners, uptake of the intervention by participants, and contextual factors. Systematic reviewers piloting the index at the Cochrane-Campbell Colloquium reported that the index was helpful for the identification of implementation data.

CONCLUSION

The Oxford Implementation Index provides a framework to help reviewers assess implementation data across trials. Reviewers can use this tool to identify implementation data, extract relevant information, and compare features of implementation across primary trials in a systematic review. The index is a work-in-progress, and future efforts will focus on refining the index, improving usability, and integrating the index with other guidance on systematic reviewing.

Accounting for Methodological, Structural, and Parameter Uncertainty in Decision-Analytic Models

Accounting for uncertainty is now a standard part of decision-analytic modeling and is recommended by many health technology agencies and published guidelines. However, the scope of such analyses is often limited, even though techniques have been developed for presenting the effects of methodological, structural, and parameter uncertainty on model results. To help bring these techniques into mainstream use, the authors present a step-by-step guide that offers an integrated approach to account for different kinds of uncertainty in the same model, along with a checklist for assessing the way in which uncertainty has been incorporated. The guide also addresses special situations such as when a source of uncertainty is difficult to parameterize, resources are limited for an ideal exploration of uncertainty, or evidence to inform the model is not available or not reliable. Methods for identifying the sources of uncertainty that influence results most are also described. Besides guiding analysts, the guide and checklist may be useful to decision makers who need to assess how well uncertainty has been accounted for in a decision-analytic model before using the results to make a decision.

Use of indirect and mixed treatment comparisons for technology assessment

Indirect and mixed treatment comparison (MTC) approaches to synthesis are logical extensions of more established meta-analysis methods. They have great potential for estimating the comparative effectiveness of multiple treatments using an evidence base of trials that individually do not compare all treatment options. Connected networks of evidence can be synthesized simultaneously to provide estimates of the comparative effectiveness of all included treatments and a ranking of their effectiveness with associated probability statements. The potential of the use of indirect and MTC methods in technology assessment is considerable, and would allow for a more consistent assessment than simpler alternative approaches. Although such models can be viewed as a logical and coherent extension of standard pair-wise meta-analysis, their increased complexity raises some unique issues with far-reaching implications concerning how we use data in technology assessment, while simultaneously raising searching questions about standard pair-wise meta-analysis. This article reviews pair-wise meta-analysis and indirect and MTC approaches to synthesis, clearly outlining the assumptions involved in each approach. It also raises the issues that the National Institute for Health and Clinical Excellence (NICE) needed to consider in updating their 2004 Guide to the Methods of Technology Appraisal, if such methods are to be used in their technology appraisals.

Clinical attrition due to biased preclinical assessments of potential efficacy

Unless it is carefully controlled, bias often distorts the results of clinical trials, usually exaggerating the magnitude of true efficacy. For that reason, procedures to limit bias have been mandated by the FDA when assessing efficacy in clinical trials. The present review shows that the effects of bias in preclinical studies are at least as large as in clinical trials, and since bias is not usually controlled in preclinical proof of concept studies, compounds that actually have little or no therapeutic potential may often be advanced into clinical trials. This possibility is supported by the fact that lack of efficacy is the single biggest reason why compounds fail in the clinic. The shift to target-based discovery during the last 10-15 years may have further increased the effects of bias on preclinical assessments of potential efficacy, and contributed to the continuing decline in clinical success rates. Procedures are available to control for bias during preclinical assessments of potential efficacy, and their use could dramatically increase clinical success rates and substantially reduce the costs of drug discovery and development.

Guidelines on how to assess the validity of results presented in subgroup analysis of clinical trials

In observational studies, identification of associations within particular subgroups is the usual method of investigation. As an exploratory method, it is the bread and butter of epidemiological research. Nearly everything that has been learned in epidemiology has been derived from the analysis of subgroups. In a randomized clinical trial, the entire purpose is the comparison of the test subjects and the controls, and when there is particular interest in the results of treatment in a certain section of trial participants, a subgroup analysis is performed. These subgroups are examined to see if they are liable to a greater benefit or risk from treatment. Thus, analyzing patient subsets is a natural part of the process of improving therapeutic knowledge through clinical trials. Nevertheless, the reliability of subgroup analysis can often be poor because of problems of multiplicity and limitations in the numbers of patients studied. The naive interpretation of the results of such examinations is a cause of great confusion in the therapeutic literature. We emphasize the need for readers to be aware that inferences based on comparisons between subgroups in randomized clinical trials should be approached more cautiously than those based on the main comparison. That is, subgroup analysis results derived from a sound clinical trial are not necessarily valid; one must not jump to conclusions and accept the validity of subgroup analysis results without an appropriate judgment.

Clinical outcomes of therapeutic agents that block the platelet glycoprotein IIb/IIIa integrin in ischemic heart disease

BACKGROUND

Several platelet glycoprotein (GP) IIb/IIIa receptor antagonists have been evaluated in clinical trials. We conducted a systematic overview (meta-analysis) to assess the effect of these compounds on death, myocardial infarction (MI), and revascularization.

METHODS AND RESULTS

ORs were calculated for 16 randomized, controlled trials of GP IIb/IIIa inhibitors. An empirical Bayesian random-effects model combined the outcomes of 32 135 patients. There was a significant mortality reduction by GP IIb/IIIa inhibitors at 48 to 96 hours, with an OR of 0.70 (95% CI, 0. 51 to 0.96; P<0.03), equivalent to a reduction of 1 death per 1000 patients treated. Mortality benefits at 30 days (OR, 0.87; 95% CI, 0. 74 to 1.02; P=0.08) and 6 months (OR, 0.97; 95% CI, 0.86 to 1.10; P=0.67) were not statistically significant. For the combined end point of death or MI, there was a highly significant (P<0.001) benefit for GP IIb/IIIa inhibitors at each time point. The 30-day OR was 0.76 (95% CI, 0.66 to 0.87), or 20 fewer events per 1000 patients treated. For the composite end point of death, MI, or revascularization, there was also a highly significant (P<0.001) benefit for GP IIb/IIIa inhibitors. At 30 days, the OR was 0.77 (95% CI, 0.68 to 0.86), or 30 fewer events per 1000 patients treated. The risk differences for death, death or MI, and composite outcomes were similar at 6 months, indicating a sustained absolute improvement. Similar benefit was seen when trials were subgrouped by therapeutic indication (percutaneous intervention versus acute coronary syndromes).

CONCLUSIONS

Application of this new therapeutic class to clinical practice promises substantial benefit for both indications.

Balneotherapy and quality assessment: interobserver reliability of the Maastricht criteria list and the need for blinded quality assessment

This study investigates aspects of the reliability of the Maastricht criteria list for quality assessment in systematic reviews, and whether blinded reviewing is necessary to prevent review bias. We used the data set of 12 articles from a systematic review concerning the efficacy of balneotherapy in patients with arthritis. Twenty reviewers participated of which two reviewers, who have been involved in developing the Maastricht criteria list, acted as reference standard. Half of all assessments were performed blindly. A high level of agreement was found between the reviewers and a high level of correlation with the reference standard. The quality scores between the blinded and unblinded assessment did not differ much. Based on the results we conclude that the Maastricht criteria list is a reliable instrument in quality assessment of clinical trials. Within the limits of this study we found no evidence that blinding is necessary to prevent review bias.

Overcoming the limitations of current meta-analysis of randomised controlled trials

For a meta-analysis to give definitive information, it should meet at least the minimum standards that would be expected of a well-designed, adequately powered, and carefully conducted randomised controlled trial. These minimum standards include both qualitative characteristics--a prospective protocol, comparable definitions of key outcomes, quality control of data, and inclusion of all patients from all trials in the final analysis--and quantitative standards--an assessment of whether the total sample is large enough to provide reliable results and the use of appropriate statistical monitoring guidelines to indicate when the results of the accumulating data of a meta-analysis are conclusive. We believe that rigorous meta-analyses undertaken according to these principles will lead to more reliable evidence about the efficacy and safety of interventions than either retrospective meta-analysis or individual trials.

Cumulating evidence from randomized trials: utilizing sequential monitoring boundaries for cumulative meta-analysis

We propose the adaptation of classical monitoring boundaries for use in cumulative meta-analysis as guidelines for deciding when accumulating evidence is statistically significant and medically convincing. The interpretation of information from a randomized controlled trial is compared with that from a meta-analysis. The concept of optimal information size for a meta-analysis is developed and used to adapt monitoring boundaries to cumulative meta-analysis.

Strengths and limitations of meta-analysis: larger studies may be more reliable

Meta-analysis of randomized controlled trials combines information from independent studies that address a similar question to provide more reliable estimates of treatment effects. At the present time, the methodology and usefulness of meta-analysis is under scrutiny. In the first part of this paper, we summarize the limitations of meta-analysis and make suggestions for improvements. In the second part, we illustrate strengths and limitations using examples of meta-analyses and subsequent large trials that address the same question. We develop the hypothesis that the size of the meta-analysis may be a useful measure of reliability. Small meta-analyses (i.e., those with less than 200 outcome events) may only be useful for summarizing the available information and generating hypotheses for future research. The results of small meta-analyses should be regarded with caution, even if the p value shows extreme statistical significance. Larger meta-analyses (i.e., those with several hundred events) are likely to be more reliable and may be clinically useful. Well-conducted meta-analyses of large trials using individual patient data may provide the best estimates of treatment effects in the cohort overall and in clinically important subgroups.

Meta-analysis and prospective meta-analysis in childhood leukemia clinical research

In this paper, we consider the role of meta-analysis and 'prospective meta-analysis' studies in childhood acute lymphocytic leukemia (ALL). In this issue, Valsecchi and Masera [1] give a thoughtful discourse, generally favorable to this approach. This article presents the opposite point of view. The aims of our article are to present the implications in clinical, rather than biostatistical terms, and to provide an extensive literature review of the subject of meta-analysis. We conclude that treatment assessments, resulting from meta-analysis of closed studies (retrospective) should be met with healthy skepticism. Trials requiring international resources should be true intergroup trials with a single coordinating center, rather than prospective meta-analysis, unless it is a question grafted onto each group's own research agenda. For example, each group might ask its own systemic control question, but a CNS protection question is asked collectively.

Lessons learned from a prospective meta-analysis

OBJECTIVE

To describe the unique aspects of and the lessons learned in planning and conducting a pooled analysis of multiple trials evaluating interventions to reduce functional decline in hospitalized older persons. Specific examples from the Hospital Outcomes Project for the Elderly (HOPE) meta-analysis are discussed.

DESIGN

A prospective meta-analysis (PMA) that compiled and pooled data from concurrently conducted clinical trials testing related but distinct interventions.

SETTING

The Data Coordinating Center for the prospective meta-analysis coordinated the collection and analysis of common outcome data from five university-affiliated hospitals and one community hospital conducting the clinical trials.

PARTICIPANTS

Acutely ill hospitalized elderly participants at least 65 to 75 years old.

INTERVENTIONS

Treatments being evaluated included exercise, physical therapy, a multidisciplinary geriatric care unit, a multidisciplinary in-hospital intervention with post-discharge care, a nursing-based geriatric care program, and a program to improve detection and evaluation of delirious patients.

CONCLUSION

The prospective meta-analysis provides selected advantages over independently conducted clinical trials and retrospective meta-analyses. It does, however, pose special design and operational challenges that must be addressed well before initiation of the individual trials. Specific issues of concern include: maintaining scientific integrity of both the individual trials and the PMA; reaching consensus on PMA goals, what data to collect, how and when to collect them and how to maintain uniformly high quality data across all sites; defining the role of the Data Coordinating Center in a multicenter project that utilizes different trials and protocols; and establishing policies concerning analyses of the pooled data, publication of pooled analyses, and ownership of the pooled database.

Meta-analysis: weighing the evidence

Use of meta-analytical (quantitative overview) techniques is now commonplace in a wide range of medical research contexts, with a rapid rise in its frequency of use being particularly apparent in the last decade. The history of meta-analyses in other fields, particularly psychology and educational research, is longer. Many methods have been proposed and used, from crude 'vote counting' of studies showing significant or non-significant results, through methods for combination of effect size estimates based on fixed or random effects models, to general linear mixed models and Bayesian methods. The history of meta-analysis and the advantages and disadvantages of various approaches to it are briefly reviewed in this paper, with reference to pharmaceutical product licence applications, other reviews of clinical trials and epidemiological studies, and health services research.

Prophylaxis of postoperative thromboembolism with low molecular weight heparins

To evaluate the thromboprophylactic use of low molecular weight heparins (LMWHs), publications from 27 orthopaedic trials and 35 studies of patients undergoing general or gynaecological surgery were scrutinized and subjected to a partial meta-analysis. In orthopaedic surgery, LMWHs were superior to placebo or dextran and at least as efficient as unfractionated heparin in the prevention of deep vein thrombosis (DVT). Compared with unfractionated heparin, one of the LMWH preparations significantly reduced the total incidence of DVT. The rate of non-fatal pulmonary embolism was 0.49 per cent in patients receiving LMWH and 1.22 per cent in controls. Seven orthopaedic patients (0.15 per cent) died from pulmonary embolism, none of whom received LMWH. In general surgery, the LMWHs were at least as efficient as unfractionated heparin, with a trend towards a lower risk of pulmonary embolism with the former. Compared with unfractionated heparin, LMWHs did not reduce the postoperative mortality rate, nor did they cause haemorrhage. LMWHs provide safe and efficient prophylaxis by administration once daily.

The structure of cardiological revolutions. James B. Herrick Lecture

Four revolutions are transforming the cardiology of our day. The first is a social revolution. It has resulted in plummeting esteem for the medical profession reflecting disenchantment coupled with the rapid emergence of the recognition that health care is a right rather than a privilege. The second revolution, interventional cardiology, has provided powerful therapeutic tools demanding technical as well as cognitive expertise. The third, the revolution in molecular and cellular biology, is transforming our understanding of mechanisms underlying disease. Because of the seminal importance of progress in basic science to advances in clinical cardiology, features of these three revolutions are being synthesized in a fourth, a revolution transforming cardiology itself. Novel approaches are needed for optimal training of clinicians with diverse areas of interest, investigators in fundamental and clinical research, house staff, and students; for optimal use of clinical and research resources; and for optimal responsiveness to the needs of patients. They must be developed with cognizance of and fidelity to our clinical and scientific heritage.

Meta-analysis: whither narrative review?

Meta-analysis refers to methodologies that are used to integrate related empirical research to arrive at conclusions not possible by reviewing individual studies, or to improve generalizations of individual studies. It is distinguished from the traditional narrative review in that statistical methodologies are applied to derive more objective conclusions than those that typify narrative reviews. Meta-analysis has been slow to appear in the literature of clinical medicine, however, particularly when viewed in the context of the exponential expansion of literature and the availability of computer technology to facilitate its conduct.