Interpreting and using clinical trials

Instructional scaffolding to improve students' skills in evaluating clinical literature

OBJECTIVE

To implement and assess the effectiveness of an activity to teach pharmacy students to critically evaluate clinical literature using instructional scaffolding and a Clinical Trial Evaluation Rubric.

DESIGN

The literature evaluation activity centered on a single clinical research article and involved individual, small group, and large group instruction, with carefully structured, evidence-based scaffolds and support materials centered around 3 educational themes: (1) the reader's awareness of text organization, (2) contextual/background information and vocabulary, and (3) questioning, prompting, and self-monitoring (metacognition).

ASSESSMENT

Students initially read the article, scored it using the rubric, and wrote an evaluation. Students then worked individually using a worksheet to identify and define 4 to 5 vocabulary/concept knowledge gaps. They then worked in small groups and as a class to further improve their skills. Finally, they assessed the same article using the rubric and writing a second evaluation. Students' rubric scores for the article decreased significantly from a mean pre-activity score of 76.7% to a post-activity score of 61.7%, indicating that their skills in identifying weaknesses in the article's study design had improved.

CONCLUSION

Use of instructional scaffolding in the form of vocabulary supports and the Clinical Trial Evaluation Rubric improved students' ability to critically evaluate a clinical study compared to lecture-based coursework alone.

Development and evaluation of an instrument for the critical appraisal of randomized controlled trials of natural products

BACKGROUND

The efficacy of natural products (NPs) is being evaluated using randomized controlled trials (RCTs) with increasing frequency, yet a search of the literature did not identify a widely accepted critical appraisal instrument developed specifically for use with NPs. The purpose of this project was to develop and evaluate a critical appraisal instrument that is sufficiently rigorous to be used in evaluating RCTs of conventional medicines, and also has a section specific for use with single entity NPs, including herbs and natural sourced chemicals.

METHODS

Three phases of the project included: 1) using experts and a Delphi process to reach consensus on a list of items essential in describing the identity of an NP; 2) compiling a list of non-NP items important for evaluating the quality of an RCT using systematic review methodology to identify published instruments and then compiling item categories that were part of a validated instrument and/or had empirical evidence to support their inclusion and 3) conducting a field test to compare the new instrument to a published instrument for usefulness in evaluating the quality of 3 RCTs of a NP and in applying results to practice.

RESULTS

Two Delphi rounds resulted in a list of 15 items essential in describing NPs. Seventeen item categories fitting inclusion criteria were identified from published instruments for conventional medicines. The new assessment instrument was assembled based on content of the two lists and the addition of a Reviewer's Conclusion section. The field test of the new instrument showed good criterion validity. Participants found it useful in translating evidence from RCTs to practice.

CONCLUSION

A new instrument for the critical appraisal of RCTs of NPs was developed and tested. The instrument is distinct from other available assessment instruments for RCTs of NPs in its systematic development and validation. The instrument is ready to be used by pharmacy students, health care practitioners and academics and will continue to be refined as required.

Scales to assess the quality of randomized controlled trials: a systematic review

BACKGROUND AND PURPOSE

The methodological quality of randomized controlled trials (RCTs) is commonly evaluated in order to assess the risk of biased estimates of treatment effects. The purpose of this systematic review was to identify scales used to evaluate the methodological quality of RCTs in health care research and summarize the content, construction, development, and psychometric properties of these scales.

METHODS

Extensive electronic database searches, along with a manual search, were performed.

RESULTS

One hundred five relevant studies were identified. They accounted for 21 scales and their modifications. The majority of scales had not been rigorously developed or tested for validity and reliability. The Jadad Scale presented the best validity and reliability evidence; however, its validity for physical therapy trials has not been supported.

DISCUSSION AND CONCLUSION

Many scales are used to evaluate the methodological quality of RCTs, but most of these scales have not been adequately developed and have not been adequately tested for validity and reliability. A valid and reliable scale for the assessment of the methodological quality of physical therapy trials needs to be developed.

Evaluating the evidence from clinical trials in chronic hepatitis C

A number of studies are being published on investigational agents for the treatment of patients with chronic hepatitis C. Before incorporating new medications into therapy or electing to postpone therapy until an investigational agent receives regulatory approval, it is important to evaluate the following specific features of the published evidence: quality of the evidence, precision of the treatment effect, magnitude of the treatment effect, magnitude of the hazards, and other factors such as risk-benefit analysis and cost, including cost-effectiveness. Previous studies of treatment of patients with the disease allow us to establish some benchmarks and provide structure to evaluate the evidence of the efficacy and safety of these newer drugs.

Recombinant human erythropoietin use in intensive care

OBJECTIVE

To review the literature concerning the role of recombinant human erythropoietin (rHuEPO) in reducing the need for transfusion in critically ill patients.

DATA SOURCES

Articles were obtained through searches of the MEDLINE database (from 1990 to June 2001) using the key words erythropoietin, epoetin alfa, anemia, reticulocytes, hemoglobin, critical care, intensive care, critical illness, and blood transfusion. Additional references were found in the bibliographies of the articles cited. The Cochrane library was also consulted.

STUDY SELECTION AND DATA EXTRACTION

Controlled, prospective, and randomized studies on the use of rHuEPO in critically ill adults were selected.

DATA SYNTHESIS

Anemia is a common complication in patients requiring intensive care. It is caused, in part, by abnormally low concentrations of endogenous erythropoietin and is mainly seen in patients with sepsis and multiple organ dysfunction syndrome, in whom inflammation mediator concentrations are often elevated. High doses of rHuEPO produce a rapid response in these patients, despite elevated cytokine concentrations. There have been 3 studies on rHuEPO administration in intensive care and 1 trial in acutely burned patients. Only 2 of these studies looked at the impact of rHuEPO administration on the need for transfusion.

CONCLUSIONS

Few randomized, controlled trials explore the role of rHuEPO in critical care. Only 1 was a large, randomized clinical trial, but it presents many limitations. Future outcome and safety studies comparing rHuEPO with placebo must include clinical endpoints such as end-organ morbidity, mortality, transfusion criteria, and pharmacoeconomic analysis. rHuEPO appears to provide an erythropoietic response. Optimal dosage and the real impact of rHuEPO on the need for transfusion in intensive care remain to be determined. Currently, based on the evidence available from the literature, rHuEPO cannot be recommended to reduce the need for red blood cell transfusions in anemic, critically ill patients.

Inhaled nitric oxide in the management of persistent pulmonary hypertension of the newborn: a meta-analysis

OBJECTIVES

To evaluate the use of inhaled nitric oxide (NO) in the management of persistent pulmonary hypertension of the newborn.

METHODS

Computerized bibliographic search on MEDLINE, CURRENT CONTENTS and LILACS covering the period from January 1990 to March 1998; review of references of all papers found on the subject. Only randomized clinical trials evaluating nitric oxide and conventional treatment were included. OUTCOMES STUDIED: death, requirement for extracorporeal membrane oxygenation (ECMO), systemic oxygenation, complications at the central nervous system and development of chronic pulmonary disease. The methodologic quality of the studies was evaluated by a quality score system, on a scale of 13 points.

RESULTS

For infants without congenital diaphragmatic hernia, inhaled NO did not change mortality (typical odds ratio: 1.04; 95% CI: 0.6 to 1.8); the need for ECMO was reduced (relative risk: 0.73; 95% CI: 0.60 to 0.90), and the oxygenation was improved (PaO2 by a mean of 53.3 mm Hg; 95% CI: 44.8 to 61.4; oxygenation index by a mean of -12.2; 95% CI: -14.1 to -9.9). For infants with congenital diaphragmatic hernia, mortality, requirement for ECMO, and oxygenation were not changed. For all infants, central nervous system complications and incidence of chronic pulmonary disease did not change.

CONCLUSIONS

Inhaled NO improves oxygenation and reduces requirement for ECMO only in newborns with persistent pulmonary hypertension who do not have diaphragmatic hernia. The risk of complications of the central nervous system and chronic pulmonary disease were not affected by inhaled NO.