Clinical trials and clinical practice: do doctors use the same criteria to judge outcome?

AIMS

The results of clinical trials often seem to have little influence on the practice of individual doctors. This could be because trial information is presented in the style of a scientific experiment which cannot often be clearly related to the context of everyday patient care. We tested the hypothesis that such framing effects would cause doctors to assess the clinical significance of treatment outcomes differently when presented as clinical trial results rather than as individual patient data.

METHODS

Fourteen rheumatologists independently reviewed the same 50 sets of data obtained from patients with rheumatoid arthritis. The data consisted of 10 commonly used clinical and laboratory variables measured before and after a period of treatment. The same data were presented in two formats on two separate occasions. The patient data format was a collection of typed sheets attributing each set of results to an individual patient. The clinical trial format was a professionally printed and bound booklet in which each set of results was laid out as summary results of a small uncontrolled clinical trial. Doctors judged the degree of improvement or deterioration and its clinical importance for each data set for both formats. These changes were converted into units of 'Clinical Importance'.

RESULTS

Although some statistically significant differences emerged in the individual doctors' judgements between the formats none of these was of a clinically important size. The median of the mean trial--patient difference between the formats for all 14 doctors was 0.035 units of clinical importance [95% CI -0.244 to 0.074].

CONCLUSIONS

This evidence does not support the hypothesis that framing effects are a major cause of the failure of clinical trials to influence clinical practice.

Physicians' judgments of the risks of cardiac procedures. Differences between cardiologists and other internists

OBJECTIVES

The authors compared judgments of the population risks of invasive cardiac procedures made by cardiologists and other internal medicine physicians. Our main hypotheses were that cardiologists' judgments would differ from those made by the other physicians and that cardiologists' judgments would be more accurate than those of other physicians.

METHODS

This was a cross-sectional survey of senior staff and physician-trainees at two teaching hospitals affiliated with a US medical school, Emergency Department physicians at a community hospital in the same metropolitan area, and senior staff and trainees at two teaching hospitals affiliated with a UK school. Judgments of the risks of severe morbidity and death due to Swan-Ganz catheterization, cardiac catheterization, percutaneous coronary angioplasty, and coronary artery bypass grafting were assessed.

RESULTS

Nineteen cardiologists judged the risks of severe morbidity due to all procedures and the risks of death due to all procedures except coronary artery bypass grafting to be significantly lower than did the 78 other internists. Cardiologists more frequently made accurate judgments of the rates of morbidity and death due to cardiac catheterization than did the other internists; other internists more frequently made accurate judgments for the rates of morbidity due to Swan-Ganz catheterization.

CONCLUSIONS

Disagreements about the risks of procedures may arise from a paucity of published data, or from an over-supply of confusing data.

Clinical judgment analysis

Judgement is central to the practice of medicine and occurs between making clinical observations and taking clinical decisions. Clinical judgment analysis has developed as a method of making statistically firm models of doctors' judgments. Computed models reveal the differential importance attached to items of clinical, social, or other data which are determinants of clinical decisions. These models can both reveal the causes of conflicts of judgment and may help resolve them in a way that unaided discussion cannot. Revealing experts' models to students speeds learning of diagnostic skills. Clinical judgment analysis offers a method of probing the judgments not just of students and doctors but also of patients who have shown systematic differences in their perceptions of risk and benefit. The power and relevance of clinical trials can be improved by the consistent application of judgment policies generated from both the trialists and those who will use their results.

How can we design trials to detect clinically important changes in disease severity?

1. Forty-eight British rheumatologists judged the change in disease activity in 50 sets of patient data drawn from life and presented as 'paper patients'. Each set comprised two values, recorded a year apart, for 10 commonly measured clinical variables. Doctors recorded the size of improvement or deterioration on a visual analogue scale (VAS) and whether the change was clinically important or not. 2. Clinical judgement policies were modelled using linear regression of the clinical variables on the VAS score. 3. Doctors showed little agreement over which patients had improved and which had not. Possible reasons could be discovered by inspecting their judgement policies. 4. The weights attributed to the clinical variables differed considerably between doctors. Furthermore weights the doctors believed they attached to the variables frequently differed from the weights in the regression models. 5. These models could be used to calculate the smallest change required in any clinical variable before it would be considered clinically important. However, the size of such changes was often outside the observed clinical range suggesting that the use of single outcome variables is unrealistic. 6. The modelling procedure described can be applied during the planning stage of the trial to participating physicians, patients, health economists or any other group having an interest in the results. The models themselves can then be used to reach a consensus policy for judging what is a successful outcome. This may be expressed as a linear combination of specific outcome measures. Its use may improve the power of clinical trials and the relevance of their results.

Residual renal function in hemodialysis patients may protect against hyperaluminemia

We investigated 106 home hemodialysis patients whose mean [+/- SEM] serum aluminum (Al) concentration was 60.9 +/- 4.1 micrograms/liter. Serum Al concentration was inversely related to daily urine output (r = -0.52, P less than 0.001). Urine volume and measurements of Al exposure were included in a multivariate analysis of serum Al concentration in the 62 patients whose urine output was greater than 10 ml/day. The multiple correlation coefficient (r) was 0.70 (P less than 0.001) and the percentage contributions to r2 (indicating the relative importance of each factor) were: urine output 57%, oral Al intake 36%, total dialysis hours 7%. The additional contribution from cumulative water Al was negligible. In a subgroup of 26 patients with a urine output exceeding 10 ml/day, urinary Al excretion averaged 15.4 micrograms/day, and renal Al clearance and serum Al concentration were inversely related (r = -0.69, P less than 0.001). We conclude that Al-containing phosphate binders were a more important source of Al than was dialysate in these patients and that residual renal function can reduce the severity of hyperaluminemia in hemodialysis patients.

The interaction between chronic oral slow-release theophylline and single-dose intravenous erythromycin

Eight volunteers were each given 300 mg of erythromycin lactobionate by i.v. infusion over 15 min in the presence and absence of chronic dosing with slow-release theophylline. Pharmacokinetic profiles were obtained for theophylline in the presence and absence of erythromycin and for erythromycin in the presence and absence of theophylline. A very small, clinically unimportant, but statistically significant increase occurred in mean (+/- S.E.M.) serum theophylline concentration from 4.9 +/- 0.3 mg/l to 5.2 +/- 0.3 mg/l in the presence of erythromycin (P = less than 0.01). The theophylline pharmacokinetic parameters did not change significantly. The only changes in erythromycin pharmacokinetics were an increase in the renal excretion (0-12 h) from 5.5 +/- 4.0 mg to 11.2 +/- 6.0 mg (P less than 0.03) and an increase in renal clearance (0-2 h) from 9.0 +/- 6.0 ml/min to 21.6 +/- 15 ml/min (P less than 0.05) in the presence of theophylline.

Gastrointestinal side effects after intravenous erythromycin lactobionate

Ten healthy normal volunteers received an intravenous infusion of erythromycin lactobionate over 60 min to a total dose of 800 mg (n = 9), and 524 mg (n = 1). Blood samples were collected at 10 min intervals for 100 min and gastric contents aspirated, via a nasogastric tube, from pre-dose to 105 min after start of infusion. Incidence and severity of three gastrointestinal symptoms (nausea, stomach discomfort and feelings of hunger), two CNS symptoms (dizziness and faintness) and a 'control' symptom (back pain) were measured using 100 mm visual analogue scales. Rate of infusion and plasma erythromycin concentration correlated with nausea (P less than 0.001) and stomach discomfort (P less than 0.001); plasma erythromycin concentration was also correlated with dizziness (P less than 0.05). Concentrations of active erythromycin in the aspirate were pH dependent. In one subject the concentration of erythromycin in the aspirate exceeded that in the plasma by 100 fold. Bile staining of samples containing the highest levels of microbiologically active erythromycin makes the origin of the erythromycin in these samples uncertain.

Inability of rheumatologists to describe their true policies for assessing rheumatoid arthritis

Eighty nine British and Australian rheumatologists took part in a study to discover how accurately they could describe their procedures for measuring disease severity in rheumatoid arthritis. The relative importance they attached to different clinical and laboratory variables showed a very wide variation, and these stated policies were generally poor at predicting their actual judgments when assessing 'paper patients' (r2 = 39%). Policies based on equal weighting of all variables, while also poor predictors (r2 = 41%), were nevertheless superior to their stated policies for 49 respondents. Policies calculated by judgment (linear regression) analysis were much more successful predictors (R2 = 73%). Unhurried, detailed interviews with four experienced rheumatologists provided carefully considered statements of assessment policy, but these also were poor predictors of routine assessments of outpatients (r2 = 34%) compared with policies calculated by clinical judgment analysis, even when these were applied to new data (R2 = 88%).

Antibiotics for otitis media: can we help doctors agree?

Wide variations in antibiotic prescribing for otitis media have suggested the need to discover the causes of the differences and help doctors reach agreement. Simulated cases--in the form of written clinical data extracts based on real patients--were used to study the diagnostic and prescribing behaviour of a group of six general practitioners. Clinical judgement analysis was used to model the way in which doctors diagnosed otitis media and their policy for using antibiotics. Most doctors performed consistently and their judgements could be fitted well to models using a small number of symptoms and signs. These models often differed from the policy they believed they were operating. This information was used as process feedback in a group discussion to help improve agreement within the practice on the management of otitis media. Some of the variation in behaviour observed at the start of the study was reduced by significant changes in that of the trainee. Other doctors changed little and some were sceptical of the validity of the experimental methods. The prospects for and difficulties of this type of analysis are discussed.

Clinical judgment in rheumatoid arthritis. III. British rheumatologists' judgments of 'change in response to therapy'

A realistic analysis of the criteria used by rheumatologists in evaluating the progress of patients suffering from rheumatoid arthritis must be based on actual clinical judgments rather than on expressed opinions. A randomly selected 15% sample of British rheumatologists (48) recorded judgements on the progress of 50 'paper' patients, based on data taken from actual patients participating in clinical trials. The rheumatologists differed markedly in their assessments of the progress of disease, with serious disagreements even when only 'clinically important' changes were considered. Some clinicians showed little consistency in their judgments of duplicate cases. Multiple regression analysis of the patient data in relation to the disease assessments provided a model of each clinician's judgment policy. These judgment policy models showed that the differences in clinical assessment were greater than could be explained by the inconsistent application of similar assessment policies, and were a consequence also of differences in the underlying judgment policies themselves. Judgments related more closely to changes in ESR and other process measures than to changes in functional ability.

Clinical judgment in rheumatoid arthritis. II. Judging 'current disease activity' in clinical practice

Two rheumatologists made judgments about 'current disease activity' in real patients and 'paper patients' with rheumatoid arthritis. Analysis of each set of judgments provides a model of judgment policy which contains only 3 clinical variables but explains over 94% of the variance in judgments. The judgment policy models differ markedly from each other and from the clinicians' own perceptions of their behaviour. Judgment policy modelling offers a means of improving co-ordination between clinical investigators within and between centres.

Clinical judgment in rheumatoid arthritis. I. Rheumatologists' opinions and the development of 'paper patients'

Opinions about the importance of various measures of disease activity in rheumatoid arthritis gathered from a survey of 20% of British rheumatologists showed a wide diversity for all clinical variables. 'Paper patients' have been developed as a method of investigating actual clinical decisions rather than expressed opinions. Assessments based on 'paper patients' correlate highly (r = +0.901) with those made on the equivalent real patients when seen in person.

Trial of three-day and ten-day courses of amoxycillin in otitis media

A randomised double-blind controlled trial compared three-day and 10-day courses of amoxycillin (25 mg/kg daily) in children with otitis media. Seventeen doctors from five centres admitted 84 children between the ages of 2 and 10 years. Symptoms and signs were measured on admission to the trial, on day 3, and on day 15. Mother's observations were recorded daily for 10 days. Audiograms were performed at four and 12 weeks after the end of the trial. The treatment groups showed little difference in the speed of resolution of symptoms and signs, the numbers of primary treatment failures, or the frequency of recurrent ear infections. There were no complications in either group. Most children with otitis media can probably be successfully and safely treated with no more than a three-day course of amoxycillin providing their progress is reviewed about the fifth or sixth day after treatment started. This policy could save over 1 million pounds annually in antibiotic costs.

Comparison of the inhibitory action of delta-9-tetrahydrocannabinol and petroleum spirit extract of herbal cannabis on human sperm motility

1 Using the trans-membrane migration method to measure drug effects on human sperm motility, it is found that delta-9-tetrahydrocannabinol is the major constituent of cannabis which inhibits sperm motility and unlike the cataleptic effect, other constituents in the petroleum spirit extract do not potentiate this effect significantly. 2 It is suggested that the mechanisms for sperm immobilization and cataleptic effects are different.

Effects of chlorpromazine and other drugs acting on the central nervous system on human sperm motility

The in vitro effects of chlorpromazine, diazepam, phenytoin and phenobarbitone on human sperm motility were investigated. Only chlorpromazine inhibits human sperm motility and the concentration which decreases sperm motility to 50% of control is 0.22 mM. Caffeine can shift the dose-response curve of chlorpromazine-inhibited sperm motility to right. We support the hypothesis that chlorpromazine acts on the cellular membrane but consider the inhibition of sperm motility an unlikely cause of decreased fertility in chlorpromazine treated patients.

Should erythromycin dose be altered in haemodialysis patients?

Erythromycin kinetics were studied in 17 patients with end stage renal failure treated with maintenance haemodialysis and 9 normal volunteers to discover if dialysis patients needed a modified dose. The elimination half life in dialysis patients (on dialysis days) was similar to that reported in normal subjects. Only small amounts of drug appeared in the dialysate, no patient loosing more than 9 mg in one dialysis. Both patients and volunteers had similar plasma concentrations 8 h after the end of a 5-day course. Protein-binding did not change significantly during dialysis and was similar to that reported in normal subjects. We conclude that dialysis patients requiring 1.5 g of erythromycin stearate daily or less can be given normal doses.

Antibiotic use in otitis media: patient simulations as an aid to audit

Diagnostic and prescribing practices in otitis media vary, and audit is difficult because doctors may not see comparable cases. Seven general practitioners took part in a pilot study to discover if simulated patients evoked the same diagnostic and treatment responses as reveal patients. Forty-eight patients entered the study over three weeks and provided data for two simulations; one included the doctor's description of the ear and the other a photograph instead. Each doctor was shown the two sets of simulations and asked to state his diagnosis and treatment. The diagnoses each doctor reached agreed significantly with those reached on the simulations of the same patients. The decisions to prescribe antibiotics also showed good agreement.

Azathioprine in rheumatoid arthritis: double-blind study of full versus half doses versus placebo

To test whether azathioprine in effective in rheumatoid arthritis in doses smaller than those normally used the drug was tested at 2 dosage levels, 2.5 and 1.25 mg/kg/day (2.5 AZ and 1.25 AZ), against placebo under double-blind conditions over 24 weeks. Dropouts were 7 out of 15 in the 2.5 AZ group, 4 out of 14 in the 1.25 AZ group, and 2 out of 13 in the placebo group. Some significant improvement occurred in all 3 groups, including those on placebo. However, the 2.5 AZ group fared significantly better than the placebo group, while the 1.25 AZ group results tended to fall between the other 2 groups. We conclude that, in order to obtain the reported effectiveness of azathioprine in rheumatoid arthritis, it is necessary to start treatment with 2.5 mg/kg/day. Halving this dosage reduces the effectiveness of the drug.

Hats for the newborn infant

The efficacy of a Gamgee-lined hat in reducing the rate of fall in rectal temperature of infants during the first 30 minutes of life was studied. The trial, which included 211 infants, was randomised, prospective, and controlled. One hundred and seven infants were exposed to overhead radiantheaters. Of these, only 30 had normal deliveries, so the analysis was confined to the infants who were not subjected to radiant heat, and in this group no conclusions were drawn about the efficacy of a hat or a radiant heater. In the 104 infants not subjected to radiant heat, body weight, initial rectal temperature, the application of hat, the environmental temperature, and the duration of exposure while naked were all found to influence measureably the rate of fall in rectal temperature during the first 30 minutes. Gamgee-lined hats should be routinely used to minimise heat loss, especially in small infants exposed at birth, during surgical operations, and during investigations necessitating prolonged exposure.

The therapeutic equivalence of oral and intravenous iron in renal dialysis patients

Oral and intravenous iron were compared in patients treated with renal dialysis by a cross-over trial. Intravenous iron was given over 2 weeks as an iron dextran (equivalent to 100 mg elemental iron). Oral iron was given daily as ferrous sulphate (equivalent to 100 mg elemental iron) in a wax matrix tablet. Each treatment period lasted 26 weeks. There was no significant difference in therapeutic or unwanted effects between the treatments.

Infected left atrial mass within anatomically normal heart

A case is presented with some features of subacute bacterial endocarditis and of left atrial myxoma. Operative removal of a 5x3 cm tumour lying above and attached to an anatomically normal mitral valve reversed a rapidly deteriorating clinical situation. Histologically the entire tumour appeared to be old thrombus, and Gram-positive cocci in its superficial strata corresponded with Streptococcus viridans previously isolated from blood cultures. Many features, including an eight-year history of intermittent neurological disturbances and recent increasing hypergammaglobulinaemia, accord with the diagnosis of atrial myxoma. However, the valvular attachment site, absence of any cardiac structural abnormality, and tumour histopathology with bacterial colonization of the tumour present a unique situation which is explored in the discussion.

A controlled trial of practolol in mild hypertension

1 Fifteen patients with diastolic blood pressures between 90 and 120 mmHg were admitted to a trial comparing placebo with practolol at doses up to 500 mg 12 hourly. 2 The trial was a double blind cross over study. Treatments were allocated at random and each treatment block lasted 12 weeks. 3 Ten patients completed the trial, and none of those who withdrew experienced serious unwanted effects. 4 Practolol caused significant falls in standing systolic and diastolic pressures and the supine diastolic pressure.

Further experience with azathioprine in rheumatoid arthritis

Azathioprine has been shown to reduce the steroid requirements of patients with severe rheumatoid arthritis. Twenty-seven patients treated with azathioprine have now been followed up for 30 months. At the end of this period only 10 were still taking the drug. Maximum steroid reduction occurred within the first 12 months of treatment. Some steroid-sparing effect seemed to persist after the drug was stopped. There was no evidence that azathioprine prevented radiological deterioration. No deaths occurred and toxic effects always reversed on stopping the drug.