51
|
Steyerberg EW, Eijkemans MJ, Harrell FE, Habbema JD. Prognostic modelling with logistic regression analysis: a comparison of selection and estimation methods in small data sets. Stat Med 2000; 19:1059-79. [PMID: 10790680 DOI: 10.1002/(sici)1097-0258(20000430)19:8<1059::aid-sim412>3.0.co;2-0] [Citation(s) in RCA: 539] [Impact Index Per Article: 22.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/17/2023]
Abstract
Logistic regression analysis may well be used to develop a prognostic model for a dichotomous outcome. Especially when limited data are available, it is difficult to determine an appropriate selection of covariables for inclusion in such models. Also, predictions may be improved by applying some sort of shrinkage in the estimation of regression coefficients. In this study we compare the performance of several selection and shrinkage methods in small data sets of patients with acute myocardial infarction, where we aim to predict 30-day mortality. Selection methods included backward stepwise selection with significance levels alpha of 0.01, 0.05, 0. 157 (the AIC criterion) or 0.50, and the use of qualitative external information on the sign of regression coefficients in the model. Estimation methods included standard maximum likelihood, the use of a linear shrinkage factor, penalized maximum likelihood, the Lasso, or quantitative external information on univariable regression coefficients. We found that stepwise selection with a low alpha (for example, 0.05) led to a relatively poor model performance, when evaluated on independent data. Substantially better performance was obtained with full models with a limited number of important predictors, where regression coefficients were reduced with any of the shrinkage methods. Incorporation of external information for selection and estimation improved the stability and quality of the prognostic models. We therefore recommend shrinkage methods in full models including prespecified predictors and incorporation of external information, when prognostic models are constructed in small data sets.
Collapse
Affiliation(s)
- E W Steyerberg
- Center for Clinical Decision Sciences, Department of Public Health, Erasmus University, Rotterdam, The Netherlands.
| | | | | | | |
Collapse
|
52
|
Steyerberg EW, Eijkemans MJ, Harrell FE, Habbema JDF. Prognostic modelling with logistic regression analysis: a comparison of selection and estimation methods in small data sets. Stat Med 2000. [DOI: 10.1002/(sici)1097-0258(20000430)19:8%3c1059::aid-sim412%3e3.0.co;2-0] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/22/2023]
|
53
|
Flanders WD, Tucker G, Krishnadasan A, Martin D, Honig E, McClellan WM. Validation of the pneumonia severity index. Importance of study-specific recalibration. J Gen Intern Med 1999; 14:333-40. [PMID: 10354252 PMCID: PMC1496595 DOI: 10.1046/j.1525-1497.1999.00351.x] [Citation(s) in RCA: 33] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/20/2022]
Abstract
OBJECTIVE To evaluate the predictive validity and calibration of the pneumonia severity-of-illness index (PSI) in patients with community-acquired pneumonia (CAP). PATIENTS Randomly selected patients (n = 1,024) admitted with CAP to 22 community hospitals. MEASUREMENTS AND MAIN RESULTS Medical records were abstracted to obtain prognostic information used in the PSI. The discriminatory ability of the PSI to identify patients who died and the calibration of the PSI across deciles of risk were determined. The PSI discriminates well between patients with high risk of death and those with a lower risk. In contrast, calibration of the PSI was poor, and the PSI predicted about 2.4 times more deaths than actually occurred in our population of patients with CAP. CONCLUSIONS We found that the PSI had good discriminatory ability. The original PSI overestimated absolute risk of death in our population. We describe a simple approach to recalibration, which corrected the overestimation in our population. Recalibration may be needed when transporting this prediction rule across populations.
Collapse
Affiliation(s)
- W D Flanders
- Department of Epidemiology, Rollins School of Public Health of Emory University, Atlanta, GA 30322, USA
| | | | | | | | | | | |
Collapse
|
54
|
McIsaac WJ, Goel V. Effect of an explicit decision-support tool on decisions to prescribe antibiotics for sore throat. Med Decis Making 1998; 18:220-8. [PMID: 9566455 DOI: 10.1177/0272989x9801800211] [Citation(s) in RCA: 37] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Studies of scoring rules for sore throat have failed to show that they lower antibiotic prescription rates. The authors studied the effect of an explicit decision-support tool, incorporating a modified score, on antibiotic-prescription decisions. Four hundred and fifty family physicians received an information package, a score card, and a recording form to use during one sore-throat encounter. The physicians randomly received either a control form or an intervention form that required them to interact with the score during the clinical recording process. There was a trend towards a reduction in antibiotic prescriptions (21%, p=0.09) in the physicians using the intervention form. A greater reduction (45%, p=0.06) was observed for patients whose probabilities of infection with group A streptococcus were low. Sore-throat-scoring rules may reduce unnecessary antibiotic prescriptions if physicians are specifically cued to use them during clinical encounters and appropriate management responses are linked to score estimates for the likelihood of group A streptococcus infection.
Collapse
Affiliation(s)
- W J McIsaac
- Mt. Sinai Family Medicine Centre, University of Toronto, Ontario, Canada.
| | | |
Collapse
|
55
|
Furukawa T, Hirai T, Kitamura T, Takahashi K. Application of the Center for Epidemiologic Studies Depression Scale among first-visit psychiatric patients: a new approach to improve its performance. J Affect Disord 1997; 46:1-13. [PMID: 9387082 DOI: 10.1016/s0165-0327(97)00079-7] [Citation(s) in RCA: 50] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/05/2023]
Abstract
BACKGROUND Although the Center for Epidemiologic Studies Depression Scale (CES-D) is an internationally popular self-rating scale for depression both in community and clinical settings, extant literature concerning its validity has several shortcomings. The present paper aimed to overcome these problems. METHODS We applied newer assessment technology of receiver operating characteristics (ROC) analyses and stratum-specific likelihood ratios (SSLRs) and cross-validated the results in the 'training' and 'testing' data sets of 591 patients representing various clinical settings all over Japan. RESULTS The ROC analyses demonstrated that the CES-D had moderate convergent and discriminant validity to detect major depressive episodes among first-visit psychiatric patients. Selecting single optimal cutoffs, however, failed to arrive at consistent results across various settings. The efficacy of the instrument was most conveniently transportable into clinical practices when converted into SSLRs, which were 0.35 (95% CI: 0.25-0.49) for the score range 0-29, 2.3 (1.8-3.1) for the score range 30-49, and 11.7 (3.1-44.0) for the scores above 50. In addition, the SSLRs proved to be generalizable not only across various clinical settings in our sample but also across psychiatric, primary care and community samples in the published reports. CONCLUSION Clinicians and clinical epidemiologists can apply the SSLRs of the CES-D to various settings to estimate the probability of suffering from a major depressive episode in a convenient and intuitive manner.
Collapse
Affiliation(s)
- T Furukawa
- Department of Psychiatry, Nagoya City University Medical School, Japan.
| | | | | | | |
Collapse
|
56
|
Aron DC, Raff H, Findling JW. Effectiveness versus efficacy: the limited value in clinical practice of high dose dexamethasone suppression testing in the differential diagnosis of adrenocorticotropin-dependent Cushing's syndrome. J Clin Endocrinol Metab 1997; 82:1780-5. [PMID: 9177382 DOI: 10.1210/jcem.82.6.3991] [Citation(s) in RCA: 39] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/04/2023]
Abstract
High dose dexamethasone suppression testing has been widely employed in the differentiation between pituitary ACTH-dependent hypercortisolism [Cushing's disease (CD)] and the ectopic ACTH syndrome. We hypothesized that the high dose dexamethasone suppression test as it is performed in practice does not improve the ability to differentiate between these two types of ACTH-dependent Cushing's syndrome. Cases were drawn from 112 consecutive patients with ACTH-dependent Cushing's syndrome, who were then classified based upon results of inferior petrosal sinus sampling for ACTH levels. Analysis of test characteristics of high dose dexamethasone suppression testing was performed in the 73 patients for whom results are available. Statistical modeling was performed using the 68 cases with complete data on all assessed variables. Logistic regression models were used to predict the probability of pituitary-dependent Cushing's syndrome (CD) given the results of high dose dexamethasone suppression testing before and after adjustment for the contribution of a series of potential covariates. Of the 112 patients with ACTH-dependent Cushing's syndrome, 15.2% had the ectopic ACTH syndrome, and the remainder had pituitary-dependent Cushing's syndrome (CD). Patients with the ectopic ACTH syndrome were significantly older (mean, 51.9 vs. 40.2), were more likely to be male (58.8% vs. 27.4%), had shorter duration of clinical findings (mean, 11.6 vs. 39.9 months), were more likely to have hypokalemia (50% vs. 8.6%), had higher baseline 24-h urinary free cortisol [mean, 8317 vs. 1164 nmol/day (3015 vs. 422 microg)] and plasma ACTH levels [mean, 47 vs. 17 pmol/L (210 vs. 78 pg/mL)] and were less likely to suppress urinary free cortisol or plasma cortisol with high dose dexamethasone using the standard criterion of 50% or more suppression compared with patients with pituitary-dependent Cushing's syndrome. Based upon the standard criterion, the sensitivity and specificity of the high dose dexamethasone suppression test for the diagnosis of pituitary-dependent Cushing's syndrome were 81.0% and 66.7%, respectively. Although the mean percent suppression was significantly greater for patients with CD than for those with the ectopic ACTH syndrome (72.2% vs. 41.3%), the range of suppression was 0-99% for each diagnosis. The area under the receiver operating characteristic curve was 0.710 (95% confidence interval, 0.541-0.879). Logistic regression models were used to evaluate the probability of CD given the responsiveness to high dose dexamethasone suppression testing before and after adjustment for the potential contributions of other factors. A model including all of the variables (age, sex, duration, presence of hypokalemia, urinary free cortisol, and plasma ACTH) had a diagnostic accuracy of 92.7%. A model including all of these variables plus a binary variable indicating whether the patient met the criterion of suppression by 50% or more resulted in 95.6% accuracy, whereas substitution of this binary variable by percent suppression resulted in a model with 94.1% accuracy. There were no statistically significant differences among these models; their values for the c statistic, which is equivalent to the area under the curve in a receiver operating characteristic analysis, were all greater than 0.9. Logistic regression models indicate that the results of the dexamethasone suppression test add little to the differential diagnosis of ACTH-dependent Cushing's syndrome, especially after taking other clinical information into account. In our patient population, the sensitivity and specificity of the dexamethasone suppression test were less than those reported by others. However, because 20-33% of cases of ectopic ACTH syndrome are misdiagnosed with these logistic regression models, other techniques are necessary to achieve greater diagnostic accuracy.
Collapse
Affiliation(s)
- D C Aron
- Division of Clinical and Molecular Endocrinology, Department of Veterans Affairs Medical Center, and Case Western Reserve University School of Medicine, Cleveland, Ohio 44106, USA.
| | | | | |
Collapse
|
57
|
Do D, West JA, Morise A, Atwood E, Froelicher V. A consensus approach to diagnosing coronary artery disease based on clinical and exercise test data. Chest 1997; 111:1742-9. [PMID: 9187202 DOI: 10.1378/chest.111.6.1742] [Citation(s) in RCA: 25] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/04/2023] Open
Abstract
OBJECTIVE To demonstrate that a consensus approach for combining prediction equations based on clinical and exercise test variables derived from different populations can stratify patients referred for possible coronary artery disease (CAD) into low-, intermediate-, and high-risk groups. DESIGN Retrospective analysis of consecutive patients with complete data from exercise testing and coronary angiography referred for evaluation of possible CAD. After derivation of a logistic equation in our own training set of patients, this equation, along with two other equations developed independently by other investigators, was validated in a test set. The validation strategy for the consensus approach included the following: (1) calculation of probability scores for each patient using each logistic equation independently; (2) determination of probability thresholds in the training set to divide the patients into three groups-low risk (prevalence CAD <5%), intermediate risk (5 to 70%), and high risk (>70% prevalence of CAD); (3) using agreement among at least two of three of the prediction equations to generate "consensus" for each patient; and (4) application of the consensus approach thresholds to the test set of patients. SETTINGS Two university-affiliated Veteran's Affairs medical centers. PATIENTS We studied 718 consecutive men between 1985 and 1995 who had coronary angiography within 3 months of an exercise treadmill test for suspected CAD. The population was randomly divided into a training set of 429 patients and a test set of 289 patients. Patients with previous myocardial infarction or coronary artery bypass surgery, valvular heart disease, left bundle branch block, or any Q waves present on their resting ECG were excluded from the study. MEASUREMENTS Recording of clinical and exercise test data along with visual interpretation of the ECG recordings on standardized forms and abstraction of visually interpreted angiographic data from clinical catheterization reports. RESULTS We demonstrated that by using simple clinical and exercise test variables, we could improve on the standard use of ECG criteria during exercise testing for diagnosing CAD. Using the consensus approach divided the test set into populations with low, intermediate, and high risk for CAD. Since the patients in the intermediate group would be sent for further testing and would eventually be correctly classified, the sensitivity of the consensus approach is 94% and the specificity is 92%. The consensus approach controls for varying disease prevalence, missing data, inconsistency in variable definition, and varying angiographic criterion for stenosis severity. The percent of correct diagnoses increased from the 67% for standard exercise ECG analysis and from the 80% for multivariable predictive equations alone to >90% correct diagnoses for the consensus approach. CONCLUSIONS The consensus approach has made population-specific logistic regression equations portable to other populations. Excellent diagnostic characteristics can be obtained using simple data and measurements. The consensus approach is best applied utilizing a programmable calculator or a computer program to simplify the process of calculating the probability of CAD using the three equations.
Collapse
Affiliation(s)
- D Do
- Cardiology Division, Veterans Affairs Palo Alto Health Care System, Stanford University, Calif 94304, USA
| | | | | | | | | |
Collapse
|
58
|
Abstract
We have demonstrated using several examples how different test characteristics can be used to assist clinicians in making better decisions for their patients. These probabilistic models may seem confusing and difficult to implement. Some general rules may help, such as SnNout and SpPin. Clinicians should know the test characteristics and decision rules for the acute problems they may face. For chronic conditions, advanced planning may be helpful. Electronic medical record systems may be able to incorporate these at the user interface. The improvements in hand-held computers may bring clinical decision-support systems directly to the point of service. We may also begin to see laboratories report test characteristics for important conditions as likelihood ratios (we already see estimates of the risk of heart disease corresponding to different lipid ratios). We also suspect that the medical literature will report likelihood ratios more frequently. As practice networks develop more sophisticated disease-tracking mechanisms, clinicians will be able to obtain estimates of disease prevalence more appropriate to their practice. Ultimately, for physicians to make better decisions, appropriate data are needed, including accurate estimates of test characteristics and of disease probability.
Collapse
Affiliation(s)
- H C Barry
- Department of Family Practice, Michigan State University, College of Human Medicine, East Lansing, USA
| | | |
Collapse
|
59
|
Morise AP, Diamond GA, Detrano R, Bobbio M, Gunel E. The effect of disease-prevalence adjustments on the accuracy of a logistic prediction model. Med Decis Making 1996; 16:133-42. [PMID: 8778531 DOI: 10.1177/0272989x9601600205] [Citation(s) in RCA: 24] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/02/2023]
Abstract
The accuracy of a logistic prediction model is degraded when it is transported to populations with outcome prevalences different from that of the population used to derive the model. The resultant errors can have major clinical implications. Accordingly, the authors developed a logistic prediction model with respect to the noninvasive diagnosis of coronary disease based on 1,824 patients who underwent exercise testing and coronary angiography, varied the prevalence of disease in various "test" populations by random sampling of the original "derivation" population, and determined the accuracy of the logistic prediction model before and after the application of a mathematical algorithm designed to adjust only for these differences in prevalence. The accuracy of each prediction model was quantified in terms of receiver operating characteristic (ROC) curve area (discrimination) and chi-square goodness-of-fit (calibration). As the prevalence of the test population diverged from the prevalence of the derivation population, discrimination improved (ROC-curve areas increased from 0.82 +/- 0.02 to 0.87 +/- 0.03; p < 0.05), and calibration deteriorated (chi-square goodness-of-fit statistics increased from 9 to 154; p < 0.05). Following adjustment of the logistic intercept for differences in prevalence, discrimination was unchanged and calibration improved (maximum chi-square goodness-of-fit fell from 154 to 16). When the adjusted algorithm was applied to three geographically remote populations with prevalences that differed from that of the derivation population, calibration improved 87%, while discrimination fell by 1%. Thus, prevalence differences produce statistically significant and potentially clinically important errors in the accuracy of logistic prediction models. These errors can potentially be mitigated by use of a relatively simple mathematical correction algorithm.
Collapse
Affiliation(s)
- A P Morise
- Department of Medicine, West Virginia University School of Medicine, Morgantown 26506, USA
| | | | | | | | | |
Collapse
|
60
|
Abstract
BACKGROUND Artificial neural networks apply non-linear statistics to pattern recognition problems. One such problem is acute myocardial infarction (AMI), a diagnosis which, in a patient presenting as an emergency, can be difficult to confirm. We report here a prospective comparison of the diagnostic accuracy of a network and that of physicians, on the same patients with suspected AMI. METHODS Emergency department physicians who evaluated 1070 patients 18 years or older presenting to the emergency department of a teaching hospital in California, USA with anterior chest pain indicated whether they thought these patients had sustained a myocardial infarction. The network analysed the patient data collected by the physicians during their evaluations and also generated a diagnosis. FINDINGS The physicians had a diagnostic sensitivity and specificity for myocardial infarction of 73.3% (95% confidence interval 63.3-83.3%) and 81.1% (78.7-83.5%), respectively, while the network had a diagnostic sensitivity and specificity of 96.0% (91.2-100%) and 96.0% (94.8-97.2%), respectively. Only 7% of patients had had an AMI, a low frequency but typical for anterior chest pain. INTERPRETATION The application of non-linear neural computational analysis via an artificial neural network to the clinical diagnosis of myocardial infarction appears to have significant potential.
Collapse
Affiliation(s)
- W G Baxt
- Department of Emergency Medicine, University of California, San Diego Medical Center, USA
| | | |
Collapse
|
61
|
Stiell IG, Greenberg GH, Wells GA, McKnight RD, Cwinn AA, Cacciotti T, McDowell I, Smith NA. Derivation of a decision rule for the use of radiography in acute knee injuries. Ann Emerg Med 1995; 26:405-13. [PMID: 7574120 DOI: 10.1016/s0196-0644(95)70106-0] [Citation(s) in RCA: 171] [Impact Index Per Article: 5.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/26/2023]
Abstract
STUDY OBJECTIVE To derive a highly sensitive decision rule for the selective use of radiography in acute knee injuries. DESIGN Prospectively administered survey. SETTING Emergency departments of two university hospitals. PARTICIPANTS Convenience sample of 1,047 adults with acute knee injuries. RESULTS Attending emergency physicians assessed each patient for 23 standardized clinical findings, which were recorded on data collection forms. A total of 127 patients was examined independently by two physicians to determine interobserver agreement. The outcome measure was fracture of the knee. Any patients who did not have ED radiography underwent a structured telephone interview to determine the possibility of a missed fracture. Those variables found to be both reliable (highest kappa values) and strongly associated with a fracture (highest chi 2 values) were further analyzed by a recursive-partitioning multivariate technique. The derived decision rule included the following variables: (1) age 55 years or older, (2) tenderness at the head of the fibula, (3) isolated tenderness of the patella, (4) inability to flex to 90 degrees, and (5) inability to bear weight both immediately and in the ED (four steps). The presence of one or more of these findings would have identified the 68 fractures in the study population with a sensitivity of 1.0 (95% confidence interval [Cl], .95 to 1.0) and a specificity of .54 (95% Cl, .51 to .57). Application of the rule would have led to a 28.0% relative reduction in the use of radiography from 68.6% to 49.4% in the study population. CONCLUSION A practical, highly sensitive, and reliable decision rule for the use of radiography in acute knee injuries has been derived. Clinical application should await prospective validation of the rule.
Collapse
Affiliation(s)
- I G Stiell
- Department of Medicine, University of Ottawa, Ontario, Canada
| | | | | | | | | | | | | | | |
Collapse
|
62
|
Do women and men receive different care for sore throats? MOTIVATION AND EMOTION 1995. [DOI: 10.1007/bf02250511] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
|
63
|
|
64
|
Poses RM, Cebul RD, Wigton RS. You can lead a horse to water--improving physicians' knowledge of probabilities may not affect their decisions. Med Decis Making 1995; 15:65-75. [PMID: 7898300 DOI: 10.1177/0272989x9501500110] [Citation(s) in RCA: 81] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/27/2023]
Abstract
OBJECTIVES To determine whether improving physicians' judgments of the probability of streptococcal pharyngitis for patients with sore throats would affect their use of antibiotics and affect the variation in such use. DESIGN Post-hoc retrospective analysis of data previously collected as part of a controlled trial. SETTINGS University student health services in Pennsylvania and Nebraska. PATIENTS Sequential patients with pharyngitis seen before and after the time clinicians received either an experimental educational intervention designed to improve probabilistic diagnostic judgments (at the Pennsylvania site) or a control intervention, a standard lecture (at the Nebraska site). The clinician-subjects were the primary case physicians practicing at either site. MEASUREMENTS Clinical variables prospectively recorded by the clinicians, probability assessments, and treatment decisions. RESULTS At the experimental site, despite marked decreases in clinicians' overestimations of disease probability after the intervention, the proportion of patients prescribed antibiotics showed a trend toward increasing: 100/290 (34.5%) pre-intervention, 90/225 (40%) post-intervention. The intervention did not decrease practice variation between individual doctors. Univariable and multivariable analyses showed no major change in the relationships between clinical variables and treatment decisions after the intervention. At the control site there was no major change in probability judgments or treatment decisions after the intervention. CONCLUSIONS Teaching physicians to make better judgments of disease probability may not alter their treatment decisions.
Collapse
Affiliation(s)
- R M Poses
- Division of General Medicine, Medical College of Virginia, Richmond
| | | | | |
Collapse
|
65
|
O'Keeffe S, Ebell MH. Prediction of failure to survive following in-hospital cardiopulmonary resuscitation: comparison of two predictive instruments. Resuscitation 1994; 28:21-5. [PMID: 7809481 DOI: 10.1016/0300-9572(94)90050-7] [Citation(s) in RCA: 50] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/27/2023]
Abstract
The purpose of this study is to compare two clinical predictive rules, the pre-arrest-morbidity (PAM) index and the prognosis-after-resuscitation (PAR) score, which predict failure to survive following in-hospital cardiopulmonary resuscitation (CPR). The study population consisted of 274 consecutive adult patients who underwent CPR at University College Hospital in Galway, Ireland over a 2-year period. The PAM and PAR scores were calculated from the most recent data available for each variable prior to cardiac arrest. Performance of the predictive scores was compared using Student's t-test, Pearson chi-square, Fisher's exact test, and receiver-operating characteristic (ROC) curves where appropriate. The PAM index identified 23 patients with a score > 4, while the PAR score identified 59 patients with a score > 5, none of whom survived. The sensitivity of the PAR score for the prediction of failure to survive was 23.7%, while that of the PAM index was 9.2%; neither index incorrectly identified a patient as a non-survivor who eventually survived. The PAR score also had a greater area under the ROC curve, although this difference was not statistically significant (P = 0.07). In summary, the PAR score performed better than the PAM index in the identification of patients who are unlikely to survive following CPR. Although further confirmation is necessary, it may provide useful prognostic information to physicians and patients involved with decisions about do-not-resuscitate orders.
Collapse
Affiliation(s)
- S O'Keeffe
- Department of Geriatric Medicine, Royal Liverpool University Hospital, UK
| | | |
Collapse
|
66
|
Birnbaum A, Gallagher J, Utkewicz M, Gennis P, Carter W. Failure to validate a predictive model for refusal of care to emergency-department patients. Acad Emerg Med 1994; 1:213-7. [PMID: 7621199 DOI: 10.1111/j.1553-2712.1994.tb02434.x] [Citation(s) in RCA: 49] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/26/2023]
Abstract
OBJECTIVE To determine whether previously developed triage criteria for refusal of care to patients presenting to an emergency department (ED) with nonurgent problems could be validated for an independent patient population. METHODS A convenience sample of 534 adults presenting to a municipal hospital ED between July 1, 1992, and October 15, 1992, who met preestablished criteria for refusal of care were entered into a prospective, observational, cohort study. The single target outcome variable was hospitalization. In order to optimize the criteria's performance, both the triage nurse and the physician caring for the patient had to agree that all criteria for "refusal of care" were specifically met. No patient was refused care, nor was a patient's management or disposition interfered with in any way by the investigators. All patients were followed until hospital admission or release from the ED. RESULTS Six (1.1%) of 534 patients (95% CI 0.4-2.4) who met the criteria for refusal of care were hospitalized. This represents a greater than 50-fold difference in incidence of hospitalization when compared with that found by other investigators, who reported that only 0.02% (95% CI 0.0004-0.04) of those patients who were refused care subsequently required hospitalization (p < 10 (-7)). CONCLUSION The authors were unable to validate a previously developed predictive model for refusal of care to patients presenting to an ED. Refusal of care to selected ED patients based on current guidelines is not a viable solution to overcrowding. Alternative strategies must be sought.
Collapse
Affiliation(s)
- A Birnbaum
- Department of Medicine, Albert Einstein College of Medicine, Bronx, NY, USA
| | | | | | | | | |
Collapse
|
67
|
Affiliation(s)
- I G Stiell
- Clinical Epidemiology Unit, Ottawa Civic Hospital, Ontario, Canada
| |
Collapse
|
68
|
McKinney WP, Heudebert GR, Harper SA, Young MJ, McIntire DD. Validation of a clinical prediction rule for the differential diagnosis of acute meningitis. J Gen Intern Med 1994; 9:8-12. [PMID: 8133355 DOI: 10.1007/bf02599135] [Citation(s) in RCA: 30] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/29/2023]
Abstract
OBJECTIVE To attempt to validate a previously reported clinical prediction rule derived to assist in distinguishing between acute bacterial meningitis and acute viral meningitis. DESIGN Retrospective chart review of patients treated at five hospitals between 1981 and 1990. The criterion standard for bacterial meningitis was a positive cerebrospinal fluid (CSF) or blood culture or a positive test for bacterial antigen in the CSF. For viral meningitis, the criterion standard was a positive viral culture from CSF, stool, or blood or a discharge diagnosis of viral meningitis with no other etiology evident. SETTING Two Department of Veterans Affairs (VA) hospitals, two county hospitals, and one private hospital, each affiliated with one of two medical schools. PATIENTS All persons aged more than 17 years who were hospitalized over a ten-year period at one of five academically affiliated hospitals for the management of acute meningitis. MEASUREMENTS AND MAIN RESULTS Sixty-two cases of bacterial meningitis and 98 cases of viral meningitis were confirmed. With all patients included, the discriminatory power of the model as measured by the area under the receiver operating characteristic curve (AUC) was 0.977 (95% CI, 0.957-0.997), compared with the AUC of 0.97 in the derivation set of the original publication. The AUCs (95% CIs) for data subsets were: Dallas cases 0.994 (0.986-1.0). Milwaukee cases 0.912 (0.834-0.990); ages 18-39 years 0.952 (0.892-1.0), ages 40-59 years 0.99 (0.951-1.0), and age > or = 60 years 0.955 (0.898-1.0). CONCLUSIONS The authors conclude that the clinical prediction rule proved robust when applied to a geographically distinct population comprised exclusively of adults. There was sustained performance of the model when applied to cases from each city and from three age strata. Prospective validation of this prediction rule will be necessary to confirm its utility in clinical practice.
Collapse
Affiliation(s)
- W P McKinney
- Department of Internal Medicine, UT/Southwestern Medical School, Dallas, Texas
| | | | | | | | | |
Collapse
|
69
|
Poses RM, Wigton RS, Cebul RD, Centor RM, Collins M, Fleischli GJ. Practice variation in the management of pharyngitis: the importance of variability in patients' clinical characteristics and in physicians' responses to them. Med Decis Making 1993; 13:293-301. [PMID: 8246701 DOI: 10.1177/0272989x9301300405] [Citation(s) in RCA: 18] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/29/2023]
Abstract
The objective of this study was to assess whether geographic differences in antibiotic-prescribing rates for patients with pharyngitis could be explained by intersite differences in patients' clinical characteristics and in how physicians responded to these clinical cues when making decisions. As part of the initial phase of a prospective controlled trial to improve physicians' diagnostic ability, the authors enrolled cohorts of consecutive patients seen at staff-model--HMO student health services in Pennsylvania and Nebraska. Physicians' decisions whether to prescribe antibiotics for 310 consecutive patients presenting with pharyngitis to the former and 214 such patients presenting to the latter at the time of the initial visit were examined. There was a large discrepancy between the antibiotic-prescribing rates at the student health services in Pennsylvania, 106/310, 32.4%, and Nebraska, 156/214, 72.9%. The clinical variables significantly independently associated with treatment at both sites in a logistic regression model were fever, adjusted odds ratio = 2.1 (95% CI = 1.1, 3.8); exudates, 5.4 (2.8, 10); palatine petechiae, 6.5 (1.5, 28); rhinorrhea, 0.46, (0.25, 0.85); and high risk of complications, 3.8 (1.04, 14). There was a significant interaction between site and anterior cervical adenopathy, 5.5 (1.6, 19); and a borderline interaction between site and rhinorrhea, 2.4 (0.89, 6.7). Site was not a significant independent predictor of treatment, 1.8 (0.45, 6.6.). Practice variation was related to geographic differences in patients' clinical characteristics and in how physicians responded to these factors when prescribing antibiotics. How physicians weight patients' clinical characteristics when making decisions may be an important element of their "practice styles."
Collapse
Affiliation(s)
- R M Poses
- Division of General Medicine, Medical College of Virginia, Richmond
| | | | | | | | | | | |
Collapse
|
70
|
|
71
|
|
72
|
Peirce JC, Cornell RG. Integrating stratum-specific likelihood ratios with the analysis of ROC curves. Med Decis Making 1993; 13:141-51. [PMID: 8483399 DOI: 10.1177/0272989x9301300208] [Citation(s) in RCA: 98] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/31/2023]
Abstract
Data used to construct receiver operating characteristic (ROC) curves and to calculate the area under the curve (ROC AUC) can be used to derive stratum-specific likelihood ratios (SSLRs) with their 95% confidence intervals (95% CIs). The purpose of this study was to determine whether useful information can be obtained by adding SSLRs to the analysis of ROC curves. The authors analyzed four previously reported sets of data: 1) serum creatine kinase (SCK) for diagnosing acute myocardial infarction (AMI) in the coronary care unit (CCU); 2) SCK in the evaluation of chest pain in the emergency center (EC); 3) four predictor variables in the diagnosis of strep throat; and 4) the ordinal assessment of computed tomographic (CT) images. Use of SCK in the CCU produced four strata that had posttest probabilities that were highly discriminating, whereas SCK in the EC resulted in only two strata with limited discriminating ability. In either study the cutpoint at which the SSLR changed from less than to greater than 1.0 was higher than the reported upper normal for the test, thereby quantitating spectrum bias. The maximum number of strata of predictor signs and symptoms for strep throat was three rather than the five used in previous studies. With a larger sample size or pooling, four strata could probably be developed. With CT images, "definitely normal," "probably normal," and "questionable" were collapsed to one negative stratum. "Probably abnormal" became the true "questionable" stratum and "definitely abnormal" was the only positive stratum. The authors conclude that additional useful information is obtained by deriving stratum-specific likelihood ratios as part of the analysis of an ROC curve.
Collapse
Affiliation(s)
- J C Peirce
- Department of Medical Education and Research, Good Samaritan Regional Medical Center, Phoenix, Arizona 85006
| | | |
Collapse
|
73
|
Emerman CL, Cydulka RK. Evaluation of high-yield criteria for chest radiography in acute exacerbation of chronic obstructive pulmonary disease. Ann Emerg Med 1993; 22:680-4. [PMID: 8457095 DOI: 10.1016/s0196-0644(05)81847-5] [Citation(s) in RCA: 37] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/30/2023]
Abstract
STUDY OBJECTIVES The objectives of this study were to assess the incidence of abnormal chest radiographs and to test the validity of previously developed high-yield criteria. There is disagreement about the need for chest radiography in acute exacerbation of chronic obstructive pulmonary disease, although high-yield criteria have been developed. DESIGN Retrospective chart review study. SETTING County-owned, university-affiliated, urban emergency department. PARTICIPANTS ED patients seen between January 1988 and July 1991 with chronic obstructive pulmonary disease. RESULTS Eight hundred forty-seven ED visits were identified; medical records were available for 742. Radiographs were not taken in 8%, leaving 685 ED visits in the study. One hundred nine patients (16%) had significant abnormalities, including 88 new infiltrates, two new lung masses, one pneumothorax, and 20 episodes of pulmonary edema. A history of congestive heart failure and fever was associated with abnormalities, as were findings of rales, pedal edema, and jugular venous distension. There was no association with WBC count, temperature, coronary artery disease, chest pain, or sputum production. Previously published high-yield criteria had a sensitivity of .76; specificity, .41; positive predictive value, 20; negative predictive value, .90; and accuracy, .47. CONCLUSION Radiographic abnormalities are common findings in acute exacerbation of chronic obstructive pulmonary disease. We found that almost one fourth of radiographic abnormalities are not predictable on the basis of previously developed high-yield criteria. Routine chest radiography should be considered in patients with acute exacerbation of chronic obstructive pulmonary disease to diagnose treatable, radiographically apparent abnormalities.
Collapse
Affiliation(s)
- C L Emerman
- Department of Emergency Medicine, MetroHealth Medical Center, Cleveland, Ohio
| | | |
Collapse
|
74
|
|
75
|
Miller ME, Langefeld CD, Tierney WM, Hui SL, McDonald CJ. Validation of probabilistic predictions. Med Decis Making 1993; 13:49-58. [PMID: 8433637 DOI: 10.1177/0272989x9301300107] [Citation(s) in RCA: 95] [Impact Index Per Article: 3.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/30/2023]
Abstract
Current advances in high-speed computing and increased availability of statistical software have led to widespread use of statistical methods for the development of computerized protocols predictive of binary health outcomes. If these predictive algorithms are to be used in settings other than those for which they were developed, e.g., applied in a different geographic setting or extrapolated for use in a slightly different population, then they should be carefully validated to ensure appropriate application. Miller et al. (Stat Med. 1991) provided a comprehensive methodology for external validation of logistic prediction models, and applied these methods in a temporal validation setting. In this article, the authors emphasize how these methods can be applied to general forms of probabilistic predictions and provide several SAS macros for computation of the desired statistics.
Collapse
Affiliation(s)
- M E Miller
- Begenstrief Institute for Health Care, Indiana University Department of Medicine, Indianapolis
| | | | | | | | | |
Collapse
|
76
|
Hoffmann S. An algorithm for a selective use of throat swabs in the diagnosis of group A streptococcal pharyngo-tonsillitis in general practice. Scand J Prim Health Care 1992; 10:295-300. [PMID: 1480870 DOI: 10.3109/02813439209014077] [Citation(s) in RCA: 13] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/27/2022] Open
Abstract
A prospective evaluation was made of an algorithm for a selective use of throat swabs in patients with sore throat in general practice. The algorithm states that a throat swab should be obtained (a) in all children younger than 15 years; (b) in patients aged 15 years or more who have pain on swallowing and at least three of four signs (enlarged or hyperaemic tonsils; exudate; enlarged or tender angular lymph nodes; and a temperature > or = 38 degrees C); and (c) in adults aged 15-44 years with pain on swallowing and one or two of the four signs, but not both cough and coryza. Group A streptococci were found by laboratory culture in 30% of throat swabs from 1783 patients. Using these results as the reference, the algorithm was 95% sensitive and 26% specific, and assigned 80% of the patients to be swabbed. Its positive and negative predictive values in this setting were 36% and 92%, respectively. It is concluded that this algorithm may be useful in general practice.
Collapse
Affiliation(s)
- S Hoffmann
- Department of Bacteriology, Statens Seruminstitut, Copenhagen, Denmark
| |
Collapse
|
77
|
Detrano R, Bobbio M, Olson H, Shandling A, Ellestad MH, Alegria E, Martinez-Caro D, Righetti A, Janosi A, Steinbrunn W. Computer probability estimates of angiographic coronary artery disease: transportability and comparison with cardiologists' estimates. COMPUTERS AND BIOMEDICAL RESEARCH, AN INTERNATIONAL JOURNAL 1992; 25:468-85. [PMID: 1395523 DOI: 10.1016/0010-4809(92)90004-t] [Citation(s) in RCA: 21] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/26/2022]
Abstract
A computer algorithm for estimating probabilities of any significant coronary obstruction and triple vessel/left main obstructions was derived, validated, and compared with the assessments of cardiac clinician angiographers. The algorithm performed at least as well as the clinicians when the latter knew the identity of the patients whose angiograms they had decided to perform. The clinicians were more accurate when they did not know the identity of the subjects but worked from tabulated objective data. Referral and value induced bias may affect physician judgment in assessing disease probability. Application of computer aids or consultation with cardiologists not directly involved with patient management may assist in more rational assessments and decision making.
Collapse
Affiliation(s)
- R Detrano
- Department of Medicine, Saint John's Cardiovascular Research Center, Harbor-UCLA Medical Center, Torrance 90502
| | | | | | | | | | | | | | | | | | | |
Collapse
|
78
|
Heckerling PS, Conant RC, Tape TG, Wigton RS. Reproducibility of predictor variables from a validated clinical rule. Med Decis Making 1992; 12:280-5; discussion 286-7. [PMID: 1484477 DOI: 10.1177/0272989x9201200406] [Citation(s) in RCA: 14] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/27/2022]
Abstract
It has been suggested that clinical prediction rules are not reproducible, and that the most important variables frequently do not appear in replicate models. The authors studied the reproducibility of a validated rule for predicting radiographic evidence of pneumonia (ROC areas for the training and validation cohorts, 0.816 and 0.821, respectively). Two hundred replicate samples of size 250 and size 500 were generated by sampling without replacement from the original training cohort of 905 patients with a 14.6% prevalence of pneumonia. Forward selection was performed among 31 candidate variables by stepwise logistic regression. Using as reproducibility criteria: 1) inclusion of all five variables from the original model in the original order; 2) inclusion of all five variables in any order; 3) inclusion of the first three variables; 4) inclusion of the first two variables; 5) inclusion of the first variable; and 6) inclusion of any of the five variables: 2.5%, 13.5%, 48.5%, 85.5%, 98.0%, and 100% of replicate models of sample size 500, respectively, met the criteria, whereas 0%, 0%, 16.5%, 49.0%, 71.5%, and 97.5% of models of sample size 250 met the criteria (all comparisons by sample size p < .0001 except for criteria 1 and 6, p = 0.07). Mean ROC areas in the training and validation samples were 0.829 and 0.791 for replicate models of sample size 500, and 0.831 and 0.779 for models of sample size 250. There was no significant difference in ROC areas between training and validation cohorts for 80.5% of models of sample size 500, and for 75.3% of models of sample size 250.(ABSTRACT TRUNCATED AT 250 WORDS)
Collapse
Affiliation(s)
- P S Heckerling
- Department of Medicine, University of Illinois, Chicago 60680
| | | | | | | |
Collapse
|
79
|
Hedges JR, Singal BM, Estep JL. The impact of a rapid screen for streptococcal pharyngitis on clinical decision making in the emergency department. Med Decis Making 1991; 11:119-24. [PMID: 1865780 DOI: 10.1177/0272989x9101100209] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/29/2022]
Abstract
The impact of a rapid streptococcal screening test (RSS) on clinical decision making in the management of patients with acute pharyngitis was evaluated. Physicians managing 95 ambulatory patients with the complaint of sore throat were asked to estimate the probability of group A beta-hemolytic streptococcal pharyngitis (GRABS) in each patient and whether they intended to treat with antibiotics both before and after the results of the RSS were known. Simultaneous throat cultures were obtained for 80 patients at the discretion of the treating physicians. Regression analysis revealed that the RSS result was an important independent predictor for the posttest decision to treat with antibiotics and for the estimated probability of disease. The mean absolute log-likelihood ratio was used to quantify the effect of the RSS on diagnostic certainty; a greater contribution to diagnostic certainty occurred when the RSS was positive. Physicians would have prescribed an antibiotic 49 times without the benefit of the RSS and prescribed an antibiotic 48 times given the RSS result. Without the RSS, the treatment decision contradicted the throat culture result in 25 cases. With the RSS, this occurred in 26 cases. The RSS, while influencing decision making, especially when positive, did not decrease the use of antibiotics.
Collapse
Affiliation(s)
- J R Hedges
- Oregon Health Sciences University, Division of Emergency Medicine, Portland 97201-3098
| | | | | |
Collapse
|
80
|
Abstract
Confidence intervals are a natural way to describe the uncertainty of post-test probability in diagnostic tests. We consider confidence intervals for two different scenarios. At a site, for example, hospital emergency room or student health centre, with measured values of disease prevalence, sensitivity and specificity available, the confidence interval is similar to results in the literature, but at a site where measured values of these indices are unavailable, we develop a method, using the values of disease prevalence, sensitivity and specificity from other sites, to obtain a confidence interval for post-test probability. We use the diagnosis of strep throat to illustrate the results. We also obtain confidence intervals from simulations to compare with the results of both scenarios.
Collapse
Affiliation(s)
- M J Monsour
- Department of Biostatistics, School of Public Health, University of North Carolina, Chapel Hill 27514
| | | | | |
Collapse
|
81
|
Baxt WG, Jones G, Fortlage D. The trauma triage rule: a new, resource-based approach to the prehospital identification of major trauma victims. Ann Emerg Med 1990; 19:1401-6. [PMID: 2240753 DOI: 10.1016/s0196-0644(05)82608-3] [Citation(s) in RCA: 113] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/30/2022]
Abstract
STUDY OBJECTIVE To develop a new trauma decision rule. DESIGN Retrospective clinical review. SETTING Level I trauma center. TYPE OF PARTICIPANTS 1,004 injured adults. MEASUREMENTS AND MAIN RESULTS A new trauma decision rule was derived from 1,004 injured adult patients using a new operational definition of major trauma. The rule, termed the Trauma Triage Rule, defines a major trauma victim as any injured adult patient whose systolic blood pressure is less than 85 mm Hg; whose motor component of the Glasgow Coma Score is less than 5; or who has sustained penetrating trauma of the head, neck, or trunk. Using the operational definition of major trauma, the rule had a sensitivity of 92% and a specificity of 92% when tested on the 1,004-patient cohort. CONCLUSION The Trauma Triage Rule may significantly reduce overtriage while only minimally increasing undertriage. This approach must be validated prospectively before it can be used in the prehospital setting.
Collapse
Affiliation(s)
- W G Baxt
- Department of Emergency Medicine, UCSD Medical Center 92103-1990
| | | | | |
Collapse
|
82
|
Connors AF, Dawson NV, Shaw PK, Montenegro HD, Nara AR, Martin L. Hemodynamic status in critically ill patients with and without acute heart disease. Chest 1990; 98:1200-6. [PMID: 2225966 DOI: 10.1378/chest.98.5.1200] [Citation(s) in RCA: 43] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/30/2022] Open
Abstract
Physicians have been urged to reduce the use of the pulmonary artery catheter. However, there are no guidelines to help the clinician make the decision to use or withhold invasive monitoring in the individual patient. This study was designed to examine the accuracy of physician estimates of cardiac function in a spectrum of patients with hemodynamic instability to determine whether differences in accuracy among subgroups would suggest subgroups of patients who could be managed without invasive measurements. Physician estimates of cardiac index were found to be sufficiently accurate in patients without acute heart disease that initial management without invasive monitoring may be appropriate in selected cases. However, due to the general inaccuracy of physician estimates, efforts to improve the accuracy of clinical judgments of cardiac function and hemodynamic status should be pursued with vigor in patients both with and without acute cardiac dysfunction.
Collapse
Affiliation(s)
- A F Connors
- Department of Medicine, Case Western Reserve University, Cleveland
| | | | | | | | | | | |
Collapse
|
83
|
McCarthy BD, Wong JB, Selker HP. Detecting acute cardiac ischemia in the emergency department: a review of the literature. J Gen Intern Med 1990; 5:365-73. [PMID: 2197378 DOI: 10.1007/bf02600409] [Citation(s) in RCA: 75] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/30/2022]
Affiliation(s)
- B D McCarthy
- Center for Cardiovascular Health Services Research, Boston, Massachusetts
| | | | | |
Collapse
|
84
|
Sox HC, Hickam DH, Marton KI, Moses L, Skeff KM, Sox CH, Neal EA. Using the patient's history to estimate the probability of coronary artery disease: a comparison of primary care and referral practices. Am J Med 1990; 89:7-14. [PMID: 2242131 DOI: 10.1016/0002-9343(90)90090-z] [Citation(s) in RCA: 61] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/30/2022]
Abstract
PURPOSE According to probability theory, the interpretation of new information should depend on the prior probability of disease. We asked if this principle applies to interpreting the history in patients with chest pain. We compared the prevalence of coronary artery disease (CAD) in patients who had similar histories but who came from populations with different disease prevalence. PATIENTS AND METHODS We studied two high-disease-prevalence populations (patients referred for coronary arteriography) and two low-disease-prevalence populations (patients from primary care practices). We used clinical characteristics of one arteriography population to develop a logistic rule for estimating the probability of coronary artery narrowing. The number of clinical findings determined the logistic score, which was proportional to the prevalence of CAD. RESULTS The prevalence of CAD was much lower in the primary care population than in the arteriography population, even when patients with similar logistic scores, and thus similar clinical histories, were compared. CONCLUSION A clinician must take account of the overall prevalence of disease in the clinical setting when using the patient's history to estimate the probability of disease. Failure to observe this caution may lead to errors in test selection and interpretation.
Collapse
Affiliation(s)
- H C Sox
- Stanford University School of Medicine, Palo Alto Veterans Administration Medical Center, California
| | | | | | | | | | | | | |
Collapse
|
85
|
Landefeld CS, McGuire E, Cohen AM. Clinical findings associated with acute proximal deep vein thrombosis: a basis for quantifying clinical judgment. Am J Med 1990; 88:382-8. [PMID: 2327426 DOI: 10.1016/0002-9343(90)90493-w] [Citation(s) in RCA: 41] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/31/2022]
Abstract
PURPOSE To identify clinical findings useful in estimating the probability of acute proximal deep vein thrombosis (DVT). PATIENTS AND METHODS The records of 355 symptomatic patients who underwent ascending venography were reviewed. Data on 76 clinical items were collected using standardized forms. Venograms were interpreted according to standard criteria and interobserver agreement was evaluated in a sample of 119 venograms. Independent clinical correlates of proximal DVT were identified using multivariate discriminant analysis in 236 randomly chosen patients; they were tested in the remaining 119 patients. RESULTS Acute proximal DVT was shown by venogram in 96 patients (27%). Five independent clinical correlates of proximal DVT--swelling above the knee of the affected leg, swelling below the knee, recent immobility, cancer, fever--predicted proximal DVT in the testing group; in patients with none, one, or two or more of these clinical findings, proximal DVT was present in 5%, 15%, and 42%, respectively. If venography had been performed only in patients with one or more of the five factors, 97% of cases of proximal DVT would have been diagnosed and venography would have been avoided in 26% of patients with normal test results. CONCLUSION These data provide a quantitative basis for estimating the probability of proximal DVT on the basis of clinical findings in symptomatic patients. How these findings can best be integrated with noninvasive testing and venography into diagnostic strategies for DVT remains to be determined.
Collapse
Affiliation(s)
- C S Landefeld
- Department of Medicine, Case Western Reserve University School of Medicine, Cleveland, Ohio
| | | | | |
Collapse
|
86
|
Baxt WG, Berry CC, Epperson MD, Scalzitti V. The failure of prehospital trauma prediction rules to classify trauma patients accurately. Ann Emerg Med 1989; 18:1-8. [PMID: 2642672 DOI: 10.1016/s0196-0644(89)80302-6] [Citation(s) in RCA: 80] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/01/2023]
Abstract
Clinical prediction rules are used extensively by most regionalized trauma systems to identify which patients have sustained major injuries. Because of reported high misclassification rates of some of these rules and the known global difficulty of transporting prediction rules, four such rules (the Trauma Score, the CRAMS Scale, the Revised Trauma Score, and the Prehospital Index) and two newly derived rules were statistically analyzed using a cohort of 2,434 injured patients. All rules accurately predicted mortality with a minimum sensitivity and specificity of 85%. However, not one of the rules was able accurately to identify surviving patients who had sustained major injuries. In this instance, no rule was able to achieve a sensitivity of at least 70% while achieving a specificity of 70%. These results suggest that the problem with trauma prediction rules lies in the inherent limitations of the clinical data on which they are based. In view of this, the usefulness of existing prehospital trauma predictive rules must be questioned.
Collapse
Affiliation(s)
- W G Baxt
- Department of Emergency Medical Services, University of California, San Diego Medical Center 92103-1990
| | | | | | | |
Collapse
|
87
|
Cebul RD. Decision making research at the interface between descriptive and prescriptive studies. Med Decis Making 1988; 8:231-2. [PMID: 3054394 DOI: 10.1177/0272989x8800800402] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.0] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/03/2023]
Affiliation(s)
- R D Cebul
- Division of General Medicine, Cleveland Metropolitan General Hospital,OH 44109
| |
Collapse
|
88
|
Clancy CM, Centor RM, Campbell MS, Dalton HP. Rational decision making based on history: adult sore throats. J Gen Intern Med 1988; 3:213-7. [PMID: 3288726 DOI: 10.1007/bf02596334] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/05/2023]
Abstract
Primary care physicians are often required to make preliminary evaluations based only on the patient's history, especially during telephone encounters about sore throats. The authors studied adults with sore throats to determine whether patients can be stratified into higher and lower risks of strep throat by history alone. They first obtained data from 517 patients seen in an emergency room. Providers graded symptoms on a four-point scale (absent, mild, moderate, or severe). Initial analyses showed that prediction based on history should include three variables: fever, difficulty in swallowing, and cough. For ease of computation, these were consolidated into one score, "history" (= fever history + difficulty in swallowing - cough). This score was used to develop a model that predicts the probability of infection with group A beta-hemolytic streptococcus, and the model's performance was tested in two additional patient groups. The predictive accuracy of the "history" score was confirmed in all patient groups, despite differences in providers and disease prevalences. Primary care physicians may use this model to help them make decisions in situations such as telephone encounters without using additional data.
Collapse
Affiliation(s)
- C M Clancy
- Department of Medicine, Medical College of Virginia, Richmond
| | | | | | | |
Collapse
|
89
|
Goldman L, Cook EF, Brand DA, Lee TH, Rouan GW, Weisberg MC, Acampora D, Stasiulewicz C, Walshon J, Terranova G. A computer protocol to predict myocardial infarction in emergency department patients with chest pain. N Engl J Med 1988; 318:797-803. [PMID: 3280998 DOI: 10.1056/nejm198803313181301] [Citation(s) in RCA: 492] [Impact Index Per Article: 13.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/05/2023]
Abstract
To achieve more appropriate triage to the coronary care unit of patients presenting with acute chest pain, we used clinical data on 1379 patients at two hospitals to construct a simple computer protocol to predict the presence of myocardial infarction. When we tested this protocol prospectively in 4770 patients at two university hospitals and four community hospitals, the computer-derived protocol had a significantly higher specificity (74 vs. 71 percent) in predicting the absence of infarction than physicians deciding whether to admit patients to the coronary care unit, and it had a similar sensitivity in detecting the presence of infarction (88.0 vs. 87.8 percent). Decisions based solely on the computer protocol would have reduced the admission of patients without infarction to the coronary care unit by 11.5 percent without adversely affecting the admission of patients in whom emergent complications developed that required intensive care. Although this protocol should not be used to override careful clinical judgment in individual cases, the computer protocol for the most part yields accurate estimates of the probability of myocardial infarction. Decisions about admission to the coronary care unit based on the protocol would have been as effective as those actually made by the unaided physicians who cared for the patients, and less costly. Whether physicians who are aided by the protocol perform better than unaided physicians cannot be determined without further study.
Collapse
Affiliation(s)
- L Goldman
- Joint Department of Medicine, Brigham and Women's Hospital, Boston, MA 02115
| | | | | | | | | | | | | | | | | | | |
Collapse
|
90
|
Chapter 7 Applications of Judgment Analysis and Cognitive Feedback to Medicine. ACTA ACUST UNITED AC 1988. [DOI: 10.1016/s0166-4115(08)62175-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register]
|
91
|
Goldman L. Multifactorial index of cardiac risk in noncardiac surgery: ten-year status report. JOURNAL OF CARDIOTHORACIC ANESTHESIA 1987; 1:237-44. [PMID: 2980950 DOI: 10.1016/s0888-6296(87)80011-x] [Citation(s) in RCA: 60] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/03/2023]
Affiliation(s)
- L Goldman
- Department of Medicine, Brigham and Women's Hospital, Boston, MA 02115
| |
Collapse
|