Does a decision aid help physicians to detect chronic obstructive pulmonary disease?

Methodological standards for the development and evaluation of clinical prediction rules: a review of the literature

Clinical prediction rules (CPRs) that predict the absolute risk of a clinical condition or future outcome for individual patients are abundant in the medical literature; however, systematic reviews have demonstrated shortcomings in the methodological quality and reporting of prediction studies. To maximise the potential and clinical usefulness of CPRs, they must be rigorously developed and validated, and their impact on clinical practice and patient outcomes must be evaluated. This review aims to present a comprehensive overview of the stages involved in the development, validation and evaluation of CPRs, and to describe in detail the methodological standards required at each stage, illustrated with examples where appropriate. Important features of the study design, statistical analysis, modelling strategy, data collection, performance assessment, CPR presentation and reporting are discussed, in addition to other, often overlooked aspects such as the acceptability, cost-effectiveness and longer-term implementation of CPRs, and their comparison with clinical judgement. Although the development and evaluation of a robust, clinically useful CPR is anything but straightforward, adherence to the plethora of methodological standards, recommendations and frameworks at each stage will assist in the development of a rigorous CPR that has the potential to contribute usefully to clinical practice and decision-making and have a positive impact on patient care.

Presentation of respiratory symptoms prior to diagnosis in general practice: a case-control study examining free text and morbidity codes

OBJECTIVE

General practitioners can record patients' presenting symptoms by using a code or free text. We compared breathlessness and wheeze symptom codes and free text recorded prior to diagnosis of ischaemic heart disease (IHD), chronic obstructive pulmonary disease (COPD) and asthma.

DESIGN

A case-control study.

SETTING

11 general practices in North Staffordshire, UK, contributing to the Consultations in Primary Care Archive consultation database.

PARTICIPANTS

Cases with an incident diagnosis of IHD, COPD or asthma in 2010 were matched to controls (four per case) with no such diagnosis. All prior consultations with codes for breathlessness or wheeze symptoms between 2004 and 2010 were identified. Free text of cases and controls were also searched for mention of these symptoms.

RESULTS

592 cases were identified, 194 (33%) with IHD, 182 (31%) with COPD and 216 (37%) with asthma. 148 (25%) cases and 125 (5%) controls had a prior coded consultation for breathlessness. Prevalence of a prior coded symptom of breathlessness or wheeze was 30% in cases, 6% in controls. Median time from first coded symptom to diagnosis among cases was 57 weeks. After adding symptoms recorded in text, prevalence rose to 62% in cases and 25% in controls. Median time from first recorded symptom increased to 144 weeks. The associations between diagnosis of cases and prior symptom codes was strong IHD relative risk ratio (RRR) 3.21 (2.15 to 4.79); COPD RRR 9.56 (6.74 to 13.60); asthma RRR 10.30 (7.17 to 14.90).

CONCLUSIONS

There is an association between IHD, COPD and asthma diagnosis and earlier consultation for respiratory symptoms. Symptoms are often noted in free text by GPs long before they are coded. Free text searching may aid investigation of early presentation of long-term conditions using GP databases, and may be an important direction for future research.

A systematic review of studies comparing diagnostic clinical prediction rules with clinical judgment

Background

Diagnostic clinical prediction rules (CPRs) are developed to improve diagnosis or decrease diagnostic testing. Whether, and in what situations diagnostic CPRs improve upon clinical judgment is unclear.

Methods and Findings

We searched MEDLINE, Embase and CINAHL, with supplementary citation and reference checking for studies comparing CPRs and clinical judgment against a current objective reference standard. We report 1) the proportion of study participants classified as not having disease who hence may avoid further testing and or treatment and 2) the proportion, among those classified as not having disease, who do (missed diagnoses) by both approaches. 31 studies of 13 medical conditions were included, with 46 comparisons between CPRs and clinical judgment. In 2 comparisons (4%), CPRs reduced the proportion of missed diagnoses, but this was offset by classifying a larger proportion of study participants as having disease (more false positives). In 36 comparisons (78%) the proportion of diagnoses missed by CPRs and clinical judgment was similar, and in 9 of these, the CPRs classified a larger proportion of participants as not having disease (fewer false positives). In 8 comparisons (17%) the proportion of diagnoses missed by the CPRs was greater. This was offset by classifying a smaller proportion of participants as having the disease (fewer false positives) in 2 comparisons. There were no comparisons where the CPR missed a smaller proportion of diagnoses than clinical judgment and classified more participants as not having the disease. The design of the included studies allows evaluation of CPRs when their results are applied independently of clinical judgment. The performance of CPRs, when implemented by clinicians as a support to their judgment may be different.

Conclusions

In the limited studies to date, CPRs are rarely superior to clinical judgment and there is generally a trade-off between the proportion classified as not having disease and the proportion of missed diagnoses. Differences between the two methods of judgment are likely the result of different diagnostic thresholds for positivity. Which is the preferred judgment method for a particular clinical condition depends on the relative benefits and harms of true positive and false positive diagnoses.

Laryngeal measurements and diagnostic tools for diagnosis of chronic obstructive pulmonary disease

PURPOSE

The purpose of this study was to evaluate sensitivity, specificity, and positive and negative likelihood ratios of laryngeal height, lung function, and diagnostic questionnaires for screening and diagnosis of chronic obstructive pulmonary disease (COPD).

METHODS

We undertook a cross-sectional study of 233 people aged between 40 and 75 years. Measured variables were age, sex, weight, height, body mass index, tobacco use, maximum laryngeal height, and spirometry, and we administered a COPD questionnaire and the Lung Function Questionnaire.

RESULTS

For laryngeal height, we found a positive likelihood ratio of 5.21, and for the Lung Function Questionnaire, we found a negative likelihood ratio of 0.10. Combining a maximum laryngeal height of ≤4 cm with Lung Function Questionnaire findings of ≤18 yielded a positive likelihood ratio of 29.06, and a negative likelihood ratio of 0.26.

CONCLUSIONS

The intrinsic validity of the lung function questionnaire makes it useful for screening. Combining Lung Function Questionnaire results and laryngeal height can help confirm or dismiss COPD.

Use of serum C reactive protein and procalcitonin concentrations in addition to symptoms and signs to predict pneumonia in patients presenting to primary care with acute cough: diagnostic study

OBJECTIVES

To quantify the diagnostic accuracy of selected inflammatory markers in addition to symptoms and signs for predicting pneumonia and to derive a diagnostic tool.

DESIGN

Diagnostic study performed between 2007 and 2010. Participants had their history taken, underwent physical examination and measurement of C reactive protein (CRP) and procalcitonin in venous blood on the day they first consulted, and underwent chest radiography within seven days.

SETTING

Primary care centres in 12 European countries.

PARTICIPANTS

Adults presenting with acute cough.

MAIN OUTCOME MEASURES

Pneumonia as determined by radiologists, who were blind to all other information when they judged chest radiographs.

RESULTS

Of 3106 eligible patients, 286 were excluded because of missing or inadequate chest radiographs, leaving 2820 patients (mean age 50, 40% men) of whom 140 (5%) had pneumonia. Re-assessment of a subset of 1675 chest radiographs showed agreement in 94% (κ 0.45, 95% confidence interval 0.36 to 0.54). Six published "symptoms and signs models" varied in their discrimination (area under receiver operating characteristics curve (ROC) ranged from 0.55 (95% confidence interval 0.50 to 0.61) to 0.71 (0.66 to 0.76)). The optimal combination of clinical prediction items derived from our patients included absence of runny nose and presence of breathlessness, crackles and diminished breath sounds on auscultation, tachycardia, and fever, with an ROC area of 0.70 (0.65 to 0.75). Addition of CRP at the optimal cut off of >30 mg/L increased the ROC area to 0.77 (0.73 to 0.81) and improved the diagnostic classification (net reclassification improvement 28%). In the 1556 patients classified according to symptoms, signs, and CRP >30 mg/L as "low risk" (<2.5%) for pneumonia, the prevalence of pneumonia was 2%. In the 132 patients classified as "high risk" (>20%), the prevalence of pneumonia was 31%. The positive likelihood ratio of low, intermediate, and high risk for pneumonia was 0.4, 1.2, and 8.6 respectively. Measurement of procalcitonin added no relevant additional diagnostic information. A simplified diagnostic score based on symptoms, signs, and CRP >30 mg/L resulted in proportions of pneumonia of 0.7%, 3.8%, and 18.2% in the low, intermediate, and high risk group respectively.

CONCLUSIONS

A clinical rule based on symptoms and signs to predict pneumonia in patients presenting to primary care with acute cough performed best in patients with mild or severe clinical presentation. Addition of CRP concentration at the optimal cut off of >30 mg/L improved diagnostic information, but measurement of procalcitonin concentration did not add clinically relevant information in this group.