What You Find Depends on How You Measure It: Reactivity of Response Scales Measuring Predecisional Information Distortion in Medical Diagnosis

Comparing GPs' antibiotic prescribing decisions to a clinical prediction rule: an online vignette study

BACKGROUND

The 'STARWAVe' clinical prediction rule (CPR) uses seven factors to guide risk assessment and antibiotic prescribing in children with cough (Short illness duration, Temperature, Age, Recession, Wheeze, Asthma, Vomiting).

AIM

To assess the influence of STARWAVe factors on GPs' unaided risk assessments and prescribing decisions.

DESIGN AND SETTING

Clinical vignettes administered to 188 UK GPs online.

METHOD

GPs were randomly assigned to view 32 (out of a possible 64) vignettes online depicting children with cough. The vignettes comprised the seven STARWAVe factors, which were varied systematically. For each vignette, GPs assessed risk of deterioration in one of two ways (sliding-scale versus risk-category selection) and indicated whether they would prescribe antibiotics. Finally, GPs saw an additional vignette, suggesting that the parent was concerned. Mixed-effects regressions were used to measure the influence of STARWAVe factors, risk-elicitation method, and parental concern on GPs' assessments and decisions.

RESULTS

Six STARWAVe risk factors correctly increased GPs' risk assessments (bssliding-scale≥0.66, odds ratios [ORs]category-selection≥1.75, Ps≤0.001), whereas one incorrectly reduced them (short illness duration: b sliding-scale -0.30, ORcategory-selection 0.80, P≤0.039). Conversely, one STARWAVe factor increased prescribing odds (temperature: OR 5.22, P<0.001), whereas the rest either reduced them (short illness duration, age, and recession: ORs≤0.70, Ps<0.001) or had no significant impact (wheeze, asthma, and vomiting: Ps≥0.065). Parental concern increased risk assessments (b sliding-scale 1.29, ORcategory-selection 2.82, P≤0.003) but not prescribing odds (P = 0.378).

CONCLUSION

GPs use some, but not all, STARWAVe factors when making unaided risk assessments and prescribing decisions. Such discrepancies must be considered when introducing CPRs to clinical practice.

The information distortion bias: implications for medical decisions

CONTEXT

Every diagnosis involves an act of decision making, which requires proper evaluation of information. However, even seemingly objective information can require interpretation, often without our conscious awareness. In this cross-cutting edge article we describe the phenomenon of leader-driven information distortion (ID) and its implications for medical education.

INFORMATION DISTORTION

Recent research indicates that one threat to good decisions is a biased interpretation of information to favour one alternative course of action over another. Once an alternative emerges as a leader during a decision there is a strong tendency to evaluate subsequent information as supporting that option. This can occur when deciding between two competing diagnoses. It is particularly a concern if diagnostic tests provide potentially ambiguous results. This leader-driven ID is pre-decisional in nature, in that it develops during a decision and involves the interpretation of information available prior to the final decision or diagnosis, with different interpretations possible depending on whichever alternative is the leader. Studies reveal that the distortion bias is pervasive in decisions, and that awareness of the act of distortion is low in decision makers.

APPLICATION TO MEDICAL EDUCATION

Empirical research has confirmed the presence of leader-driven ID in hypothetical diagnoses made by physicians. ID creates two threats to medical decisions: First, it can make a diagnosis sticky in that it is resistant to being overturned by contradictory information. Second, it can promote unwarranted certainty in a diagnosis. The outcome may be premature closure, unnecessary testing or incorrect treatment, resulting in delayed or missed diagnoses.

METHODS

This paper summarises research related to leader-driven ID in medical and professional decisions and discusses various approaches directed towards reducing ID. A framework and language are provided for thinking about and discussing ID in medical decisions and medical education. Courses of action for mitigating the effects of ID are suggested.

Requirements and validation of a prototype learning health system for clinical diagnosis

INTRODUCTION

Diagnostic error is a major threat to patient safety in the context of family practice. The patient safety implications are severe for both patient and clinician. Traditional approaches to diagnostic decision support have lacked broad acceptance for a number of well-documented reasons: poor integration with electronic health records and clinician workflow, static evidence that lacks transparency and trust, and use of proprietary technical standards hindering wider interoperability. The learning health system (LHS) provides a suitable infrastructure for development of a new breed of learning decision support tools. These tools exploit the potential for appropriate use of the growing volumes of aggregated sources of electronic health records.

METHODS

We describe the experiences of the TRANSFoRm project developing a diagnostic decision support infrastructure consistent with the wider goals of the LHS. We describe an architecture that is model driven, service oriented, constructed using open standards, and supports evidence derived from electronic sources of patient data. We describe the architecture and implementation of 2 critical aspects for a successful LHS: the model representation and translation of clinical evidence into effective practice and the generation of curated clinical evidence that can be used to populate those models, thus closing the LHS loop.

RESULTS/CONCLUSIONS

Six core design requirements for implementing a diagnostic LHS are identified and successfully implemented as part of this research work. A number of significant technical and policy challenges are identified for the LHS community to consider, and these are discussed in the context of evaluating this work: medico-legal responsibility for generated diagnostic evidence, developing trust in the LHS (particularly important from the perspective of decision support), and constraints imposed by clinical terminologies on evidence generation.