Diagnostic testing and treatment under ambiguity: using decision analysis to inform clinical practice

Identification of a Theory-Practice Gap in the Education of Biomedical Scientists

Introduction

The Biomedical Scientist (BMS) role is established in healthcare, working in laboratory environments to provide diagnostic testing and to monitor treatment effects on a patients' health. The profession is subject to several professional standards which highlight the importance of working in the best interests of the patient and service user. However, Biomedical Scientists have little or no patient contact. This study aimed to determine how Biomedical Scientists evidence that they meet the professional standards and support the achievement of patient outcomes.

Materials and Methods

This study utilised a Delphi method to explore the opinions of professional stakeholders to determine whether there was consensus for how this professional group contributes to patient outcomes and offers evidence that they are working in the best interests of the patient. The qualitative 1st round of the study consisted of focus groups and interviews with staff and students on the BSc Biomedical Science awards, Professional, Statutory and Regulatory body (PSRB) representatives and Biomedical Scientists from the National Health Service (NHS). The first-round responses were analysed using thematic analysis which then generated attitude statements which participants scored using a 5-point Likert scale in the 2nd round. Consensus or divergence of opinion was determined based upon a 70% consensus level within each participant group and overall.

Results

Following analysis of the 2nd round data, there was divergence of opinion across all stakeholders, with consensus rates being highest in the Biomedical Scientist group (72.7% of statements reached 70% consensus), followed by the student group (54.5% of statements reached 70% consensus) and lowest in the academic group (40.9% of statements reached 70% consensus).

Discussion

This demonstrates a theory-practice gap in both the academic and student groups, suggesting that graduates are insufficiently prepared for their post-graduate role. This gap was particularly evident when discussing topics such as how Biomedical Scientists contribute to patient care, professional registration and working as part of the multi-disciplinary team (MDT). The identification of a theory-practice gap in the education of Biomedical Scientists is a novel finding, indicating that students may graduate with insufficient understanding of the Biomedical Scientist role.

Improving methods of clinical practice guidelines: From guidelines to pathways to fast-and-frugal trees and decision analysis to develop individualised patient care

BACKGROUND

Current methods for developing clinical practice guidelines have several limitations: they are characterised by the "black box" operation-a process with defined inputs and outputs but an incomplete understanding of its internal workings; they have "the integration problem"-a lack of framework for explicitly integrating factors such as patient preferences and trade-offs between benefits and harms; they generate one recommendation at a time that typically are not connected in a coherent analytical framework; and they apply to "average" patients, while clinicians and their patients seek advice tailored to individual circumstances.

METHODS

We propose augmenting the current guideline development method by converting evidence-based pathways into fast-and-frugal decision trees (FFTs) and integrating them with generalised decision curve analysis to formulate clear, individualised management recommendations.

RESULTS

We illustrate the process by developing recommendations for the management of heparin-induced thrombocytopenia (HIT). We converted evidence-based pathways for HIT, developed by the American Society of Hematology, into an FFT. Here, we consider only thrombotic complications and major bleeding. We leveraged the predictive potential of FFTs to compare the effects of argatroban, bivalirudin, fondaparinux, and direct oral anticoagulants (DOACs) using generalised decision curve analysis. We found that DOACs were superior to other treatments if the FFT-predicted probability of HIT exceeded 3%.

CONCLUSIONS

The proposed analytical framework connects guidelines, pathways, FFTs, and decision analysis, offering risk-tailored personalised recommendations and addressing current guideline development critiques.

Using measures of race to make clinical predictions: Decision making, patient health, and fairness

The use of race measures in clinical prediction models is contentious. We seek to inform the discourse by evaluating the inclusion of race in probabilistic predictions of illness that support clinical decision making. Adopting a static utilitarian framework to formalize social welfare, we show that patients of all races benefit when clinical decisions are jointly guided by patient race and other observable covariates. Similar conclusions emerge when the model is extended to a two-period setting where prevention activities target systemic drivers of disease. We also discuss non-utilitarian concepts that have been proposed to guide allocation of health care resources.

Patient-centered appraisal of race-free clinical risk assessment

Until recently, there has been a consensus that clinicians seeking to assess patient risks of illness should condition risk assessments on all observed patient covariates with predictive power. The broad idea is that knowing more about patients enables more accurate predictions of their health risks and, hence, better clinical decisions. This consensus has recently unraveled with respect to a specific covariate, namely race. There have been increasing calls for race-free risk assessment, arguing that using race to predict health risks contributes to racial disparities and inequities in health care. In some medical fields, leading institutions have recommended race-free risk assessment. An important open question is how race-free risk assessment would affect the quality of clinical decisions. Considering the matter from the patient-centered perspective of medical economics yields a disturbing conclusion: Race-free risk assessment would harm patients of all races.

Embracing Uncertainty: The Value of Partial Identification in Public Health and Clinical Research

INTRODUCTION

This paper describes the methodology of partial identification and its applicability to empirical research in preventive medicine and public health.

METHODS

The authors summarize findings from the methodologic literature on partial identification. The analysis was conducted in 2020-2021.

RESULTS

The applicability of partial identification methods is demonstrated using 3 empirical examples drawn from published literature.

CONCLUSIONS

Partial identification methods are likely to be of considerable interest to clinicians and others engaged in preventive medicine and public health research.

Effect of Strict and Soft Policy Interventions on Laboratory Diagnostic Testing in Ontario, Canada: A Bayesian Structural Time Series Analysis

Applications of behavioral economics targeted at optimizing laboratory utilization among physicians have been implemented in Ontario through different types of nonfinancial interventions. Strict policy interventions restrict Ontario Health Insurance Plan (OHIP) payment for tests to patients with specific conditions or limit ordering to particular physician specialties, while soft policy interventions involve modifications to the laboratory requisition form. This study evaluates the effectiveness of these interventions in terms of changing physician ordering behavior for eight tests that were subject to a strict or soft policy intervention during the study period. We use a Bayesian structural time series model applied to Ontario laboratory claims data for FY2006 through FY2017. Results show a 16-75% reduction in laboratory services with a strict policy intervention and an 8-36% reduction in laboratory services with a soft policy intervention. Although the overall magnitude of change was smaller for soft policy interventions, interventions designed with soft or strict policy mechanisms addressing laboratory utilization management are effective at influencing physicians' test ordering behavior.

Elective cardiovascular care in the era of the COVID-19 pandemic: managing tragic choices

The COVID-19 pandemic has led to significant morbidity and mortality globally. As health systems grapple with caring for patients affected with COVID-19, cardiovascular procedures that are deemed ‘elective’ have been postponed. Guidelines concerning which cardiac procedures should be performed during the pandemic vary by specialty and geography in the USA. We propose a clinical heuristic to guide individual physicians and governing bodies in their decision making regarding which cardiac procedures should be performed during the COVID-19 pandemic using the behavioural economics concept of heuristics and ecological rationality.

Forming COVID-19 Policy Under Uncertainty

This paper presents my thinking and concerns about formation of COVID-19 policy. Policy formation must cope with substantial uncertainties about the nature of the disease, the dynamics of transmission, and behavioral responses. Data uncertainties limit our knowledge of the past trajectory and current state of the pandemic. Data and modeling uncertainties limit our ability to predict the impacts of alternative policies. I explain why current epidemiological and macroeconomic modeling cannot deliver realistically optimal policy. I describe my recent work quantifying basic data uncertainties that make policy analysis difficult. I discuss approaches for policy choice under uncertainty and suggest adaptive policy diversification.

Tuberculosis diagnosis and treatment under uncertainty

In 2017, 1.6 million people worldwide died from tuberculosis (TB). A new TB diagnostic test-Xpert MTB/RIF from Cepheid-was endorsed by the World Health Organization in 2010. Trials demonstrated that Xpert is faster and has greater sensitivity and specificity than smear microscopy-the most common sputum-based diagnostic test. However, subsequent trials found no impact of introducing Xpert on morbidity and mortality. We present a decision-theoretic model of how a clinician might decide whether to order Xpert or other tests for TB, and whether to treat a patient, with or without test results. Our first result characterizes the conditions under which it is optimal to perform empirical treatment; that is, treatment without diagnostic testing. We then examine the implications for decision making of partial knowledge of TB prevalence or test accuracy. This partial knowledge generates ambiguity, also known as deep uncertainty, about the best testing and treatment policy. In the presence of such ambiguity, we show the usefulness of diversification of testing and treatment.

Intraocular pressure increases after complex simulated surgical procedures in residents: an experimental study

BACKGROUND

Surgeons' overload is one of the main causes of medical errors that might compromise patient safety. Due to the drawbacks of current options to monitor surgeons' load, new, sensitive, and objective indices of task (over)load need to be considered and tested. In non-health-care scenarios, intraocular pressure (IOP) has been proved to be an unbiased physiological index, sensitive to task complexity (one of the main variables related to overload), and time on task. In the present study, we assessed the effects of demanding and complex simulated surgical procedures on surgical and medical residents' IOP.

METHODS

Thirty-four surgical and medical residents and healthcare professionals took part in this study (the experimental group, N = 17, and the control group, N = 17, were matched for sex and age). The experimental group performed two simulated bronchoscopy procedures that differ in their levels of complexity. The control group mimicked the same hand-eye movements and posture of the experimental group to help control for the potential effects of time on task and re-measurement on IOP. We measured IOP before and after each procedure, surgical performance during procedures, and perceived task complexity.

RESULTS

IOP increased as consequence of performing the most complex procedure only in the experimental group. Consistently, residents performed worse and reported higher perceived task complexity for the more complex procedure.

CONCLUSIONS

Our data show, for the first time, that IOP is sensitive to residents' task load, and it could be used as a new index to easily and rapidly assess task (over)load in healthcare scenarios. An arousal-based explanation is given to describe IOP variations due to task complexity.

Reasonable patient care under uncertainty

This paper discusses how limited ability to predict illness and treatment response may affect the welfare achieved in patient care. The discussion covers both decentralized clinical decision making and care that adheres to clinical practice guidelines. I explain why predictive ability has been limited, calling attention to questionable methodological practices in the research that supports evidence-based medicine. I summarize research on identification whose objective is to yield credible prediction of patient outcomes. Recognizing that uncertainty will continue to afflict medical decision making, I apply basic decision theory to suggest reasonable decision criteria with well-understood welfare properties. Previous research on medical decision making has largely embraced Bayesian decision theory. I summarize research studying the minimax-regret criterion, which seeks uniformly near-optimal decisions.

Transforming clinical practice guidelines and clinical pathways into fast-and-frugal decision trees to improve clinical care strategies

BACKGROUND

Contemporary delivery of health care is inappropriate in many ways, largely due to suboptimal Q5 decision-making. A typical approach to improve practitioners' decision-making is to develop evidence-based clinical practice guidelines (CPG) by guidelines panels, who are instructed to use their judgments to derive practice recommendations. However, mechanisms for the formulation of guideline judgments remains a "black-box" operation-a process with defined inputs and outputs but without sufficient knowledge of its internal workings.

METHODS

Increased explicitness and transparency in the process can be achieved by implementing CPG as clinical pathways (CPs) (also known as clinical algorithms or flow-charts). However, clinical recommendations thus derived are typically ad hoc and developed by experts in a theory-free environment. As any recommendation can be right (true positive or negative), or wrong (false positive or negative), the lack of theoretical structure precludes the quantitative assessment of the management strategies recommended by CPGs/CPs.

RESULTS

To realize the full potential of CPGs/CPs, they need to be placed on more solid theoretical grounds. We believe this potential can be best realized by converting CPGs/CPs within the heuristic theory of decision-making, often implemented as fast-and-frugal (FFT) decision trees. This is possible because FFT heuristic strategy of decision-making can be linked to signal detection theory, evidence accumulation theory, and a threshold model of decision-making, which, in turn, allows quantitative analysis of the accuracy of clinical management strategies.

CONCLUSIONS

Fast-and-frugal provides a simple and transparent, yet solid and robust, methodological framework connecting decision science to clinical care, a sorely needed missing link between CPGs/CPs and patient outcomes. We therefore advocate that all guidelines panels express their recommendations as CPs, which in turn should be converted into FFTs to guide clinical care.

Misdiagnosis of obstetrical cases and the clinical and cost consequences to patients: a cross-sectional study of urban providers in the Philippines

BACKGROUND

Misdiagnosis may be a significant and under-recognized quality of care problem. In birthing facilities located in anurban Philippine setting, we investigated the diagnostic accuracy for three obstetric conditions: cephalopelvic disproportion (CPD), post-partum hemorrhage (PPH), and pre-eclampsia.

DESIGN

Identical simulated cases were used to measure diagnostic accuracy for every provider (n=103). We linked misdiagnosis - identified by the simulated cases - to obstetrical complications of the patients at the participating facilities. Patient-level data on health outcomes and costs were obtained from medical records and follow-home in-person interviews.

RESULTS

The prevalence of misdiagnosis among obstetric providers was 29.8% overall, 25% for CPD, 33% for PPH, and 31% for pre-eclampsia. Linking provider decision-making to patients, we found those who misdiagnosed the simulated cases were more likely to have patients with a complication (OR 2.96; 95% CI 1.39-3.77) compared with those who did not misdiagnose. Complicated patients were significantly less likely to be referred to a hospital immediately, were more likely to be readmitted to a hospital after delivery, had significantly higher medical costs, and lost more income than non-complicated patients.

CONCLUSION

Diagnosis is arguably the most important task a clinician performs because it determines the subsequent course of evaluation and treatment, with the direct and indirect costs of diagnostic error, placing large financial burdens on the patient.

The optimal treatment of an infectious disease with two strains

This paper explores the optimal treatment of an infectious disease in a Susceptible-Infected-Susceptible model, where there are two strains of the disease and one strain is more infectious than the other. The strains are perfectly distinguishable, instantly diagnosed and equally costly in terms of social welfare. Treatment is equally costly and effective for both strains. Eradication is not possible, and there is no superinfection. In this model, we characterise two types of fixed points: coexistence equilibria, where both strains prevail, and boundary equilibria, where one strain is asymptotically eradicated and the other prevails at a positive level. We derive regimes of feasibility that determine which equilibria are feasible for which parameter combinations. Numerically, we show that optimal policy exhibits switch points over time, and that the paths to coexistence equilibria exhibit spirals, suggesting that coexistence equilibria are never the end points of optimal paths.