Accuracy of Practitioner Estimates of Probability of Diagnosis Before and After Testing

Opportunity and Opportunism in Artificial-Intelligence-Powered Data Extraction: A Value-Centered Approach

Radiologists' traditional role in the diagnostic process is to respond to specific clinical questions and reduce uncertainty enough to permit treatment decisions. This charge is rapidly evolving due to forces such as artificial intelligence [AI], big data [opportunistic imaging, imaging prognostication], and advanced diagnostic technologies. A new "modernistic" paradigm is emerging whereby radiologists, in conjunction with computer algorithms, will be tasked with extracting as much information from imaging data as possible, often without a specific clinical question being posed and independent of any stated clinical need. In addition, AI algorithms are increasingly able to predict long-term outcomes using data from seemingly normal examinations, enabling AI-assisted prognostication. As these algorithms become a standard component of radiology practice, the sheer amount of information they demand will increase the need for streamlined workflows, communication, and data management techniques. In addition, the provision of such information raises reimbursement, liability, and access issues. Guidelines will be needed to ensure all patients have access to the benefits of this new technology and guarantee mined data do not inadvertently create harm. In this article, we discuss challenges and opportunities relevant to radiologists in this changing landscape, with an emphasis on ensuring that radiologists provide high-value care.

Validating, Implementing, and Monitoring Machine Learning Solutions in the Clinical Laboratory Safely and Effectively

BACKGROUND

Machine learning solutions offer tremendous promise for improving clinical and laboratory operations in pathology. Proof-of-concept descriptions of these approaches have become commonplace in laboratory medicine literature, but only a scant few of these have been implemented within clinical laboratories, owing to the often substantial barriers in validating, implementing, and monitoring these applications in practice. This mini-review aims to highlight the key considerations in each of these steps.

CONTENT

Effective and responsible applications of machine learning in clinical laboratories require robust validation prior to implementation. A comprehensive validation study involves a critical evaluation of study design, data engineering and interoperability, target label definition, metric selection, generalizability and applicability assessment, algorithmic fairness, and explainability. While the main text highlights these concepts in broad strokes, a supplementary code walk-through is also provided to facilitate a more practical understanding of these topics using a real-world classification task example, the detection of saline-contaminated chemistry panels.Following validation, the laboratorian's role is far from over. Implementing machine learning solutions requires an interdisciplinary effort across several roles in an organization. We highlight the key roles, responsibilities, and terminologies for successfully deploying a validated solution into a live production environment. Finally, the implemented solution must be routinely monitored for signs of performance degradation and updated if necessary.

SUMMARY

This mini-review aims to bridge the gap between theory and practice by highlighting key concepts in validation, implementation, and monitoring machine learning solutions effectively and responsibly in the clinical laboratory.

Every Crisis Is an Opportunity: Advancing Blood Culture Stewardship During a Blood Culture Bottle Shortage

The current manufacturing disruption of BACTEC blood culture bottles has drawn attention to diagnostic stewardship around blood culture utilization. In this perspective, we offer strategies for implementing blood culture stewardship using a graded approach based on a hospital's blood culture bottle supply. These strategies should inform plans to mitigate the impact of the shortage on patient care and reinforce fundamental principles of blood culture stewardship.

Alzheimer disease blood biomarkers: considerations for population-level use

In the past 5 years, we have witnessed the first approved Alzheimer disease (AD) disease-modifying therapy and the development of blood-based biomarkers (BBMs) to aid the diagnosis of AD. For many reasons, including accessibility, invasiveness and cost, BBMs are more acceptable and feasible for patients than a lumbar puncture (for cerebrospinal fluid collection) or neuroimaging. However, many questions remain regarding how best to utilize BBMs at the population level. In this Review, we outline the factors that warrant consideration for the widespread implementation and interpretation of AD BBMs. To set the scene, we review the current use of biomarkers, including BBMs, in AD. We go on to describe the characteristics of typical patients with cognitive impairment in primary care, who often differ from the patient populations used in AD BBM research studies. We also consider factors that might affect the interpretation of BBM tests, such as comorbidities, sex and race or ethnicity. We conclude by discussing broader issues such as ethics, patient and provider preference, incidental findings and dealing with indeterminate results and imperfect accuracy in implementing BBMs at the population level.

From pre-test and post-test probabilities to medical decision making

BACKGROUND

A central goal of modern evidence-based medicine is the development of simple and easy to use tools that help clinicians integrate quantitative information into medical decision-making. The Bayesian Pre-test/Post-test Probability (BPP) framework is arguably the most well known of such tools and provides a formal approach to quantify diagnostic uncertainty given the result of a medical test or the presence of a clinical sign. Yet, clinical decision-making goes beyond quantifying diagnostic uncertainty and requires that that uncertainty be balanced against the various costs and benefits associated with each possible decision. Despite increasing attention in recent years, simple and flexible approaches to quantitative clinical decision-making have remained elusive.

METHODS

We extend the BPP framework using concepts of Bayesian Decision Theory. By integrating cost, we can expand the BPP framework to allow for clinical decision-making.

RESULTS

We develop a simple quantitative framework for binary clinical decisions (e.g., action/inaction, treat/no-treat, test/no-test). Let p be the pre-test or post-test probability that a patient has disease. We show thatr ∗ = ( 1 - p ) / p represents a critical value called a decision boundary. In terms of the relative cost of under- to over-acting, r ∗ represents the critical value at which action and inaction are equally optimal. We demonstrate how this decision boundary can be used at the bedside through case studies and as a research tool through a reanalysis of a recent study which found widespread misestimation of pre-test and post-test probabilities among clinicians.

CONCLUSIONS

Our approach is so simple that it should be thought of as a core, yet previously overlooked, part of the BPP framework. Unlike prior approaches to quantitative clinical decision-making, our approach requires little more than a hand-held calculator, is applicable in almost any setting where the BPP framework can be used, and excels in situations where the costs and benefits associated with a particular decision are patient-specific and difficult to quantify.

[The expert panel for Script Concordance Tests: A truly adequate reference?]

The Script Concordance Tests (SCTs) are an examination modality introduced by decree in the French National Ranking Exam for medical students in 2024. Their objective is to evaluate clinical reasoning in situations of uncertainty. In practice, SCTs assess the impact of new information on the probability of a hypothesis formulated a priori based on an authentic clinical scenario. This approach resembles probabilistic (or Bayesian) reasoning. Due to the uncertainty associated with the explored clinical situation, SCTs do not compare the student's response to an expected one in a theoretical knowledge reference. Instead, the distribution of responses from a panel of experienced physicians is used to establish the question's scoring scale. Literature data suggest that physicians, even experienced ones, like most humans, often exhibit biased intuitive probabilistic reasoning. These biases raise questions about the relevance of using expert panel responses as scoring scales for SCTs.

Preferences for Management of Pediatric Pneumonia: A Clinician Survey of Artificially Generated Patient Cases

BACKGROUND

It is unknown which factors are associated with chest radiograph (CXR) and antibiotic use for suspected community-acquired pneumonia (CAP) in children. We evaluated factors associated with CXR and antibiotic preferences among clinicians for children with suspected CAP using case scenarios generated through artificial intelligence (AI).

METHODS

We performed a survey of general pediatric, pediatric emergency medicine, and emergency medicine attending physicians employed by a private physician contractor. Respondents were given 5 unique, AI-generated case scenarios. We used generalized estimating equations to identify factors associated with CXR and antibiotic use. We evaluated the cluster-weighted correlation between clinician suspicion and clinical prediction model risk estimates for CAP using 2 predictive models.

RESULTS

A total of 172 respondents provided responses to 839 scenarios. Factors associated with CXR acquisition (OR, [95% CI]) included presence of crackles (4.17 [2.19, 7.95]), prior pneumonia (2.38 [1.32, 4.20]), chest pain (1.90 [1.18, 3.05]) and fever (1.82 [1.32, 2.52]). The decision to use antibiotics before knowledge of CXR results included past hospitalization for pneumonia (4.24 [1.88, 9.57]), focal decreased breath sounds (3.86 [1.98, 7.52]), and crackles (3.45 [2.15, 5.53]). After revealing CXR results to clinicians, these results were the sole predictor associated with antibiotic decision-making. Suspicion for CAP correlated with one of 2 prediction models for CAP (Spearman's rho = 0.25). Factors associated with a greater suspicion of pneumonia included prior pneumonia, duration of illness, worsening course of illness, shortness of breath, vomiting, decreased oral intake or urinary output, respiratory distress, head nodding, focal decreased breath sounds, focal rhonchi, fever, and crackles, and lower pulse oximetry.

CONCLUSIONS

Ordering preferences for CXRs demonstrated similarities and differences with evidence-based risk models for CAP. Clinicians relied heavily on CXR findings to guide antibiotic ordering. These findings can be used within decision support systems to promote evidence-based management practices for pediatric CAP.

Lessons in clinical reasoning - pitfalls, myths, and pearls: a woman brought to a halt

OBJECTIVES

Limitations in human cognition commonly result in clinical reasoning failures that can lead to diagnostic errors. A metacognitive structured reflection on what clinical findings fit and/or do not fit with a diagnosis, as well as how discordance of data can help advance the reasoning process, may reduce such errors.

CASE PRESENTATION

A 60-year-old woman with Hashimoto thyroiditis, diabetes, and generalized anxiety disorder presented with diffuse arthralgias and myalgias. She had been evaluated by physicians of various specialties and undergone multiple modalities of imaging, as well as a electromyography/nerve conduction study (EMG/NCS), leading to diagnoses of fibromyalgia, osteoarthritis, and lumbosacral plexopathy. Despite treatment for these conditions, she experienced persistent functional decline. The only definitive alleviation of her symptoms identified was in the few days following intra-articular steroid injections for osteoarthritis. On presentation to our institution, she appeared fit with a normal BMI. She was a long-time athlete and had been training consistently until her symptoms began. Prediabetes had been diagnosed the year prior and her A1c progressed despite lifestyle modifications and 10 pounds of intentional weight loss. She reported fatigue, intermittent nausea without emesis, and reduced appetite. Examination revealed intact strength and range of motion in both the shoulders and hips, though testing elicited pain. She had symmetric hyperreflexia as well as a slowed, rigid gait. Autoantibody testing revealed strongly positive serum GAD-65 antibodies which were confirmed in the CSF. A diagnosis of stiff-person syndrome was made. She had an incomplete response to first-line therapy with high-dose benzodiazepines. IVIg was initiated with excellent response and symptom resolution.

CONCLUSIONS

Through integrated commentary on the diagnostic reasoning process from clinical reasoning experts, this case underscores the importance of frequent assessment of fit along with explicit explanation of dissonant features in order to avoid misdiagnosis and halt diagnostic inertia. A fishbone diagram is provided to visually demonstrate the major factors that contributed to the diagnostic error. The case discussant demonstrates the power of iterative reasoning, case progression without commitment to a single diagnosis, and the dangers of both explicit and implicit bias. Finally, this case provides clinical teaching points in addition to a pitfall, myth, and pearl specific to overcoming diagnostic inertia.

Game-based learning to improve diagnostic accuracy: a pilot randomized-controlled trial

OBJECTIVES

Perform a pilot study of online game-based learning (GBL) using natural frequencies and feedback to teach diagnostic reasoning.

METHODS

We conducted a multicenter randomized-controlled trial of computer-based training. We enrolled medical students, residents, practicing physicians and nurse practitioners. The intervention was a 45 min online GBL training vs. control education with a primary outcome of score on a scale of diagnostic accuracy (composed of 10 realistic case vignettes, requesting estimates of probability of disease after a test result, 0-100 points total).

RESULTS

Of 90 participants there were 30 students, 30 residents and 30 practicing clinicians. Of these 62 % (56/90) were female and 52 % (47/90) were white. Sixty were randomized to GBL intervention and 30 to control. The primary outcome of diagnostic accuracy immediately after training was better in GBL (mean accuracy score 59.4) vs. control (37.6), p=0.0005. The GBL group was then split evenly (30, 30) into no further intervention or weekly emails with case studies. Both GBL groups performed better than control at one-month and some continued effect at three-month follow up. Scores at one-month GBL (59.2) GBL plus emails (54.2) vs. control (33.9), p=0.024; three-months GBL (56.2), GBL plus emails (42.9) vs. control (35.1), p=0.076. Most participants would recommend GBL to colleagues (73 %), believed it was enjoyable (92 %) and believed it improves test interpretation (95 %).

CONCLUSIONS

In this pilot study, a single session with GBL nearly doubled score on a scale of diagnostic accuracy in medical trainees and practicing clinicians. The impact of GBL persisted after three months.

Breaking Boundaries in Pneumonia Diagnostics: Transitioning from Tradition to Molecular Frontiers with Multiplex PCR

The advent of rapid molecular microbiology testing has revolutionized infectious disease diagnostics and is now impacting pneumonia diagnosis and management. Molecular platforms offer highly multiplexed assays for diverse viral and bacterial detection, alongside antimicrobial resistance markers, providing the potential to significantly shape patient care. Despite the superiority in sensitivity and speed, debates continue regarding the clinical role of multiplex molecular testing, notably in comparison to standard methods and distinguishing colonization from infection. Recent guidelines endorse molecular pneumonia panels for enhanced sensitivity and rapidity, but implementation requires addressing methodological differences and ensuring clinical relevance. Diagnostic stewardship should be leveraged to optimize pneumonia testing, emphasizing pre- and post-analytical strategies. Collaboration between clinical microbiologists and bedside providers is essential in developing implementation strategies to maximize the clinical utility of multiplex molecular diagnostics in pneumonia. This narrative review explores these multifaceted issues, examining the current evidence on the clinical performance of multiplex molecular assays in pneumonia, and reflects on lessons learned from previous microbiological advances. Additionally, given the complexity of pneumonia and the sensitivity of molecular diagnostics, diagnostic stewardship is discussed within the context of current literature, including implementation strategies that consider pre-analytical and post-analytical modifications to optimize the clinical utility of advanced technologies like multiplex PCR.

Optimizing the data in direct access testing: information technology to support an emerging care model

Direct access testing (DAT) is an emerging care model that provides on-demand laboratory services for certain preventative, diagnostic, and monitoring indications. Unlike conventional testing models where health care providers order tests and where sample collection is performed onsite at the clinic or laboratory, most interactions between DAT consumers and the laboratory are virtual. Tests are ordered and results delivered online, and specimens are frequently self-collected at home with virtual support. Thus, DAT depends on high-quality information technology (IT) tools and optimized data utilization to a greater degree than conventional laboratory testing. This review critically discusses the United States DAT landscape in relation to IT to highlight digital challenges and opportunities for consumers, health care systems, providers, and laboratories. DAT offers consumers increased autonomy over the testing experience, cost, and data sharing, but the current capacity to integrate DAT as a care option into the conventional patient-provider model is lacking and will require innovative approaches to accommodate. Likewise, both consumers and health care providers need transparent information about the quality of DAT laboratories and clinical decision support to optimize appropriate use of DAT as a part of comprehensive care. Interoperability barriers will require intentional approaches to integrating DAT-derived data into the electronic health records of health systems nationally. This includes ensuring the laboratory results are appropriately captured for downstream data analytic pipelines that are used to satisfy population health and research needs. Despite the data- and IT-related challenges for widespread incorporation of DAT into routine health care, DAT has the potential to improve health equity by providing versatile, discreet, and affordable testing options for patients who have been marginalized by the current limitations of health care delivery in the United States.

From Pre-test and Post-test Probabilities to Medical Decision Making

Background

A central goal of modern evidence-based medicine is the development of simple and easy to use tools that help clinicians integrate quantitative information into medical decision-making. The Bayesian Pre-test/Post-test Probability (BPP) framework is arguably the most well known of such tools and provides a formal approach to quantify diagnostic uncertainty given the result of a medical test or the presence of a clinical sign. Yet, clinical decision-making goes beyond quantifying diagnostic uncertainty and requires that that uncertainty be balanced against the various costs and benefits associated with each possible decision. Despite increasing attention in recent years, simple and flexible approaches to quantitative clinical decision-making have remained elusive.

Methods

We extend the BPP framework using concepts of Bayesian Decision Theory. By integrating cost, we can expand the BPP framework to allow for clinical decision-making.

Results

We develop a simple quantitative framework for binary clinical decisions (e.g., action/inaction, treat/no-treat, test/no-test). Let p be the pre-test or post-test probability that a patient has disease. We show that r * = ( 1 - p ) / p represents a critical value called a decision boundary. In terms of the relative cost of under- to over-acting, r * represents the critical value at which action and inaction are equally optimal. We demonstrate how this decision boundary can be used at the bedside through case studies and as a research tool through a reanalysis of a recent study which found widespread misestimation of pre-test and post-test probabilities among clinicians.

Conclusions

Our approach is so simple that it should be thought of as a core, yet previously overlooked, part of the BPP framework. Unlike prior approaches to quantitative clinical decision-making, our approach requires little more than a hand-held calculator, is applicable in almost any setting where the BPP framework can be used, and excels in situations where the costs and benefits associated with a particular decision are patient-specific and difficult to quantify.

Family physicians overestimate diagnosis probabilities regardless of the test results

Introduction

As useful tools for clinical decision-making, diagnostic tests require careful interpretation in order to prevent underdiagnosis, overdiagnosis or misdiagnosis. The aim of this study was to explore primary care practitioners' understanding and interpretation of the probability of disease before and after test results for six common clinical scenarios.

Methods

This cross-sectional study was conducted with 414 family physicians who were working at primary care in Istanbul via face-to-face interviews held between November 2021 and March 2022. The participants were asked to estimate the probability of diagnosis in six clinical scenarios provided to them. Clinical scenarios were about three cancer screening cases (breast, cervical and colorectal), and three infectious disease cases (pneumonia, urinary tract infection, and COVID-19). For each scenario participants estimated the probability of the diagnosis before application of a diagnostic test, after a positive test result, and after a negative test result. Their estimates were compared with the true answers derived from relevant guidelines.

Results

For all scenarios, physicians' estimates were significantly higher than the scientific evidence range. The minimum overestimation was positive test result for COVID-19 and maximum was pre-test case for cervical cancer. In the hypothetical control question for prevalence and test accuracy, physicians estimated disease probability as 95.0% for a positive test result and 5.0% for a negative test result while the correct answers were 2.0 and 0%, respectively (p < 0.001).

Discussion

Comparing the scientific evidence, overestimation in all diagnostic scenarios, regardless of if the disease is an acute infection or a cancer, may indicate that the probabilistic approach is not conducted by the family physicians. To prevent inaccurate interpretation of the tests that may lead to incorrect or unnecessary treatments with adverse consequences, evidence-based decision-making capacity must be strengthened.

Healthcare providers consistently overestimate the diagnostic probability of ventilator-associated pneumonia

OBJECTIVE

To assess the accuracy of provider estimates of ventilator-associated pneumonia (VAP) diagnostic probability in various clinical scenarios.

DESIGN

We conducted a clinical vignette-based survey of intensive care unit (ICU) physicians to evaluate provider estimates of VAP diagnostic probability before and after isolated cardinal VAP clinical changes and VAP diagnostic test results. Responses were used to calculate imputed diagnostic likelihood ratios (LRs), which were compared to evidence-based LRs.

SETTING

Michigan Medicine University Hospital, a tertiary-care center.

PARTICIPANTS

This study included 133 ICU clinical faculty and house staff.

RESULTS

Provider estimates of VAP diagnostic probability were consistently higher than evidence-based diagnostic probabilities. Similarly, imputed LRs from provider-estimated diagnostic probabilities were consistently higher than evidence-based LRs. These differences were most notable for positive bronchoalveolar lavage culture (provider-estimated LR 5.7 vs evidence-based LR 1.4; P < .01), chest radiograph with air bronchogram (provider-estimated LR 6.0 vs evidence-based LR 3.6; P < .01), and isolated purulent endotracheal secretions (provider-estimated LR 1.6 vs evidence-based LR 0.8; P < .01). Attending physicians and infectious disease physicians were more accurate in their LR estimates than trainees (P = .04) and non-ID physicians (P = .03).

CONCLUSIONS

Physicians routinely overestimated the diagnostic probability of VAP as well as the positive LRs of isolated cardinal VAP clinical changes and VAP diagnostic test results. Diagnostic stewardship initiatives, including educational outreach and clinical decision support systems, may be useful adjuncts in minimizing VAP overdiagnosis and ICU antibiotic overuse.

Predicting seizure recurrence after an initial seizure-like episode from routine clinical notes using large language models: a retrospective cohort study

BACKGROUND

The evaluation and management of first-time seizure-like events in children can be difficult because these episodes are not always directly observed and might be epileptic seizures or other conditions (seizure mimics). We aimed to evaluate whether machine learning models using real-world data could predict seizure recurrence after an initial seizure-like event.

METHODS

This retrospective cohort study compared models trained and evaluated on two separate datasets between Jan 1, 2010, and Jan 1, 2020: electronic medical records (EMRs) at Boston Children's Hospital and de-identified, patient-level, administrative claims data from the IBM MarketScan research database. The study population comprised patients with an initial diagnosis of either epilepsy or convulsions before the age of 21 years, based on International Classification of Diseases, Clinical Modification (ICD-CM) codes. We compared machine learning-based predictive modelling using structured data (logistic regression and XGBoost) with emerging techniques in natural language processing by use of large language models.

FINDINGS

The primary cohort comprised 14 021 patients at Boston Children's Hospital matching inclusion criteria with an initial seizure-like event and the comparison cohort comprised 15 062 patients within the IBM MarketScan research database. Seizure recurrence based on a composite expert-derived definition occurred in 57% of patients at Boston Children's Hospital and 63% of patients within IBM MarketScan. Large language models with additional domain-specific and location-specific pre-training on patients excluded from the study (F1-score 0·826 [95% CI 0·817-0·835], AUC 0·897 [95% CI 0·875-0·913]) performed best. All large language models, including the base model without additional pre-training (F1-score 0·739 [95% CI 0·738-0·741], AUROC 0·846 [95% CI 0·826-0·861]) outperformed models trained with structured data. With structured data only, XGBoost outperformed logistic regression and XGBoost models trained with the Boston Children's Hospital EMR (logistic regression: F1-score 0·650 [95% CI 0·643-0·657], AUC 0·694 [95% CI 0·685-0·705], XGBoost: F1-score 0·679 [0·676-0·683], AUC 0·725 [0·717-0·734]) performed similarly to models trained on the IBM MarketScan database (logistic regression: F1-score 0·596 [0·590-0·601], AUC 0·670 [0·664-0·675], XGBoost: F1-score 0·678 [0·668-0·687], AUC 0·710 [0·703-0·714]).

INTERPRETATION

Physician's clinical notes about an initial seizure-like event include substantial signals for prediction of seizure recurrence, and additional domain-specific and location-specific pre-training can significantly improve the performance of clinical large language models, even for specialised cohorts.

FUNDING

UCB, National Institute of Neurological Disorders and Stroke (US National Institutes of Health).

Artificial Intelligence vs Clinician Performance in Estimating Probabilities of Diagnoses Before and After Testing

This diagnostic study compares the performance of artificial intelligence (AI) with that of human clinicians in estimating the probability of diagnoses before and after testing.

Diagnostic Accuracy of a Real-Time Host-Protein Test for Infection

OBJECTIVE

Determining infection etiology can be difficult because viral and bacterial diseases often manifest similarly. A host protein test that computationally integrates the circulating levels of TNF-related apoptosis-induced ligand, interferon γ-induced protein-10, and C-reactive protein to differentiate between bacterial and viral infection (called MMBV) demonstrated high performance in multiple prospective clinical validation studies. Here, MMBV's diagnostic accuracy is evaluated in febrile children for whom physicians were uncertain about etiology when applied at the physician's discretion.

METHODS

Patients aged 3 months to 18 years were retrospectively recruited (NCT03075111; SPIRIT study; 2014-2017). Emergency department physician's etiological suspicion and certainty level were recorded in a questionnaire at blood-draw. MMBV results are based on predefined score thresholds: viral/non-bacterial etiology (0 ≤ score <35), equivocal (35 ≤ score ≤65), and bacterial or coinfection (65 < score ≤100). Reference standard etiology (bacterial/viral/indeterminate) was adjudicated by 3 independent experts based on all available patient data. Experts were blinded to MMBV. MMBV and physician's etiological suspicion were assessed against the reference standard.

RESULTS

Of 3003 potentially eligible patients, the physicians were uncertain about infection etiology for 736 of the cases assigned a reference standard (128 bacterial, 608 viral). MMBV performed with sensitivity 89.7% (96/107; 95% confidence interval 82.4-94.3) and specificity 92.6% (498/538; 95% confidence interval 90.0-94.5), significantly outperforming physician's etiological suspicion (sensitivity 49/74 = 66.2%, specificity 265/368 = 72.0%; P < .0001). MMBV equivocal rate was 12.4% (91/736).

CONCLUSIONS

MMBV was more accurate in determining etiology compared with physician's suspicion and had high sensitivity and specificity according to the reference standard.

Effect of a Peer Comparison and Educational Intervention on Medical Test Conversation Quality: A Randomized Clinical Trial

Importance

Medical test overuse and resulting care cascades represent a costly, intractable problem associated with inadequate patient-clinician communication. One possible solution with potential for broader benefits is priming routine, high-quality medical test conversations.

Objective

To assess if a peer comparison and educational intervention for physicians and patients improved medical test conversations during annual visits.

Design, Setting, and Participants

Randomized clinical trial and qualitative evaluation at an academic medical center conducted May 2021 to October 2022. Twenty primary care physicians (PCPs) were matched-pair randomized. For each physician, at least 10 patients with scheduled visits were enrolled. Data were analyzed from December 2022 to September 2023.

Interventions

In the intervention group, physicians received previsit emails that compared their low-value testing rates with those of peer PCPs and included point-of-care-accessible guidance on medical testing; patients received previsit educational materials via email and text message. Control group physicians and patients received general previsit preparation tips.

Main outcomes and measures

The primary patient outcome was the Shared Decision-Making Process survey (SDMP) score. Secondary patient outcomes included medical test knowledge and presence of test conversation. Outcomes were compared using linear regression models adjusted for patient age, gender, race and ethnicity, and education. Poststudy interviews with intervention group physicians and patients were also conducted.

Results

There were 166 intervention group patients and 148 control group patients (mean [SD] patient age, 50.2 [15.3] years; 210 [66.9%] female; 246 [78.3%] non-Hispanic White). Most patients discussed at least 1 test with their physician (95.4% for intervention group; 98.3% for control group; difference, -2.9 percentage points; 95% CI, -7.0 to 1.2 percentage points). There were no statistically significant differences in SDMP scores (2.11 out of 4 for intervention group; 1.97 for control group; difference, 0.14; 95% CI, -0.25 to 0.54) and knowledge scores (2.74 vs 2.54 out of 4; difference, 0.19; 95% CI, -0.05 to 0.43). In poststudy interviews with 3 physicians and 16 patients, some physicians said the emails helped them reexamine their testing approach while others noted competing demands. Most patients said they trusted their physicians' advice even when inconsistent with educational materials.

Conclusions and Relevance

In this randomized clinical trial of a physician-facing and patient-facing peer comparison and educational intervention, there was no significant improvement in medical test conversation quality during annual visits. These results suggest that future interventions to improve conversations and reduce overuse and cascades should further address physician adoption barriers and leverage patient-clinician relationships.

Trial Registration

ClinicalTrials.gov Identifier: NCT04902664.

The incidence and factors of inappropriate rapid antigen test usage for group A streptococcus

INTRODUCTION

Although rapid antigen tests (RADTs) for group A streptococcus (GAS) can help diagnose group A streptococcal pharyngitis, little is known about the inappropriate use of these RADTs.

METHODS

This retrospective observational study compared the appropriate vs. inappropriate use of RADTs in patients who had a RADT between January 2019 and August 2022. RADTs for patients with a low Centor score of 0-1 point were deemed inappropriate.

RESULTS

Of the 1015 patients, 380 (37.4%) had inappropriate RADTs. Patients with asthma were associated with an increased risk of inappropriate testing. In contrast, during the coronavirus 2019 pandemic, outpatients and residents were associated with a reduced risk of inappropriate testing. Consequent to the inappropriate use of RADTs, 162 (16.0%) patients received potentially inappropriate antibiotics.

CONCLUSIONS

Our results suggest that diagnostic stewardship for pharyngitis, including education for healthcare workers, is needed to reduce inappropriate test ordering and prevent unnecessary care.

The Ability of Self-Report Methods to Accurately Diagnose Attention Deficit Hyperactivity Disorder: A Systematic Review

OBJECTIVE

To identify and analyze all studies validating rating scales or interview-based screeners commonly used to evaluate ADHD in adults.

METHOD

A systematic literature search identified all studies providing diagnostic accuracy statistics, including sensitivity and specificity, supplemented by relevant articles or test manuals referenced in reviewed manuscripts.

RESULTS

Only 20 published studies or manuals provided data regarding sensitivity and specificity when tasked with differentiating those with and without ADHD. While all screening measures have excellent ability to correctly classify non-ADHD individuals (with negative predictive values exceeding 96%), false positive rates were high. At best, positive predictive values in clinical samples reached 61%, but most fell below 20%.

CONCLUSION

Clinicians cannot rely on scales alone to diagnose ADHD and must undertake more rigorous evaluation of clients who screen positive. Furthermore, relevant classification statistics must be included in publications to help clinicians make statistically defensible decisions. Otherwise, clinicians risk inappropriately diagnosing ADHD.

Biochemical Assessment of Pheochromocytoma and Paraganglioma

Pheochromocytoma and paraganglioma (PPGL) require prompt consideration and efficient diagnosis and treatment to minimize associated morbidity and mortality. Once considered, appropriate biochemical testing is key to diagnosis. Advances in understanding catecholamine metabolism have clarified why measurements of the O-methylated catecholamine metabolites rather than the catecholamines themselves are important for effective diagnosis. These metabolites, normetanephrine and metanephrine, produced respectively from norepinephrine and epinephrine, can be measured in plasma or urine, with choice according to available methods or presentation of patients. For patients with signs and symptoms of catecholamine excess, either test will invariably establish the diagnosis, whereas the plasma test provides higher sensitivity than urinary metanephrines for patients screened due to an incidentaloma or genetic predisposition, particularly for small tumors or in patients with an asymptomatic presentation. Additional measurements of plasma methoxytyramine can be important for some tumors, such as paragangliomas, and for surveillance of patients at risk of metastatic disease. Avoidance of false-positive test results is best achieved by plasma measurements with appropriate reference intervals and preanalytical precautions, including sampling blood in the fully supine position. Follow-up of positive results, including optimization of preanalytics for repeat tests or whether to proceed directly to anatomic imaging or confirmatory clonidine tests, depends on the test results, which can also suggest likely size, adrenal vs extra-adrenal location, underlying biology, or even metastatic involvement of a suspected tumor. Modern biochemical testing now makes diagnosis of PPGL relatively simple. Integration of artificial intelligence into the process should make it possible to fine-tune these advances.

Clinician Testing and Treatment Thresholds for Management of Urinary Tract Infection

Greater understanding of clinical decision thresholds may improve inappropriate testing and treatment of urinary tract infection (UTI). We used a survey of clinicians to examine UTI decision thresholds. Although overestimates of UTI occurred, testing and treatment thresholds were generally rational, were lower than previously reported, and differed by type of clinician.

Understanding and Implementing Diagnostic Stewardship: A Guide for Resident Physicians in the Era of Antimicrobial Resistance

Antimicrobial resistance (AMR) poses a significant global health challenge, exacerbated by the COVID-19 pandemic. Antimicrobial stewardship programs (ASPs) are crucial in managing this crisis, with diagnostic stewardship (DS) emerging as a key component. DS refers to the appropriate use of diagnostic tests to optimize patient outcomes, improve antimicrobial use, and combat multi-drug-resistant (MDR) organisms. Despite its potential, understanding and application of DS remain ambiguous in multiple respects, which, however, do not directly implicate the implementation of such initiatives. DS is particularly important for resident physicians who are often at the forefront of patient care and can significantly influence future AMR strategies. This review provides a comprehensive overview of DS, discussing its importance, potential challenges, and future directions. It emphasizes the need for resident physicians to understand DS principles and integrate them into their clinical practice from the beginning of their careers. The review also highlights the role of various stakeholders in implementing DS and the importance of continuous education and training. Ultimately, DS is not just a clinical tool but a philosophy of care, essential for a more responsive, humane, and effective healthcare system.

Developing a diagnosis calculator to estimate the probability of bacterial pneumonia

Misdiagnosis of bacterial pneumonia increases risk of exposure to inappropriate antibiotics and adverse events. We developed a diagnosis calculator (https://calculator.testingwisely.com) to inform clinical diagnosis of community-acquired bacterial pneumonia using objective indicators, including incidence of disease, risk factors, and sensitivity and specificity of diagnostic tests, that were identified through literature review.

Clinical characteristics of hospitalized children with community-acquired pneumonia and respiratory infections: Using machine learning approaches to support pathogen prediction at admission

BACKGROUND

Acute respiratory infections (ARIs) are common in children. We developed machine learning models to predict pediatric ARI pathogens at admission.

METHODS

We included hospitalized children with respiratory infections between 2010 and 2018. Clinical features were collected within 24 h of admission to construct models. The outcome of interest was the prediction of 6 common respiratory pathogens, including adenovirus, influenza virus types A and B, parainfluenza virus (PIV), respiratory syncytial virus (RSV), and Mycoplasma pneumoniae (MP). Model performance was estimated using area under the receiver operating characteristic curve (AUROC). Feature importance was measured using Shapley Additive exPlanation (SHAP) values.

RESULTS

A total of 12,694 admissions were included. Models trained with 9 features (age, event pattern, fever, C-reactive protein, white blood cell count, platelet count, lymphocyte ratio, peak temperature, peak heart rate) achieved the best performance (AUROC: MP 0.87, 95% CI 0.83-0.90; RSV 0.84, 95% CI 0.82-0.86; adenovirus 0.81, 95% CI 0.77-0.84; influenza A 0.77, 95% CI 0.73-0.80; influenza B 0.70, 95% CI 0.65-0.75; PIV 0.73, 95% CI 0.69-0.77). Age was the most important feature to predict MP, RSV and PIV infections. Event patterns were useful for influenza virus prediction, and C-reactive protein had the highest SHAP value for adenovirus infections.

CONCLUSION

We demonstrate how artificial intelligence can assist clinicians identify potential pathogens associated with pediatric ARIs upon admission. Our models provide explainable results that could help optimize the use of diagnostic testing. Integrating our models into clinical workflows may lead to improved patient outcomes and reduce unnecessary medical costs.

"Pivot and Cluster Strategy" in the light of Kahneman's "Decision Hygiene" template

Pivot and cluster strategy (PCS) is a cognitive forcing strategy designed to achieve diagnostic accuracy through the analytical deployment of a cluster of differential diagnoses (Cluster) specific to the initial most likely diagnosis (Pivot) recalled by a clinical diagnostician. This approach has been widely implemented and has effectively decreased diagnostic errors. Kahneman et al. have introduced innovative notions of noise and decision hygiene. Noise refers to the variance of errors, with numerous individuals' errors in judgment pointing in different directions. They suggest a "Decision Hygiene" (DH) template, w preventative technique meant to reduce noise in decision-making. This paper introduced an interpretation of the existing strategy of PCS from new perspectives of noise and DH, which would allow us to further understand the usefulness of PCS, thereby contributing to a positive effect on the quality of diagnosis.

Treatment Patterns and Unmet Needs in the Management of Alopecia Areata: Results of a Physician's Survey in the Middle East

INTRODUCTION

Alopecia areata (AA) is an autoimmune disease characterized by nonscarring hair loss involving the scalp, face, and/or body. Literature on the prevalence, patient characteristics, management approaches, and challenges faced by patients with AA across the Middle East is limited. Therefore, a greater understanding of the current AA landscape within the region is needed. This cross-sectional study surveyed dermatologists from four countries to assess dermatologists' perspectives on the prevalence of AA within the Middle East, as well as patient characteristics, unmet needs, and management strategies.

METHODS

This blinded, quantitative, observational study surveyed practicing dermatologists in Egypt, Lebanon, Saudi Arabia, and the United Arab Emirates. The survey was conducted between September 2021 and January 2022 and comprised 47 closed-ended, multiple-choice questions as well as Likert scale responses. These questions assessed the characteristics of physicians and the patients in their practices, physicians' familiarity with treatment, and physicians' treatment approaches.

RESULTS

The estimated prevalence of AA varied across the region. Across all age groups treated for AA, the majority of patients had AA of mild severity (pediatric: 63%; adolescent: 60%; adult: 54%) and the scalp was reported as the most affected area (65%). Potent topical corticosteroids were the most frequently used treatment for mild to moderate and severe AA (92% and 78%, respectively). There was a lack of awareness of investigative treatments, with only 33% of dermatologists aware of these options. The greatest unmet needs in treating AA included long-term disease control, improved efficacy, faster onset of action, and better safety profiles (62%, 53%, 52%, and 51%, respectively).

CONCLUSIONS

This study provided insight into the diagnosis and management of AA in the Middle East. Treatment strategies were similar regardless of the severity of AA. Long-term disease control and improved efficacy and safety profiles were identified as key unmet needs in the treatment of AA.

Development and validation of an electronic health records-based opioid use disorder algorithm by expert clinical adjudication among patients with prescribed opioids

BACKGROUND

In the US, over 200 lives are lost from opioid overdoses each day. Accurate and prompt diagnosis of opioid use disorders (OUD) may help prevent overdose deaths. However, international classification of disease (ICD) codes for OUD are known to underestimate prevalence, and their specificity and sensitivity are unknown. We developed and validated algorithms to identify OUD in electronic health records (EHR) and examined the validity of OUD ICD codes.

METHODS

Through four iterations, we developed EHR-based OUD identification algorithms among patients who were prescribed opioids from 2014 to 2017. The algorithms and OUD ICD codes were validated against 169 independent "gold standard" EHR chart reviews conducted by an expert adjudication panel across four healthcare systems. After using 2014-2020 EHR for validating iteration 1, the experts were advised to use 2014-2017 EHR thereafter.

RESULTS

Of the 169 EHR charts, 81 (48%) were reviewed by more than one expert and exhibited 85% expert agreement. The experts identified 54 OUD cases. The experts endorsed all 11 OUD criteria from the Diagnostic and Statistical Manual of Mental Disorders-5, including craving (72%), tolerance (65%), withdrawal (56%), and recurrent use in physically hazardous conditions (50%). The OUD ICD codes had 10% sensitivity and 99% specificity, underscoring large underestimation. In comparison our algorithm identified OUD with 23% sensitivity and 98% specificity.

CONCLUSIONS AND RELEVANCE

This is the first study to estimate the validity of OUD ICD codes and develop validated EHR-based OUD identification algorithms. This work will inform future research on early intervention and prevention of OUD.

Evidence of a cognitive bias in the quantification of COVID-19 with CT: an artificial intelligence randomised clinical trial

Chest computed tomography (CT) has played a valuable, distinct role in the screening, diagnosis, and follow-up of COVID-19 patients. The quantification of COVID-19 pneumonia on CT has proven to be an important predictor of the treatment course and outcome of the patient although it remains heavily reliant on the radiologist's subjective perceptions. Here, we show that with the adoption of CT for COVID-19 management, a new type of psychophysical bias has emerged in radiology. A preliminary survey of 40 radiologists and a retrospective analysis of CT data from 109 patients from two hospitals revealed that radiologists overestimated the percentage of lung involvement by 10.23 ± 4.65% and 15.8 ± 6.6%, respectively. In the subsequent randomised controlled trial, artificial intelligence (AI) decision support reduced the absolute overestimation error (P < 0.001) from 9.5% ± 6.6 (No-AI analysis arm, n = 38) to 1.0% ± 5.2 (AI analysis arm, n = 38). These results indicate a human perception bias in radiology that has clinically meaningful effects on the quantitative analysis of COVID-19 on CT. The objectivity of AI was shown to be a valuable complement in mitigating the radiologist's subjectivity, reducing the overestimation tenfold.Trial registration: https://Clinicaltrial.gov . Identifier: NCT05282056, Date of registration: 01/02/2022.

Predicting survival and neurological outcome in out-of-hospital cardiac arrest using machine learning: the SCARS model

BACKGROUND

A prediction model that estimates survival and neurological outcome in out-of-hospital cardiac arrest patients has the potential to improve clinical management in emergency rooms.

METHODS

We used the Swedish Registry for Cardiopulmonary Resuscitation to study all out-of-hospital cardiac arrest (OHCA) cases in Sweden from 2010 to 2020. We had 393 candidate predictors describing the circumstances at cardiac arrest, critical time intervals, patient demographics, initial presentation, spatiotemporal data, socioeconomic status, medications, and comorbidities before arrest. To develop, evaluate and test an array of prediction models, we created stratified (on the outcome measure) random samples of our study population. We created a training set (60% of data), evaluation set (20% of data), and test set (20% of data). We assessed the 30-day survival and cerebral performance category (CPC) score at discharge using several machine learning frameworks with hyperparameter tuning. Parsimonious models with the top 1 to 20 strongest predictors were tested. We calibrated the decision threshold to assess the cut-off yielding 95% sensitivity for survival. The final model was deployed as a web application.

FINDINGS

We included 55,615 cases of OHCA. Initial presentation, prehospital interventions, and critical time intervals variables were the most important. At a sensitivity of 95%, specificity was 89%, positive predictive value 52%, and negative predictive value 99% in test data to predict 30-day survival. The area under the receiver characteristic curve was 0.97 in test data using all 393 predictors or only the ten most important predictors. The final model showed excellent calibration. The web application allowed for near-instantaneous survival calculations.

INTERPRETATION

Thirty-day survival and neurological outcome in OHCA can rapidly and reliably be estimated during ongoing cardiopulmonary resuscitation in the emergency room using a machine learning model incorporating widely available variables.

FUNDING

Swedish Research Council (2019-02019); Swedish state under the agreement between the Swedish government, and the county councils (ALFGBG-971482); The Wallenberg Centre for Molecular and Translational Medicine.

Evaluation of an Opt-Out Protocol for Antibiotic De-Escalation in Patients With Suspected Sepsis: A Multicenter, Randomized, Controlled Trial

BACKGROUND

Sepsis guidelines recommend daily review to de-escalate or stop antibiotics in appropriate patients. This randomized, controlled trial evaluated an opt-out protocol to decrease unnecessary antibiotics in patients with suspected sepsis.

METHODS

We evaluated non-intensive care adults on broad-spectrum antibiotics despite negative blood cultures at 10 US hospitals from September 2018 through May 2020. A 23-item safety check excluded patients with ongoing signs of systemic infection, concerning or inadequate microbiologic data, or high-risk conditions. Eligible patients were randomized to the opt-out protocol vs usual care. Primary outcome was post-enrollment antibacterial days of therapy (DOT). Clinicians caring for intervention patients were contacted to encourage antibiotic discontinuation using opt-out language. If continued, clinicians discussed the rationale for continuing antibiotics and de-escalation plans. To evaluate those with zero post-enrollment DOT, hurdle models provided 2 measures: odds ratio of antibiotic continuation and ratio of mean DOT among those who continued antibiotics.

RESULTS

Among 9606 patients screened, 767 (8%) were enrolled. Intervention patients had 32% lower odds of antibiotic continuation (79% vs 84%; odds ratio, 0.68; 95% confidence interval [CI], .47-.98). DOT among those who continued antibiotics were similar (ratio of means, 1.06; 95% CI, .88-1.26). Fewer intervention patients were exposed to extended-spectrum antibiotics (36% vs 44%). Common reasons for continuing antibiotics were treatment of localized infection (76%) and belief that stopping antibiotics was unsafe (31%). Thirty-day safety events were similar.

CONCLUSIONS

An antibiotic opt-out protocol that targeted patients with suspected sepsis resulted in more antibiotic discontinuations, similar DOT when antibiotics were continued, and no evidence of harm.

CLINICAL TRIALS REGISTRATION

NCT03517007.

Medical comorbidities and ethnicity impact plasma Alzheimer's disease biomarkers: Important considerations for clinical trials and practice

INTRODUCTION

Despite the clinical implementation, there remain significant gaps in our knowledge regarding the impact of race/ethnicity or common medical comorbidity on plasma Alzheimer's disease (AD) biomarkers.

METHODS

Plasma biomarkers of amyloid beta (Aβ)_40, Aβ₄₂ , total tau, and neurofilament light chain (NfL) were measured across cognitively normal Mexican Americans (n = 445) and non-Hispanic Whites (n = 520).

RESULTS

Dyslipidemia was associated with elevated Aβ₄₀ (P = .01) and Aβ₄₂ (P = .001) while hypertension was associated with elevated Aβ₄₀ (P = .003), Aβ₄₂ (P < .001), and total tau (P = .002) levels. Diabetes was associated with higher Aβ₄₀ (P < .001), Aβ₄₂ (P < .001), total tau (P < .001), and NfL (P < .001) levels. Chronic kidney disease (CKD) was associated with elevations in Aβ₄₀ (P < .001), Aβ₄₂ (P < .001), total tau (P < .001), and NfL (P < .001) levels. Mexican Americans had significantly lower Aβ₄₀ (P < .001) and higher total tau (P = .005) levels.

DISCUSSION

Plasma AD biomarkers vary significantly in association with common medical comorbidities as well as ethnicity. These findings are important for those using these biomarkers in clinical practice and clinical trials.

Common things are common, but what is common? Incorporating probability information into differential diagnosis

The well-known clinical axiom declaring that 'common things are common' attests to the pivotal role of probability in diagnosis. Despite the popularity of this and related axioms, there is no operationalized definition of a common disease, and no practicable way of incorporating actual disease frequencies into differential diagnosis. In this essay, we aim to disambiguate the definition of a common (or rare) disease and show that incidence-not prevalence-is the proper metric of disease frequency for differential diagnosis. We explore how numerical estimates of disease frequencies based on incidence can be incorporated into differential diagnosis as well as the inherent limitations of this method. These concepts have important implications for diagnostic decision making and medical education, and hold promise as a method to improve diagnostic accuracy.

Impact of Organism Reporting from Endotracheal Aspirate Cultures on Antimicrobial Prescribing Practices in Mechanically Ventilated Pediatric Patients

Endotracheal aspirate cultures (EACs) help diagnose lower respiratory tract infections in mechanically ventilated patients but are limited by contamination with normal microbiota and variation in laboratory reporting. Increased use of EACs is associated with increased antimicrobial prescribing, but the impact of microbiology reporting on prescribing practices is unclear. This study was a retrospective analysis of EACs from mechanically ventilated patients at Children's Hospital Colorado (CHCO) admitted between 1 January 2019 and 31 December 2019. Chart review was performed to collect all culture and Gram stain components, as well as antibiotic use directed to organisms in culture. Reporting concordance was determined for each organism using American Society for Microbiology guidelines. Days of therapy were calculated for overreported and guideline-concordant organisms. A multivariable model was used to assess the relationship between organism reporting and total days of therapy. Overall, 448 patients with 827 EACs were included in this study. Among patients with tracheostomy, 25 (8%) organisms reported from EACs were overreported and contributed 48 days of excess therapy, while 227 (29%) organisms from the EACs of endotracheally intubated patients were overreported, contributing 472 excess days of therapy. After adjustment, organism overreporting was associated with a >2-fold-higher rate of antimicrobial therapy than guideline-concordant reporting (incident rate ratio [IRR], 2.83; 95% confidence interval [CI], 1.23, 6.53; P < 0.05). Overreported organisms from respiratory cultures contribute to excess antimicrobial therapy exposure in mechanically ventilated patients. Microbiology laboratories have an opportunity to mitigate antimicrobial overuse through standardized reporting practices.

Diagnostic accuracy of clinical signs, symptoms and point-of-care testing for early adenoviral conjunctivitis

CLINICAL RELEVANCE

This study identifies key signs and symptoms of acute conjunctivitis, that when combined with a point-of-care test, can improve clinician accuracy of diagnosing adenoviral conjunctivitis.

BACKGROUND

Adenoviral conjunctivitis is a common ocular infection with the potential for high economic impact due to widespread outbreaks and subsequent furloughs from work and school. In this report, we describe clinical signs and participant-reported symptoms that most accurately identify polymerase chain reaction (PCR)-confirmed adenoviral conjunctivitis.

METHODS

Adults with 'red eye' symptoms of four days or less were enrolled. Participants rated 10 ocular symptoms from 0 (not bothersome) to 10 (very bothersome), and indicated the presence or absence of systemic flu-like symptoms. Clinicians determined the presence or absence of swollen lymph nodes and rated the severity of eight ocular signs using a 5-point scale. An immunoassay targeting adenovirus antigen was utilised for the point-of-care test, and conjunctival swab samples were obtained for subsequent adenovirus detection by PCR analyses. Univariate and multivariate logistic regression models were used to identify symptoms and signs associated with PCR-confirmed adenoviral conjunctivitis. The diagnostic accuracy of these clinical findings, and the potential benefit of incorporating point-of-care test results, was assessed by calculating areas under the receiver operating characteristic curves (AUC).

RESULTS

Clinician-rated bulbar conjunctival redness, participant-rated eyelid swelling and overall ocular discomfort had the best predictive value in the multivariate logistic regression model with an AUC of 0.83. The addition of the point-of-care test results to these three clinical sign/symptom scores improved diagnostic accuracy, increasing the AUC to 0.94.

CONCLUSIONS

Conjunctival redness severity and participant-reported eyelid swelling and overall discomfort, along with adenoviral point-of-care test results, were highly predictive in identifying individuals with PCR-confirmed adenoviral conjunctivitis. Improved diagnostic accuracy by clinicians at the initial presenting visit could prevent unnecessary work furloughs and facilitate earlier treatment decisions.

Geographic Upon Noninfectious Diseases Accounting for Fever of Unknown Origin (FUO): A Systematic Review and Meta-analysis

Background

Diagnostic outcomes for fever of unknown origin (FUO) remain with notable numbers of undiagnosed cases. A recent systemic review and meta-analysis of studies reported geographic variation in FUO-related infectious diseases. Whether geography influences types of FUO noninfectious diagnoses deserves examination.

Methods

Medline (PubMed), Embase, Scopus, and Web of Science databases were searched systematically using medical subject headings published from January 1, 1997, to March 31, 2021. Prospective clinical studies investigating participants meeting adult FUO defining criteria were selected if they assessed final diagnoses. Meta-analyses were based on the random-effects model according to World Health Organization (WHO) geographical regions.

Results

Nineteen studies with significant heterogeneity were analyzed, totaling 2,667 participants. Noninfectious inflammatory disorders had a pooled estimate at 20.0% (95%CI: 17.0-23.0%). Undiagnosed illness had a pooled estimate of 20.0% (95%CI: 14.0-26.0%). The pooled estimate for cancer was 15.0% (95%CI: 12.0-18.0%). Miscellaneous conditions had a pooled estimate of 6.0% (95%CI: 4.0-8.0%). Noninfectious inflammatory disorders and miscellaneous conditions were most prevalent in the Western Pacific region with a 27.0% pooled estimate (95%CI: 20.0-34.0%) and 9.0% (95%CI: 7.0-11.0%), respectively. The highest pooled estimated for cancer was in the Eastern Mediterranean region at 25.0% (95%CI: 18.0-32.0%). Adult-onset Still’s disease (114 [58.5%]), systemic lupus (52 [26.7%]), and giant-cell arteritis (40 [68.9%]) predominated among the noninfectious inflammatory group. Lymphoma (164 [70.1%]) was the most common diagnosis in the cancer group.

Conclusions

In this systematic review and meta-analysis, noninfectious disease diagnostic outcomes varied among WHO-defined geographies. Evaluation of FUO should consider local variations in disease prevalence.

Clinical diagnosis of seasonal influenza by physicians: a retrospective observational study

OBJECTIVE

To elucidate the diagnostic accuracy of pretest probability of influenza (%) by physicians and the factors affecting the clinical diagnosis.

DESIGN

Retrospective, single-centre observational study.

SETTING

A community primary care clinic in Japan.

PARTICIPANTS

The participants were recruited from a database of studies conducted during the influenza season from December 2017 to April 2019.

PRIMARY OUTCOME MEASURE

Sensitivity and specificity of the physician's clinical diagnosis of influenza recorded in the medical record as pretest probability.

RESULTS

A total of 335 patients (median age, 31 years; male, 66.6%) were analysed in this study. The area under the curve (AUC) of the physician's pretest probability was 0.77. At a cut-off value of 30%, the sensitivity and negative likelihood ratio were 92.0% (95% CI 86.7 to 95.7) and 0.19 (95% CI 0.11 to 0.33), respectively. At a cut-off value of 80%, the specificity and positive likelihood ratio were 90.8% (95% CI 85.4 to 94.6) and 4.01 (95% CI 2.41 to 6.66), respectively. The AUCs of patients who had and had not taken any medications before visiting the clinic were 0.77 (95% CI 0.69 to 0.85) and 0.78 (95% CI 0.71 to 0.84), respectively. The AUCs of patients with type A and B influenza were 0.78 (95% CI 0.72 to 0.84) and 0.76 (95% CI 0.70 to 0.82), respectively. The AUCs of vaccinated and unvaccinated patients were 0.80 (95% CI 0.72 to 0.88) and 0.76 (95% CI 0.63 to 0.89), respectively. The AUC for patients less than 12 hours after onset was 0.69 (95% CI 0.51 to 0.88), and that for patients aged younger than 6 years was 0.69 (95% CI 0.49 to 0.88).

CONCLUSIONS

The physician's pretest probability of influenza (%) may be useful for both definitive and exclusionary diagnoses within the limits of our study.

Integration of artificial intelligence and plasma steroidomics with laboratory information management systems: application to primary aldosteronism

Objectives

Mass spectrometry-based steroidomics combined with machine learning (ML) provides a potentially powerful approach in endocrine diagnostics, but is hampered by limitations in the conveyance of results and interpretations to clinicians. We address this shortcoming by integration of the two technologies with a laboratory information management systems (LIMS) model.

Methods

The approach involves integration of ML algorithm-derived models with commercially available mathematical programming software and a web-based LIMS prototype. To illustrate clinical utility, the process was applied to plasma steroidomics data from 22 patients tested for primary aldosteronism (PA).

Results

Once mass spectrometry data are uploaded into the system, automated processes enable generation of interpretations of steroid profiles from ML models. Generated reports include plasma concentrations of steroids in relation to age- and sex-specific reference intervals along with results of ML models and narrative interpretations that cover probabilities of PA. If PA is predicted, reports include probabilities of unilateral disease and mutations of KCNJ5 known to be associated with successful outcomes of adrenalectomy. Preliminary results, with no overlap in probabilities of disease among four patients with and 18 without PA and correct classification of all four patients with unilateral PA including three of four with KCNJ5 mutations, illustrate potential utility of the approach to guide diagnosis and subtyping of patients with PA.

Conclusions

The outlined process for integrating plasma steroidomics data and ML with LIMS may facilitate improved diagnostic-decision-making when based on higher-dimensional data otherwise difficult to interpret. The approach is relevant to other diagnostic applications involving ML.

Clinician Attitudes and Beliefs Associated with More Aggressive Diagnostic Testing

BACKGROUND

Variation in clinicians' diagnostic test utilization is incompletely explained by demographics and likely relates to cognitive characteristics. We explored clinician factors associated with diagnostic test utilization.

METHODS

We used a self-administered survey of attitudes, cognitive characteristics, and reported likelihood of test ordering in common scenarios; frequency of lipid and liver testing in patients on statin therapy. Participants were 552 primary care physicians, nurse practitioners, and physician assistants from practices in 8 US states across 3 regions, from June 1, 2018 to November 26, 2019. We measured Testing Likelihood Score: the mean of 4 responses to testing frequency and self-reported testing frequency in patients on statins.

RESULTS

Respondents were 52.4% residents, 36.6% attendings, and 11.0% nurse practitioners/physician assistants; most were white (53.6%) or Asian (25.5%). Median age was 32 years; 53.1% were female. Participants reported ordering tests for a median of 20% (stress tests) to 90% (mammograms) of patients; Testing Likelihood Scores varied widely (median 54%, interquartile range 43%-69%). Higher scores were associated with geography, training type, low numeracy, high malpractice fear, high medical maximizer score, high stress from uncertainty, high concern about bad outcomes, and low acknowledgment of medical uncertainty. More frequent testing of lipids and liver tests was associated with low numeracy, high medical maximizer score, high malpractice fear, and low acknowledgment of uncertainty.

CONCLUSIONS

Clinician variation in testing was common, with more aggressive testing consistently associated with low numeracy, being a medical maximizer, and low acknowledgment of uncertainty. Efforts to reduce undue variations in testing should consider clinician cognitive drivers.

Analysis of Physicians' Probability Estimates of a Medical Outcome Based on a Sequence of Events

Importance

The probability of a conjunction of 2 independent events is the product of the probabilities of the 2 components and therefore cannot exceed the probability of either component; violation of this basic law is called the conjunction fallacy. A common medical decision-making scenario involves estimating the probability of a final outcome resulting from a sequence of independent events; however, little is known about physicians' ability to accurately estimate the overall probability of success in these situations.

Objective

To ascertain whether physicians are able to correctly estimate the overall probability of a medical outcome resulting from 2 independent events.

Design, Setting, and Participants

This survey study consisted of 3 separate substudies, in which 215 physicians were asked via internet-based survey to estimate the probability of success of each of 2 components of a diagnostic or prognostic sequence as well as the overall probability of success of the 2-step sequence. Substudy 1 was performed from April 2 to 4, 2021, substudy 2 from November 2 to 11, 2021, and substudy 3 from May 13 to 19, 2021. All physicians were board certified or board eligible in the primary specialty germane to the substudy (ie, obstetrics and gynecology for substudies 1 and 3 and pulmonology for substudy 2), were recruited from a commercial survey service, and volunteered to participate in the study.

Exposures

Case scenarios presented in an online survey.

Main Outcomes and Measures

Respondents were asked to provide their demographic information in addition to 3 probability estimates. The first substudy included a scenario describing a brow presentation discovered during labor; the 2 conjuncts were the probabilities that the brow presentation would resolve and that the delivery would be vaginal. The second substudy involved a diagnostic evaluation of an incidentally discovered pulmonary nodule; the 2 conjuncts were the probabilities that the patient had a malignant condition and that a technically successful transthoracic needle biopsy would reveal a malignant condition. The third substudy included a modification of the first substudy in an attempt to debias the conjunction fallacy prevalent in the first substudy. Respondents' own probability estimates of the individual events were used to calculate the mathematically correct conjunctive probability.

Results

Among 215 respondents, the mean (SD) age was 54.0 (9.5) years; 142 respondents (66.0%) were male. Data on race and ethnicity were not collected. A total of 168 physicians (78.1%) estimated the probability of the 2-step sequence to be greater than the probability of at least 1 of the 2 component events. Compared with the product of their 2 estimated components, respondents overestimated the combined probability by 12.8% (95% CI, 9.6%-16.1%; P < .001) in substudy 1, 19.8% (95% CI, 16.6%-23.0%; P < .001) in substudy 2, and 18.0% (95% CI, 13.4%-22.5%; P < .001) in substudy 3, results that were mathematically incoherent (ie, formally illogical and mathematically incorrect).

Conclusions and Relevance

In this survey study of 215 physicians, respondents consistently overestimated the combined probability of 2 events compared with the probability calculated from their own estimates of the individual events. This biased estimation, consistent with the conjunction fallacy, may have substantial implications for diagnostic and prognostic decision-making.

'Diagnostic downshift': clinical and system consequences of extrapolating secondary care testing tactics to primary care

Numerous drivers push specialist diagnostic approaches down to primary care ('diagnostic downshift'), intuitively welcomed by clinicians and patients. However, primary care's different population and processes result in under-recognised, unintended consequences. Testing performs poorer in primary care, with indication creep due to earlier, more undifferentiated presentation and reduced accuracy due to spectrum bias and the 'false-positive paradox'. In low-prevalence settings, tests without near-100% specificity have their useful yield eclipsed by greater incidental or false-positive findings. Ensuing cascades and multiplier effects can generate clinician workload, patient anxiety, further low-value tests, referrals, treatments and a potentially nocebic population 'disease' burden of unclear benefit. Increased diagnostics earlier in pathways can burden patients and stretch general practice (GP) workloads, inducing downstream service utilisation and unintended 'market failure' effects. Evidence is tenuous for reducing secondary care referrals, providing patient reassurance or meaningfully improving clinical outcomes. Subsequently, inflated investment in per capita testing, at a lower level in a healthcare system, may deliver diminishing or even negative economic returns. Test cost poorly represents 'value', neglecting under-recognised downstream consequences, which must be balanced against therapeutic yield. With lower positive predictive values, more tests are required per true diagnosis and cost-effectiveness is rarely robust. With fixed secondary care capacity, novel primary care testing is an added cost pressure, rarely reducing hospital activity. GP testing strategies require real-world evaluation, in primary care populations, of all downstream consequences. Test formularies should be scrutinised in view of the setting of care, with interventions to focus rational testing towards those with higher pretest probabilities, while improving interpretation and communication of results.