Management Reasoning: Empirical Determination of Key Features and a Conceptual Model

Large Language Model Influence on Management Reasoning: A Randomized Controlled Trial

Importance

Large language model (LLM) artificial intelligence (AI) systems have shown promise in diagnostic reasoning, but their utility in management reasoning with no clear right answers is unknown.

Objective

To determine whether LLM assistance improves physician performance on open-ended management reasoning tasks compared to conventional resources.

Design

Prospective, randomized controlled trial conducted from 30 November 2023 to 21 April 2024.

Setting

Multi-institutional study from Stanford University, Beth Israel Deaconess Medical Center, and the University of Virginia involving physicians from across the United States.

Participants

92 practicing attending physicians and residents with training in internal medicine, family medicine, or emergency medicine.

Intervention

Five expert-developed clinical case vignettes were presented with multiple open-ended management questions and scoring rubrics created through a Delphi process. Physicians were randomized to use either GPT-4 via ChatGPT Plus in addition to conventional resources (e.g., UpToDate, Google), or conventional resources alone.

Main Outcomes and Measures

The primary outcome was difference in total score between groups on expert-developed scoring rubrics. Secondary outcomes included domain-specific scores and time spent per case.

Results

Physicians using the LLM scored higher compared to those using conventional resources (mean difference 6.5 %, 95% CI 2.7-10.2, p<0.001). Significant improvements were seen in management decisions (6.1%, 95% CI 2.5-9.7, p=0.001), diagnostic decisions (12.1%, 95% CI 3.1-21.0, p=0.009), and case-specific (6.2%, 95% CI 2.4-9.9, p=0.002) domains. GPT-4 users spent more time per case (mean difference 119.3 seconds, 95% CI 17.4-221.2, p=0.02). There was no significant difference between GPT-4-augmented physicians and GPT-4 alone (-0.9%, 95% CI -9.0 to 7.2, p=0.8).

Conclusions and Relevance

LLM assistance improved physician management reasoning compared to conventional resources, with particular gains in contextual and patient-specific decision-making. These findings indicate that LLMs can augment management decision-making in complex cases.

Trial registration

ClinicalTrials.gov Identifier: NCT06208423; https://classic.clinicaltrials.gov/ct2/show/NCT06208423.

'Walking together': How relationships shape physicians' clinical reasoning

INTRODUCTION

The clinical reasoning literature has increasingly considered context as an important influence on physicians' thinking. Physicians' relationships with patients, and their ongoing efforts to maintain these relationships, are important influences on how clinical reasoning is contextualised. The authors sought to understand how physicians' relationships with patients shaped their clinical reasoning.

METHODS

Drawing from constructivist grounded theory, the authors conducted semi-structured interviews with primary care physicians. Participants were asked to reflect on recent challenging clinical experiences, and probing questions were used to explore how participants attended to or leveraged relationships in conjunction with their clinical reasoning. Using constant comparison, three investigators coded transcripts, organising the data into codes and conceptual categories. The research team drew from these codes and categories to develop theory about the phenomenon of interest.

RESULTS

The authors interviewed 15 primary care physicians with a range of experience in practice and identified patient agency as a central influence on participants' clinical reasoning. Participants drew from and managed relationships with patients while attending to patients' agency in three ways. First, participants described how contextualised illness constructions enabled them to individualise their approaches to diagnosis and management. Second, participants managed tensions between enacting their typical approaches to clinical problems and adapting their approaches to foster ongoing relationships with patients. Finally, participants attended to relationships with patients' caregivers, seeing these individuals' contributions as important influences on how their clinical reasoning could be enacted within patients' unique social contexts.

CONCLUSION

Clinical reasoning is influenced in important ways by physicians' efforts to both draw from, and maintain, their relationships with patients and patients' caregivers. Such efforts create tensions between their professional standards of care and their orientations toward patient-centredness. These influences of relationships on physicians' clinical reasoning have important implications for training and clinical practice.

Reasoning Report: Engineering Case Conferences to Maximize Clinical Reasoning Education for All Learners

Case conferences, specifically those in which an unknown case is presented and discussed, are widely utilized in the delivery of medical education. However, the format of case conferences is not always optimized to engage and challenge audience members' clinical reasoning (CR). Based on the current conception of CR and our experience, we provide recommendations on how to better engineer case conferences to maximize CR education for learners at all levels through case selection, conference format, and intentional case construction.

Management reasoning and patient-clinician interactions: Insights from shared decision-making and simulated outpatient encounters

PURPOSE

To expand understanding of patient-clinician interactions in management reasoning.

METHODS

We reviewed 10 videos of simulated patient-clinician encounters to identify instances of problematic and successful communication, then reviewed the videos again through the lens of two models of shared decision-making (SDM): an 'involvement-focused' model and a 'problem-focused' model. Using constant comparative qualitative analysis we explored the connections between these patient-clinician interactions and management reasoning.

RESULTS

Problems in patient-clinician interactions included failures to: encourage patient autonomy; invite the patient's involvement in decision-making; convey the health impact of the problem; explore and address concerns and questions; explore the context of decision-making (including patient preferences); meet the patient where they are; integrate situational preferences and priorities; offer >1 viable option; work with the patient to solve a problem of mutual concern; explicitly agree to a final care plan; and build the patient-clinician relationship. Clinicians' 'management scripts' varied along a continuum of prioritizing clinician vs patient needs. Patients also have their own cognitive scripts that guide their interactions with clinicians. The involvement-focused and problem-focused SDM models illuminated distinct, complementary issues.

CONCLUSIONS

Management reasoning is a deliberative interaction occurring in the space between individuals. Juxtaposing management reasoning alongside SDM generated numerous insights.

Script concordance test to assess diagnostic and management reasoning in acute medicine

Clinical reasoning, an essential skill for patient care, can be difficult to assess. We created and validated a script concordance test (SCT) to assess clinical reasoning in acute medicine. This tool was used to provide feedback and targeted remediation for Postgraduate-Year-1 (PGY1) doctors, guide teaching and learning, and facilitate programme evaluation.

Management reasoning scripts: Qualitative exploration using simulated physician-patient encounters

INTRODUCTION

Management reasoning is distinct from diagnostic reasoning and remains incompletely understood. The authors sought to empirically investigate the concept of management scripts.

METHODS

In November 2020, 4 investigators each reviewed 10 video clips of simulated outpatient physician-patient encounters, and used a coding form to document observations about management reasoning. The team used constant comparative analysis to integrate empirically-grounded insights with theories related to cognitive scripts and Type 1/Type 2 thinking.

RESULTS

Management scripts are precompiled conceptual knowledge structures that represent and connect management options and clinician tasks in a temporal or logical sequence. Management scripts appear to differ substantially from illness scripts. Management scripts varied in quality (in content, sequence, flexibility, and fluency) and generality. The authors empirically identified six key features (components) of management scripts: the problem (diagnosis); management options; preferences, values, and constraints; education needs; interactions; and encounter flow. The authors propose a heuristic framework describing script activation, selection, instantiation with case-specific details, and application to guide development of the management plan. They further propose that management reasoning reflects iterative, back-and-forth involvement of both Type 1 (non-analytic, effortless) and Type 2 (analytic, effortful) thinking. Type 1 thinking likely influences initial script activation, selection, and initial instantiation. Type 2 increasingly influences subsequent script revisions, as activation, selection, and instantiation become more deliberate (effortful) and more hypothetical (involving mental simulation).

DISCUSSION

Management scripts constitute a key feature of management reasoning, and could represent a new target for training in clinical reasoning (distinct from illness scripts).