Medicine and the McNamara fallacy

The Shadow Economy of Effort: Unintended Consequences of Pass/Fail Grading on Medical Students' Clinical Education and Patient Care Skills

ABSTRACT

The shift to pass/fail grading in undergraduate medical education was designed to reduce medical students' stress. However, this change has given rise to a "shadow economy of effort," as students move away from traditional didactic and clinical learning to engage in increasing numbers of research, volunteer, and work experiences to enhance their residency applications. These extracurricular efforts to secure a residency position are subphenomena of the hidden curriculum. Medical schools do not officially require all the activities students need to be most competitive for residency selection; therefore, students, as rational actors, participate in the activities they think will most help them succeed.Here, the authors frame residency application and selection as a complex adaptive system (CAS), which self-organizes without centralized control or hierarchical intent. Individuals in a CAS operate in environments marked by volatility, randomness, and uncertainty-all of which are abundant in the residency selection process. Outcomes in such systems, like the development of a shadow economy, are novel, emergent, and cannot always be anticipated. To address these challenges, the authors suggest the need for deep understanding of the system's elements, interrelationships, and dynamics, including feedback loops and emergent properties. Optimizing the results of a CAS requires incentivizing outcomes over activities, ensuring open information flow, and engaging in continuous monitoring and evaluation.The current pass/fail era and resultant shadow economy of effort risk creating a triple harm by devaluing clinical excellence, burning out medical students, and potentially producing superficial or, worse, inauthentic academic and community work. Medical educators must optimize residency application and selection for cooperative outcomes and design incentives to ensure the outputs of medical education align student, institutional, patient, and societal goals. Without a set of predictive "answers," the authors suggest a process of determining actions to advance this ultimate aim and reduce harm.

Competency Based Medical Education (CBME) in India: Clinicians' Multifaceted Challenges

In the recent past, in order to achieve our national goal of 'health for all', National Medical Commission and Government of India made competency based medical education (CBME) mandatory in all Indian medical colleges. By undergoing CBME, the Indian Medical Graduate is expected to achieve the five roles of Clinician, Communicator, Leader and Member of the health care, Professional and Lifelong learner. This article is an overview of CBME, its relevant terminologies and knowledge-gaps. The purpose is to share these changes in medical education with practicing Otolaryngologists of India. It also discusses the various challenges regarding this already- suggested implementation process for clinical academicians, which could be counterproductive to the expected national aim and goal.

Defence Engagement (Health): a historical perspective

Defence Engagement (DE) (Health) themes are reviewed in this paper through two historical perspectives set in very different contexts. The first-person narrative in Guerrilla Surgeon by Lindsay Rogers outlines the experience of a medical officer building capacity with Tito's Partisans behind enemy lines in Second World War Yugoslavia. This is contrasted with a more academic evaluation of strategic and medical benefits of DE (Health) delivered by the US military during the Vietnam War in Robert Wilensky's Military Medicine to Win Hearts and Minds: Aid to Civilians in the Vietnam War Both texts infer that clear objectives, supported by effective strategic communication, are required for the impact of DE (Health) to be fully realised. Wilensky, in particular, noted that the US military medical effort in Vietnam had no measurable impact on health or political goals in the conflict. Rogers' experience on a more individual level speaks to the promise of DE (Health) delivery contrasted against the lack of regional objectives and cites the loss of British influence when Soviet propaganda was more cohesive and coordinated, resulting in the shift in Partisan loyalty despite British efforts in supplying the bulk of military and medical material. While neither author offers a definitive guide on DE (Health), they offer clear examples of themes that should be considered and demonstrate the importance of evaluating activity and maintaining the historical record to provide an evidence base for future work. This is an article commissioned for the Defence Engagement special issue of BMJ Military Health.

The Day the Electronic Medical Records System Went Down

This essay describes the author’s experience of improved patient care and collaboration the day the electronic medical records system went down.

The physician's role in reducing health disparities for persons with epilepsy and intellectual disability: "it's not just epilepsy…you really have to take a deeper dive."

Epilepsy affects approximately 25 % of people with intellectual disability (ID). Despite this high prevalence, evidence of health disparity exists in healthcare access and health outcomes for this population. Patients with ID experience additional challenges in accessing appropriate epilepsy care, and are at greater risk of experiencing inappropriate prescribing, polypharmacy and misdiagnosis compared with the general population. The expectations, attitudes and actions of physicians are key in addressing health inequalities, particularly those which disproportionately impact a specific group of patients, such as patients with ID and epilepsy. This qualitative study aimed to explore the views of specialist physicians as to why they believe this patient group are at a disadvantage when it comes to accessing appropriate epilepsy care, and how physicians can intervene to ensure that patients with ID are given equal access to suitable epilepsy care, and equal opportunity to achieve the best possible treatment outcomes. Semi-structured interviews were carried out with six physicians, located in six countries, who specialise in the care of persons with ID who have epilepsy. Interviews sought views on prognostic expectations, experiences of disparities in epilepsy care, and suggestions for advocacy interventions. Interviews were analysed using reflexive thematic analysis. Three core themes and nine subthemes were identified. Core themes included (1) 'Nervousness in care and treatment,' which reflected participants' descriptions of a nervousness by colleagues when treating epilepsy in patients with ID. (2) 'Taking a deeper dive' captured the harmful effects of accepting "common dogma," as well as the issue of a lack of clarity around treatment pathways for patients with epilepsy and ID. (3) 'Teach me' illustrated the importance of shared expertise, reflective practice and continued research and advocacy. Findings reflected participants' recommendations to address disparities in epilepsy care for patients with ID. These recommendations highlighted education and training, taking time to learn how to communicate in different ways, and regular reflection on personal assumptions and biases as important contributors to addressing inequalities in epilepsy care for patients with ID. It is hoped that findings will prompt those providing epilepsy care to reflect on their own practice and identify ways in which they might intervene to minimise inadvertent harm and reduce health disparities in epilepsy care for patients with ID.

Evaluation of Digitalisation in Healthcare and the Quantification of the "Unmeasurable"

Evaluating healthcare digitalisation, where technology implementation and adoption transforms existing socio-organisational processes, presents various challenges for outcome assessments. Populations are diverse, interventions are complex and evolving over time, meaningful comparisons are difficult as outcomes vary between settings, and outcomes take a long time to materialise and stabilise. Digitalisation may also have unanticipated impacts. We here discuss the limitations of evaluating the digitalisation of healthcare, and describe how qualitative and quantitative approaches can complement each other to facilitate investment and implementation decisions. In doing so, we argue how existing approaches have focused on measuring what is easily measurable and elevating poorly chosen values to inform investment decisions. Limited attention has been paid to understanding processes that are not easily measured even though these can have significant implications for contextual transferability, sustainability and scale-up of interventions. We use what is commonly known as the McNamara Fallacy to structure our discussions. We conclude with recommendations on how we envisage the development of mixed methods approaches going forward in order to address shortcomings.

Enhanced Stress Resilience Training for UK Surgical Trainees; Effect and Evolution Evaluated

INTRODUCTION

Core Surgical Training (CST) programs are associated with high burnout. This study aimed to assess the influence of Enhanced Stress Resilience Training (ESRT) over a 2-year period in a single UK Statutory Education Body.

METHOD

CSTs participated in 5-weeks of formal ESRT to address work stressors. The primary outcome measure was career progression related to curriculum metrics and National Training Number (NTN) appointment. Secondary measures related to burnout using validated psychological inventories.

RESULTS

Of 42 CSTs, 13 engaged fully with ESRT (31.0%; male 8, female 5, median age 28 year.), 11 engaged partially, and 18 did not. ESRT engagement was associated with better NTN appointment (ESRT 8/13 (61.5%) vs. not 1/18 (5.6%), p = 0.025), less burnout [aMBI; mean 5.14 (SD ± 2.35) vs. 3.14 (±2.25), F 6.637, p = 0.002, ηp2=0.167], less stress [PSS-10; 19.22 (±5.91) vs. 15.79 (±5.47), F 8.740, p < 0.001, ηp2=0.200], but more mindfulness [CAMS-R; 19.22 (±5.91) vs. 20.57 (±2.93), F 3.201, p = 0.047, ηp2=0.084]. On multivariable analysis, Improving Surgical Training (run-through CST) program (OR 5.2 (95% CI 1.42-28.41, p = 0.022), MRCS pass (OR 17.128 (95% CI 1.48-197.11, p = 0.023) and ESRT engagement (OR 13.249, 95% CI 2.08-84.58, p = 0.006) were independently associated with NTN success.

DISCUSSION

ESRT was associated with less stress and burnout, better mindfulness, and most importantly 13-fold better career progression.

Dead rats, dopamine, performance metrics, and peacock tails: Proxy failure is an inherent risk in goal-oriented systems

When a measure becomes a target, it ceases to be a good measure. For example, when standardized test scores in education become targets, teachers may start "teaching to the test," leading to breakdown of the relationship between the measure - test performance - and the underlying goal - quality education. Similar phenomena have been named and described across a broad range of contexts, such as economics, academia, machine learning, and ecology. Yet it remains unclear whether these phenomena bear only superficial similarities, or if they derive from some fundamental unifying mechanism. Here, we propose such a unifying mechanism, which we label proxy failure. We first review illustrative examples and their labels, such as the "cobra effect," "Goodhart's law," and "Campbell's law." Second, we identify central prerequisites and constraints of proxy failure, noting that it is often only a partial failure or divergence. We argue that whenever incentivization or selection is based on an imperfect proxy measure of the underlying goal, a pressure arises that tends to make the proxy a worse approximation of the goal. Third, we develop this perspective for three concrete contexts, namely neuroscience, economics, and ecology, highlighting similarities and differences. Fourth, we outline consequences of proxy failure, suggesting it is key to understanding the structure and evolution of goal-oriented systems. Our account draws on a broad range of disciplines, but we can only scratch the surface within each. We thus hope the present account elicits a collaborative enterprise, entailing both critical discussion as well as extensions in contexts we have missed.

The pursuit of fairness in assessment: Looking beyond the objective

Health professions education has undergone significant changes over the last few decades, including the rise of competency-based medical education, a shift to authentic workplace-based assessments, and increased emphasis on programmes of assessment. Despite these changes, there is still a commonly held assumption that objectivity always leads to and is the only way to achieve fairness in assessment. However, there are well-documented limitations to using objectivity as the 'gold standard' to which assessments are judged. Fairness, on the other hand, is a fundamental quality of assessment and a principle that almost no one contests. Taking a step back and changing perspectives to focus on fairness in assessment may help re-set a traditional objective approach and identify an equal role for subjective human judgement in assessment alongside objective methods. This paper explores fairness as a fundamental quality of assessments. This approach legitimises human judgement and shared subjectivity in assessment decisions alongside objective methods. Widening the answer to the question: 'What is fair assessment' to include not only objectivity but also expert human judgement and shared subjectivity can add significant value in ensuring learners are better equipped to be the health professionals required of the 21st century.

A win-win scenario? Restrictive policies from alternative standpoints

PURPOSE

In this viewpoint article, the authors consider the challenges in implementing restrictive policies, with a particular focus on how these policies are experienced, in practice, from alternative standpoints.

DESIGN/METHODOLOGY/APPROACH

The authors draw on social science studies of decommissioning work to highlight how patient and official versions of value often vary, creating difficulties and distrust as restrictive policies are implemented. Patients and the public are well aware that financial calculations are somehow embedded in concepts of "evidence" and "value" but are usually unfamiliar with the social infrastructures that produce and utilise such concepts. The authors discuss with reference to a contemporary restrictive programme in England.

FINDINGS

While policymakers and researchers frequently present restrictive policies as "win-win" scenarios (achieving both cost-savings for healthcare services and improved patient care), social science analyses highlight the potential for tensions and controversies between stakeholders. The authors recognise that cost containment is a necessary component of policymaking work but argue that policymakers and researchers should seek to map (and make visible) the socially organised reasoning, systems and processes that are involved in enacting restrictive policies. Although transparency may pose challenges, it is important for informed democratic engagement, allowing legitimate scrutiny of whose voices are being heard and interests served (the "winners" and "losers").

ORIGINALITY/VALUE

The authors argue for social science analyses that explore overuse, value and restrictive practices from alternative (e.g. patient) standpoints. These can provide important insights to help identify priorities for intervention and support better communication.

Diversity, Complexity and Ordinality: Mental Health Services Outside the Institutions-Service Users' and Professionals' Experience-Based Practices and Knowledges, and New Public Management

In conjunction with the dismantling of psychiatric hospitals, social workers have been commissioned to help service users in their daily living in their homes and in the community. The consequences of these changes for experience-based knowledge and practices in their contexts remain relatively unknown. In this study, eighteen service users and the social workers they described as helpful for them were interviewed. The interviews were recorded, transcribed, and analyzed using Thematic Analysis. The following themes emerged: “Here, there and everywhere”, “Doing, being, becoming”, “Talking” and “Order, planning and improvisation” concerning the contradictions service users and professionals mentioned about their practices and the conditions imposed by managerial methods connected to New Public Management. Finally, “Spontaneous planned complexity” was chosen as our overarching theme to characterize the new knowledge and practices which have been developed. The displacement of the place for the encounter and the introduction of non-medicalized professions have allowed community-based practices and thus the co-creation and emergence of new knowledge about the service users as persons and the professionals as qualified professionals. The challenge remains for managers to have trust in their colleagues and not impose rigid rules, schematized methods, and repeated controls.

Correlation does not imply geomorphic causation in data-driven landslide susceptibility modelling - Benefits of exploring landslide data collection effects

Data-driven landslide susceptibility models formally integrate spatial landslide information with explanatory environmental variables that describe predisposing factors of slope instability. Well-performing models are commonly utilized to identify landslide-prone terrain or to understand the causes of slope instability. In most cases, however, the available landslide data is affected by spatial biases (e.g. underrepresentation of landslides far from infrastructure or in forests) and does therefore not perfectly represent the spatial distribution of past slope instabilities. Literature shows that implications of such data flaws are frequently ignored. This study was built upon landslide information that systematically relates to damage-causing and infrastructure-threatening events in South Tyrol, Italy (7400 km²). The created models represent three conceptually different strategies to deal with biased landslide information. The aims were to demonstrate why an inference of geomorphic causation from apparently well-performing models is invalid under common landslide data bias conditions (Model 1), to test a novel bias-adjustment approach (Model 2) and to exploit the underlying data bias to model areas likely affected by potentially damaging landslides (Model 3; intervention index), instead of landslide susceptibility. The study offers a novel perspective on how biases in landslide data can be considered within data-driven models by focusing not only on the process under investigation (landsliding), but also on the circumstances that led to the registration of landslide information (data collection effects). The results were evaluated in terms of statistical relationships, variable importance, predictive performance, and geomorphic plausibility. The results revealed that none of the models reflected landslide susceptibility. Despite partly high predictive performances, the models were unable to create geomorphically plausible spatial predictions. The impact-oriented intervention index, however, enabled to identify damage-causing landslides with high accuracy. We conclude that the frequent practice of inferring geomorphic causation from well-performing models without accounting for data limitations is invalid.

Statistical, Clinical, Methodological Evaluation of Local Recurrence Following Transanal Total Mesorectal Excision for Rectal Cancer: A Systematic Review

BACKGROUND

A recent Norwegian moratorium challenged the status quo of transanal total mesorectal excision for rectal cancer by reporting increased early multifocal local recurrences.

OBJECTIVE

The aim of this systematic review and meta-analysis was to evaluate the local recurrence rates following transanal total mesorectal excision as well as to assess statistical, clinical, and methodological bias in reports published to date.

DATA SOURCES

The PubMed and MEDLINE (via Ovid) databases were systematically searched.

STUDY SELECTION

Descriptive or comparative studies reporting rates of local recurrence at a median follow-up of 6 months (or more) after transanal total mesorectal excision were included.

INTERVENTIONS

Patients underwent transanal total mesorectal excision.

MAIN OUTCOME MEASURES

Local recurrence was any recurrence located in the pelvic surgery site. The untransformed proportion method of 1-arm meta-analysis was utilized. Untransformed percent proportion with 95% confidence interval was reported. Ad hoc meta-regression with the Omnibus test was utilized to assess risk factors for local recurrence. Among-study heterogeneity was evaluated: statistically by I2 and τ2, clinically by summary tables, and methodologically by a 33-item questionnaire.

RESULTS

Twenty-nine studies totaling 2906 patients were included. The pooled rate of local recurrence was 3.4% (2.7%-4.0%) at an average of 20.1 months with low statistical heterogeneity (I2 = 0%). Meta-regression yielded no correlation between complete total mesorectal excision quality (p = 0.855), circumferential resection margin (p = 0.268), distal margin (p = 0.886), and local recurrence rates. Clinical heterogeneity was substantial. Methodological heterogeneity was linked to the excitement of novelty, loss aversion, reactivity to criticism, indication for transanal total mesorectal excision, nonprobability sampling, circular reasoning, misclassification, inadequate follow-up, reporting bias, conflict of interest, and self-licensing.

LIMITATIONS

The studies included had an observational design and limited sample and follow-up.

CONCLUSION

This systematic review found a pooled rate of local recurrence of 3.4% at 20 months. However, given the substantial clinical and methodological heterogeneity across the studies, the evidence for or against transanal total mesorectal excision is inconclusive at this time.

Future Treatment of Sleep Disorders: Syndromic Approach Versus Management of Treatable Traits?

Sleep disorders are categorized in line with traditional taxonomy. This conventional approach allows adequate management of many patients. Failure of treatment, however, may be due to nonspecificity of symptoms, coincidental association between symptoms and pathophysiological endotype, as well as co-occurrence of different pathologic mechanisms affecting sleep. Complex phenotypes often do not respond well to standard therapeutic interventions. In these cases, the clinical workup should aim at identifying treatable traits that will likely improve under targeted therapy. The challenge for sleep medicine is to further develop this innovative approach that is driven by the principles of systems medicine.

Towards a rigorous understanding of societal responses to climate change

A large scholarship currently holds that before the onset of anthropogenic global warming, natural climatic changes long provoked subsistence crises and, occasionally, civilizational collapses among human societies. This scholarship, which we term the 'history of climate and society' (HCS), is pursued by researchers from a wide range of disciplines, including archaeologists, economists, geneticists, geographers, historians, linguists and palaeoclimatologists. We argue that, despite the wide interest in HCS, the field suffers from numerous biases, and often does not account for the local effects and spatiotemporal heterogeneity of past climate changes or the challenges of interpreting historical sources. Here we propose an interdisciplinary framework for uncovering climate-society interactions that emphasizes the mechanics by which climate change has influenced human history, and the uncertainties inherent in discerning that influence across different spatiotemporal scales. Although we acknowledge that climate change has sometimes had destructive effects on past societies, the application of our framework to numerous case studies uncovers five pathways by which populations survived-and often thrived-in the face of climatic pressures.

Complex systems, part I: why 42 is rarely, if ever, the ultimate answer

We describe the different categories of systems and systems thinking, and illustrate why almost all clinical interactions constitute simultaneously complex and complicated systems, so-called "wicked systems". We also discuss why they are not amenable to quantitative analysis. With the use of comparisons and illustrations we show some of the problems that reductionist metrics create, and support concerns regarding quantitative fallacy. The systematic correlation of data in medicine was one of the earliest achievements of the discipline. Recently, however, the overwhelming bias towards an evidence base, which relies almost entirely on randomised controlled trials, has created a reductionist view that often excludes important aspects of medicine and, in particular, surgery. We must now move away from thinking that is controlled by the "tyranny of metrics" to embrace complex-systems thinking, and work across disciplines. We outline the arguments for this and give clinical examples from oral and maxillofacial surgery.

Nothing matters: the significance of the unidentifiable, the superficial and nonsense

Purpose: The aim of this study is to explore the ways in which “small things” may be of importance for people with mental health difficulties.

Method: Empirical material from three different studies was reanalysed through a phenomenological, dialogical, approach.

Results: We discovered some paradoxical aspects of small things: i.e., they could be about “something” that was difficult or even impossible to identify. The unidentifiable could be about bodily, sensual experiences that are superficial (i.e., belonging to the surface). The interaction with others highlighted as significant could be about doing something fun, talking nonsense or kidding around, and hence not at all about making sense of something—a kind of important nonsense. We summarize these aspects in three themes: the importance of the unidentifiable, the superficial and nonsense. These aspects can be regarded as small things—even “nothings”—that make it possible “to stay in the world”.

Conclusion: We elaborate on the findings in relation to the following: Gumbrecht’s critique of the prevailing hermeneutic world-view with its idea that “interpretation is humankind’s exclusive way of relating to the world”, Ingold’s idea that social life is lived in relations of “interfacility” and hence a turn to surfaces is needed for a “restoration of social life”, and Biesta’s idea of existence as “coming into the world in the presence of others”.

Digital Diabetes Data and Artificial Intelligence: A Time for Humility Not Hubris

In the future artificial intelligence (AI) will have the potential to improve outcomes diabetes care. With the creation of new sensors for physiological monitoring sensors and the introduction of smart insulin pens, novel data relationships based on personal phenotypic and genotypic information will lead to selections of tailored, effective therapies that will transform health care. However, decision-making processes based exclusively on quantitative metrics that ignore qualitative factors could create a quantitative fallacy. Difficult to quantify inputs into AI-based therapeutic decision-making processes include empathy, compassion, experience, and unconscious bias. Failure to consider these "softer" variables could lead to important errors. In other words, that which is not quantified about human health and behavior is still part of the calculus for determining therapeutic interventions.