Actor-network theory and the OSCE: formulating a new research agenda for a post-psychometric era

Revised paper ASIM-D-21-00055R1: "Consulting properly rather than acting": advocating for real patient involvement in summative OSCEs

Background

In this “Advancing simulation practice” article, we offer an expose of the involvement of real patients in Objective Structured Clinical Examinations (OSCEs), inviting educators who traditionally involve solely SPs in their summative OSCEs to consider the practice. The need for standardisation in summative assessments can make educators understandably wary to try this, even if the rhetoric to involve real patients is accepted. We offer this as an instance of the tussle between standardisation and validity experienced throughout health professions education.

Main text

We offer our experience and empirical evidence of this simulation practice, based on an institutional ethnographic examination of the involvement of real patients in summative OSCEs from an undergraduate medical school in the UK. Our critique demonstrates the merits of this approach as an assessment environment closer to the real clinical environments where these soon-to-be doctors interact in a more authentic way with real patients and their illness experiences. We balance this against the extra work required for all involved and suggest the biggest challenge is in the reorientation work required for both Faculty and students who are institutionalised to expect standardisation above all in assessment.

Conclusion

We advocate for involving real patients in summative OSCEs and hope that readers may feel compelled and empowered to foster this shift in mindset required to introduce this practice into their assessments.

Materials matter: Understanding the importance of sociomaterial assemblages for OSCE candidate performance

INTRODUCTION

The OSCE is a sociomaterial assemblage-a meshing together of human and material components producing multiple effects. Materials matter because they shape candidate performance, with potentially calamitous career consequences if materials influence performance unjustly. Although the OSCE literature refers to materials, few papers study the sociomateriality of OSCEs. Therefore, we explored OSCE stakeholders' talk about sociomaterial assemblages to better understand their importance for candidate performance.

METHODS

We conducted 15 focus groups with OSCE candidates (n = 42), examiners (n = 20) and simulated patients (n = 17) after an Australian postgraduate nursing OSCE. Sociomateriality informed our team-based framework analysis of data.

RESULTS

Participants identified a multiplicity of OSCE materials (objects, technologies and spaces) thought to matter for candidate performance. Candidates' unfamiliarity with materials and missing or malfunctioning materials were reported to yield numerous negative impacts (eg cognitive overload, negative affect, time-wasting), thereby adversely affecting candidate performance. Both examiners and candidates made micro-adjustments to sociomaterial assemblages during the OSCE in order to make it work (eg candidates saying what they would do rather than doing it). Sometimes, such tinkering extended so far that sociomaterial assemblages were ruptured (eg examiners ignoring rubrics to help pass candidates), potentially influencing OSCE standardisation.

DISCUSSION

Our novel empirical study extends previous conceptual work by illustrating wide-ranging sociomaterial assemblages influencing OSCE candidate performance. Further research is now needed employing sociomaterial approaches to further elucidate sociomaterial entanglements in diverse OSCEs. We encourage OSCE stakeholders to become more attuned to the productive nature of materials within all stages of OSCE design and implementation.

The hidden labours of designing the Objective Structured Clinical Examination: a Practice Theory study

Objective Structured Clinical Examinations (OSCEs) have become ubiquitous as a form of assessment in medical education but involve substantial resource demands and considerable local variation. A detailed understanding of the processes by which OSCEs are designed and administered could improve feasibility and sustainability. This exploration of OSCE design is informed by Practice Theory, which suggests assessment design processes are dynamic, social and situated activities. The overall purpose is to provide insights that inform on-the-ground OSCE administration. Fifteen interviews were conducted with OSCE academics and administrators from three medical schools in Australia, the United Kingdom and Canada. Drawing from post-qualitative inquiry, Schatzki's Practice Theory was used both as a sensibility and as an analytic framework. OSCE design was characterised by planning activities, administration activities, negotiation activities and bureaucratic activities; it involves significant and resource-intensive effort in negotiation and coordination. There was considerable local variation but at the same time activities were remarkably consonant across national boundaries. There was a tension between general understandings such as reliability and validity that underpin the OSCE and the improvisational practices associated with design and administration. Our findings highlighted the role of blueprints as a key coordinating artefact but with too many rules and procedures prompting cycles of bureaucracy and complexity. Emphasising coordination rather than standardisation might ease workloads, support adaptation to local environments and prevent an overly reductive approach to this assessment format.

Learning environments for interprofessional education: A micro-ethnography of sociomaterial assemblages in team-based learning

BACKGROUND

Teamwork and collaboration are central to interprofessional education but fostering these attributes in large undergraduate cohorts is challenging.

OBJECTIVES

This study aimed to examine the complexities of IPE group learning processes by examining how the material and intersubjective intertwine when newly formed interprofessional groups (Chinese medicine, medicine, nursing, pharmacy, and social work) synchronously engaged with face-to-face and online learning in a blended, team-based learning environment.

METHODS

It was a micro-ethnography study using a sociomaterial theoretical lens. We selected two undergraduate interprofessional healthcare student groups within a large scale programme for contrastive video analysis of synchronous spatial and physical configurations, associated talk, and online activity.

RESULTS

Video analysis of evolving physical configurations indicated that Group B was spatially more evenly grouped, and physically orientated to an identifiable leader, despite their blinded peer evaluations indicating distributed leadership. Group A faced a critical event at a public forum leading to spatial disruption breaking into subgroups and isolates; however, this group identified one member as a defined leader in the peer evaluations.

CONCLUSIONS

Based on online scores, we found that peer identification of leaders may influence learning processes but not learning outcomes in the first IPE team meeting. The design of the physical and virtual learning environments contributed to the developing, sociomaterial processes of group cohesion in interprofessional team-based learning.

Thinking Sociomaterially: Why Matter Matters in Medical Education

Sociomaterial perspectives in research are those that encourage researchers to focus their inquiry on the relationships between people (social) and things (material), rather than focusing solely on people. The unique possibility of sociomaterial perspectives is increasingly recognized in health professions education scholarship. In an effort to support those who may be interested in engaging with the principles of sociomaterialism in their own work, the authors have developed a primer on this paradigm of research. This Invited Commentary-one of several exploring different philosophies of science-offers an overview of the ontological, epistemological, axiological, and methodological foundations of sociomateriality. The authors then put these ideas into action, highlighting the philosophical foundations of sociomaterial perspectives in a sample case study that tells the story of Lee, a resident involved in a medical error.

Approaching culture in medical education: Three perspectives

BACKGROUND

The notion of culture is increasingly invoked in the medical education literature as a key influence on how educational strategies unfold, and culture change is frequently identified as a necessary precursor to progress. A meaningful perspective on what culture means is often missing from these discussions, however. Without a theoretically grounded notion of culture, calls for culture change are challenging to interpret and to act upon.

OBJECTIVE

In this cross-cutting edge paper, we explore how culture has been defined and theorised using three lenses: the organisational perspective; the identity perspective, and the practice perspective. We consider what each perspective might offer to medical education researchers.

RESULTS

Each of these perspectives draws on a range of disciplinary influences, and none represents a singular theory of culture. Broadly, the organisational perspective directs our attention to the shared assumptions and values that bind individuals within an organisation. It tends to view culture through a strategic lens; culture may be either a barrier to or a facilitator of the changes that are inevitably required of an organisation if it is to maintain its relevance. The identity perspective, particularly the notion of figured worlds, alerts us to the power of communal narratives to shape how individuals see themselves within particular cultural worlds. The practice perspective emphasises what actually occurs in practice, avoiding symbolic ideas about culture and shared values and instead privileging activity and human-material networks or arrangements.

CONCLUSIONS

These diverse perspectives share a common thread- they shift our research gaze beyond the individual, allowing us instead to see how those individuals form organisations, inhabit cultural worlds and constitute practices. They afford substance and direction for explorations of culture, and thus offer the promise of a more nuanced understanding of some of medical education's most challenging problems.

Actor-network theory and ethnography: Sociomaterial approaches to researching medical education

Medical education is a messy tangle of social and material elements. These material entities include tools, like curriculum guides, stethoscopes, cell phones, accreditation standards, and mannequins; natural elements, like weather systems, disease vectors, and human bodies; and, objects, like checklists, internet connections, classrooms, lights, chairs and an endless array of others.We propose that sociomaterial approaches to ethnography can help us explore taken for granted, or under-theorized, elements of a situation under study, thereby enabling us to think differently. In this article, we describe ideas informing Actor-Network Theory approaches, and how these ideas translate into how ethnographic research is designed and conducted. We investigate epistemological (what we can know, and how) positioning of the researcher in an actor-network theory informed ethnography, and describe how we tailor ethnographic methods-document and artefact analysis; observation; and interviews-to align with a sociomaterial worldview.Untangling sociomaterial scenarios can offer a novel perspective on myriad contemporary medical education issues. These issues include examining how novel tools (e.g. accreditation standards, assessment tools, mannequins, videoconferencing technologies) and spaces (e.g. simulation suites, videoconferenced lecture theatres) used in medical education impact how teaching and learning actually happen in these settings.

Examining the clinical learning environment through the architectural avenue

Medical education has traditionally focused on the learners, the educators, and the curriculum, while tending to overlook the role of the designed environment. Experience indicates, however, that processes and outcomes of medical education are sensitive to the qualities and disposition of the spaces in which it occurs. This includes the clinical education within the patient care environment, termed the clinical learning environment (CLE). Recognition of this has informed the design of some new clinical learning spaces for the past decade. Competency-based clinical education can drive design requirements that differ materially from those associated with general purpose educational or clinical spaces. In this article, we outline two conceptual frameworks: (i) materialist spatiality and (ii) actor-network theory and consider how they can guide the design of spaces to support competency-based medical education and to guide the evaluation and discussion of the educational impacts of the spaces once built. We illustrate the use of these frameworks through discussion of the educational ambitions that underpinned the design of some recent clinical educational spaces. We close with practical points for consideration by educators and designers.