How Do Thresholds of Principle and Preference Influence Surgeon Assessments of Learner Performance?

When words are your scalpel, what and how information is exchanged may be differently salient to assessors

PURPOSE

Variable assessments of learner performances can occur when different assessors determine different elements to be differently important or salient. How assessors determine the importance of performance elements has historically been thought to occur idiosyncratically and thus be amenable to assessor training interventions. More recently, a main source of variation found among assessors was two underlying factors that were differently emphasised: medical expertise and interpersonal skills. This gave legitimacy to the theory that different interpretations of the same performance may represent multiple truths. A faculty development activity introducing assessors to entrustable professional activities in which they estimated a learner's level of readiness for entrustment provided an opportunity to qualitatively explore assessor variation in the context of an interaction and in a setting in which interpersonal skills are highly valued.

METHODS

Using a constructivist grounded theory approach, we explored variation in assessment processes among a group of palliative medicine assessors who completed a simulated direct observation and assessment of the same learner interaction.

RESULTS

Despite identifying similar learner strengths and areas for improvement, the estimated level of readiness for entrustment varied substantially among assessors. Those who estimated the learner as not yet ready for entrustment seemed to prioritise what information was exchanged and viewed missed information as performance gaps. Those who estimated the learner as ready for entrustment seemed to prioritise how information was exchanged and viewed the same missed information as personal style differences or appropriate clinical judgement. When presented with a summary, assessors expressed surprise and concern about the variation.

CONCLUSION

A main source of variation among our assessors was the differential salience of performance elements that align with medical expertise and interpersonal skills. These data support the theory that when assessing an interaction, differential salience for these two factors may be an important and perhaps inevitable source of assessor variation.

Medical competence as a multilayered construct

BACKGROUND

The conceptualisation of medical competence is central to its use in competency-based medical education. Calls for 'fixed standards' with 'flexible pathways', recommended in recent reports, require competence to be well defined. Making competence explicit and measurable has, however, been difficult, in part due to a tension between the need for standardisation and the acknowledgment that medical professionals must also be valued as unique individuals. To address these conflicting demands, a multilayered conceptualisation of competence is proposed, with implications for the definition of standards and approaches to assessment.

THE MODEL

Three layers are elaborated. This first is a core layer of canonical knowledge and skill, 'that, which every professional should possess', independent of the context of practice. The second layer is context-dependent knowledge, skill, and attitude, visible through practice in health care. The third layer of personalised competence includes personal skills, interests, habits and convictions, integrated with one's personality. This layer, discussed with reference to Vygotsky's concept of Perezhivanie, cognitive load theory, self-determination theory and Maslow's 'self-actualisation', may be regarded as the art of medicine. We propose that fully matured professional competence requires all three layers, but that the assessment of each layer is different.

IMPLICATIONS

The assessment of canonical knowledge and skills (Layer 1) can be approached with classical psychometric conditions, that is, similar tests, circumstances and criteria for all. Context-dependent medical competence (Layer 2) must be assessed differently, because conditions of assessment across candidates cannot be standardised. Here, multiple sources of information must be merged and intersubjective expert agreement should ground decisions about progression and level of clinical autonomy of trainees. Competence as the art of medicine (Layer 3) cannot be standardised and should not be assessed with the purpose of permission to practice. The pursuit of personal excellence in this level, however, can be recognised and rewarded.

Will Any Road Get You There? Examining Warranted and Unwarranted Variation in Medical Education

Undergraduate and graduate medical education have long embraced uniqueness and variability in curricular and assessment approaches. Some of this variability is justified (warranted or necessary variation), but a substantial portion represents unwarranted variation. A primary tenet of outcomes-based medical education is ensuring that all learners acquire essential competencies to be publicly accountable to meet societal needs. Unwarranted variation in curricular and assessment practices contributes to suboptimal and variable educational outcomes and, by extension, risks graduates delivering suboptimal health care quality. Medical education can use lessons from the decades of study on unwarranted variation in health care as part of efforts to continuously improve the quality of training programs. To accomplish this, medical educators will first need to recognize the difference between warranted and unwarranted variation in both clinical care and educational practices. Addressing unwarranted variation will require cooperation and collaboration between multiple levels of the health care and educational systems using a quality improvement mindset. These efforts at improvement should acknowledge that some aspects of variability are not scientifically informed and do not support desired outcomes or societal needs. This perspective examines the correlates of unwarranted variation of clinical care in medical education and the need to address the interdependency of unwarranted variation occurring between clinical and educational practices. The authors explore the challenges of variation across multiple levels: community, institution, program, and individual faculty members. The article concludes with recommendations to improve medical education by embracing the principles of continuous quality improvement to reduce the harmful effect of unwarranted variation.

The Interactive Relationship Between Instructor Perceptions and Learner Personality on Surgical Skills Performance

OBJECTIVE

The objective of this study was to examine the association between learner personality and capacity to be trained (i.e., performance improvement) on a surgical task, and how instructor perceptions of the learners' capacity to be trained interact with learner personality and performance during training and feedback. There is meaningful heterogeneity in the degree of learner surgical skills acquisition despite receiving the same amount of training. While learner personality may independently contribute to skill acquisition, the instructor-learner feedback process is also important to consider. To better understand this interpersonal relationship, it is necessary to also consider instructor factors (i.e., perceptions), and how this may contribute to learner variability in skills training.

DESIGN

This exploratory study employed a prospective two-phase design. Medical and non-medical undergraduate students (N = 62) completed measures of personality and participated in two 20-minute training sessions with expert feedback 2 weeks apart, performing an end-to-side anastomosis on a low-fidelity model. Learner performance and instructors' perceptions of a learner's capacity to be trained were assessed.

PARTICIPANTS

Sixty-two medical and non-medical undergraduate students.

RESULTS

There was a significant interaction between learner Extraversion and instructor's perceptions of learner capacity to be trained. Higher learner Extraversion was only associated with an increase in performance improvement for those who were considered trainable (OR = 4.83, p = 0.017). Post hoc analysis revealed a significant difference in the amount of feedback provided to participants who were considered trainable (M = 9.45) versus not trainable (M = 16.48).

CONCLUSIONS

This study highlights the importance of both individual learner factors and instructor perceptions on surgical skill acquisition.

The pursuit of fairness in assessment: Looking beyond the objective

Health professions education has undergone significant changes over the last few decades, including the rise of competency-based medical education, a shift to authentic workplace-based assessments, and increased emphasis on programmes of assessment. Despite these changes, there is still a commonly held assumption that objectivity always leads to and is the only way to achieve fairness in assessment. However, there are well-documented limitations to using objectivity as the 'gold standard' to which assessments are judged. Fairness, on the other hand, is a fundamental quality of assessment and a principle that almost no one contests. Taking a step back and changing perspectives to focus on fairness in assessment may help re-set a traditional objective approach and identify an equal role for subjective human judgement in assessment alongside objective methods. This paper explores fairness as a fundamental quality of assessments. This approach legitimises human judgement and shared subjectivity in assessment decisions alongside objective methods. Widening the answer to the question: 'What is fair assessment' to include not only objectivity but also expert human judgement and shared subjectivity can add significant value in ensuring learners are better equipped to be the health professionals required of the 21st century.

The Supervisor's Toolkit: Strategies of Supervisors to Entrust and Regulate Autonomy of Residents in the Operating Room

OBJECTIVE

To identify what strategies supervisors use to entrust autonomy during surgical procedures and to clarify the consequences of each strategy for a resident's level of autonomy.

BACKGROUND

Entrusting autonomy is at the core of teaching and learning surgical procedures. The better the level of autonomy matches the learning needs of residents, the steeper their learning curves. However, entrusting too much autonomy endangers patient outcome, while entrusting too little autonomy results in expertise gaps at the end of training. Understanding how supervisors regulate autonomy during surgical procedures is essential to improve intraoperative learning without compromising patient outcome.

METHODS

In an observational study, all the verbal and nonverbal interactions of 6 different supervisors and residents were captured by cameras. Using the iterative inductive process of conversational analysis, each supervisor initiative to guide the resident was identified, categorized, and analyzed to determine how supervisors affect autonomy of residents.

RESULTS

In the end, all the 475 behaviors of supervisors to regulate autonomy in this study could be classified into 4 categories and nine strategies: I) Evaluate the progress of the procedure: inspection (1), request for information (2), and expressing their expert opinion (3); II) Influence decision-making: explore (4), suggest (5), or declare the next decision (6); III) Influence the manual ongoing action: adjust (7), or stop the resident's manual activity (8); IV) take over (9).

CONCLUSIONS

This study provides new insights into how supervisors regulate autonomy in the operating room. This insight is useful toward analyzing whether supervisors meet learning needs of residents as effectively as possible.

Implementation and Use of Workplace-Based Assessment in Clinical Learning Environments: A Scoping Review

PURPOSE

Workplace-based assessment (WBA) serves a critical role in supporting competency-based medical education (CBME) by providing assessment data to inform competency decisions and support learning. Many WBA systems have been developed, but little is known about how to effectively implement WBA. Filling this gap is important for creating suitable and beneficial assessment processes that support large-scale use of CBME. As a step toward filling this gap, the authors describe what is known about WBA implementation and use to identify knowledge gaps and future directions.

METHOD

The authors used Arksey and O'Malley's 6-stage scoping review framework to conduct the review, including: (1) identifying the research question; (2) identifying relevant studies; (3) study selection; (4) charting the data; (5) collating, summarizing, and reporting the results; and (6) consulting with relevant stakeholders.

RESULTS

In 2019-2020, the authors searched and screened 726 papers for eligibility using defined inclusion and exclusion criteria. One hundred sixty-three met inclusion criteria. The authors identified 5 themes in their analysis: (1) Many WBA tools and programs have been implemented, and barriers are common across fields and specialties; (2) Theoretical perspectives emphasize the need for data-driven implementation strategies; (3) User perceptions of WBA vary and are often dependent on implementation factors; (4) Technology solutions could provide useful tools to support WBA; and (5) Many areas of future research and innovation remain.

CONCLUSIONS

Knowledge of WBA as an implemented practice to support CBME remains constrained. To remove these constraints, future research should aim to generate generalizable knowledge on WBA implementation and use, address implementation factors, and investigate remaining knowledge gaps.

Fingerprints of Teaching Interactions: Capturing and Quantifying How Supervisor Regulate Autonomy of Residents in the Operating Room

OBJECTIVE

Supervisors and residents agree that entrusted autonomy is central to learning in the Operating Room (OR), but supervisors and residents hold different opinions about entrustment: residents regularly experience that they receive insufficient autonomy while supervisors feel their guiding is not appreciated as teaching. These opinions are commonly grounded on general experiences and perceptions, instead of real-time supervisors' regulatory behaviors as procedures unfold. To close that gap, we captured and analyzed when and to what level supervisors award or restrain autonomy during procedures. Furthermore, we constructed fingerprints, an instrument to visualize entrustment of autonomy by supervisors in the OR that allows us to reflect on regulation of autonomy and discuss teaching interactions.

DESIGN

All interactions between supervisors and residents were captured by video and transcribed. Subsequently a multistage analysis was performed: (1) the procedure was broken down into 10 steps, (2) for each step, type and frequency of strategies by supervisors to regulate autonomy were scored, (3) the scores for each step were plotted into fingerprints, and (4) fingerprints were analyzed and compared.

SETTING

University Medical Centre Groningen (the Netherlands).

PARTICIPANTS

Six different supervisor-resident dyads.

RESULTS

No fingerprint was alike: timing, frequency, and type of strategy that supervisors used to regulate autonomy varied within and between procedures. Comparing fingerprints revealed that supervisors B and D displayed more overall control over their program-year 5 residents than supervisors C and E over their program-year 4 residents. Furthermore, each supervisor restrained autonomy during steps 4 to 6 but with different intensities.

CONCLUSIONS

Fingerprints show a high definition view on the unique dynamics of real-time autonomy regulation in the OR. One fingerprint functions as a snapshot and serves a purpose in one-off teaching and learning. Multiple snapshots of one resident quantify autonomy development over time, while multiple snapshots of supervisors may capture best teaching practices to feed train-the-trainer programs.

Scut to Scholarship: Can Operative Notes be Educationally Useful?

OBJECTIVE

Efforts to implement competency-based medical education require new sources of workplace-based evidence of growth in learning. We used qualitative analysis of operative notes to explore procedural variation in a simple surgical procedure.

DESIGN

We used a grounded theory-based mixed methods approach to depict intersurgeon procedural variation. Our grounded theory approach to analysis included follow up interviews with surgeons and residents to probe their understandings of the reasons for variation in the dictated notes and the current and potential utility of operative notes as a reliable source of data for learning and assessment.

SETTING

Publicly funded tertiary care otolaryngology-head & neck surgery residency program in Ontario, Canada PARTICIPANTS: Using maximum variability sampling, all surgeons performing tonsillectomy in the department (n = 6) contributed operative notes from 65 tonsillectomies, 5 intraoperative observations, and 4 semi-structured interviews. An additional 3 residents from various levels of training contributed semistructured interviews.

RESULTS

Intersurgeon procedural variations persist even in simple surgical procedures such as tonsillectomy. Operative notes appear to capture procedural variations in a limited way. Surgeons and resident make informal educational use of the clerical work of writing and assessing operative notes, but optimization will be required to shift such hidden work into the formal educational domain.

CONCLUSIONS

The implementation of competency-based medical education requires surgical educators to both eliminate low-yield tasks for learning and to find new opportunities for multiple low-stakes assessment. Analysis of operative notes may become a high-yield strategy for learning and assessment if residents and surgeons are coached to use operative notes more reliably and efficiently.

Residents Think in the "Now" and Supervisors Think Ahead in the Operating Room. A Survey Study About Task Perception of Residents and Supervising Surgeons

OBJECTIVE

Progressive autonomous task performance is the cornerstone of teaching residents in the operating room, where they are entrusted with autonomy when they meet their supervisors' preferences. To optimize the teaching, supervisors need to be aware of how residents experience parts of the procedure. This study provides insight into how supervisors and residents perceive different tasks of a single surgical procedure.

DESIGN

In this qualitative survey study a cognitive task analysis (CTA) of supervisors and residents for the 47 tasks of an uncemented total hip arthroplasty was executed. Both groups rated the level of attention they would assign to each task and were asked to explain attention scores of 4 or 5.

SETTING

University Medical Centre Groningen (the Netherlands) and its 5 affiliated teaching hospitals.

PARTICIPANTS

Seventeen supervising surgeons and 21 residents.

RESULTS

Normal attention (median attention score 3) was assigned by supervisors to 34 tasks (72.3%) and by residents to 35 tasks (74.5 %). Supervisors rated 12 tasks (25.6%) and residents 9 tasks (19.1%) with a median attention score of 4. In general, supervisors associated high attention with patient outcome and prevention of complications, while residents associated high attention with "effort."

CONCLUSIONS

Supervisors and residents assigned attention to tasks for different reasons. Supervisors think ahead and emphasize patient outcome and prevention of complications when they indicate high attention, while residents think in the "now" and raise attention to execute the tasks themselves. The results of this study allow residents and supervisors to anticipate preferences: residents are able to appreciate why supervisors increase attention to specific tasks, and supervisors obtain information on which tasks require individual guidance of residents. This information can contribute to improve the learning climate in the operating room and task-specific procedural training.

"Languaging" tacit judgment in formal postgraduate assessment: the documentation of ad hoc and summative entrustment decisions

While subjective judgment is recognized by the health professions education literature as important to assessment, it remains difficult to carve out a formally recognized role in assessment practices for personal experiences, gestalts, and gut feelings. Assessment tends to rely on documentary artefacts-like the forms, standards, and policies brought in under competency-based medical education, for example-to support accountability and fairness. But judgment is often tacit in nature and can be more challenging to surface in explicit (and particularly written) form. What is needed is a nuanced approach to the incorporation of judgment in assessment such that it is neither in danger of being suppressed by an overly rigorous insistence on documentation nor uncritically sanctioned by the defense that it resides in a black box and that we must simply trust the expertise of assessors. The concept of entrustment represents an attempt to effect such a balance within current competency frameworks by surfacing judgments about the degree of supervision learners need to care safely for patients. While there is relatively little published data about its implementation as yet, one readily manifest variation in the uptake of entrustment relates to the distinction between ad hoc and summative forms. The ways in which these forms are languaged, together with their intended purposes and guidelines for their use, point to directions for more focused empirical inquiry that can inform current and future uptake of entrustment in competency-based medical education and the responsible and meaningful inclusion of judgment in assessment more generally.

Use of Resident-Sensitive Quality Measure Data in Entrustment Decision Making: A Qualitative Study of Clinical Competency Committee Members at One Pediatric Residency

PURPOSE

Resident-sensitive quality measures (RSQMs) are quality measures that are likely performed by an individual resident and are important to care quality for a given illness of interest. This study sought to explore how individual clinical competency committee (CCC) members interpret, use, and prioritize RSQMs alongside traditional assessment data when making a summative entrustment decision.

METHOD

In this constructivist grounded theory study, 19 members of the pediatric residency CCC at Cincinnati Children's Hospital Medical Center were purposively and theoretically sampled between February and July 2019. Participants were provided a deidentified resident assessment portfolio with traditional assessment data (milestone and/or entrustable professional activity ratings as well as narrative comments from 5 rotations) and RSQM performance data for 3 acute, common diagnoses in the pediatric emergency department (asthma, bronchiolitis, and closed head injury) from the emergency medicine rotation. Data collection consisted of 2 phases: (1) observation and think out loud while participants reviewed the portfolio and (2) semistructured interviews to probe participants' reviews. Analysis moved from close readings to coding and theme development, followed by the creation of a model illustrating theme interaction. Data collection and analysis were iterative.

RESULTS

Five dimensions for how participants interpret, use, and prioritize RSQMs were identified: (1) ability to orient to RSQMs: confusing to self-explanatory, (2) propensity to use RSQMs: reluctant to enthusiastic, (3) RSQM interpretation: requires contextualization to self-evident, (4) RSQMs for assessment decisions: not sticky to sticky, and (5) expectations for residents: potentially unfair to fair to use RSQMs. The interactions among these dimensions generated 3 RSQM data user profiles: eager incorporation, willing incorporation, and disinclined incorporation.

CONCLUSIONS

Participants used RSQMs to varying extents in their review of resident data and found such data helpful to varying degrees, supporting the inclusion of RSQMs as resident assessment data for CCC review.

Unusual suspects: Real-time physiological evaluation of stressors during laparoscopic donor nephrectomy

INTRODUCTION

The purpose of this study was to document the variability of faculty surgeon electrodermal activity (EDA) peaks during laparoscopic donor nephrectomy (LDN) to determine the effect of case difficulty and learner expertise on the stress response.

METHODS

EDA for a single faculty surgeon was captured over 15 LDN cases using an Empatica E4 wristband. During each case, one of three transplant fellows (novice, intermediate, or expert level LDN expertise) participated. Difficulty was rated preoperatively as "low/moderate/high" by the faculty. EDA peaks were collected and analyzed; the frequency and magnitude of EDA peaks, case difficulty, and fellow expertise were compared using a two-way factorial ANOVA.

RESULTS

The main effects of learner expertise (F[2, 308]=11.27, p<0.001) and difficulty rating (F[2, 414]=15.13, p<0.001) were significant. The interaction between difficulty and expertise on faculty EDA peaks was also significant (F[3, 391]=14.29, p<0.001). The novice fellow resulted in higher faculty EDA levels compared to intermediate and expert fellows on low-difficulty cases, but not moderate- or high-difficulty cases.

CONCLUSIONS

This is the first report examining faculty surgeon EDA across cases of varying difficulty and varying learner expertise during a high-stakes operation. EDA levels were inversely proportional to the expertise of the learner and case difficulty, suggestive of a significant impact of learner autonomy on faculty stress response.

Resident-Sensitive Quality Measures in the Pediatric Emergency Department: Exploring Relationships With Supervisor Entrustment and Patient Acuity and Complexity

PURPOSE

This study explores the associations between resident-sensitive quality measures (RSQMs) and supervisor entrustment as well as between RSQMs and patient acuity and complexity for encounters in the pediatric emergency department (PED) in which residents are caring for patients.

METHOD

Pediatric residents rotating through Cincinnati Children's Hospital Medical Center PED as well as supervising pediatric emergency medicine faculty and fellows were recruited during the 2017-2018 academic year for the purpose of collecting the following data from the residents' patient encounters for 3 illnesses (acute asthma exacerbation, bronchiolitis, and closed head injury [CHI]): supervisor entrustment decision rating, RSQMs relevant to the care provided, and supervisor patient acuity and complexity ratings. To measure the association of RSQM composite scores with the other variables of interest, mixed models were used.

RESULTS

A total of 83 residents cared for 110 patients with asthma, 112 with bronchiolitis, and 77 with CHI. Entrustment decision ratings were positively associated with asthma RSQM composite scores (beta coefficient = 0.03; P < .001). There was no significant association between RSQM composite scores and entrustment decision ratings for bronchiolitis or CHI. RSQM composite scores were significantly higher when acuity was also higher and significantly lower when acuity was also lower for both asthma (P < .001) and bronchiolitis (P = .01). However, RSQM composite scores were almost identical between levels of acuity for CHI (P = .94). There were no significant differences in RSQM composite scores when complexity varied.

CONCLUSION

This study found limited associations between RSQM composite scores and entrustment decision ratings but offers insight into how RSQMs could be used for the purposes of resident assessment and feedback.

Impact of industry 4.0 to create advancements in orthopaedics

Scientists and health professional are focusing on improving the medical sciences for the betterment of patients. The fourth industrial revolution, which is commonly known as Industry 4.0, is a significant advancement in the field of engineering. Industry 4.0 is opening a new opportunity for digital manufacturing with greater flexibility and operational performance. This development is also going to have a positive impact in the field of orthopaedics. The purpose of this paper is to present various advancements in orthopaedics by the implementation of Industry 4.0. To undertake this study, we have studied the available literature extensively on Industry 4.0, technologies of Industry 4.0 and their role in orthopaedics. Paper briefly explains about Industry 4.0, identifies and discusses the major technologies of Industry 4.0, which will support development in orthopaedics. Finally, from the available literature, the paper identifies twelve significant advancements of Industry 4.0 in orthopaedics. Industry 4.0 uses various types of digital manufacturing and information technologies to create orthopaedics implants, patient-specific tools, devices and innovative way of treatment. This revolution is to be useful to perform better spinal surgery, knee and hip replacement, and invasive surgeries.

Perceptions on Competence by Design in urology

INTRODUCTION

The Royal College of Physicians and Surgeons of Canada has begun implementing Competence by Design (CBD). However, it is unclear how much urology trainees and faculty know about CBD, their attitudes towards this change, and their willingness to embrace and participate in this new model of training.

METHODS

This cross-sectional study was conducted through an online survey, which was administered to all trainees and faculty at Canadian urology programs prior to the implementation of CBD. The final survey consisted of eight demographic questions, 17 five-point Likert items, one visual analog scale question, 11 multiple selection questions, and two open-ended questions.

RESULTS

A total of 74 participants (38 faculty and 36 trainees) across 12 universities responded, with a completion rate of 82.4%. This corresponded to an overall response rate of 20.5%. Overall, there was a lack of resounding enthusiasm towards this shift to CBD in urology. Although both trainees and faculty had overall positive perceptions of CBD on assessment, teaching, and readiness, most agreed that this transition will be costly and associated with increased requirements for time, funding, and administrative support. Furthermore, there were significant concerns regarding the lack of valid assessment tools and evidence for the validity of entrustable professional activities.

CONCLUSIONS

While this survey has demonstrated an appreciation for the benefits of CBD, challenges are equally anticipated. CBD in urology will be a fertile research area; this study has identified several important educational questions regarding the model's effectiveness and consequences, thus, providing collaborative opportunities among all Canadian programs.

Unravelling residents' and supervisors' workplace interactions: an intersubjectivity study

CONTEXT

Successful engagement between residents and supervisors lies at the core of workplace learning, a process that is not exempt from challenge. Clinical encounters have unique learning potential as they offer opportunities to achieve a shared understanding between the resident and supervisor of how to accomplish a common goal. How residents and supervisors develop such a mutual understanding is an issue that has received limited attention in the literature. We used the 'intersubjectivity' concept as a novel conceptual framework to analyse this issue.

METHODS

We conducted a constructivist grounded theory study in an anaesthesiology department in Bogota, Colombia, using focus groups and field observations. Eleven residents of different training levels and 18 supervisors with varying years of teaching experience participated in the study. Through iterative data analysis, collection and constant comparison, we constructed the final results.

RESULTS

We found that residents and supervisors achieved a shared understanding by adapting to one another in the process of providing patient care. Continuous changes in the composition of resident-supervisor dyads exposed them to many procedural variations, to which they responded by engaging in various adaptation patterns that included compliance by residents with supervisors' directions, negotiation by residents of supervisors' preferences, and the sharing of decision making. In the process, the resident played an increasingly key role as a member of the supervisory dyad. Additionally, experiencing these adaptation patterns repeatedly resulted in the creation of a working repertoire: an attuned working code used by the members of each supervisory dyad to work together as a team.

CONCLUSIONS

The development of shared understanding between residents and supervisors entailed experiencing diverse adaptation patterns which resulted in the creation of working repertoires. Seeing supervisory interactions as adaptation processes has essential theoretical and practical implications regarding workplace learning in postgraduate settings. Our findings call for further exploration to understand learning in postgraduate education as a social process.

Development and validation of surgical training tool: cystectomy assessment and surgical evaluation (CASE) for robot-assisted radical cystectomy for men

BACKGROUND

We aimed to develop a structured scoring tool: cystectomy assessment and surgical evaluation (CASE) that objectively measures and quantifies performance during robot-assisted radical cystectomy (RARC) for men.

METHODS

A multinational 10-surgeon expert panel collaborated towards development and validation of CASE. The critical steps of RARC in men were deconstructed into nine key domains, each assessed by five anchors. Content validation was done utilizing the Delphi methodology. Each anchor was assessed in terms of context, score concordance, and clarity. The content validity index (CVI) was calculated for each aspect. A CVI ≥ 0.75 represented consensus, and this statement was removed from the next round. This process was repeated until consensus was achieved for all statements. CASE was used to assess de-identified videos of RARC to determine reliability and construct validity. Linearly weighted percent agreement was used to assess inter-rater reliability (IRR). A logit model for odds ratio (OR) was used to assess construct validation.

RESULTS

The expert panel reached consensus on CASE after four rounds. The final eight domains of the CASE included: pelvic lymph node dissection, development of the peri-ureteral space, lateral pelvic space, anterior rectal space, control of the vascular pedicle, anterior vesical space, control of the dorsal venous complex, and apical dissection. IRR > 0.6 was achieved for all eight domains. Experts outperformed trainees across all domains.

CONCLUSION

We developed and validated a reliable structured, procedure-specific tool for objective evaluation of surgical performance during RARC. CASE may help differentiate novice from expert performances.