General considerations regarding assessment of professional behaviour

Unprofessional behaviour of GP residents leading to a dismissal dispute: characteristics and outcomes of those who appeal

BACKGROUND

Recognition of poor performance in General Practice trainees is important because underperformance compromises patients' health and safety. However, in General Practice, research on persistent underperformance while in training and its ultimate consequences is almost completely lacking. We aim to explore the unprofessional behaviours of residents in General Practice who were dismissed from training and who litigated against dismissal.

METHODS

We performed a structured analysis using open-source data from all General Practice cases before the Conciliation Board of the Royal Dutch Medical Association between 2011 and 2020. Anonymised law cases about residents from all Dutch GP training programmes were analysed in terms of the quantitative and qualitative aspects related to performance.

RESULTS

Between 2011 and 2020, 24 residents who were dismissed from training challenged their programme director's decision. Dismissed residents performed poorly in several competencies, including communication, medical expertise and most prominently, professionalism. Over 90% of dismissed residents failed on professionalism. Most lacked self-awareness and/or failed to profit from feedback. Approximately 80% failed on communication, and about 60% on medical expertise as well. A large majority (more than 80%) of dismissed residents had previously participated in some form of remediation.

CONCLUSIONS

Deficiencies in both professionalism and communication were the most prevalent findings among the dismissed General Practice residents. These two deficiencies overlapped considerably. Dismissed residents who challenged their programme director's decision were considered to lack self-awareness, which requires introspection and the appreciation of feedback from others.

A "Red Flag" system adds value to medical school admissions interviews

INTRODUCTION

Non-cognitive traits should be considered when selecting candidates to study medicine. However, evaluating these traits remains difficult. We explored whether measuring undesirable non-cognitive behaviour ('Red Flags') added value to a medical school admissions system. Red Flags included rudeness, ignoring the contributions of others, disrespectful behaviour, or poor communication.

METHODS

Following an admissions interview testing non-cognitive attributes in 648 applicants to a UK medical school, we measured the association between interview score and Red Flag frequency. We tested linear and polynomial regression models to evaluate whether the association was linear or non-linear.

RESULTS

In total, 1126 Red Flags were observed. While Red Flags were concentrated among low-scorers, candidates in the highest- and second-highest deciles for interview score still received Red Flags (six and twenty-two, respectively). The polynomial regression model indicated candidates with higher scores received fewer Red Flags, but the association was not linear (F(3644) = 159.8, p = .001, adjusted R2 = 0.42).

CONCLUSIONS

The non-linear association between interview score and Red Flag frequency shows some candidates with desirable non-cognitive attributes will still display undesirable-or even exclusionary-non-cognitive attributes. Recording Red Flag behaviour reduces the likelihood such candidates will be offered a place at medical school.

Adaptation of the professionalism mini-evaluation exercise instrument into Turkish: a validity and reliability study

BACKGROUND

There is an ongoing search for standardized scales appropriate for each culture to evaluate professionalism, which is one of the basic competencies of a physician. The Professionalism Mini-evaluation Exercise (P-MEX) instrument was originally developed in Canada to meet this need. In this study, it was aimed to adapt the P-MEX to Turkish and to evaluate the validity and reliability of the Turkish version.

METHODS

A total of 58 residents at Bakirkoy Dr. Sadi Konuk Training and Research Hospital were assessed with the Turkish version of P-MEX by 24 raters consisting of faculty members, attending physicians, peer residents, and nurses during patient room visits, outpatient clinic and group practices. For construct validity, the confirmatory factor analysis was performed. For reliability, Cronbach's alpha scores were calculated. Generalizibility and decision studies were undertaken to predict the reliability of the validated tool under different conditions. After the administration of P-MEX was completed, the participants were asked to provide feedback on the acceptability, feasibility, and educational impact of the instrument.

RESULTS

A total of 696 forms were obtained from the administration of P-MEX. The content validity of P-MEX was found to be appropriate by the faculty members. In the confirmatory factor analysis of the original structure of the 24-item Turkish scale, the goodness-of-fit parameters were calculated as follows: CFI = 0.675, TLI = 0.604, and RMSEA = 0.089. In the second stage, the factors on which the items loaded were changed without removing any item, and the model was modified. For the modified model, the CFI, TLI, and RMSEA values were calculated as 0.857, 0.834, and 0.057, respectively. The decision study on the results obtained from the use of P-MEX in a Turkish population revealed the necessity to perform this evaluation 18 times to correctly evaluate professionalism with this instrument. Cronbach's alpha score was 0.844. All the faculty members provided positive feedback on the acceptability, feasibility, and educational impact of the adapted P-MEX.

CONCLUSION

The findings of this study showed that the Turkish version of P-MEX had sufficient validity and reliability in assessing professionalism among residents. Similarly, the acceptability and feasibility of the instrument were found to be high, and it had a positive impact on education.

TRIAL REGISTRATION

2020/249, Bakirkoy Dr. Sadi Konuk Training and Research Hospital.

Evaluation of an Instrument to Assess Students' Personal and Professional Development During the Faculty Advising Process

Objective. To evaluate a tool designed to assess Doctor of Pharmacy (PharmD) students' personal and professional development prior to beginning advanced pharmacy practice experiences (APPEs).Methods. A five-item instrument, entitled the Faculty Advisor's Assessment of the Advisee (FAAA) tool, was developed to assess and monitor pharmacy students' progress over the three-year didactic curriculum. Question anchors were created to describe characteristics exhibited by the student that matched categories of not engaged, beginning, emerging, or engaged. Possible FAAA composite scores ranged from 7 to 20. Using the FAAA tool, faculty advisors assessed their advisees' values, engagement, self-awareness, professionalism, and leadership in 2017, 2018, and 2019. Individual and aggregate cohort reports were run and data for each of the three years were matched with students. To determine if the FAAA showed progression in assessed dimensions in the students during the first, second, and third professional (P1, P2, and P3) years, a Friedman test was performed. Cronbach alpha was used to assess the reliability of the instrument.Results. The data of 93 students were matched for the P1 through the P3 years. Median (IQR) for the FAAA composite score levels for the P1, P2, and P3 were 13 (12-16), 17 (15-19) and 18 (16-20), respectively. Significant differences existed at all timepoints compared, including from the P1 to P2, P2 to P3, and P1 to P3 years. The reliability of the FAAA scale was strong across all three years (winter 2017, α=0.87; winter 2018, α=0.89; and winter 2019, α=0.87). All items appeared worthy of retention as removal did not significantly increase their reliability.Conclusion. A five-item tool which assesses pharmacy students' personal and professional development during the first three years of a PharmD program could be used by faculty advisors to assess student's progress across the didactic curriculum.

Professional practice behaviour: Identification and validation of key indicators

Introduction

Professional behaviour is regarded as an important competency for occupational therapy practice, yet little guidance exists for indicators underpinning development or remediation in the educational or practice settings. This study sought to confirm the content validity of observable professional behaviour indicators from an existing evaluation framework for representativeness and relevance for occupational therapy practice.

Methods

A modified Delphi approach was conducted with expert panellists ( n = 30) consisting of regulators, administrators, faculty members, practitioners, and students for professional behaviour indicator consensus, together with a cross-sectional survey of practitioners ( n = 119). Fleiss’ κ and χ2 contingency tables were completed for agreement across panellists, and between panellist and survey groups. Cross-case qualitative analyses identified facilitators and barriers for professional behaviour practice.

Results

Content validity of 17 professional behaviour indicators was achieved, with >85% agreement from the expert panellists and the cross-sectional survey group. Main professional behaviour reporting issues in practice included fear of reprisal, lack of formal policies, and an unsupportive culture. Support from others, documented workplace policies, and self-regulation/duty to monitor were the critical facilitators for supporting professional behaviour in practice.

Conclusion

The professional behaviour indicators in this study offer observable behaviours from which assessment rubrics or tools may be developed. Further study is warranted.

Development and Validation of a Simplified Chinese Version of the Assessment Tool for Students' Perceptions of Medical Professionalism

Professionalism is crucial in all professions and is particularly important in the medical field. Measuring students' perceptions of professionalism can help to form education targeting the enhancement of professionalism. This study aimed to validate an effective assessment tool for the measurement of medical students' perceptions of medical professionalism in mainland China. The cross-sectional survey was conducted in three medical colleges in Guangdong, China. Of the 2103 eligible medical students, 1976 responded, and 1856 questionnaires were deemed valid. Students from clinical medicine in these three medical colleges were randomly selected by cluster sampling. First, a Simplified Chinese Version questionnaire to measure Student's Perception of Medical Professionalism (SCV-SPMP) was constructed. Second, questionnaires from 1856 students majoring in clinical medicine at three medical colleges were included in the analysis. Third, exploratory factor analysis, Cronbach's alpha, item-subscale correlation, and confirmatory factor analysis were conducted to test the validity and reliability of the SCV-SPMP. Nine items were eliminated following exploratory factor analysis, and four subscales were extracted from the analysis. All internal consistency reliability exceeded the minimum standard. The overall Cronbach's alpha was 0.94, and four subscales' alphas were 0.82 (Accountability and excellence), 0.81 (Duty), 0.89 (Honor and integrity), and 0.85 (Practice habits and respect for others), respectively. The model fit was good. The convergent validity and discriminant validity were acceptable. The modified SCV-SPMP was found to be a valid and reliable tool to capture the main features of Chinese students' perceptions of medical professionalism in four dimensions, and it provides a quantitative method for the measurement of the students' perceptions in mainland China..

Self-Assessment of Preparedness among Critical Care Trainees Transitioning from Fellowship to Practice

This study evaluated the subjective assessment of preparedness needs of critical care trainees and recent graduates between 2013 and 2014. A questionnaire was developed and validated by the subcommittee of the In-Training Section of Society of Critical Care Medicine (SCCM). The survey was deployed twice between December 2013 and January 2014 via email to any trainee or individual graduated from a critical care fellowship within the previous three years. Six percent (180) of all individuals completed the survey, and 67% of respondents had recently interviewed for a job. Northeast was the preferred location for a job (47%), and academia was favored over private practice (80% vs. 15%). Of the respondents that secured an interview, 55% felt prepared for the interview, 67% felt prepared to build an adequate job portfolio, 33% received formal guidance from their mentor/training program. 89% of total respondents agreed it is important to participate in a formal training course in job search, portfolio development, and interviewing process. The preferred sources of training were equally distributed between their home institution, webinars, and SCCM. There is an ongoing need in education regarding the transition period from fellowship to practice.

Descriptors for unprofessional behaviours of medical students: a systematic review and categorisation

BACKGROUND

Developing professionalism is a core task in medical education. Unfortunately, it has remained difficult for educators to identify medical students' unprofessionalism, because, among other reasons, there are no commonly adopted descriptors that can be used to document students' unprofessional behaviour. This study aimed to generate an overview of descriptors for unprofessional behaviour based on research evidence of real-life unprofessional behaviours of medical students.

METHODS

A systematic review was conducted searching PubMed, Ebsco/ERIC, Ebsco/PsycINFO and Embase.com from inception to 2016. Articles were reviewed for admitted or witnessed unprofessional behaviours of undergraduate medical students.

RESULTS

The search yielded 11,963 different studies, 46 met all inclusion criteria. We found 205 different descriptions of unprofessional behaviours, which were coded into 30 different descriptors, and subsequently classified in four behavioural themes: failure to engage, dishonest behaviour, disrespectful behaviour, and poor self-awareness.

CONCLUSIONS

This overview provides a common language to describe medical students' unprofessional behaviour. The framework of descriptors is proposed as a tool for educators to denominate students' unprofessional behaviours. The found behaviours can have various causes, which should be explored in a discussion with the student about personal, interpersonal and/or institutional circumstances in which the behaviour occurred. Explicitly denominating unprofessional behaviour serves two goals: [i] creating a culture in which unprofessional behaviour is acknowledged, [ii] targeting students who need extra guidance. Both are important to avoid unprofessional behaviour among future doctors.

Assessing medical professionalism: A systematic review of instruments and their measurement properties

BACKGROUND

Over the last three decades, various instruments were developed and employed to assess medical professionalism, but their measurement properties have yet to be fully evaluated. This study aimed to systematically evaluate these instruments' measurement properties and the methodological quality of their related studies within a universally acceptable standardized framework and then provide corresponding recommendations.

METHODS

A systematic search of the electronic databases PubMed, Web of Science, and PsycINFO was conducted to collect studies published from 1990-2015. After screening titles, abstracts, and full texts for eligibility, the articles included in this study were classified according to their respective instrument's usage. A two-phase assessment was conducted: 1) methodological quality was assessed by following the COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN) checklist; and 2) the quality of measurement properties was assessed according to Terwee's criteria. Results were integrated using best-evidence synthesis to look for recommendable instruments.

RESULTS

After screening 2,959 records, 74 instruments from 80 existing studies were included. The overall methodological quality of these studies was unsatisfactory, with reasons including but not limited to unknown missing data, inadequate sample sizes, and vague hypotheses. Content validity, cross-cultural validity, and criterion validity were either unreported or negative ratings in most studies. Based on best-evidence synthesis, three instruments were recommended: Hisar's instrument for nursing students, Nurse Practitioners' Roles and Competencies Scale, and Perceived Faculty Competency Inventory.

CONCLUSION

Although instruments measuring medical professionalism are diverse, only a limited number of studies were methodologically sound. Future studies should give priority to systematically improving the performance of existing instruments and to longitudinal studies.

Reaching Consensus on Measuring Professional Behaviour in Physical Therapy Objective Structured Clinical Examinations

Purpose: We determined which professional behaviours (PBs) are important and feasible to measure in an objective structured clinical examination (OSCE) intended to assess the hands-on skills and knowledge of students in a Canadian physical therapy (PT) program. Methods: We used a modified Delphi technique to identify the criteria required to assess PBs in PT students during an OSCE. We conducted a focus group to better understand the results of the modified Delphi process. Results: Experienced local OSCE examiners participated in the modified Delphi panel, which consisted of two rounds of surveys: round 1 (n=12) and round 2 (n=10). A total of 31 PBs were reduced to 18 through the two rounds. Five of the panellists participated in the focus group, reduced the 18 PBs to 15, and then identified 4 as clinical skills. Participants categorized the remaining 11 as mixed PBs and clinical skills (1 item), PBs (4 items), or communication skills (6 items). Conclusion: This study provides preliminary evidence to support the feasibility and importance of evaluating 5 PB items in practical skills OSCEs for entry-to-practice PT students.

The Desired Concept Maps and Goal Setting for Assessing Professionalism in Medicine

Due to the multi-dimensional characteristics of professionalism, no single assessment modality has shown to reliably assess professionalism. This review aims to describe some of the popular assessment tools that are being used to assess professionalism with a view to formulate a framework of assessment of professionalism in medicine. In December 2015, the online research databases of MEDLINE, the Educational Resources Information Center (ERIC), Elton Bryson Stephens Company (EBSCO), SCOPUS, OVID and PsychINFO were searched for full-text English language articles published during 2000 to 2015. MeSH terms "professionalism" AND "duty" AND "assessment" OR "professionalism behavioural" AND "professionalism-cognitive" were used. The research articles that assessed professionalism across medical fields along with other areas of competencies were included. A final list of 35 articles were selected for this review. Several assessment tools are available for assessing professionalism that includes, but not limited to, mini clinical evaluation exercise, standardised direct observation of procedural skills, professionalism mini-evaluation exercise, multi-source feedback and 360 degree evaluation, and case based discussions. Because professionalism is a complex construct, it is less likely that a single assessment strategy will adequately measure it. Since every single assessment tool has its own weaknesses, triangulation involving multiple tools can compensate the shortcomings associated with any single approach. Assessment of professionalism necessitates a combination of modalities at individual, interpersonal, societal, and institutional levels and should be accompanied by feedback and motivational reflection that will, in turn, lead to behaviour and identity formation. The assessment of professionalism in medicine should meet the criteria of validity, reliability, feasibility and acceptability. Educators are urged to enhance the depth and quality of assessment instruments in the existing medical curricula for ensuring validity and reliability of assessment tools for professionalism.

Measuring professional behaviour in canadian physical therapy students' objective structured clinical examinations: an environmental scan

PURPOSE

To identify professional behaviours measured in objective structured clinical examinations (OSCEs) by Canadian university physical therapy (PT) programs.

METHOD

A cross-sectional telephone survey was conducted to review current practice and determine which OSCE items Canadian PT programs are using to measure PT students' professional behaviours. Telephone interviews using semi-structured questions were conducted with individual instructors responsible for courses that included an OSCE as part of the assessment component.

RESULTS

Nine PT programmes agreed to take part in the study, and all reported conducting at least one OSCE. The number and characteristics of OSCEs varied both within and across programs. Participants identified 31 professional behaviour items for use in an OSCE; these items clustered into four categories: communication (n=14), respect (n=10), patient safety (n=4), and physical therapists' characteristics (n=3).

CONCLUSIONS

All Canadian entry-level PT programmes surveyed assess professional behaviours in OSCE-type examinations; however, the content and style of assessment is variable. The local environment should be considered when determining what professional behaviours are appropriate to assess in the OSCE context in individual programmes.

Responding to moderate breaches in professionalism: an intervention for medical students

Much has been written about how we understand, teach and evaluate professionalism in medical training. Less often described are explicit responses to mild or moderate professionalism concerns in medical students. To address this need, Baylor College of Medicine created a mechanism to assess professionalism competency for medical students and policies to address breaches in professional behavior. This article describes the development of an intervention using a guided reflection model, student responses to the intervention, and how the program evolved into a credible resource for deans and other educational leaders.

Professionalism and Communication Education in Pediatric Critical Care Medicine: The Learner Perspective

OBJECTIVE

Communication and professionalism are often challenging to teach, and the impact of the use of a given approach is not known. We undertook this investigation to establish pediatric critical care medicine (PCCM) trainee perception of education in professionalism and communication and to compare their responses from those obtained from PCCM fellowship program directors.

METHODS

The Education in Pediatric Intensive Care (E.P.I.C.) Investigators used the modified Delphi technique to develop a survey examining teaching of professionalism and communication. After piloting, the survey was sent to all 283 PCCM fellows in training in the United States.

RESULTS

Survey response rate was 47% (133 of 283). Despite high rates of teaching overall, deficiencies were noted in all areas of communication and professionalism assessed. The largest areas of deficiency included not being specifically taught how to communicate: as a member of a nonclinical group (reported in 24%), across a broad range of socioeconomic and cultural backgrounds (19%) or how to provide consultation outside of the intensive care unit (17%). Only 50% of fellows rated education in communication as "very good/excellent." However, most felt confident in their communication abilities. For professionalism, fellows reported not being taught accountability (12%), how to conduct a peer review (12%), and how to handle potential conflict between personal beliefs, circumstances, and professional values (10%). Fifty-seven percent of fellows felt that their professionalism education was "very good/excellent," but nearly all expressed confidence in these skills. Compared with program directors, fellows reported more deficiencies in both communication and professionalism.

CONCLUSIONS

There are numerous components of communication and professionalism that PCCM fellows perceive as not being specifically taught. Despite these deficiencies, fellow confidence remains high. Substantial opportunities exist to improve teaching in these areas.

Are pediatric critical care medicine fellowships teaching and evaluating communication and professionalism?

OBJECTIVES

To describe the teaching and evaluation modalities used by pediatric critical care medicine training programs in the areas of professionalism and communication.

DESIGN

Cross-sectional national survey.

SETTING

Pediatric critical care medicine fellowship programs.

SUBJECTS

Pediatric critical care medicine program directors.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Survey response rate was 67% of program directors in the United States, representing educators for 73% of current pediatric critical care medicine fellows. Respondents had a median of 4 years experience, with a median of seven fellows and 12 teaching faculty in their program. Faculty role modeling or direct observation with feedback were the most common modalities used to teach communication. However, six of the eight (75%) required elements of communication evaluated were not specifically taught by all programs. Faculty role modeling was the most commonly used technique to teach professionalism in 44% of the content areas evaluated, and didactics was the technique used in 44% of other professionalism content areas. Thirteen of the 16 required elements of professionalism (81%) were not taught by all programs. Evaluations by members of the healthcare team were used for assessment for both competencies. The use of a specific teaching technique was not related to program size, program director experience, or training in medical education.

CONCLUSIONS

A wide range of techniques are currently used within pediatric critical care medicine to teach communication and professionalism, but there are a number of required elements that are not specifically taught by fellowship programs. These areas of deficiency represent opportunities for future investigation and improved education in the important competencies of communication and professionalism.

Teaching and assessing veterinary professionalism

The teaching and assessment of professional behaviors and attitudes are important components of veterinary curricula. This article aims to outline some important considerations and concepts which will be useful for veterinary educators reviewing or developing this topic. A definition or framework of veterinary professionalism must be decided upon before educators can develop relevant learning outcomes. The interface between ethics and professionalism should be considered, and both clinicians and ethicists should deliver professionalism teaching. The influence of the hidden curriculum on student development as professionals should also be discussed during curriculum planning because it has the potential to undermine a formal curriculum of professionalism. There are several learning theories that have relevance to the teaching and learning of professionalism; situated learning theory, social cognitive theory, adult learning theory, reflective practice and experiential learning, and social constructivism must all be considered as a curriculum is designed. Delivery methods to teach professionalism are diverse, but the teaching of reflective skills and the use of early clinical experience to deliver valid learning opportunities are essential. Curricula should be longitudinal and integrated with other aspects of teaching and learning. Professionalism should also be assessed, and a wide range of methods have the potential to do so, including multisource feedback and portfolios. Validity, reliability, and feasibility are all important considerations. The above outlined approach to the teaching and assessment of professionalism will help ensure that institutions produce graduates who are ready for the workplace.

Assessment of professionalism: a consolidation of current thinking

CONTEXT

Professionalism has become a hot topic in medical education. Professionalism needs to be assessed if it is to be viewed as both positive and relevant.

OBJECTIVES

The assessment of professionalism is an evolving field. This review aims to consolidate current thinking.

IMPLICATIONS

Assessment of professionalism has progressed from an initial focus on the development and attainment of professional identity, through identifying areas of deficiency, to the attainment of a set of identifiable positive attributes and behaviours. It is now beginning to recognise the challenge of assessing a multi-dimensional construct, looking beyond the measurement of behaviour to embrace a diversity of approaches.

CONCLUSIONS

Professionalism should be assessed longitudinally. It requires combinations of different approaches, assessing professionalism at individual, interpersonal and societal/institutional levels. Increasing the depth and the quality of reliability and validity of existing programmes in various contexts may be more appropriate than concentrating on developing new instruments. Increasing the number of tests and the number of relevant contexts will increase the reliability of the result. Similarly increasing the number of observers increases reliability. Feedback, encouraging reflection, can promote change in behaviour and identity formation.

Assessing professionalism within dental education; the need for a definition

Professionalism is a broad competency needed by dentists to act effectively and efficiently and is seen as a central part of both undergraduate and postgraduate curricula. Assessment is vital in education to assess progress and direct future learning. It is also an essential part of good professional regulation, which depends upon high quality assessment to maintain credibility. Educators must produce clear expectations that students can strive for. Thus dental educators are required to understand precisely what is meant by "professionalism" in relation to dentistry in order to both teach and assess it. The aim of this paper is to explain the importance of professionalism, explore its features and its assessment as described in the literature. The paper concludes that without a validated definition of this construct, assessment of professionalism within dental education will be compromised.

Integrating professionalism into the curriculum: AMEE Guide No. 61

Professional values and behaviours are intrinsic to all medical practice, yet remain one of the most difficult subjects to integrate explicitly into a curriculum. Professionalism for the twenty-first century raises challenges not only to adapting the course to changing societal values but also for instilling skills of ongoing self-directed continuous development in trainees for future revalidation. This Guide is based on the contemporary available literature and focuses on instilling Professionalism positively into both undergraduate and postgraduate training deliberately avoiding the more negative aspects of Fitness to Practise. The literature on Professionalism is extensive. An evidence-based approach has been taken throughout. We have selected only some of the available publications to offer practical advice. Comprehensive reviews are available elsewhere (van Mook et al. 2009a-g). This Guide takes a structured stepwise approach and sequentially addresses: (i) agreeing an institutional definition, (ii) structuring the curriculum to integrate learning across all years, (iii) suggesting learning models, (iv) harnessing the impact of the formal, informal and hidden curricula and (v) assessing the learning. Finally, a few well-evaluated case studies for both teaching and assessment have been selected to illustrate our recommendations.

A multirater instrument for the assessment of simulated pediatric crises

BACKGROUND

Few validated instruments exist to measure pediatric code team skills. The goal of this study was to develop an instrument for the assessment of resuscitation competency and self-appraisal using multirater and gap analysis methodologies.

METHODS

Multirater assessment with gap analysis is a robust methodology that enables the measurement of self-appraisal as well as competency, offering faculty the ability to provide enhanced feedback. The Team Performance during Simulated Crises Instrument (TPDSCI) was grounded in the Accreditation Council for Graduate Medical Education competencies. The instrument contains 5 competencies, each assessed by a series of descriptive rubrics. It was piloted during a series of simulation-based interdisciplinary pediatric crisis resource management education sessions. Course faculty assessed participants, who also did self-assessments. Internal consistency and interrater reliability were analyzed using Cronbach α and intraclass correlation (ICC) statistics. Gap analysis results were examined descriptively.

RESULTS

Cronbach α for the instrument was between 0.72 and 0.69. The overall ICC was 0.82. ICC values for the medical knowledge, clinical skills, communication skills, and systems-based practice were between 0.87 and 0.72. The ICC for the professionalism domain was 0.22. Further examination of the professionalism competency revealed a positive skew, 43 simulated sessions (98%) had significant gaps for at least one of the competencies, 38 sessions (86%) had gaps indicating self-overappraisal, and 15 sessions (34%) had gaps indicating self-underappraisal.

CONCLUSIONS

The TPDSCI possesses good measures of internal consistency and interrater reliability with respect to medical knowledge, clinical skills, communication skills, systems-based practice, and overall competence in the context of simulated interdisciplinary pediatric medical crises. Professionalism remains difficult to assess. These results provide an encouraging first step toward instrument validation. Gap analysis reveals disparities between faculty and self-assessments that indicate inadequate participant self-reflection. Identifying self-overappraisal can facilitate focused interventions.

Systematic review on mentoring and simulation in laparoscopic colorectal surgery

OBJECTIVE

To identify and evaluate the influence of mentoring and simulated training in laparoscopic colorectal surgery (LCS) and define the key components for learning advanced technical skills.

BACKGROUND

Laparoscopic colorectal surgery is a complex procedure, often being self-taught by senior surgeons. Educational issues such as inadequate training facilities or a shortfall of training fellowships may result in a slow uptake of LCS. The effectiveness of mentored and simulated training, however, remains unclear.

METHODS

We conducted a systematic search, using Ovid databases. Four study categories were identified: mentored versus nonmentored cases, training case selection, simulation, and assessment. We performed a meta-analysis and a mixed model regression on the difference of the main outcome measures (conversion rates, morbidity, and mortality) for mentored trainees and expert surgeons. We also compared conversion rates of mentored and nonmentored. Meta-analysis of risk factors for conversion was performed using published and unpublished data sets requested from various investigators. For studies on simulation, we compared scores of surveys on the perception of different training courses.

RESULTS

Thirty-seven studies were included. Pooled weighted outcomes of mentored cases (n = 751) showed a lower conversion rate (13.3% vs 20.5%, P = 0.0332) compared with nonmentored cases (n = 695). Compared to expert case series (n = 5313), there was no difference in conversion (P = 0.2835), anastomotic leak (P = 0.8342), or mortality (P = 0.5680). A meta-analysis of training case selection data (n = 4444) revealed male sex (P < 0.0001), previous abdominal surgery (P = 0.0200), a BMI greater than 30 (P = 0.0050), an ASA of less than 2 (P < 0.0001), colorectal cancer (P < 0.0001) and intra-abdominal fistula (P < 0.0001), but not older than 64 years (P = 0.4800), to significantly increase conversion risk. Participants on cadaveric courses were highly satisfied with the teaching value yet trainees on an animal course gave less positive feedback. Structured assessment for LCS has been partially implemented.

CONCLUSION

This review and meta-analysis supports evidence that trainees can obtain similar clinical results like expert surgeons in laparoscopic colorectal surgery if supervised by an experienced trainer. Cadaveric models currently provide the best value for training in a simulated environment. There remains a need for further research into technical skills assessment and the educational value of simulated training.

Development, validation and implementation of a monitoring tool for training in laparoscopic colorectal surgery in the English National Training Program

INTRODUCTION

The National Training Program for laparoscopic colorectal surgery (LCS) provides supervised training to colorectal surgeons in England. The purpose of this study was to create, validate, and implement a method for monitoring training progression in laparoscopic colorectal surgery that met the requirements of a good assessment tool.

METHODS

A generic scale for different tasks in LCS was created under the guidance of a national expert group. The scores were defined by the extent to which the trainees were dependent on support (1 = unable to perform, 5 = unaided (benchmark), 6 = proficient). Trainers were asked to rate their trainees after each supervised case; trainees completed a similar self-assessment form. Construct validity was evaluated comparing scores of trainees at different experience levels (1-5, 6-10, 11-15, 16+) using the Wilcoxon signed-rank test and ANOVA. Internal consistency was determined by Crohnbach's alpha, interrater reliability by comparing peer- and self-assessment (interclass correlation coefficient, ICC). Proficiency gain curves were plotted using CUSUM charts.

RESULTS

Analysis included 610 assessments (333 by trainers and 277 by trainees). There was high interrater reliability (ICC = 0.867), internal consistency (α = 0.920), and construct validity [F(3,40) = 6.128, p < 0.001]. Detailed analysis of proficiency gain curves demonstrates that theater setup, exposure, and anastomosis were performed independently after 5 to 15 sessions, and the dissection of the vascular pedicle took 24 cases. Mobilization of the colon and of the splenic/hepatic flexure took more than 25 procedures. Median assessment time was 3.3 (interquartile range (IQR) 1-5) minutes and the tool was accepted as useful [median score 5 of 6 (IQR 4-5)].

DISCUSSION

A valid and reliable monitoring tool for surgical training has been implemented successfully into the National Training Program. It provides a description of an individualized proficiency gain curve in terms of both the level of support required and the competency level achieved.

Bad apples spoil the barrel: Addressing unprofessional behaviour

Given the changes in society we are experiencing, the increasing focus on patient-centred care and acknowledgement that medical education including professionalism issues needs to continue not only in the residency programmes but also throughout the doctors career, is not surprising. Although most of the literature on professionalism pertains to learning and teaching professionalism issues, addressing unprofessional behaviour and related patient safety issues forms an alternative or perhaps complementary approach. This article describes the possibility of selecting applicants for a medical school based on personality characteristics, the attention to professional lapses in contemporary undergraduate training, as well as the magnitude, aetiology, surveillance and methods of dealing with reports of unprofessional behaviour in postgraduate education and CME.