Leadership competencies for medical education and healthcare professions: population-based study

OBJECTIVE

To identify and empirically investigate the dimensions of leadership in medical education and healthcare professions.

DESIGN

A population-based design with a focus group and a survey were used to identify the perceived competencies for effective leadership in medical education.

SETTING

The focus group, consisting of five experts from three countries (Austria n=1; Germany n=2; Switzerland n=2), was conducted (all masters of medical education), and the survey was sent to health professionals from medical schools and teaching hospitals in six countries (Austria, Canada, Germany, Switzerland, the UK and the USA).

PARTICIPANTS

The participants were educators, physicians, nurses and other health professionals who held academic positions in medical education. A total of 229 completed the survey: 135 (59.0%) women (mean age=50.3 years) and 94 (41.0%) men (mean age=51.0 years).

MEASURES

A 63-item survey measuring leadership competencies was developed and administered via electronic mail to participants.

RESULTS

Exploratory principal component analyses yielded five factors accounting for 51.2% of the variance: (1) social responsibility, (2) innovation, (3) self-management, (4) task management and (5) justice orientation. There were significant differences between physicians and other health professionals on some factors (Wilk's λ=0.93, p<0.01). Social responsibility was rated higher by other health professionals (M=71.09) than by physicians (M=67.12), as was innovation (health professionals M=80.83; physicians M=76.20) and justice orientation (health professionals M=21.27; physicians M=20.46).

CONCLUSIONS

The results of the principal component analyses support the theoretical meaningfulness of these factors, their coherence, internal consistency and parsimony in explaining the variance of the data. Although there are some between-group differences, the competencies appear to be stable and coherent.

A review of student evaluation of teaching: applications to veterinary medical education

Student evaluation of teaching is ubiquitous to teaching in colleges and universities around the world. Since the implementation of student evaluations in the 1970s in the US, considerable research has been devoted to their appropriate use as a means of judging the effectiveness of teaching. The present article aims to (1) examine the evidence for the reliability, validity, and utility of student ratings; (2) provide seven guidelines for ways to identify effective instruction, given that the purpose of student evaluation is to assess effective teaching; and (3) conclude with recommendations for the integration of student ratings into the continuous evaluation of veterinary medical education.

A generalizability analysis of a veterinary school Multiple Mini Interview: effect of number of interviewers, type of interviewers, and number of stations

BACKGROUND

The number of Multiple Mini Interview (MMI) stations and the type and number of interviewers required for an acceptable level of reliability for veterinary admissions requires investigation.

PURPOSE

The goal is to investigate the reliability of the 2009 MMI admission process at the University of Calgary.

METHODS

Each applicant (n = 103; female = 80.6%; M age = 23.05 years, SD = 3.96) participated in a 7-station MMI. Applicants were rated independently by 2 interviewers, a faculty member, and a community veterinarian, within each station (total interviewers/applicant N = 14). Interviewers scored applicants on 3 items, each on a 5-point anchored scale.

RESULTS

Generalizability analysis resulted in a reliability coefficient of G = 0.79. A Decision study (D-study) indicated that 10 stations with 1 interviewer would produce a G = 0.79 and 8 stations with 2 interviewers would produce a G = 0.81; however, these have different resource requirements. A two-way analysis of variance showed that there was a nonsignificant main effect of interviewer type (between faculty member and community veterinarian) on interview scores, F(1, 1428) = 3.18, p = .075; a significant main effect of station on interview scores, F(6, 1428) = 4.34, p < .001; and a nonsignificant interaction effect between interviewer-type and station on interview scores, F(6, 1428) = 0.74, p = .62.

CONCLUSIONS

Overall reliability was adequate for the MMI. Results from the D-study suggest that the current format with 7 stations provides adequate reliability given that there are enough interviewers; to achieve the same G-coefficient 1 interviewer per station with 10 stations would suffice and reduce the resource requirements. Community veterinarians and faculty members demonstrated an adequate level of agreement in their assessments of applicants.

The Coordinators of Psychiatric Education (COPE) Residency In-Training Exam: a preliminary psychometric assessment

OBJECTIVE

The Coordinators of Psychiatric Education (COPE) Residency In-Training Exam is a formative exam for Canadian psychiatric residents that was reconstructed using assessment best practices. An assessment of psychometric properties was subsequently performed on the exam to ensure preliminary validity and reliability.

METHODS

An exam blueprint was developed based on the 2007 Royal College objectives for psychiatric training. A minimum pass level was established using a modified Angoff method. The exam was administered to all Canadian psychiatric residents in postgraduate years 2 to 5 with test reliability (Cronbach alpha) and item analysis performed. Exam validity was assessed through blueprint adherence and cross-year resident performance analysis.

RESULTS

Four hundred two exams were suitable for analysis. The overall mean score for all residents was 69.6% (SD=8.5) with significant differences in total scores between each of the postgraduate year groups, with consistently better performance with increasing time in residency. Cronbach alpha was 0.79.

CONCLUSION

The present study provides preliminary support that the reconstructed COPE Residency In-Training Exam demonstrates adequate reliability and validity, including showing the capacity to discriminate between levels of training.

Assessment of first-year veterinary students' clinical skills using objective structured clinical examinations

The DVM program at the University of Calgary offers a Clinical Skills course each year for the first three years. The course is designed to teach students the procedural skills required for entry-level general veterinary practice. Objective Structured Clinical Examinations (OSCEs) were used to assess students' performance on these procedural skills. A series of three OSCEs were developed for the first year. Content was determined by an exam blueprint, exam scoring sheets were created, rater training was provided, a mock OSCE was performed with faculty and staff, and the criterion-referencing Ebel method was used to set cut scores for each station using two content experts. Each station and the overall exam were graded as pass or fail. Thirty first-year DVM students were assessed. Content validity was ensured by the exam blueprint and expert review. Reliability (coefficient α) of the stations from the three OSCE exams ranged from 0.0 to 0.71. The three exam reliabilities (Generalizability Theory) were, for OSCE 1, G=0.56; OSCE 2, G=0.37; and OSCE 3, G=0.32. Preliminary analysis has suggested that the OSCEs demonstrate face and content validity, and certain stations demonstrated adequate reliability. Overall exam reliability was low, which reflects issues with first-time exam delivery. Because this year was the first that this course was taught and this exam format was used, work continues in the program on the teaching of the procedural skills and the development and revision of OSCE stations and scoring checklists.

Using standardized essays in the veterinary medicine admissions process: are the ratings reliable and valid?

The reliability and validity of using essays for veterinary medical school admissions requires investigation. We explored the reliability and construct validity of a structured essay station in the 2009 admission process at the University of Calgary Faculty of Veterinary Medicine. One hundred three applicants (female=80.6%, male=19.4%; mean age=23.05 years, SD=3.96) participated. Each applicant wrote a one-hour supervised essay (750 words). Essays were rated independently by two randomly assigned raters (n=16). Raters scored essays on four items, each on a five-point anchored scale. Nine essays were scored by all raters to perform a decision study. Generalizability analysis resulted in a reliability coefficient of 0.55. The decision study indicated that three raters and four items produces a G of 0.68. Essay score correlated with interview score (r=0.30, p<0.01) but not with GPA (r=0.05, p=ns). Overall reliability was adequate and higher than what has been reported for unsupervised written submissions. Results from the decision study suggest that three raters with four items provide adequate reliability. Correlations with interviews and grade point average provide evidence of construct validity. A time-limited essay with a clear scoring protocol results in adequate reliability and some validity.

Aptitude, achievement and competence in medicine: a latent variable path model

To develop and test a latent variable path model of general achievement, aptitude for medicine and competence in medicine employing data from the Medical College Admission Test (MCAT), pre-medical undergraduate grade point average (UGPA) and demographic characteristics for competence in pre-clinical and measures of competence (United States Licensure Examination {USMLE} Steps 1, 2, and 3). Data were gathered on 839,710 participants from 1991 to 2000 on demographic and school variables, UGPA, MCAT subtest scores and Steps 1, 2, and 3 of the United Stated Licensure Examination (USMLE). However, subsets of the total 839,710 participants included in the database were used for various analyses and the testing of a latent variable path model (LVPA). A number of preliminary descriptive and inferential techniques were used to confirm previous hypotheses and stated relationships amongst the variables of interest to the present study. Through development and testing of a latent variable path model, three latent variables measured by UGPA (general achievement), subscales of the MCAT (aptitude for medicine), and Steps 1, 2, and 3 of the USMLE (competence in medicine) were identified which resulted in a comparative fit index = .932 of the model to a large sample (n = 20,714). In a confirmatory latent variable path model we were able to identify theoretical constructs, aptitude for medicine, general achievement, and competence in medicine and their interrelationships. These are distinct but interrelated latent variables.

Medical school curricula: do curricular approaches affect competence in medicine?

BACKGROUND AND OBJECTIVES

US medical school curricula continually undergo reform. The effect of formal curricular approaches (course organization and pedagogical techniques) on competence in medicine as measured by the United States Medical Licensing Examinations (USMLE) Step 1, 2, and 3 is not fully understood. The purpose of this study was to investigate the effects of formal curricular approaches in a latent variable path analysis model of achievement-aptitude-competence in medicine.

METHODS

Using Association of American Medical Colleges (AAMC) and USMLE longitudinal data (1994-2004) for 116 medical schools, structural equation modeling was used to study latent variable path models assessing the impact of curriculum on competence in medicine (n=9,332).

RESULTS

A latent variable path model consisting of three latent variables measured by undergraduate grade point average (general achievement), Medical College Admission Test subscores (aptitude for medicine), and USMLE Step 1-3 (competence in medicine) was used to assess the impact of curriculum on competence in medicine. Two models were tested; one resulted in a Comparative Fit Index=.931 with a path coefficient of 0.04 from curriculum to competence in medicine. While there was a good fit of the data to the final model, the type of school curriculum did not significantly influence competence in medicine since it accounted for less than 1% of the variation in student performance on the USMLE.

CONCLUSIONS

Various formal curricular approaches have little differential effect on students' performance on the USMLE.

Validity, reliability, and defensibility of assessments in veterinary education

In this article, we provide an introduction to and overview of issues of validity, reliability, and defensibility related to measurement of student performance in veterinary medical education. Validity has to do with the extent to which the instrument measures whatever it is supposed to measure, reliability has to do with the consistency of measurement, and defensibility deals with the appropriate standards of the assessment procedure. An explanation of the methods that can be used to determine reliability and validity are given and examples of how they have been used in recent research findings are provided. Veterinary educators should have an understanding of each of these concepts because evaluation of veterinary students and performance measures provides evidence of utility of assessment tools, informs teaching practices, and can guide curriculum development and revision.

Assessment of applicants to the veterinary curriculum using a multiple mini-interview method

This study describes the development, implementation, and psychometric assessment of the multiple mini-interview (MMI) for the inaugural class of veterinary medicine applicants at the University of Calgary Faculty of Veterinary Medicine (UCVM). The MMI is a series of approximately five to 12 10-minute interviews that consist of situational events. Applicants are given a scenario and asked to work through an issue or behavioral-type questions that are meant to assess one attribute (e.g., empathy) at a time. This structure allows for multiple assessments of the applicants by trained interviewers on the same questions. MMI scenario development was based on a review of the noncognitive attributes currently assessed by the 31 veterinary schools across Canada and the United States and the goals and objectives of UCVM. The noncognitive attributes of applicants (N=110) were assessed at five stations, by two interviewers within each station, on three items using a standardized rating form on an anchored 1-5 scale. The method was determined to be reliable (G-coefficient=0.88) and demonstrated evidence of validity. The MMI score did not correlate with grade-point average (r=0.12, p=0.22). While neither the applicants nor interviewers had participated in an MMI format before, both groups reported the process to be acceptable in a post-interview questionnaire. This analysis provides preliminary evidence of the reliability, validity, and acceptability of the MMI in assessing the noncognitive attributes of applicants for veterinary medical school admissions.

Xenon anesthesia impairs hepatic oxygenation and perfusion in healthy pigs

BACKGROUND

Over the last 15 years, there has been growing interest in the noble gas xenon as a new inhalational anesthetic. This is due to its favorable pharmacological properties such as short onset and offset, as well as its hemodynamic stability. However, most volatile anesthetics appear to play an important role in the multi-factorial etiology of perioperative liver injury by decreasing liver blood flow with a subsequent reduction of hepatic oxygen supply. However, the effects of the anesthetic gas xenon on hepatic perfusion and oxygenation have not been completely investigated.

METHODS

Following ethical approval, 18 anesthetized and acutely monitored pigs were randomly assigned to the two following groups: 9 animals received xenon anesthesia in increasing inspiratory concentrations of 0%, 20%, 50%, and 65% in addition to their basic intravenous anesthesia; 9 animals served as a control group. Measurement points for systemic and regional hemodynamic and oxygenation parameters were performed 30 min after changing the xenon concentration.

RESULTS

Xenon elicited dose-dependent systemic hemodynamic changes such that the mean arterial pressure did not change, while the heart rate and cardiac output decreased by about 30%, thereby indicating an increase in the systemic vascular resistance. Portal venous blood flow decreased, while hepatic arterial blood flow was unchanged. The oxygen supply of the liver was reduced, but not the rate of indocyanine plasma disappearance from the liver. Furthermore, the increase of liver surface pO2 to systemic hyperoxia was absent, and hepatic lactate uptake was reduced.

CONCLUSION

Xenon, in addition to basic intravenous anesthesia, elicited a decrease in heart rate and cardiac output and an increase in mean arterial pressure. Similar to volatile anesthetics, xenon does reduce portal venous flow and influences hepatic tissue oxygenation. In contrast, hepatic arterial blood flow remains stable in the presence of xenon, and no changes in the hepatic arterial buffer responses were evident. Xenon does affect hepatic perfusion and oxygenation.

How much do differences in medical schools influence student performance? A longitudinal study employing hierarchical linear modeling

BACKGROUND

Medical school curricula have undergone considerable change in the past half century. There is little evidence, however, for the impact of various curricula and educational policies on student learning once incoming performance and the nonrandom nature of students nested within schools has been accounted for.

PURPOSE

To investigate effects of school variables on United States Medical Licensing Examination (USMLE) Step 1-3 scores over an 11-year period (1994-2004).

METHODS

Using Association of American Medical Colleges and USMLE longitudinal data for 116 medical schools, hierarchical linear modeling was used to study the effects of school variables on Step 1-3.

RESULTS

Mean unadjusted between school variance was 14.74%, 10.50%, and 11.25%, for USMLE Step 1-3. When student covariates were included, between-school variation was less than 5%. The variance accounted for in student performance by the student covariates ranged from 27.58% to 36.51% for Step 1,16.37% to 24.48% for Step 2 and 19.22% to 25.32% for Step 3.The proportion of the between-school variance that was accounted for by the student covariates ranged between 81.22% and 88.26% for Step 1, 48.44% and 79.77% for Step 2, and 68.41% and 80.78% for Step 3 [corrected]. School-level variables did not consistently predict for adjusted mean school Step performance.

CONCLUSIONS

Individual student differences account for most of the variation in USMLE performance with small contributions from between-school variation and even smaller contribution from curriculum and educational policies.

Patients' self-evaluation after 4-12 weeks following xenon or propofol anaesthesia

BACKGROUND AND OBJECTIVE

The aim of this study was to assess postoperative patients' self-evaluation after xenon anaesthesia compared to total intravenous anaesthesia with propofol.

METHODS

160 patients aged 18-60 yr, ASA I-II undergoing elective surgery took part in this randomised-controlled trial. After approval by the local Ethics Committee and as soon as the patients had given their written informed consent, they were randomly allocated to either the xenon (n = 80) or propofol (n = 80) group. In both groups remifentanil was used as opioid. The postoperative patients' self-evaluation was assessed with a double-blind telephone poll. Early spatial orientation, patients' self-evaluation of anaesthesia, choice of the same anaesthesia for future operations and recall of uncomfortable feelings after anaesthesia were determined.

RESULTS

116 Patients were analysed, 63 in the xenon and 53 in the propofol group. The two study groups were comparable with respect to age, weight, height, gender and ASA classification. The two groups indicated similar values in the early spatial orientation at the onset of recovery and thereafter. Patients' self-evaluation of anaesthesia with main emphasis at high marks and repetition of the same anaesthesia if necessary were similar in both groups. Recalls of uncomfortable feelings were comparable but not for postoperative pain and appetite/thirst which appeared with a significantly higher incidence in the xenon group.

CONCLUSIONS

Patients' self-evaluation and memory of early spatial orientation following xenon anaesthesia are comparable to propofol.

Randomized controlled trial of the haemodynamic and recovery effects of xenon or propofol anaesthesia

BACKGROUND

There is limited clinical experience with xenon in a large number of patients. We present intra- and postoperative haemodynamic and recovery data comparing xenon and total intravenous anaesthesia with propofol.

METHODS

A total of 160 patients aged 18-60 years (ASA I and II) undergoing elective surgery took part in this prospective non-blinded randomized controlled trial. After local ethics committee approval and written informed consent, patients were allocated randomly to either the xenon or the propofol group. Anaesthesia was induced with propofol and remifentanil and was maintained with xenon at 60% (minimal alveolar concentration 0.95) or with propofol 0.1-0.12 mg kg(-1) min(-1). Remifentanil was titrated to clinical need in both groups.

RESULTS

The two study groups were comparable with respect to age, weight, height, gender and ASA classification. Baseline in heart rate and systolic arterial pressure (SAP) were comparable in both groups. Following induction, SAP initially decreased but returned to baseline values over 15 min in the xenon group and differed significantly from the propofol group. Heart rate decreased significantly only in the xenon group and remained at stable values. Occurrence and duration of hypertension, hypotension and bradycardia showed no significant difference between groups. Patient recovery time in the post-anaesthetic care unit and recovery from anaesthesia was similar in the two groups.

CONCLUSIONS

After induction the xenon/opioid regimen maintains systolic blood pressure at baseline levels and a low heart rate. No differences between groups were found in haemodynamic stability during anaesthesia. Recovery from xenon anaesthesia was similar to that observed in the propofol group.

Effects of xenon anaesthesia on intestinal oxygenation in acutely instrumented pigs

BACKGROUND

Xenon is a narcotic gas that might be able to replace volatile anaesthetics or nitrous oxide due to its favourable pharmacological properties, such as providing haemodynamic stability. Intestinal oxygenation is affected by most volatile anaesthetics as a result of cardiodepressive effects. Reducing oxygenation of the gut might be a factor leading to perioperative organ dysfunction. This animal study was designed to assess the effects of xenon on intestinal oxygenation.

METHODS

After ethical approval, 24 anaesthetized, acutely instrumented pigs were randomly assigned to three groups: nine animals received xenon anaesthesia with inspiratory concentrations of 0, 20, 50 and 65% in addition to their basic i.v. anaesthesia, nine animals served as a study control group, and five animals were used to assess model stability. Measurement of systemic and regional haemodynamic and oxygenation parameters was made 30 min after changing the xenon concentration.

RESULTS

Xenon elicited dose-dependent systemic haemodynamic changes: heart rate and cardiac output decreased by 30%, while mean arterial pressure was stable. Superior mesenteric artery blood flow was lower in the xenon group. Vascular resistance of the superior mesenteric artery increased. The small intestinal oxygen supply decreased with increasing xenon concentration; the mucosal tissue oxygen partial pressure decreased but did not reach hypoxic (<5 mm Hg) values. Serosal tissue oxygen partial pressure was maintained.

CONCLUSIONS

Xenon, in addition to basic i.v. anaesthesia, elicited a decrease in cardiac output and maintained mean arterial pressure. Intestinal oxygenation was maintained, although regional macrohaemodynamic perfusion decreased. Xenon does not impair intestinal oxygenation under physiological conditions.

Xenon, a modern anaesthesia gas

Xenon is an interesting anesthetic as it appears to lack negative inotropicy and vasodilatation, giving great advantages to both patients with limited cardiovascular reserve or those who require hemodynamic stability. It has low toxicity and is not teratogenic. Xenon gives rapid induction and recovery, due to its low blood/gas partition coefficient (0.15), and has a MAC of 63%. Several vitro studies showed that Xenon may protect neural cells against ischaemic injury. Its low blood solubility can take to diffusion hypoxia if Xenon is not substituted by 100% oxygen at the end of anesthesia. It has been shown that, compared to other anesthetic regimens, Xenon anesthesia produces the highest regional blood flow in the brain, liver, kidney and intestine. In conclusion, the most important positive effects of Xenon are cardiovascular stability, cerebral protection and favourable pharmacokinetics. Negative points are high cost and the limited number of ventilators supplying Xenon.

Increased airway resistance during xenon anaesthesia in pigs is attributed to physical properties of the gas

BACKGROUND

In this study we investigated the effects of the physical properties of xenon on respiratory mechanisms in pigs.

METHODS

With institutional approval, 10 female pigs (mean 25.2 (SD 2.5) kg) were anaesthetized with thiopental, remifentanil, and pancuronium. Gas flow and pressure were recorded continuously at the proximal end of the tracheal tube during constant flow ventilation for control, with 100% oxygen (control), followed by 1.5% isoflurane in 70/30% nitrogen/oxygen, 1.0% isoflurane in 70/30% nitrous oxide/oxygen, and 70/30% xenon/oxygen in random order. Compliance (C) and resistance (R) were calculated using a single compartment model. Resistance was corrected for gas viscosities eta and also for densities pho and viscosities eta as (pho*eta)(1/2) to compare assumptions of laminar and mixed flow in the airways.

RESULTS

With constant flow ventilation, xenon increases inspiratory pressure compared with other gas mixtures. There were no significant differences in resistance, corrected for laminar or mixed flow, between the gas mixtures. Xenon anaesthesia did not affect compliance.

CONCLUSIONS

The increase in airway pressure observed with xenon anaesthesia is attributed completely to its higher density and viscosity. Therefore, determination of airway resistance must take into account the physical properties of the gas. Xenon does not exert any major effect on airway diameter.

Hosts manipulated by one parasite incur additional costs from infection by another parasite

Manipulation of host phenotype by parasites often serves to increase the predation rate of definitive hosts on intermediate hosts. For intermediate hosts, the indirect consequences of manipulation may extend beyond the direct increase in predation, however. Metacercariae of the trematode Curtuteria australis encyst in the foot of New Zealand cockles, Austrovenus stutchburyi, and stunt its growth, rendering cockles incapable of burrowing into the sediments. Here, we show that cockles manipulated by C. australis are 5 times more likely to be infected by the castrating sporocysts of another trematode than normal, nonmanipulated cockles. Our results indicate that the consequences for C. australis-manipulated cockles are far more important than a simple increase in the risk of predation and that indirect repercussions of manipulation can be as severe as direct ones.