Mapping hospital data to characterize residents' educational experiences

BACKGROUND

Experiential learning through patient care is fundamental to graduate medical education. Despite this, the actual content to which trainees are exposed in clinical practice is difficult to quantify and is poorly characterized. There remains an unmet need to define precisely how residents' patient care activities inform their educational experience.  METHODS: Using a recently-described crosswalk tool, we mapped principal ICD-10 discharge diagnosis codes to American Board of Internal Medicine (ABIM) content at four training hospitals of a single Internal Medicine (IM) Residency Program over one academic year to characterize and compare residents' clinical educational experiences. Frequencies of broad content categories and more specific condition categories were compared across sites to profile residents' aggregate inpatient clinical experiences and drive curricular change.

RESULTS

There were 18,604 discharges from inpatient resident teams during the study period. The crosswalk captured > 95% of discharges at each site. Infectious Disease (ranging 17.4 to 39.5% of total discharges) and Cardiovascular Disease (15.8 to 38.2%) represented the most common content categories at each site. Several content areas (Allergy/Immunology, Dermatology, Obstetrics/Gynecology, Ophthalmology, Otolaryngology/Dental Medicine) were notably underrepresented (≤ 1% at each site). There were significant differences in the frequencies of conditions within most content categories, suggesting that residents experience distinct site-specific clinical content during their inpatient training.

CONCLUSIONS

There were substantial differences in the clinical content experienced by our residents across hospital sites, prompting several important programmatic and curricular changes to enrich our residents' hospital-based educational experiences.

Experience and Education in Residency Training: Capturing the Resident Experience by Mapping Clinical Data

PROBLEM

Internal medicine training programs operate under the assumption that the 3-year residency training period is sufficient for trainees to achieve the depth and breadth of clinical experience necessary for independent practice; however, the medical conditions to which residents are exposed in clinical practice are not easily measured. As a result, residents' clinical educational experiences are poorly understood.

APPROACH

A crosswalk tool (a repository of International Classification of Diseases [ICD]-10 codes linked to medical content areas) was developed to query routinely collected inpatient principal diagnosis codes and translate them into an educationally meaningful taxonomy. This tool provides a robust characterization of residents' inpatient clinical experiences.

OUTCOMES

This pilot study has provided proof of principle that the crosswalk tool can effectively map 1 year of resident-attributed diagnosis codes to both the broad content category level (e.g., "cardiovascular disease") and to the more specific condition category level (e.g., "myocardial disease"). The authors uncovered content areas in their training program that are overrepresented and some that are underrepresented relative to material on the American Board of Internal Medicine (ABIM) Certification Exam.

NEXT STEPS

The crosswalk tool introduced here translated residents' patient care activities into discrete, measurable educational content and enabled 1 internal medicine residency program to characterize residents' inpatient educational experience with a high degree of resolution. Leaders of other programs seeking to profile the clinical exposure of their trainees may adopt this strategy. Such clinical content mapping drives innovation in the experiential curriculum, enables comparison across practice sites, and lays the groundwork to test associations between individual clinical exposure and competency-based outcomes, which, in turn, will allow medical educators to draw conclusions regarding how clinical experience reflects clinical competency.

Mapping the Clinical Experience of a New York City Residency Program During the COVID-19 Pandemic

The COVID-19 pandemic has dramatically disrupted the educational experience of medical trainees. However, a detailed characterization of exactly how trainees' clinical experiences have been affected is lacking. Here, we profile residents' inpatient clinical experiences across the four training hospitals of NYU's Internal Medicine Residency Program during the pandemic's first wave. We mined ICD-10 principal diagnosis codes attributed to residents from February 1, 2020, to May 31, 2020. We translated these codes into discrete medical content areas using a newly developed "crosswalk tool." Residents' clinical exposure was enriched in infectious diseases (ID) and cardiovascular disease content at baseline. During the pandemic's surge, ID became the dominant content area. Exposure to other content was dramatically reduced, with clinical diversity repopulating only toward the end of the study period. Such characterization can be leveraged to provide effective practice habits feedback, guide didactic and self-directed learning, and potentially predict competency-based outcomes for trainees in the COVID era.

Collaborating Across Private, Public, Community, and Federal Hospital Systems: Lessons Learned from the Covid-19 Pandemic Response in NYC

In the spring of 2020, New York City became the epicenter of the Covid-19 pandemic in the United States. During the peak from March 10 to May 1, the hospitals of the Department of Medicine at New York University Grossman School of Medicine — including an academic, private hospital in a community setting, a private hospital, an affiliated public hospital, and an affiliated Veterans Affairs federal hospital — cared for more than 5,000 hospitalized patients with Covid-19. Each hospital encountered unique challenges based on its own resources, affiliations, size, and patient populations. However, with ongoing collaboration, leadership developed protocols applicable across sites. Despite these four hospitals being distinct, these collaborations resulted in many strategies that can be applied to a wide variety of medical centers that must rapidly respond to the unprecedented challenges created by the Covid-19 pandemic. In this article, the authors describe strategies for communication, surge planning, clinical care, and staff wellness.

Theory-guided teaching: Implementation of a clinical reasoning curriculum in residents

Introduction: Educators have theorized that interventions grounded in dual process theory (DPT) and script theory (ST) may improve the diagnostic reasoning process of physicians but little empirical evidence exists. Methods: In this quasi-experimental study, we assessed the impact of a clinical reasoning (CR) curriculum grounded in DPT and ST on medicine residents participating in one of three groups during a 6-month period: no, partial, or full intervention. Residents completed the diagnostic thinking inventory (DTI) at baseline and 6 months. At 6 months, participants also completed a post-survey assessing application of concepts to cases. Results: There was a significant difference between groups in application of concepts (no intervention 1.6 (0.65) compared to partial 2.3 (0.81) and full 2.2 (0.91), p = 0.05), as well as describing cases in problem representation format (no intervention 1.2 (0.38) and partial 1.5 (0.55) compared to full 2.1 (0.93), p = 0.004). There was no significant difference in change in DTI scores (no intervention 7.0 (16.3), partial 8.8 (9.8), full 7.8 (12.0)). Conclusions: Residents who participated in a CR curriculum grounded in DPT and ST were effective in applying principles of CR in cases from their practice. To our knowledge, this is the first workplace-based CR educational intervention study showing differences in the reasoning process residents apply to patients.

Faculty feedback that begins with resident self-assessment: motivation is the key to success

CONTEXT

The seeking and incorporating of feedback are necessary for continuous performance improvement in medicine. We know that beginning feedback conversations with resident self-assessment may reduce some of the tensions experienced by faculty staff. However, we do not fully understand how residents experience feedback that begins with self-assessment, and whether any existing theoretical frameworks can explain their experiences.

METHODS

We conducted a constructivist grounded theory study exploring physical medicine and rehabilitation residents' experiences as they engaged in a structured self-assessment and faculty staff feedback programme. Utilising purposive sampling, we conducted 15 individual interviews and analysed verbatim transcripts iteratively. We implemented several procedures to enhance the credibility of the findings and the protection of participants during recruitment, data collection and data analysis. After defining the themes, we reviewed a variety of existing frameworks to determine if any fitted the data.

RESULTS

Residents valued self-assessment followed by feedback (SAFF) and had clear ideas of what makes the process useful. Time pressures and poor feedback quality could lead to a process of 'just going through the motions'. Motivation coloured residents' experiences, with more internalised motivation related to a more positive experience. There were no gender- or year of training-related patterns.

CONCLUSIONS

Self-determination theory provided the clearest lens for framing our findings and fitted into a conceptual model linking the quality of the SAFF experience and residents' motivational loci. We identified several study limitations including time in the field, evolving characteristics of the SAFF programme and the absence of faculty voices. We believe that by better understanding residents' experiences of SAFF, educators may be able to tailor the feedback process, enhance clinical performance and ultimately improve patient care.

The Courteous Consult: A CONSULT Card and Training to Improve Resident Consults

BACKGROUND

Communication and courtesy are important elements of consultations, but there is limited published data about the quality of trainee consults.

OBJECTIVES

This study assessed residents' views on consult interactions, evaluated the impact of the consult interactions on patient care, and developed and implemented a pocket card and training on trainee consults.

METHODS

We surveyed resident and fellow physicians at Mount Sinai Hospital to assess perceptions, created a CONSULT card that uses a mnemonic for key elements, and developed a training session for how to call consults. We also conducted a consult training session using the CONSULT card as part of orientation in 2011 for all interns. We assessed the acceptability, feasibility, and sustainability of this intervention.

RESULTS

Of 1001 trainees, 403 (40%) responded. Respondents reported that the most important components of calling consults included giving patient name, medical record number, and location (91%), and giving a clear question/reason (89%). Respondents also reported that these behaviors are done consistently for only 64%, and 10% of consults, respectively. Trainees reported that consult interactions affect the timeliness of treatment (62%), timeliness of tests performed (57%), appropriateness of diagnosis (56%), and discharge planning (49%). Approximately 300 interns attended the consult training session, and their feedback demonstrated acceptability and utility of the session.

CONCLUSIONS

Trainees believe that consult interactions impact patient care, but important components of the consult call are often missing. Our training and CONSULT card is an acceptable, feasible, and novel training intervention. Once developed, the training session and CONSULT card require minimal faculty time to deliver.

What factors create a humanistic doctor? A nationwide survey of fourth-year medical students

PURPOSE

The authors sought to develop a conceptual framework of the factors that most influence medical students' development of humanism and to explore students' opinions regarding the role these factors play in developing or inhibiting humanism.

METHOD

In 2006-2007, the authors conducted 16 focus groups with fourth-year students and first-year residents at four universities to design a conceptual framework. They used the framework to develop a survey, which they administered to fourth-year medical students at 20 U.S. medical schools in 2007-2008.

RESULTS

Data from 80 focus-group participants suggested that the key influences on students' development of humanism were their authentic, unique, and participatory experiences before and during medical school, and the opportunity to process these experiences. Students who completed the survey (N = 1,170) reported that experiences of greatest intensity (e.g., being involved in a case where the patient dies), participatory learning experiences (e.g., volunteer work, international clinical rotations), and positive role models had the greatest effect on their development of humanism, whereas stressful conditions, such as a busy workload or being tired or postcall, inhibited their humanism. Women and students going into primary care placed significantly greater importance on experiences promoting humanism than did men and those not going into primary care. In addition, students with lower debt burdens viewed such experiences as more important than did those with higher debt burdens.

CONCLUSIONS

Students viewed a variety of factors as influencing their development of humanism. This research provides a starting point for enhancing curricula to promote humanism.

International medical school faculty development: the results of a needs assessment survey among medical educators in China

To explore the need for faculty development among Chinese medical educators. Leaders at each medical school in China were asked to complete a 123-item survey to identify interest in various topics and barriers and perceived benefits of participating in faculty development programs. Interest levels were high for all topics. Experience with Hospital Management and Research positively correlated with interest in learning more (p < 0.001). Ninety-two percent believe that international experiences are very or extremely important to medical educators' career advancement. Chinese medical education faculty members have a strong interest in faculty development programs.

Critical events in the lives of interns

BACKGROUND

Early residency is a crucial time in the professional development of physicians. As interns assume primary care for their patients, they take on new responsibilities. The events they find memorable during this time could provide us with insight into their developing professional identities.

OBJECTIVE

To evaluate the most critical events in the lives of interns.

PARTICIPANTS

Forty-one internal medicine residents at one program participated in a two-day retreat in the fall of their first year. Each resident provided a written description of a recent high point, low point, and patient conflict.

MEASUREMENTS

We used a variant of grounded theory to analyze these critical incidents and determine the underlying themes of early internship. Independent inter-rater agreement of >90% was achieved for the coding of excerpts.

MAIN RESULTS

The 123 critical incidents were clustered into 23 categories. The categories were further organized into six themes: confidence, life balance, connections, emotional responses, managing expectations, and facilitating teamwork. High points were primarily in the themes of confidence and connections. Low points were dispersed more generally throughout the conceptual framework. Conflicts with patients were about negotiating the expectations inherent in the physician-patient relationship.

CONCLUSION

The high points, low points, and conflicts reported by early residents provide us with a glimpse into the lives of interns. The themes we have identified reflect critical challenges interns face the development of their professional identity. Program directors could use this process and conceptual framework to guide the development and promotion of residents' emerging professional identities.

Quality of life, optimism/pessimism, and knowledge and attitudes toward HIV Screening among pregnant women in Ghana

OBJECTIVE

We sought to explore optimism/pessimism, knowledge of HIV, and attitudes toward HIV screening and treatment among Ghanaian pregnant women.

METHOD

Pregnant women in Accra, Ghana, completed a self-administered questionnaire including the Life Orientation Test-Revised (LOT-R, an optimism/pessimism measure), an HIV knowledge and screening attitudes questionnaire, the Short Form 12 (SF-12, a measure of health-related quality of life [HRQOL]), and a demographic questionnaire. Data were analyzed using t-tests, ANOVA, correlations, and the chi2 test.

RESULTS

There were 101 participants; 28% were nulliparous. Mean age was 29.7 years, and mean week of gestation was 31.8. All women had heard of AIDS, 27.7% had been tested for HIV before this pregnancy, 46.5% had been tested during this pregnancy, and 59.4% of the sample had ever been tested for HIV. Of those not tested during this pregnancy, 64.2% were willing to be tested. Of all respondents, 89% said they would get tested if antiretroviral drugs (ARVs) were readily available and might prevent maternal-to-child transmission. Neither optimism/pessimism nor HRQOL was associated with attitudes toward HIV screening. Optimism was negatively correlated with HIV knowledge (p = .001) and was positively correlated with having never been tested before this pregnancy (p = .007).

CONCLUSION

The relationship between optimism/pessimism and HIV knowledge and screening behavior is worthy of further study using larger samples and objective measures of testing beyond self-report.

Teaching humanism

As the "passion that animates authentic professionalism," humanism must be infused into medical education and clinical care as a central feature of medicine's professionalism movement. In this article, we discuss a current definition of humanism in medicine. We will also provide detailed descriptions of educational programs intended to promote humanism at a number of medical schools in the United States (and beyond) and identify the key factors that make these programs effective. Common elements of programs that effectively teach humanism include: (1) opportunities for students to gain perspective in the lives of patients; (2) structured time for reflection on those experiences; and (3) focused mentoring to ensure that these events convert to positive, formative learning experiences. By describing educational experiences that both promote and sustain humanism in doctors, we hope to stimulate the thinking of other medical educators and to disseminate the impact of these innovative educational programs to help the profession meet its obligation to provide the public with humanistic physicians.

Can there be a single system for peer assessment of professionalism among medical students? A multi-institutional study

PURPOSE

Peer assessment is a valuable source of information about medical students' professionalism. How best to facilitate peer assessment of students' professional behavior remains to be answered, however. This report extends previous research through a multi-institutional study of students' perspectives about system characteristics for peer assessment of professionalism. It examines whether students from different schools and year levels prefer different characteristics of peer assessment to assess each other candidly, or whether a single system can be designed. It then identifies the characteristics of the resulting preferred system(s).

METHOD

At the beginning of academic year 2004-2005, students (1,661 of 2,115; 78%) in years one through four at four schools replied to a survey about which peer assessment characteristics - related to, for example, who receives the assessment, its anonymity, and timing - would prevent or encourage their participation. Multivariate analysis of variance was used to detect differences among institutions and students from each year level.

RESULTS

Students across year levels and schools generally agreed about the characteristics of peer assessment. They prefer a system that is 100% anonymous, provides immediate feedback, focuses on both unprofessional and professional behaviors, and uses peer assessment formatively while rewarding exemplary behavior and addressing serious repetitive professional lapses. The system, they emphasize, must be embedded in a supportive environment.

CONCLUSIONS

Students' agreement about peer-assessment characteristics suggests that one system can be created to meet the majority of students' preferences. Once implemented, the system should be monitored for student acceptability to maximize participation and to determine the formative and summative value of the process.

Patients speak: what's really important about bedside interactions with physician teams

BACKGROUND

Although it may take up relatively little time, hospitalized patients' interactions with their physicians are important elements of their hospital experience.

PURPOSE

We conducted a qualitative study to explore what is important to patients about bedside interactions with their physician teams.

METHODS

We interviewed medical inpatients and used grounded theory methodology to analyze interview transcripts.

RESULTS

We recruited 17 patients from an academic medical center including a university hospital and a Veterans Administration (VA) hospital. We found that important aspects of patient-team interactions included the exchange of information, evidence of caring from their team, involvement in teaching, knowing the team members, and bedside manner. Patients also described team characteristics that were important to them such as team attributes and intrateam collaboration.

CONCLUSIONS

This model clarifies the aspects of bedside interactions that patients value and will allow physicians to focus their limited time to the benefit of their patients.

Ethics and professionalism: what does a resident need to learn?

Training in ethics and professionalism is a fundamental component of residency education, yet there is little empirical information to guide curricula. The objective of this study is to describe empirically derived ethics objectives for ethics and professionalism training for multiple specialties. Study design is a thematic analysis of documents, semi-structured interviews, and focus groups conducted in a setting of an academic medical center, Veterans Administration, and community hospital training more than 1000 residents. Participants were 84 informants in 13 specialties including residents, program directors, faculty, practicing physicians, and ethics committees. Thematic analysis identified commonalities across informants and specialties. Resident and nonresident informants identified consent, interprofessional relationships, family interactions, communication skills, and end-of-life care as essential components of training. Nonresidents also emphasized formal ethics instruction, resource allocation, and self-monitoring, whereas residents emphasized the learning environment and resident-attending interactions. Conclusions are that empirically derived learning needs for ethics and professionalism included many topics, such as informed consent and resource allocation, relevant for most specialties, providing opportunities for shared curricula and resources.

Setting school-level outcome standards

BACKGROUND

To establish international standards for medical schools, an appropriate panel of experts must decide on performance standards. A pilot test of such standards was set in the context of a multidimensional (multiple-choice question examination, objective structured clinical examination, faculty observation) examination at 8 leading schools in China.

METHODS

A group of 16 medical education leaders from a broad array of countries met over a 3-day period. These individuals considered competency domains, examination items, and the percentage of students who could fall below a cut-off score if the school was still to be considered as meeting competencies. This 2-step process started with a discussion of the borderline school and the relative difficulty of a borderline school in achieving acceptable standards in a given competency domain. Committee members then estimated the percentage of students falling below the standard that is tolerable at a borderline school and were allowed to revise their ratings after viewing pilot data.

RESULTS

Tolerable failure rates ranged from 10% to 26% across competency domains and examination types. As with other standard-setting exercises, standard deviations from initial to final estimates of the tolerable failure rates fell, but the cut-off scores did not change significantly. Final, but not initial cut-off scores were correlated with student failure rates (r = 0.59, P = 0.03).

DISCUSSION

This paper describes a method to set school-level outcome standards at an international level based on prior established standard-setting methods. Further refinement of this process and validation using other examinations in other countries will be needed to achieve accurate international standards.

Disciplinary action by medical boards and prior behavior in medical school

BACKGROUND

Evidence supporting professionalism as a critical measure of competence in medical education is limited. In this case-control study, we investigated the association of disciplinary action against practicing physicians with prior unprofessional behavior in medical school. We also examined the specific types of behavior that are most predictive of disciplinary action against practicing physicians with unprofessional behavior in medical school.

METHODS

The study included 235 graduates of three medical schools who were disciplined by one of 40 state medical boards between 1990 and 2003 (case physicians). The 469 control physicians were matched with the case physicians according to medical school and graduation year. Predictor variables from medical school included the presence or absence of narratives describing unprofessional behavior, grades, standardized-test scores, and demographic characteristics. Narratives were assigned an overall rating for unprofessional behavior. Those that met the threshold for unprofessional behavior were further classified among eight types of behavior and assigned a severity rating (moderate to severe).

RESULTS

Disciplinary action by a medical board was strongly associated with prior unprofessional behavior in medical school (odds ratio, 3.0; 95 percent confidence interval, 1.9 to 4.8), for a population attributable risk of disciplinary action of 26 percent. The types of unprofessional behavior most strongly linked with disciplinary action were severe irresponsibility (odds ratio, 8.5; 95 percent confidence interval, 1.8 to 40.1) and severely diminished capacity for self-improvement (odds ratio, 3.1; 95 percent confidence interval, 1.2 to 8.2). Disciplinary action by a medical board was also associated with low scores on the Medical College Admission Test and poor grades in the first two years of medical school (1 percent and 7 percent population attributable risk, respectively), but the association with these variables was less strong than that with unprofessional behavior.

CONCLUSIONS

In this case-control study, disciplinary action among practicing physicians by medical boards was strongly associated with unprofessional behavior in medical school. Students with the strongest association were those who were described as irresponsible or as having diminished ability to improve their behavior. Professionalism should have a central role in medical academics and throughout one's medical career.

Maximizing participation in peer assessment of professionalism: the students speak

BACKGROUND

Medical students have unique information about peers' professionalism but are reluctant to share it through peer assessment.

METHOD

Students (231 of 375; 62%) in one school replied to a survey about whether various characteristics of peer assessment (e.g., who receives the assessment, its anonymity, implications for the classmate) would prevent or encourage their participation.

RESULTS

Sixty-six percent of the students agreed that there should be peer assessment of professionalism as long as the assessment reflected their preferences for how the assessment should take place. Some of their preferences included reporting unprofessional behavior to an impartial counselor, a 100% anonymous process, and having the classmate receive corrective instruction. Students across year levels generally agreed about the characteristics of peer assessment. Men and women disagreed about some characteristics.

CONCLUSION

Most students are willing to participate in peer assessment as long as their preferences are taken into consideration.

Medical students' views on peer assessment of professionalism

BACKGROUND

Although peer assessment holds promise for assessing professionalism, reluctance and refusal to participate have been noted among learners and practicing physicians. Understanding the perspectives of potential participants may therefore be important in designing and implementing effective peer assessment.

OBJECTIVE

To identify factors that, according to students themselves, will encourage or discourage participation in peer assessment.

DESIGN

A qualitative study using grounded theory to interpret views shared during 16 focus groups that were conducted by leaders using a semi-structured guide.

PARTICIPANTS

Sixty-one students in Years 1, 3, and 4 in 2 mid-western public medical schools.

RESULTS

Three themes students say would promote or discourage peer assessment emerged: personal struggles with peer assessment, characteristics of the assessment system itself, and the environment in which the system operates. Students struggle with reporting an unprofessional peer lest they bring harm to the peer, themselves, or their clinic team or work group. Who receives the assessment and gives the peer feedback and whether it is formative or summative and anonymous, signed, or confidential are important system characteristics. Students' views of characteristics promoting peer assessment were not unanimous. Receptivity to peer reports and close positive relationships among students and between students and faculty mark an environment conducive to peer assessment, students say.

CONCLUSIONS

The study lays a foundation for creating acceptable peer assessment systems among students by soliciting their views. Merely introducing an assessment tool will not result in students' willingness to assess each other.

The economic impact of quarantine: SARS in Toronto as a case study

Objectives

Over time, quarantine has become a classic public health intervention and has been used repeatedly when newly emerging infectious diseases have threatened to spread throughout a population. Here, we weigh the economic costs and benefits associated with implementing widespread quarantine in Toronto during the SARS outbreaks of 2003.

Methods

We compared the costs of two outbreak scenarios: in Scenario A, SARS is able to transmit itself throughout a population without any significant public health interventions. In Scenario B, quarantine is implemented early on in an attempt to contain the virus. By evaluating these situations, we can investigate whether or not the use of quarantine is justified by being either cost-saving, life saving, or both.

Results

Our results indicate that quarantine is effective in containing newly emerging infectious diseases, and also cost saving when compared to not implementing a widespread containment mechanism.

Conclusions

This paper illustrates that it is not only in our humanitarian interest for public health and healthcare officials to remain aggressive in their response to newly emerging infections, but also in our collective economic interest. Despite somewhat daunting initial costs, quarantine saves both lives and money.

Initial use of a novel instrument to measure professionalism in surgical residents

BACKGROUND

No universally accepted method to measure professionalism exists. We developed an instrument to measure specific aspects of professionalism in surgical residents.

METHODS

Professionalism was deconstructed into 15 domains. Behavioral descriptors were determined for extreme and selected intermediate anchors. It became evident that residents could "go too far" in some professional behaviors. Therefore, although a 7-point continuous ordinal scale forms the framework, a score of 7 does not necessarily indicate the ideal. This characteristic minimizes the problem of inflated ratings.

RESULTS

The instrument was utilized by attending faculty to evaluate residents and also by residents as a self-evaluation. Calculated from ordinal values, mean (SD) of the ratings across domains for faculty evaluations of residents was 4.95 (0.38) while mean for self-evaluations was 4.95 (0.39). Reliability was high (coefficient alpha 0.85).

CONCLUSIONS

This instrument provides a means to measure professional behaviors during surgical residency. Repeated use will be required to thoroughly establish validity and reliability.

Ensuring global standards for medical graduates: a pilot study of international standard-setting

Increasing physician and patient mobility has led to a move toward internationalization of standards for physician competence. The Institute for International Medical Education proposed a set of outcome-based standards for student performance, which were then measured using three assessment tools in eight leading schools in China: a 150-item multiple-choice examination, a 15-station OSCE and a 16-item faculty observation form. The purpose of this study was to empanel a group of experts to determine whether international student-level performance standards could be set. The IIME convened an international panel of experts in student education with specialty and geographic diversity. The group was split into two, with each sub-group establishing standards independently. After a discussion of the borderline student, the sub-groups established minimally acceptable cut-off scores for performance on the multiple-choice examination (Angoff and Hofstee methods), the OSCE station and global rating performance (modified Angoff method and holistic criterion reference), and faculty observation domains (holistic criterion reference). Panelists within each group set very similar standards for performance. In addition, the two independent parallel panels generated nearly identical performance standards. Cut-off scores changed little before and after being shown pilot data but standard deviations diminished. International experts agreed on a minimum set of competences for medical student performance. In addition, they were able to set consistent performance standards with multiple examination types. This provides an initial basis against which to compare physician performance internationally.

The prediction of professional behaviour

OBJECTIVE

The purpose of this study was to establish outcome measures for professionalism in medical students and to identify predictors of these outcomes.

DESIGN

Retrospective cohort study.

SETTING

A US medical school.

PARTICIPANTS

All students entering in 1995 and graduating within 5 years.

MEASURES

Outcome measures included review board identification of professionalism problems and clerkship evaluations for items pertaining to professionalism. Pre-clinical predictor variables included material from the admissions application, completion of required course evaluations, students' self-reporting of immunisation compliance, students' performance on standardised patient (SP) exercises, and students' self-assessed performance on SP exercises.

RESULTS

The outcome measures of clerkship professionalism scores were found to be highly reliable (alpha 0.88-0.96). No data from the admissions material was found to be predictive of professional behaviour in the clinical years. Using multivariate regression, failing to complete required course evaluations (B = 0.23) and failing to report immunisation compliance (B = 0.29) were significant predictors of unprofessional behaviour found by the review board in subsequent years. Immunisation non-compliance predicted low overall clerkship professional evaluation scores (B = - 0.34). Student self-assessment accuracy (SP score minus self-assessed score) (B = 0.03) and immunisation non-compliance (B = 0.54) predicted the internal medicine clerkship professionalism score.

CONCLUSIONS

This study identifies a set of reliable, context-bound outcome measures in professionalism. Although we searched for predictors of behaviour in the admissions application and other domains commonly felt to be predictive of professionalism, we found significant predictors only in domains where students had had opportunities to demonstrate conscientious behaviour or humility in self-assessment.

Bedside interactions from the other side of the bedrail

OBJECTIVE

To assess the importance to patients of various aspects of bedside interactions with physician teams.

DESIGN

Cross-sectional survey.

SETTING

VA hospital.

PATIENTS

Ninety-seven medical inpatients.

INTERVENTION

Survey of 44 questions including short answer, multiple choice, and Likert-type questions.

MEASUREMENTS AND MAIN RESULTS

Data analysis included descriptive statistics. The sample was predominantly male, with a mean age of 62. Overall satisfaction with the hospital experience and with the team of doctors were both high (95% and 96% reported being very or mostly satisfied, respectively). Patients reported learning about several issues during their interactions with the teams; the 3 most highly rated areas were new problems, tests that will be done, and treatments that will be done. Most patients (76%) felt that their teams cared about them very much. Patients were made comfortable when the team showed that they cared, listened, and appeared relaxed (reported by 63%, 57%, and 54%, respectively). Patients were made uncomfortable by the team using language they did not understand (22%) and when several people examined them at once (13%). Many (58%) patients felt personally involved in teaching. The majority of patients liked having medical students and residents involved in their care (69% and 64%, respectively).

CONCLUSIONS

Patients have much to teach about what is important about interacting with physician teams. Although patients' reactions to team interactions are generally positive, patients are different with respect to what makes them comfortable and uncomfortable. Taking their preferences into account could improve the experience of being in a teaching hospital.

Trading Health Services Across Borders: GATS, Markets, And Caveats

The General Agreement on Trade in Services (GATS) is the result of an ongoing process of opening national markets to foreign health services within an international framework of trade agreements that prohibit discriminatory treatment of foreign suppliers. Developing markets are growing, as is demand for health care services, and most of this demand is being met by the private market. The globalization of health services requires the resources of the academic and corporate sectors of the developed world for equitable and sustainable growth. Health services trade should be seen as a tool for achieving these goals, rather than as an end in itself.

A content analysis of e-mail communication between patients and their providers: patients get the message

OBJECTIVE

E-mail use in the clinical setting has been slow to diffuse for several reasons, including providers' concerns about patients' inappropriate and inefficient use of the technology. This study examined the content of a random sample of patient-physician e-mail messages to determine the validity of those concerns.

DESIGN

A qualitative analysis of patient-physician e-mail messages was performed.

MEASUREMENTS

A total of 3,007 patient-physician e-mail messages were collected over 11 months as part of a randomized, controlled trial of a triage-based e-mail system in two primary care centers (including 98 physicians); 10% of messages were randomly selected for review. Messages were coded across such domains as message type, number of requests per e-mail, inclusion of sensitive content, necessity of a physician response, and message tone.

RESULTS

The majority (82.8%) of messages addressed a single issue. The most common message types included information updates to the physicians (41.4%), prescription renewals (24.2%), health questions (13.2%), questions about test results (10.9%), referrals (8.8%), "other" (including thank yous, apologies) (8.8%), appointments (5.4%), requests for non-health-related information (4.8%), and billing questions (0.3%). Overall, messages were concise, formal, and medically relevant. Very few (5.1%) included sensitive content, and none included urgent messages. Less than half (43.2%) required a physician response.

CONCLUSION

A triage-based e-mail system promoted e-mail exchanges appropriate for primary care. Most patients adhered to guidelines aimed at focusing content, limiting the number of requests per message, and avoiding urgent requests or highly sensitive content. Thus, physicians' concerns about the content of patients' e-mails may be unwarranted.

The physical examination of patients with abdominal pain: the long-term effect of adding standardized patients and small-group feedback to a lecture presentation

BACKGROUND

One of the most effective methods for teaching physical diagnosis may be standardized patient instructors.

PURPOSE

To determine if a lecture plus standardized patient instructors with small-group sessions is more effective than a lecture alone for teaching the evaluation of patients with abdominal pain.

METHODS

Control (class of 2001) and intervention (class of 2002) groups both attended a lecture on the abdominal examination. The intervention group then underwent an exercise with standardized patient instructors and a review session with surgical faculty. An evaluation 18 months later used standardized patient instructors to complete evaluations assessing history-taking and physical examination skills.

RESULTS

The intervention group performed significantly better than the control group on both the history and the physical examination subscales.

CONCLUSION

It is possible to have an important, measurable, and lasting effect on physical examination skills by adding standardized patient instructors and small-group discussion to a lecture presentation.

The assessment of Global Minimum Essential Requirements in medical education

Using an international network of experts in medical education, the Institute for International Medical Education (IIME) developed the Global Minimum Essential Requirements (GMER) as a set of competence-based outcomes for graduating students. To establish a set of tools to evaluate these competences, the IIME then convened a Task Force of international experts on assessment that reviewed the GMER. After screening 75 potential assessment tools, they identified three that could be used most effectively. Of the 60 competences envisaged in the GMER, 36 can be assessed using a 150-item multiple-choice question (MCQ) examination, 15 by using a 15-station objective structured clinical examination (OSCE), and 17 by using a 15-item faculty observation form. In cooperation with eight leading medical schools in China, the MCQ, OSCE and Faculty Observation Form were developed to be used in an assessment program that is scheduled to be given to all seven-year students in October 2003.

Expanding the boundaries of medical education: evidence for cross-cultural exchanges

PROBLEM STATEMENT AND BACKGROUND

Cross-cultural experiences are in increasing demand by both graduate and undergraduate medical students, yet the benefits of these experiences are not clearly established.

METHOD

A review of the literature was conducted to identify articles on the outcomes of cross-cultural experiences. Themes were identified and categorized into domains.

RESULTS

Forty-two studies were found; 27 articles used qualitative methods, nine used quantitative methods, and six used both. Most (24) were from the nursing literature, 18 were from the medical literature. All studies reported positive outcomes along four domains: students' professional development, students' personal development, medical school benefits, and host population benefits.

CONCLUSIONS

Studies reviewed were primarily case controlled or case series. Future research is needed that more clearly defines outcome measures and uses more rigorous methods. Although results suggest positive outcomes in all domains, additional research is needed before cross-cultural rotations can be supported based on evidence.

Effect of a triage-based E-mail system on clinic resource use and patient and physician satisfaction in primary care: a randomized controlled trial

OBJECTIVES

E-mail communication between patients and their providers has diffused slowly in clinical practice. To address concerns about the use of this technology, we performed a randomized controlled trial of a triage-based e-mail system in primary care. DESIGN AND PATIENTS/PARTICIPANTS: Physicians in 2 university-affiliated primary care centers were randomized to a triage-based e-mail system promoted to their patients. E-mails from patients of intervention physicians were routed to a central account and parsed to the appropriate staff for response. Control group physicians and their patients did not have access to the system. We collected information on patient e-mail use, phone calls, and visit distribution by physician over the 10 months and performed physician and patient surveys to examine attitudes about communication.

RESULTS

E-mail volume was greater for intervention versus control physicians (46 weekly e-mails per 100 scheduled visits vs 9 in the control group at the study midpoint; P <.01) but there were no between-group differences in phone volume (67 weekly phone calls per 100 scheduled visits vs 55 in the control group; P =.45) or rates of patient no-shows (5% in both groups; P =.77). Intervention physicians reported more favorable attitudes toward electronic communication than did control physicians but there were no differences in attitudes toward patient or staff communication in general. There were few between-group differences in patient attitudes toward electronic communication or communication in general.

CONCLUSIONS

E-mail generated through a triage-based system did not appear to substitute for phone communication or to reduce visit no-shows in a primary care setting. Physicians' attitudes toward electronic communication were improved, but physicians' and patients' attitudes toward general communication did not change. Growth of e-mail communication in primary care settings may not improve the efficiency of clinical care.

Bridging the electronic divide: patient and provider perspectives on e-mail communication in primary care

OBJECTIVE

To determine e-mail utilization patterns and attitudes toward e-mail use among primary care physicians and their ambulatory outpatient clinic patients.

STUDY DESIGN

Cross-sectional baseline survey.

PARTICIPANTS AND METHODS

Participants included 476 consecutive outpatient clinic patients, 126 general medical and family practice physicians, and 16 clinical and office staff from 2 large primary care centers within an academic teaching system. They completed a survey about e-mail usage patterns and their attitudes toward using e-mail for patient-provider communication.

RESULTS

More than half of patients (52.1%) were self-defined e-mail users, yet only 10.5% of those users had ever used e-mail to communicate with their doctors. Seventy percent of all patients said they would be willing to use e-mail to communicate with their doctors. Overall, patients were concerned about logistics, such as whether the message would get to the right person and how long it would take to get a response. Physicians and staff were more optimistic than patients about the potential for e-mail to improve the doctor-patient relationship. Patient e-mail users, patient e-mail nonusers, physicians, and staff reported low levels of concern about the security and privacy of e-mail.

CONCLUSIONS

Patient-provider e-mail may diffuse slowly into the primary care clinical practice setting because of patient concerns about efficiency and effectiveness and whether e-mail use will improve their relationship with providers. Managed care organizations that plan to build e-mail and Web-based patient portals will need to promote these technologies in a way that educates both patient and providers about their appropriate use.

Using a multimedia tool to improve cardiac auscultation knowledge and skills

OBJECTIVE

Today's medical school graduates have significant deficits in physical examination skills. Medical educators have been searching for methods to effectively teach and maintain these skills in students. The objective of this study was to determine if an auscultation curriculum centered on a portable multimedia CD-ROM was effective in producing and maintaining significant gains in cardiac auscultatory skills.

DESIGN

Controlled cohort study.

PARTICIPANTS

All 168 third-year medical students at 1 medical school in an academic medical center.

INTERVENTIONS

Students were tested before and after exposure to 1 or more elements of the auscultation curriculum: teaching on ward/clinic rotations, CD-ROM comprehensive cases with follow-up seminars, and a CD-ROM 20-case miniseries. The primary outcome measures were student performance on a 10-item test of auscultation skill (listening and identifying heart sound characteristics) and a 30-item test of auscultation knowledge (factual questions about auscultation). A subset of students was tested for attenuation effects 9 or 12 months after the intervention.

RESULTS

Compared with the control group (1 month clinical rotation alone), students who were also exposed to the CD-ROM 20-case miniseries had significant improvements in auscultation skills scores (P < .05), but not knowledge. Additional months of clerkship, comprehensive CD-ROM cases, and follow-up seminars increased auscultation knowledge beyond the miniseries alone (P < .05), but did not further improve auscultation skills. Students' auscultation knowledge diminished one year after the intervention, but auscultation skills did not.

CONCLUSION

In addition to the standard curriculum of ward and conference teaching, portable multimedia tools may help improve quality of physical examination skills.

Teaching the one-minute preceptor. A randomized controlled trial

OBJECTIVE

The One-Minute Preceptor (OMP) model of faculty development is used widely to improve teaching, but its effect on teaching behavior has not been assessed. We aim to evaluate the effect of this intervention on residents' teaching skills.

DESIGN

Randomized controlled trial.

SETTING

Inpatient teaching services at both a tertiary care hospital and a Veterans Administration Medical Center affiliated with a University Medical Center.

PARTICIPANTS

Participants included 57 second- and third-year internal medicine residents that were randomized to the intervention group (n = 28) or to the control group (n = 29).

INTERVENTION

The intervention was a 1-hour session incorporating lecture, group discussion, and role-play.

MEASUREMENTS AND MAIN RESULTS

Primary outcome measures were resident self-report and learner ratings of resident performance of the OMP teaching behaviors. Residents assigned to the intervention group reported statistically significant changes in all behaviors (P <.05). Eighty-seven percent of residents rated the intervention as "useful or very useful" on a 1-5 point scale with a mean of 4.28. Student ratings of teacher performance showed improvements in all skills except "Teaching General Rules." Learners of the residents in the intervention group reported increased motivation to do outside reading when compared to learners of the control residents. Ratings of overall teaching effectiveness were not significantly different between the 2 groups.

CONCLUSIONS

The OMP model is a brief and easy-to-administer intervention that provides modest improvements in residents' teaching skills.

Comparison of resident and medical student evaluation of faculty teaching

Recognizing and rewarding teaching faculty are increasingly important to medical schools and are often hampered by low perceived reliability and validity of measures of teaching ability. The purpose of this study was to cross-validate two independently generated measures of teaching from medical students and residents. A total of 2,318 medical student and 4,425 resident scores for single-item measures of teaching ability for 129 teaching faculty members of a department of internal medicine over a 6-year period were compared. Results showed that average teaching scores were higher for medical students than residents. Rank order of faculty were within 2 quintiles for the two groups for over 90% of faculty. Highly discordant evaluations were seen for only 8% of faculty. The authors conclude the general concordance of two independent measures of teaching ability adds evidence to the existing literature of the validity of single-item measures of teaching ability from two different types of learners.

Is there a relationship between attending physicians' and residents' teaching skills and students' examination scores?

PURPOSE

Faculty development programs and faculty incentive systems have heightened the need to validate a connection between the quality of teaching and students' learning. This study was designed to determine the association between attending physicians' and residents' teacher ratings and their students' examination scores.

METHOD

From a database of 362 students, 138 faculty, and 107 residents in internal medicine, student-faculty (n = 476) and student-resident (n = 474) pairs were identified. All students were in their third year, rotating on inpatient general medicine and cardiology services, July 1994 through June 1996, at a single institution. The outcome measure for students' knowledge was the NBME Subject Examination in internal medicine. To control for students' baseline knowledge, the predictors were scores on the USMLE Step 1 and a sequential examination (a clinically-based pre- and post-clerkship examination). Teaching abilities of faculty and residents were rated by a global item on the post-clerkship evaluation. Faculty's ratings used only scores from prior to the study period; residents' ratings included those scores students gave during the study period.

RESULTS

Multivariate analyses showed faculty's teaching ratings were a small but significant predictor of the increase in students' knowledge. Residents' teaching ratings did not predict an increase in students' knowledge.

CONCLUSION

Attending faculty's clinical teaching ability has a positive and significant effect on medical students' learning.