To the point: medical education reviews evaluation in context: assessing learners, teachers, and training programs

Discourse on Exposure to Pornography Content Online Between Arab Adolescents and Parents: Qualitative Study on its Impact on Sexual Education and Behavior

Background

The internet revolution of the 21st century has made sexual content available and accessible on a scale that has never existed before. Many studies have indicated that the use of pornography was associated with more permissive sexual attitudes and tended to be linked with stronger gender-stereotypical sexual beliefs. It also seemed to be associated with other risky behaviors and sexual promiscuity. Pornography exposure in conservative societies leads to conflicts with religious and cultural taboos.

Objective

The aim of this study was to characterize the barriers and difficulties that prevent sexual discourse in the Arab society and enable pornography viewing according to the perceptions of adolescents and mothers.

Methods

This study involved qualitative research methods and in-depth interviews with 40 participants. This study included 20 Arab adolescents, sampled by 2 age groups (14-16 years and 16-18 years), and 20 mothers of adolescents from both sexes.

Results

The findings indicate that mothers “turn a blind eye” to porn viewing and sexual activity by boys; however, they show a sweeping prohibition and denial of such behavior by girls. Boys reported viewing porn routinely, whereas girls denied doing so, but admitted that their female friends watched porn. The study also found that boys experienced guilt during and after viewing porn as a result of the clash between modernity and traditional values. The mothers and adolescents emphasized the need for an open sexual discourse to reduce violent behaviors such as Web-based sexual harassment, including sending videos and pictures of naked girls, often accompanied by threats and blackmail.

Conclusions

It is necessary to find a way to encourage a significant sexual discourse to prevent the violent consequences of its absence in the Arab society. A controlled, transparent, and critical sexual discourse could help youth make more informed decisions concerning the search for sexual content, porn viewing, and sexual behavior.

Community medicine teaching and evaluation: scope of betterment

There have been rapid and extensive changes in the way assessment is conducted in medical education. Assessment brings about standardization of the manner in which the syllabus is to be implemented and also gives guidelines regarding the teaching pattern, especially when the student is going to rotate through various departments in a medical college. Community Medicine is an important branch of medicine concerned with the health of populations. Existing forms of assessment of community medicine education mainly consists of internal [formative] assessment and final (summative) examination. Advantages of the present system is the similarity of the methods used for internal assessments and final examinations and is relatively easily done since only the knowledge application and recall ability of the student in theory and practical are assessed. Disadvantages in the current evaluation system are neglecting the assessment of psychomotor, affective and communication skills. Evaluation systems can be improved by implementing techniques to assess psychomotor skills, presentation and communication skills, organizational skills and the student's ability to work in a team. Regular feedback from students should be taken periodically for the betterment of Community Medicine education. This article is meant to sensitise the academic experts in medical education to plan better need based methods of assessment in the subject of Community Medicine, in relation to the new MCI 2012 Regulations, in order to make it a better learning experience for the students.

Narrative descriptions should replace grades and numerical ratings for clinical performance in medical education in the United States

Background: In medical education, evaluation of clinical performance is based almost universally on rating scales for defined aspects of performance and scores on examinations and checklists. Unfortunately, scores and grades do not capture progress and competence among learners in the complex tasks and roles required to practice medicine. While the literature suggests serious problems with the validity and reliability of ratings of clinical performance based on numerical scores, the critical issue is not that judgments about what is observed vary from rater to rater but that these judgments are lost when translated into numbers on a scale. As the Next Accreditation System of the Accreditation Council on Graduate Medical Education (ACGME) takes effect, medical educators have an opportunity to create new processes of evaluation to document and facilitate progress of medical learners in the required areas of competence.

Proposal and initial experience: Narrative descriptions of learner performance in the clinical environment, gathered using a framework for observation that builds a shared understanding of competence among the faculty, promise to provide meaningful qualitative data closely linked to the work of physicians. With descriptions grouped in categories and matched to milestones, core faculty can place each learner along the milestones' continua of progress. This provides the foundation for meaningful feedback to facilitate the progress of each learner as well as documentation of progress toward competence.

Implications: This narrative evaluation system addresses educational needs as well as the goals of the Next Accreditation System for explicitly documented progress. Educators at other levels of education and in other professions experience similar needs for authentic assessment and, with meaningful frameworks that describe roles and tasks, may also find useful a system built on descriptions of learner performance in actual work settings.

Conclusions: We must place medical learning and assessment in the contexts and domains in which learners do clinical work. The approach proposed here for gathering qualitative performance data in different contexts and domains is one step along the road to moving learners toward competence and mastery.

Teaching in the ICU

Teaching in the fast-paced, high-pressure environment of the ICU can be very demanding. Thus, the educator-intensivist must learn teaching strategies that are time-efficient, simple, and successful. In this article, we provide an overview of the current and relevant teaching theories and highlight potential obstacles and limitations to teaching in the ICU. In the second part, we discuss a sample of simple approaches to optimize the ICU-rotation curriculum as well as effective techniques to improve teaching, while not compromising quality of care.

New tools for systematic evaluation of teaching qualities of medical faculty: results of an ongoing multi-center survey

Background

Tools for the evaluation, improvement and promotion of the teaching excellence of faculty remain elusive in residency settings. This study investigates (i) the reliability and validity of the data yielded by using two new instruments for evaluating the teaching qualities of medical faculty, (ii) the instruments' potential for differentiating between faculty, and (iii) the number of residents' evaluations needed per faculty to reliably use the instruments.

Methods and Materials

Multicenter cross-sectional survey among 546 residents and 629 medical faculty representing 29 medical (non-surgical) specialty training programs in the Netherlands. Two instruments—one completed by residents and one by faculty—for measuring teaching qualities of faculty were developed. Statistical analyses included factor analysis, reliability and validity exploration using standard psychometric methods, calculation of the numbers of residents' evaluations needed per faculty to achieve reliable assessments and variance components and threshold analyses.

Results

A total of 403 (73.8%) residents completed 3575 evaluations of 570 medical faculty while 494 (78.5%) faculty self-evaluated. In both instruments five composite-scales of faculty teaching qualities were detected with high internal consistency and reliability: learning climate (Cronbach's alpha of 0.85 for residents' instrument, 0.71 for self-evaluation instrument, professional attitude and behavior (0.84/0.75), communication of goals (0.90/0.84), evaluation of residents (0.91/0.81), and feedback (0.91/0.85). Faculty tended to evaluate themselves higher than did the residents. Up to a third of the total variance in various teaching qualities can be attributed to between-faculty differences. Some seven residents' evaluations per faculty are needed for assessments to attain a reliability level of 0.90.

Conclusions

The instruments for evaluating teaching qualities of medical faculty appear to yield reliable and valid data. They are feasible for use in medical residencies, can detect between-faculty differences and supply potentially useful information for improving graduate medical education.

Good clinical teachers likely to be specialist role models: results from a multicenter cross-sectional survey

CONTEXT

Medical educational reform includes enhancing role modelling of clinical teachers. This requires faculty being aware of their role model status and performance. We developed the System for Evaluation of Teaching Qualities (SETQ) to generate individualized feedback on previously defined teaching qualities and role model status for faculty in (non) academic hospitals.

OBJECTIVES

(i) To examine whether teaching qualities of faculty were associated with their being seen as a specialist role model by residents, and (ii) to investigate whether those associations differed across residency years and specialties.

METHODS & MATERIALS

Cross-sectional questionnaire survey amongst 549 Residents of 36 teaching programs in 15 hospitals in the Netherlands. The main outcome measure was faculty being seen as specialist role models by residents. Statistical analyses included (i) Pearson's correlation coefficients and (ii) multivariable logistic generalized estimating equations to assess the (adjusted) associations between each of five teaching qualities and 'being seen as a role model'.

RESULTS

407 residents completed a total of 4123 evaluations of 662 faculty. All teaching qualities were positively correlated with 'being seen as a role model' with correlation coefficients ranging from 0.49 for 'evaluation of residents' to 0.64 for 'learning climate' (P<0.001). Faculty most likely to be seen as good role models were those rated highly on 'feedback' (odds ratio 2.91, 95% CI: 2.41-3.51), 'a professional attitude towards residents' (OR 2.70, 95% CI: 2.34-3.10) and 'creating a positive learning climate' (OR 2.45, 95% CI: 1.97-3.04). Results did not seem to vary much across residency years. The relative strength of associations between teaching qualities and being seen as a role model were more distinct when comparing specialties.

CONCLUSIONS

Good clinical educators are more likely to be seen as specialist role models for most residents.

Towards effective evaluation and reform in medical education: a cognitive and learning sciences perspective

Health professions education is dealing with major transformations in light of the changing nature of the health care delivery system, including the use of technology for "just in time" delivery of care, evidence-based practice, personalized medical care and learning, as health professionals strive to integrate biomedical advances and clinical practice. This has forced the medical education community to reassess the current teaching and learning practices and more importantly, the evaluation of the medical education process. There have been recent advances in cognitive and learning sciences theories, some of which can inform medical educators about best teaching and learning practices and their impact on the evaluation process. An understanding of these theories provides a sound rationale for choosing specific instructional strategies and choosing evaluation measures that assess the curricular objectives. The review begins with an overview of evaluation and assessment in education, followed by an overview of major theories from the cognitive and learning sciences. Next, the role of cognitive and learning sciences theories in informing the process of medical education evaluation is discussed, including its impact on student learning, performance and professional competence, as well as recommendations for reform of medical curricula based on such theories. The paper continues with the elaboration of current trends in health sciences education, particularly medical education, and available evidence for the impact on student learning and performance as well as areas where more research is needed.

Are successful resident clinicians good teachers? Multidimensional resident assessment

BACKGROUND

GME training programs must demonstrate residents are prepared as both teachers and clinicians. We examined the relationship between faculty evaluations of residents' clinical performance and student evaluations of residents' teaching performance.

METHOD

Concordance tables for mean ratings and qualitative analysis of comments among 95 residents receiving evaluations by 267 faculty and 106 students.

RESULTS

A total of 88% of residents received concordant ratings from faculty and students. When discordant, faculty gave higher ratings (P = .003). Ninety and ninety-one percent of faculty and student comments exhibited a positive tone.

CONCLUSIONS

Faculty and students tended to agree on the successful and unsuccessful residents, supporting the validity of rating scale evaluations. Future research might focus on how to tailor resident remediation when discordant ratings or comments occur.

A review of portfolio use in residency programs and considerations before implementation

BACKGROUND

Portfolios, often described as collections of evidence, are discussed as a means of teaching or assessing the Accreditation Council for Graduate Medical Education competencies. Yet, it is unclear how many residency programs utilize portfolios. The purpose of this article is to (a) review the literature on portfolio use in graduate medical education; (b) examine efficacy of portfolio use, based upon studies in the field; and (c) offer a discussion of considerations for implementing portfolios.

SUMMARY

Two searches of PubMed, OVID, JSTOR, SCOPUS, and FirstSearch Wilson Select were conducted between October 2006 and April 2007 to identify studies and articles related to portfolio usage. Thirty-nine articles met criteria and were reviewed.

CONCLUSIONS

There is wide variation in how portfolios are utilized within U.S. residency programs. The challenge for graduate medical education is to create consensus on the definition and purpose of portfolios, such that best practices in portfolio implementation and assessment can be achieved.

Evaluation, grading, and use of the RIME vocabulary on internal medicine clerkships: results of a national survey and comparison to other clinical clerkships

BACKGROUND

Evaluation methods within and across clerkships are rapidly evolving, including greater emphasis or frameworks for descriptive evaluation and direct observation of competence.

PURPOSE

The purpose of this study is to describe current evaluation methods, use of the Reporter-Interpreter-Manager/Educator (RIME) framework, and grade assignment by internal medicine clerkship directors.

METHODS

In 2005, the Clerkship Directors in Internal Medicine surveyed its 109 institutional members. Topics included evaluation methods and grade contribution, use of evaluation sessions and/or RIME, and grade assignment (criterion referenced or normative).

RESULTS

Response rate was 81% (88/109). The evaluation methods were as follows: teachers' evaluations, 93% (64% of grade); National Board of Medical Examiners subject examination, 81% (25% of grade); faculty written exam, 34% (14% of grade); objective structured clinical examinations, 32% (12% of grade); direct observation, 22% (7% of grade). RIME is used by 42% of respondents. Many clerkship directors (43%) meet with teachers to discuss student performance. Criterion-referenced grading is used by 59%, and normative grading is used by 27%. Unsatisfactory grades are given for examination failures (72%), unprofessional behavior (49%), poor clinical performance (42%), and failure to meet requirements (18%).

CONCLUSIONS

Internal medicine clerkship directors emphasize description and observation of students. RIME and discussions with teachers are becoming commonplace.

The interns' learning assessment in obstetrics and gynecology department of Zahedan University of Medical Sciences

OBJECTIVE

One of the aims of management priorities in medical universities is the evaluation of learning in educational departments in order to prevent educational retardation and to improve the quality of education. The aim of this study was to evaluate the interns' learning in the obstetrics and gynecology (O&G) department at Zahedan University of Medical Sciences (ZUMS).

MATERIALS AND METHODS

The study was performed in ZUMS, Iran, in 2002-2003 on all interns at the O&G department, including 30 men and 40 women. For data collection, a questionnaire was used and included some questions regarding the common emergencies and diseases in O&G, together with different learning indicators such as reading, observation, hearing, management, and the capability of management. The data were analyzed using descriptive statistics, tables, t test, and chi-square test using the SPSS software.

RESULTS

The mean percentages of learning indicators of observation, bedside teaching, supervised management, and personal management in the common emergencies and diseases of O&G in male interns were significantly lower than those in female interns. Also, the mean percentages of managing capabilities were 12% and 70.5% in common emergencies and 14.2% and 59.3% in common diseases for male and female interns, respectively. The chi-square test showed a significant difference between the mean percentages of the managing capabilities in male and female interns for the majority of the common emergencies and diseases. Also, the chi-square test revealed a significant relationship between the learning indicators and the interns' managing capabilities for common emergencies and diseases.

CONCLUSION

Some learning indicators in the male interns were very low. This needs urgent improvement of the learning quality in the O&G department, especially for the male interns, particularly those who are supposed to work in the deprived areas of the country after graduation in the public service.

To the point: Medical education review of the RIME method for the evaluation of medical student clinical performance

This article, the sixth in the ongoing To The Point Series produced by the Association of Professors of Gynecology and Obstetrics Undergraduate Medical Education Committee, reviews the Reporter-Interpreter-Manager-Educator (RIME) method for the evaluation of student clinical performance on the obstetrics and gynecology rotation. This article discusses the inherent challenges of descriptive narrative evaluation and the superiority of the RIME method in producing meaningful evaluation of and feedback for students. The use of the method to fulfill Liaison Committee on Medical Education standards and implementation of the method are described.

To the point: medical education reviews--dealing with student difficulties in the clinical setting

Learners with cognitive and behavioral difficulties are particularly challenging in the clinical setting. Cognitive difficulties in the clinical realm may relate to knowledge deficits and/or weak problem solving skills. Behavioral difficulties resulting from attitudinal or motivational problems may manifest themselves as specific unprofessional acts either committed or omitted in the course of caring for patients as well as in unprofessional behavior. A disproportionate amount of time and energy can be expended addressing the needs of such students. This paper reviews several types of difficulties encountered by educators and suggests strategies for preventing, assessing, and working effectively with challenging students in the clinical setting. Specific attention is directed to impaired students.

Implementation and evaluation of a genetics curriculum to improve obstetrician-gynecologist residents' knowledge and skills in genetic diagnosis and counseling

OBJECTIVE

This study was undertaken to develop, implement, and evaluate a genetics curriculum for obstetrician-gynecologist residents.

STUDY DESIGN

We prospectively evaluated the effect of a genetics curriculum on obstetrician-gynecologist residents' knowledge and skills. Residents completed a needs assessment and pretest. Educational intervention included 2 3-hour didactic sessions with 1 hour of lecture followed by case discussion and 1 3-hour session of experiential learning using standardized patients who evaluated residents' knowledge and skills in taking family history, drawing genetic pedigrees, and counseling patients. Posttest scores were compared with pretest scores.

RESULTS

Needs assessment was completed by all 40 obstetrics and gynecology residents and identified limited and variable genetics education in medical school. Twenty-eight of 40 residents attended the entire educational intervention and completed the pretest and posttest, and 25 of 28 showed improved test scores. Residents stated that they were more confident in their ability to take a family history, record a 3-generation pedigree, and counsel patients about genetic conditions after completion of the genetics curriculum.

CONCLUSION

This multifaceted genetics curriculum improved residents' knowledge of genetics as well as their confidence in applying genetic concepts as assessed by the pretest and posttest and by their comments in the debrief session.