Exporting a technical skills evaluation technology to other sites

BACKGROUND

The Objective Structured Assessment of Technical Skill (OSATS) is a multistation performance-based examination that assesses the technical skills of surgery residents. This study explores the implementation issues involved in remote administration of the OSATS focusing on feasibility and the psychometric properties of the examination.

METHODS

An eight-station OSATS was administered to surgical residents in Los Angeles and Chicago. The University of Toronto and the local institutions shared responsibility for organization and administration of the examination.

RESULTS

There was good reliability for both the checklist (alpha = 0.68 for LA, 0.73 for Chicago) and global rating forms (alpha = 0.82 for both sites). Both iterations also showed evidence of construct validity, with a significant effect of training year for the checklist and global rating forms at both sites (analysis of variance: F = 8.66 to 19.93, P <0.01). Despite some challenges, the model of central organization and peripheral delivery was effective for the administration of the examinations.

CONCLUSIONS

Two iterations of the OSATS at remote sites demonstrated psychometric properties that are highly consistent with previously reported data suggesting that the examination is portable. Both faculty and residents indicated satisfaction with the examination experience. A model of central administration with peripheral delivery was feasible and effective.

University and practice-based physicians' input on the content of a surgical curriculum

BACKGROUND

The specific knowledge and skills students learn during surgical rotations are reconsidered in light of recent changes in medical school curricula. The purpose of this study was to determine the priorities of a surgical curriculum based on input from three groups; surgical faculty (SF), primary care faculty (PCF), and community-based , practicing primary care physicians (PCP).

METHODS

A questionnaire was developed in which SF (n=54), PCF (n=85), and PCP (n=876) were asked to rank the importance of 145 areas of knowledge and 48 areas of clinical skills on a 5-point Likert-type scale. Responses were rank ordered by the mean of importance ratings for each group. Differences among groups were evaluated using ANOVA.

RESULTS

Response rates were best for faculty (100%) SF, 88% PCF, 61% PCP). All three groups were best considered general surgery related topics and general skills very important. Primary care physicians and PCF consistently ranked otolaryngology, ophthalmology, and orthopedic topics and skills higher than did SF. Surgery faculty ranked invasive surgical procedures higher than did PCP while PCP ranked orthopedic procedural skills more highly.

CONCLUSIONS

There is significant overlap among physicians about what medical students should learn during surgical rotation. Differences between groups centered on surgical subspecialty knowledge and clinical skills. These results provide a broad perspective about required subjects for a core surgical clerkship curriculum, which should include surgical subspecialty training.

A prospective randomized trial of a residents-as-teachers training program

PURPOSE

To develop, implement, and evaluate a course for improving the teaching skills of surgery residents.

METHOD

Responses from residents at four general surgery training programs to a needs assessment survey were used to develop a two-day course for improving teaching skills. Residents at two surgical training programs were randomly assigned to experimental and control groups, and experimental residents participated in and evaluated the newly devised course. Six to seven months later, experimental and control residents' teaching performances were evaluated using a five-station objective structured teaching evaluation (OSTE). Differences between the residents' performances were calculated using Mann-Whitney U, chi-square analysis, or Fisher's exact test.

RESULTS

Participating residents rated the course highly. They considered the interactive nature of the course its greatest strength. As measured by the OSTE, the performances of the residents differed least significantly in the feedback station, where the residents in the experimental groups showed significant improvement on only one of seven items at one institution, and only one of nine items at the other. The greatest differences occurred in the microskills teaching station, where the residents at one institution performed significantly better than did their control counterparts on four of five items and in overall performance.

CONCLUSION

This study demonstrates the value of a needs assessment in developing a course to improve residents' teaching skills. Such courses must provide active learning with opportunities for practicing skills and, following the course, ongoing feedback to maintain changes in teaching behaviors. The curriculum developed in this study has been put into a transportable form that includes an instructor's manual providing guidelines and suggestions for implementation.

Strategies for efficient and effective teaching in the ambulatory care setting

Medical education in the ambulatory care setting is characterized in part by the question of how to ensure educational effectiveness while simultaneously providing high-quality, cost-effective patient care. The constraints associated with managed care have only served to escalate the intensity of this dilemma. However, in spite of the difficulties faced by ambulatory care preceptors, there are educationally sound and time-efficient strategies clinical teachers may employ to improve ambulatory care education. Emphasizing the basic three-step process of planning, teaching, and reflection, the authors describe five such strategies: "wave" scheduling, orienting learners to patients, having learners do their case presentations in the examination room, employing the microskills of the "one-minute preceptor," and effectively reflecting on one's teaching in order to develop effective teaching scripts. Research in ambulatory care learning has indicated that learners must be given significant roles in patient care and that preceptors must observe trainees as they care for patients so that they can provide trainees with helpful feedback. Employing these strategies in the ambulatory care setting will help educators to accomplish these two objectives while minimizing disruption to cost-effective, high-quality clinical practice.