Surgeon communication behaviors that lead patients to not recommend the surgeon to family members or friends: Analysis and impact

BACKGROUND

This study analyzes specific elements of physician communication that lead patients to not recommend surgeons to family members or friends (FMoFs).

METHODS

Patients completed questionnaires after surgery clinic encounters. Questionnaires addressed whether surgeons used optimal communication behaviors and whether patients would recommend the surgeon.

RESULTS

A total of 1,514 questionnaires were completed for 39 surgeons. Patients reported the following communication lapses: failure to ask whether the patient had questions (6.9% of occasions), failure to sit down (6.5%), use of words patients could not understand (5%), failure to educate patients about their condition (4.3%), failure to introduce themselves (4%), lack of interest in patients as persons (2.4%), and inadequacies in answering questions (2%). Surgeons omitted at least one of these optimal behaviors in 16.3% of encounters. Surgeons were not recommended in 1.7% of encounters. Twelve surgeons (31%) were not recommended on at least 1 occasion. Behaviors omitted most commonly in encounters where patients wouldn't recommend surgeons included failure to show interest in the patient (52%), explain their medical condition (52%), invite questions (40%), and answer questions (36%).

CONCLUSIONS

Extrapolating these results to 1,618 patient visits/surgeon/year, results in the following number of patients annually who do not recommend their surgeons: 15 for failure to adequately explain their medical condition, 15 for failure to show interest in them, 11 for failure to ask if the patient had questions, and 10 for failure to answer questions. Considering the ripple effect due to the number of a patient's FMoFs, surgeons should be aware of the significant impact of even occasional lapses in optimal communication behaviors.

See one, do one, teach one--exploring the core teaching beliefs of medical school faculty

This paper explores the core teaching beliefs of medical school faculty and establishes whether these beliefs differ among basic science, clinical, and instruction specialist faculty. One hundred and twenty-five medical school teachers who were members of professional organizations dedicated to the improvement of medical school teaching completed a Q-sort of 56 statements reflecting their core teaching beliefs. The statements described beliefs about motivation, knowledge and skill acquisition, retention, feedback, transfer, teacher characteristics, and teaching strategies. Q-sorts were completed by 37 basic scientists (30% of respondents), 59 clinicians (47%) and 29 instruction specialists (23%) working in medical schools. Fifty-two participants were classroom teachers (42%), 66 were classroom and clinical teachers (53%), and seven reported that they do not teach (6%). The Q-sort results indicate how medical school faculty members differ in their core beliefs about teaching and learning. Thirty-two respondents (26%) focused on the student as a person first. Eight (6%) were content oriented. Thirty-four (27%) were performance oriented; their focus was on having students learn and apply knowledge and skills to accomplish clinical tasks. Fifty-one respondents (41%) were found to have a blend of these viewpoints. Respondents' type of training or type of teaching did not provide a reliable indication of core teaching beliefs classification.

An investigation of medical student reactions to feedback: a randomised controlled trial

BACKGROUND

Medical educators have indicated that feedback is one of the main catalysts required for performance improvement. However, medical students appear to be persistently dissatisfied with the feedback that they receive. The purpose of this study was to evaluate learning outcomes and perceptions in students who received feedback compared to those who received general compliments.

METHODS

All subjects received identical instruction on two-handed surgical knot-tying. Group 1 received specific, constructive feedback on how to improve their knot-tying skill. Group 2 received only general compliments. Performance was videotaped before and after instruction and after feedback. Subjects completed the study by indicating their global level of satisfaction. Three faculty evaluators observed and scored blinded videotapes of each performance. Intra-observer agreement among expert ratings of performance was calculated using 2-way random effects intraclass correlation (ICC) methods. Satisfaction scores and performance scores were compared using paired samples t-tests and independent samples t-tests.

RESULTS

Performance data from 33 subjects were analysed. Inter-rater reliability exceeded 0.8 for ratings of pre-test, pre-intervention and post-intervention performances. The average performance of students who received specific feedback improved (21.98 versus 15.87, P<0.001), whereas there was no significant change in the performance score in the group who received only compliments (17.00 versus 15.39, P=0.181) The average satisfaction rating in the group that received compliments was significantly higher than the group that received feedback (6.00 versus 5.00, P=0.005).

DISCUSSION

Student satisfaction is not an accurate measure of the quality of feedback. It appears that satisfaction ratings respond to praise more than feedback, while learning is more a function of feedback.

Feasibility, reliability and validity of an operative performance rating system for evaluating surgery residents

BACKGROUND

Resident evaluation traditionally involves global assessments including clinical performance, professional behavior, technical skill, and number of procedures performed. These evaluations lack objective assessment of operative skills. We describe an operative performance rating system (OPRS) designed to provide objective operative performance ratings using a sentinel procedure format.

METHODS

Ten-item procedure-specific rating instruments were developed. Items included technical skills, operative decision making, and general items. A 1 to 5 (5 = excellent) scale was used for evaluation. Six procedures had sufficient forms returned to allow evaluation. Inter-rater reliability was determined by having faculty evaluators view 2 videotaped operations.

RESULTS

Return rates for the Internet-based form were full-time faculty (92%), volunteer faculty (27%), and overall (67%). Reliability, (average interitem correlation), and total procedures evaluated were excisional biopsy, 0.90, (0.48), 77; open inguinal herniorraphy, 0.94, (0.62), 51; laparoscopic cholecystectomy, 0.95, (0.64), 75; small-bowel and colon resection, 0.92, (0.58), 30; parathyroidectomy, 0.70, (0.19), 30; and lumpectomy, 0.92, (0.51), 38. Years of training accounted for 25% to 57% of the variation in scores. Inter-rater variability was observed; however, the average rater agreement was reliable.

CONCLUSIONS

Internet-based management made obtaining the data feasible. The OPRS complements traditional evaluations by providing objective assessment of operative decision-making and technical skills. Interitem correlations indicate the average rating of items provides a reliable indicator of resident performance. The OPRS is useful in tracking resident development throughout postgraduate training and offers a structured means of certifying operative skills.

Forecasting residents' performance--partly cloudy

The authors offer a practical guide for improving the appraisal of a resident's performance. They identify six major factors that compromise the process of observing, measuring, and characterizing a resident's current performance, forecasting future performance, and making decisions about the resident's progress. Factors that compromise any of these steps lead to individual and collective uncertainty and decrease faculty confidence when making decisions on a resident's progress. The six factors, addressed in order of importance, are inaccuracies due to (1) incomplete sampling of performance, (2) rater memory constraints, (3) hidden performance deficits of the resident, (4) lack of performance benchmarks, (5) faculty members' hesitancy to act on negative performance information, and (6) systematic rater error. The description of each factor is followed by a number of specific suggestions on what residency programs can do to eliminate or minimize the impact of these factors. While this article is couched in the context of the performance evaluation of residents, everything included pertains to measuring and appraising medical students' and practicing physicians' clinical performance as well.

Improving resident performance assessment data: numeric precision and narrative specificity

PURPOSE

To evaluate the use of a systems approach for diagnosing performance assessment problems in surgery residencies, and intervene to improve the numeric precision of global rating scores and the behavioral specificity of narrative comments.

METHOD

Faculty and residents at two surgery programs participated in parallel before- and-after trials. During the baseline year, quality assurance data were gathered and problems were identified. During two subsequent intervention years, an educational specialist at each program intervened with an organizational change strategy to improve information feedback loops. Three quality-assurance measures were analyzed: (1) percentage return rate of forms, (2) generalizability coefficients and 95% confidence intervals of scores, and (3) percentage of forms with behaviorally specific narrative comments.

RESULTS

Median return rates of forms increased significantly from baseline to intervention Year 1 at Site A (71% to 100%) and Site B (75% to 100%), and then remained stable during Year 2. Generalizability coefficients increased between baseline and intervention Year 1 at Site A (0.65 to 0.85) and Site B (0.58 to 0.79), and then remained stable. The 95% confidence interval around resident mean scores improved at Site A from baseline to intervention Year 1 (0.78 to 0.58) and then remained stable; at Site B, it remained constant throughout (0.55 to 0.56). The median percentage of forms with behaviorally specific narrative comments at Site A increased significantly from baseline to intervention Years 1 and 2 (50%, 57%, 82%); at Site B, the percentage increased significantly in intervention Year 1, and then remained constant (50%, 60%, 67%).

CONCLUSIONS

Diagnosing performance assessment system problems and improving information feedback loops improved the quality of resident performance assessment data at both programs.

Assuring the reliability of resident performance appraisals: More items or more observations?

BACKGROUND

The tendency to add items to resident performance rating forms has accelerated due to new ACGME competency requirements. This study addresses the relative merits of adding items versus increasing number of observations. The specific questions addressed are (1) what is the reliability of single items used to assess resident performance, (2) what effect does adding items have on reliability, and (3) how many observations are required to obtain reliable resident performance ratings.

METHODS

Surgeon ratings of resident performance were collected for 3 years. The rating instrument had 3 single items representing clinical performance, professional behavior, and comparisons to other house staff. Reliability analyses were performed separately for each year, and variance components were pooled across years to compute overall reliability coefficients.

RESULTS

Single-item resident performance rating scales were equivalent to multiple-item scales using conventional reliability standards. Increasing the number of rating items had little effect on reliability. Increasing the number of observations had a much larger effect.

CONCLUSIONS

Program directors should focus on increasing the number of observations per resident to improve performance sampling and reliability of assessment. Increasing the number of rating items had little effect on reliability and is unlikely to assess new ACGME competencies adequately.

The effects of group dynamics on resident progress committee deliberations

BACKGROUND

Prior investigations suggest that resident progress decisions by committee provide a broader perspective on performance and result in less grade inflation. There is countervailing concern that group dynamics may compromise quality of progress decisions.

PURPOSE

To determine whether and how group dynamics compromise decision making about resident progress.

METHODS

Researchers recorded and analyzed participant comments during a resident progress committee meeting. End-of-rotation (EOR) evaluations were analyzed and compared to progress committee meeting results.

RESULTS

EOR and progress committee comments were similar in content. The ratio of specific to general comments was higher for EOR evaluations (5:1) than for progress committee meetings (2:1). EOR evaluation comments provided more supporting evidence for assertions. Individual progress committee participants did not dominate discussion or sway decision making. Participant progress committee meeting comments were consistent with their EOR comments failing to support the presence of progress committee meeting "feeding frenzies."

CONCLUSIONS

Results suggest that progress committee meeting group dynamics do not seriously compromise the validity of resident progress decisions.

Prognostic value of resident clinical performance ratings

BACKGROUND

This study investigated the concurrent and predictive validity of end-of-rotation (EOR) clinical performance ratings.

STUDY DESIGN

Surgeon EOR ratings of residents were collected and compared with end-of-year (EOY) progress decisions and to EOR and EOY confidential judgments of resident ability to provide patient care without direct supervision.

RESULTS

Eighty percent to 85% of EOR ratings were Excellent or Very Good. Five percent or fewer were Fair or Poor. Almost all residents receiving Excellent or Very Good EOR ratings also received positive EOR judgments about ability to provide patient care without direct supervision. Residents rated Fair or Poor received negative EOR judgments about ability to provide patient care without direct supervision. As the cumulative percent of Good, Fair, and Poor EOR ratings increased, the number of residents promoted without stipulations at the end of the year decreased and the percentage of faculty members who judged the residents capable of providing effective patient care without direct supervision at the end of the year declined. All residents receiving 40% or more EOR ratings below Very Good had stipulations associated with their promotion.

CONCLUSIONS

Despite use of descriptive anchors on the scale, clinical performance ratings have no direct meaning. Their meaning needs to be established in the same manner as is done in setting normal values for diagnostic tests, ie, by establishing the relationship between EOR ratings and practice outcomes.

Do individual attendings' post-rotation performance ratings detect residents' clinical performance deficiencies?

PURPOSE

To determine whether attending physicians' post-rotation performance ratings and written comments detect surgery residents' clinical performance deficits.

METHOD

Residents' performance records from 1997-2002 in the Department of Surgery, Southern Illinois University School of Medicine, were reviewed to determine the percentage of times end-of-rotation performance ratings and/or comments detected deficiencies leading to negative end-of-year progress decisions.

RESULTS

Thirteen of 1,986 individual post-rotation ratings (0.7%) nominally noted a deficit. Post-rotation ratings of "good" or below were predictive of negative end-of-year progress decisions. Eighteen percent of residents determined to have some deficiency requiring remediation received no post-rotation performance ratings indicating that deficiency. Written comments on post-rotation evaluation forms detected deficits more accurately than did numeric ratings. Physicians detected technical skills performance deficits more frequently than applied knowledge and professional behavior deficits. More physicians' post-rotation numeric ratings contradicted performance deficits than supported them. More written comments supported deficits than contradicted them in the technical skills area. In the applied knowledge and professional behavior areas, more written comments contradicted deficits than supported them.

CONCLUSIONS

A large percentage of performance deficiencies only became apparent when the attending physicians discussed performance at the annual evaluation meetings. Annual evaluation meetings may (1) make patterns of residents' behavior apparent that were not previously apparent to individual physicians, (2) provide evidence that strengthens the individual attending's preexisting convictions about residents' performance deficiencies, or (3) lead to erroneous conclusions. The authors believe deficiencies were real and that their findings can be explained by a combination of reasons one and two.

Variables influencing medical student learning in the operating room

BACKGROUND

The operating room (OR) is an important venue where surgeons do much of medical student teaching and yet there has been little work evaluating variables that influence learning in this unique environment. We designed this study to identify variables that affected medical student learning in the OR.

METHODS

We developed a questionnaire based on surgery faculty observations of learning in the OR. The medical students completed the questionnaire on 114 learning episodes in the OR. Pearson correlation coefficient was used to establish the strength of association between various variables and the student's overall perception of learning.

RESULTS

The students evaluated 27 variables that might impact their learning in the OR. Strong correlations were identified between the attending physician's attitude, interactions and teaching ability in the OR and the environment being conducive to learning.

CONCLUSIONS

Surgical faculty behavior is a powerful determinant of student perceptions of what provides for a favorable learning environment in the OR.

A lay person versus a trained endoscopist: can the preop endoscopy simulator detect a difference?

The purpose of this study was to establish construct validation of a flexible sigmoidoscopy simulator by comparing training-level grouped subjects. These included clerical staff (n = 10), residents (n = 19), and experts (n = 5). Each participant performed 3 scopes. The ANOVA group-based results for trainer-measured variables are shown in Table 1. These results demonstrate that the flexible sigmoidoscopy simulator distinguished the trained from the untrained and the resident from the expert. Although there was no statistically significant differences between the senior residents and the experts, the expert commonly outperformed the residents. Establishing the transferability of simulator training to real life is next. If the transfer of skill can be established, it may give rise to a new skills training approach.

Self-assessment in simulation-based surgical skills training

BACKGROUND

Simulation-based training provides minimal feedback and relies heavily on self-assessment. Research has shown medical trainees are poor self-assessors. The purpose of this study was to examine trainees' ability to self-assess technical skills using a simulation-trainer.

METHODS

Twenty-one medical students performed 10 repetitions of a simulated task. After each repetition they estimated their time and errors made. These were compared with the simulator data.

RESULTS

Task time (P < 0.0001) and errors made (P < 0.0001) improved with repetition. Both self-assessment curves reflected their actual performance curves (P < 0.0001). Self-assessment of time did not improve in accuracy (P = 0.26) but error estimation did (P = 0.01) when compared with actual performance.

CONCLUSIONS

Novices demonstrated improved skill acquisition using simulation. Their estimates of performance and accuracy of error estimation improved with repetition. Clearly, practice enhances technical skill self-assessment. These results support the notion of self-directed skills training and could have significant implications for residency training programs.

Who are the surgery clerkship directors and what are their educational needs?

BACKGROUND

The surgery clerkship director is a key individual in the surgery department's educational mission and yet there has been no prior effort to describe this group or identify their learning needs. The purpose of this study was to develop a demographic profile and an educational needs assessment for surgery clerkship directors.

METHODS

A survey instrument was designed based on existing literature and distributed to surgery clerkship directors in the United States and Canada.

RESULTS

Surveys were returned from 108 subjects (77%). The majority of clerkship directors strongly agree that directing is a positive experience but express concern that the job demands may impede their professional careers. The perceived educational needs identified related primarily to the development and management of the student education curriculum.

CONCLUSIONS

Surgery clerkship directors are experienced academic surgeons who report high levels of satisfaction. They identify a number of important educational needs of the position and express concern about the requirements of the position on their academic careers.

The impact of a program for systematically recognizing and rewarding academic performance

PURPOSE

To describe an academic performance incentive system (APIS) and faculty perception of it; explore the impacts of incentive level, faculty rank, clinical practice volume, and administrative responsibility on academic productivity; and describe the APIS's use in maintaining congruence between department mission and activities.

METHOD

A list of teaching, research, and academic service activities was developed, which full-time faculty (n = 33) used to report activities. Clinical faculty members received incentive income based on credits earned. APIS initially distributed 1% of practice plan receipts (subsequently increased to 3% and then 5%). Productivity was measured by differences in APIS points achieved. Satisfaction of all faculty participants was measured by survey.

RESULTS

Faculty members (n = 20) who participated throughout averaged 22 credits per month (nine to 42 credits), and quarterly incentive bonuses ranged from 145 US dollars to 6,128 US dollars. Average credits earned per month were 24 for the 1% incentive, 23 for the 3% incentive, and 20 for the 5% incentive. Faculty members with administrative responsibilities were as productive academically as were their non-administrative counterparts. Senior faculty members were as productive as junior faculty. Faculty members who were more productive clinically were more productive academically. Seventy percent of respondents reported they were either very satisfied or somewhat satisfied with the APIS. Seventy-eight percent felt that the APIS accurately reflected their academic productivity. Most respondents (81%) felt that the amount of money allocated to the incentive system was appropriate (15% felt it should be increased and one respondent recommended reduction).

CONCLUSIONS

The APIS system has been well accepted by faculty and allows for data-driven discussion of the department's mission and activities.

Addressing the new competencies for residents' surgical training

In July 2001 the Accreditation Council for Graduate Medical Education (ACGME) charged U.S. residency training programs to implement a curriculum and evaluation plan covering six competencies. The authors describe the curriculum and evaluation strategy of the first surgical training program developed to meet the competencies, and list each competency and the teaching method and measurement instruments used. Implementation began July 1, 2001, and the program was fully operational on July 1, 2002. Meeting the curriculum challenges required modification of the existing curriculum and the addition of new instructional units. Nine additional evaluation instruments were needed. The largest investment was in planning and implementation, a one-time development cost. Staff workload increased by 252 hours; this is expected to be a continuing annual requirement. Faculty workload increased by two hours per resident and each resident's workload increased by 112 hours per year (2.3 hours per week). The transition was smoother than expected. Faculty and residents' buy-in was crucial. Faculty and residents were alerted to upcoming changes at the beginning of the year in a grand rounds presentation on the ACGME competencies and the approach to meeting requirements. Updates were presented periodically. The authors recommend that residency programs engaged in similar efforts make effective use of instruments developed elsewhere and collaborate with other programs rather than develop everything locally. The program's benefits include time savings and the availability of validity data and norms to inform decision making on residents' and program progress.

Cognitive, social and environmental sources of bias in clinical performance ratings

BACKGROUND

Global ratings based on observing convenience samples of clinical performance form the primary basis for appraising the clinical competence of medical students, residents, and practicing physicians. This review explores cognitive, social, and environmental factors that contribute unwanted sources of score variation (bias) to clinical performance evaluations.

SUMMARY

Raters have a 1 or 2-dimensional conception of clinical performance and do not recall details. Good news is reported more quickly and fully than bad news, leading to overly generous performance evaluations. Training has little impact on accuracy and reproducibility of clinical performance ratings.

CONCLUSIONS

Clinical performance evaluation systems should assure broad, systematic sampling of clinical situations; keep rating instruments short; encourage immediate feedback for teaching and learning purposes; encourage maintenance of written performance notes to support delayed clinical performance ratings; give raters feedback about their ratings; supplement formal with unobtrusive observation; make promotion decisions via group review; supplement traditional observation with other clinical skills measures (e.g., Objective Structured Clinical Examination); encourage rating of specific performances rather than global ratings; and establish the meaning of ratings in the manner used to set normal limits for clinical diagnostic investigations.

Familial vocal fold paralysis

Familial clustering of congenital bilateral abductor vocal fold paralysis has been reported very rarely. So far, only a handful of cases have been reported, mostly with the autosomal dominant of X-linked recessive mode of inheritance. We describe the cases of a brother and sister, who presented with neonatal stridor due to bilateral abductor vocal fold paralysis. First-degree parental consanguinity suggests an autosomal recessive mode of inheritance. Karyotype analysis revealed a paracentric balanced inversion of chromosome 13 in both cases, that was also present in the unaffected mother. An updated review of the literature on this interesting but rare condition is also presented.

Large-volume, coolable spectroscopic cell for aerosol studies

We have constructed a coolable spectroscopic cell for characterizing the physical and chemical properties of simulated atmospheric aerosol particles. The cell is designed for experiments in which the refractive indices, freezing temperatures, and the phase and chemical composition of a wide range of aerosol types are measured. The relatively large volume (0.075 m(3)) of the cell reduces wall-aerosol interactions and allows the aerosol residence time to exceed 2 h. The cell has been optically interfaced to Fourier-transform spectrometers to record broadband infrared, visible, and ultraviolet extinction spectra of aerosol particles and gas-phase components over a range of temperatures (180-300 K). The data generated with the cell have applications in remote sensing, radiative transfer models, heterogeneous atmospheric chemistry, and pollution studies.

Body image and weight consciousness among South Asian, Italian and general population women in Britain

Italians in Britain have low rates of coronary heart disease while South Asians have high rates, which correspond to a tendency to central abdominal fat deposition and overweight. World variations in attitudes to body size are thought to be related to economic security. This cross-sectional study employed a range of measures including photographic silhouettes of known BMI to investigate the attitudes of 259 South Asian, Italian and general population women (aged 20-42 years) towards body size. Migrants are compared with British-born minority members. Our results indicate that although migrant South Asians were less happy with their weight than migrant Italians, fewer had tried to lose weight in the past or had experienced external pressures to change their bodies. More migrant South Asians than Italians or general population women equated one of the four largest shapes (BMI 28-38) with health and successful reproduction. All groups wanted to resemble one of the two thinnest shapes, equating them with longevity, likelihood of marriage and job success. British-born South Asians generally showed a considerable degree of convergence towards general population women's negative attitudes to large body size, but British-born Italians' attitudes were significantly more negative even than general population women. The study's conclusions were that South Asian health beliefs are an important focus of resistance to slimness. The tendency of migrant South Asians to equate large size with health contrasts with the opposing views of Italian and general population women. British-born South Asians' views are modifying from those of migrants, but significant differences remain when compared with general population women and British-born Italians. Present differences in economic security offer only a partial explanation; South Asian attitudes may be explained by economic insecurity in the past.