Combined formative and summative professional behaviour assessment approach in the bachelor phase of medical school: a Dutch perspective

BACKGROUND

Teaching and assessment of professional behaviour (PB) has been receiving increasing attention in the educational literature and educational practice. Although the focus tends to be summative aspects, it seems perfectly feasible to combine formative and summative approaches in one procedural approach.

AIMS AND METHOD

Although, many examples of frameworks of professionalism and PB can be found in the literature, most originate from North America, and only few are designed in other continents. This article presents the framework for PB that is used at Maastricht medical school, the Netherlands.

RESULTS

The approach to PB used in the Dutch medical schools is described with special attention to 4 years (2005-2009) of experience with PB education in the first 3 years of the 6-year undergraduate curriculum of Maastricht medical school. Future challenges are identified.

CONCLUSIONS

The adages 'Assessment drives learning' and 'They do not respect what you do not inspect' [Cohen JJ. 2006. Professionalism in medical education, an American perspective: From evidence to accountability. Med Educ 40, 607-617] suggest that formative and summative aspects of PB assessment can be combined within an assessment framework. Formative and summative assessments do not represent contrasting but rather complementary approaches. The Maastricht medical school framework combines the two approaches, as two sides of the same coin.

Quality of communication during telephone triage at Dutch out-of-hours centres

OBJECTIVE

To assess the quality of communication skills of triagists, working at out-of-hours (OOH) centres, and to determine the correlation between the communication score and the duration of the telephone consultation.

METHODS

Telephone incognito standardised patients (TISPs) called 17 OOH centres presenting different clinical cases. The assessment of communication skills was carried out using the RICE-communication rating list. The duration of each telephone consultation was determined.

RESULTS

The mean overall score for communication skills was 35% of the maximum feasible. Triagists usually asked questions about the clinical situation correctly and little about the patients' personal situation, perception of the problem or expectation. Advice about the outcome of triage and self-care advice was usually given without checking for patients' understanding and acceptance of the advice. Calls were often handled in an unstructured way, without summarizing or clarifying the different steps within the consultation. There was a positive correlation of 0.86 (p<0.01) between the overall communication score and the duration of the telephone consultation.

CONCLUSION

Assessment of communication skills of triagists revealed specific shortcomings and learning points to improve the quality of communication skills during telephone triage.

PRACTICE IMPLICATIONS

Training in telephone consultation should focus more on patient-centred communication with active listening, active advising and structuring the call. Apart from adequate communication skills, triagists need sufficient time for telephone consultation to enable high quality performance.

Quality of clinical aspects of call handling at Dutch out of hours centres: cross sectional national study

OBJECTIVE

To assess the quality of telephone triage by following the consecutive phases of its care process and the quality of the clinical questions asked about the patient's clinical condition, of the triage outcome, of the content of the home management advice, and of the safety net advice given at out of hours centres.

DESIGN

Cross sectional national study using telephone incognito standardised patients.

SETTING

The Netherlands.

PARTICIPANTS

17 out of hours centres.

MAIN OUTCOME MEASURES

Percentages of clinical obligatory questions asked and items within home management and safety net advice, both in relation to pre-agreed standards, and of care advice given in relation to the required care advice.

RESULTS

The telephone incognito standardised patients presented seven clinical cases three times each over a period of 12 months, making a total of 357 calls. The mean percentage of obligatory questions asked compared with the standard was 21%. Answers to questions about the clinical condition were not always correctly evaluated from a clinical viewpoint, either by triagists or by general practitioners. The quality of information on home management and safety net advice varied, but it was consistently poor for all cases and for all out of hours centres. Triagists achieved the appropriate triage outcome in 58% of calls.

CONCLUSION

In determining the outcome of the care process, triagists often reached a conclusion after asking a minimal number of questions. By analysing the quality of different phases within the process of telephone triage, evaluation of whether an appropriate triage outcome has been arrived at by means of good clinical reasoning or by an educated guess is possible. In terms of enhancing the overall clinical safety of telephone triage, apart from obtaining an appropriate clinical history, adequate home management and safety net advice must also be given.

Assessing communication skills of clinical call handlers working at an out-of-hours centre: development of the RICE rating scale

BACKGROUND

Out-of-hours centres provide telephone support to patients with medical problems. In most of these centres specially-trained nurses handle incoming telephone calls. They assess patients' needs, the degree of urgency, and determine the level of care required. Assessment of the medical problem and the quality of 'care-by-phone' depend on the medical and communication skills of the call handlers.

AIM

To develop a valid, reliable, and practical rating scale to evaluate the communication skills of call handlers working at an out-of-hours centre and to improve quality of communication.

DESIGN OF STUDY

Qualitative study with focus groups followed by validation of the rating scale and measurement of reliability (internal consistency).

SETTING

Out-of-hours centres in the Netherlands.

METHOD

A focus group developed the rating scale. Experts with experience in training and evaluating communication skills of medical students and GPs commented on the scale to ensure content validity. The reliability of the rating scale was tested in a pilot in which ten specially-trained assessors scored six telephone calls each.

RESULTS

The scale, known as the RICE rating scale, has 17 items divided over four different phases of the telephone consultation: Reason for calling; Information gathering; Conclusion; and Evaluation (RICE). Content validity of the scale was assessed by two experts. Reliability of the scale tested in the pilot was 0.73 (Cronbach's alpha).

CONCLUSION

Establishing a rating scale to assess the communication skills of call handlers which meets common scientific demands, such as content validity and reliability, proved successful. This instrument can be used to give feedback to call handlers.

Fielding incognito standardised patients as 'known' patients in a controlled trial in general practice

BACKGROUND

Incognito standardised patients (SPs) have only been used to represent new patients so far. The few trials with incognito SPs provide little detail on the method used for fielding them.

OBJECTIVE

To establish the feasibility of introducing SPs as 'known' patients (i.e. patients who have previously visited the practice) into practices, to indicate the required practice preparations, and to describe the various aspects of using SPs in a pretest/post-test design.

METHODS

We used incognito SPs as known patients in a controlled trial to assess the practice behaviour of 49 trainees. The SPs received a 2-day training in role playing and completing checklists. We compiled comprehensive practice information folders of each practice to be visited. Real personal data and faked medical data of SPs were inserted in the filing system of each practice to be visited. Apart from SP roles with slightly different reasons-for-encounter and different SPs, the same training protocol, checklists and practice information folders were used in the post-test.

RESULTS

The SPs carried out 287 visits in 50 practices. All practices prepared the patient records for the SP visits in a fairly authentic practice style. Trainees detected the SP in 74 visits. The main reasons for detection were imperfections in patient records and aspects of SP roles or role playing.

CONCLUSION

Fielding incognito SP visits as known patients was feasible but labour-intensive. Preparing the SP patient records and familiarising SPs with the interior of practices represented new elements in the use of SPs. The pretest/post-test format made their use more efficient instead of complicating it.

GPs' needs for practice-oriented nutrition education; a Delphi study among Dutch GPs

BACKGROUND

Published guidelines and lists of topics in nutrition education for GPs are affected by practical drawbacks, which prevent them from being easily implemented in vocational training curricula.

OBJECTIVE

Our aim was to draw up a concise priority list of disease-related nutrition topics reflecting the needs among Dutch GPs for nutrition education geared to everyday practice, which can be used to decide on the main topics and educational content of computer-based instruction.

METHODS

This was a two-round Delphi study, using pilot-tested questionnaires carried out among 41 GPs who attended a national CME meeting on nutrition in The Netherlands. Sum scores of, and summarized comments on nutrition topics and their frequencies in top-10 lists drawn up by respondents were the main outcome measures.

RESULTS

The response in both Delphi rounds was high (93 and 95%). Analysis of questionnaires in both rounds yielded a priority list of disease-related nutrition topics, with the 14 highest ranking topics identical in both rounds. A fifteenth topic, only found in the second questionnaire, completed the priority list. A group of experts assessed this list as fully acceptable.

CONCLUSION

It was possible to compile a priority list of disease-related nutrition topics reflecting the needs of GPs for nutrition education. Based on the results of the Delphi study, we generated the following list of five priority topics: weight problems; diabetes mellitus; hypercholesterolaemia; intestinal complaints; and hypertension. This list of five and the summarized comments of respondents will determine the main topics and educational content of a computer-based instruction we are currently developing.

Controlled trial of effect of computer-based nutrition course on knowledge and practice of general practitioner trainees

BACKGROUND

Nutrition education is not an integral part of either undergraduate or postgraduate medical education. Computer-based instruction on nutrition might be an attractive and appropriate tool to fill this gap.

OBJECTIVE

The study objective was to assess the degree to which computer-based instruction on nutrition improves factual knowledge and practice behavior of general practitioner (GP) trainees.

DESIGN

We carried out a controlled experimental study, using a 79-item knowledge test and 3 incognito standardized patients' visits in a pre- and posttest design with 49 first-year GP trainees. The experimental group (n = 25) received an average of 6 h of a newly developed computer-based instruction on nutrition. The control subjects (n = 24) took the standard vocational training program.

RESULTS

The percentage of correct answers on the knowledge test increased from 30% at pretest to 42% at posttest in the experimental group, and from 36% to 37% in the control group. Analysis of covariance, with the pretest scores as covariate, showed a significant experimental versus control group difference at posttest: 9.2% (P = 0.002). The mean percentage of correctly performed items during the 3 standardized patients' visits (assessed by checklists) showed an increase in the experimental group from 20% at pretest to 36% at posttest, whereas the control group changed from 20% to 22%. Analysis of covariance, with the pretest scores as covariate, revealed a significant group difference at posttest: 13.7% (P < 0.001).

CONCLUSION

The computer-based instruction proved its effectiveness, both by increasing factual knowledge and by substantially enhancing GP trainees' practice behavior on the subject of nutrition.