General principles to consider when designing a clinical communication assessment program

Do the teaching, practice and assessment of clinical communication skills align?

BACKGROUND

Evidence indicates that communication skills teaching learnt in the classroom are not often readily transferable to the assessment methods that are applied nor to the clinical environment. An observational study was conducted to objectively evaluate students' communication skills in different learning environments. The study sought to investigate the extent to which the communication skills demonstrated by students in classroom, clinical, and assessment settings align.

METHOD

A mixed methods study was conducted to observe and evaluate students during the fourth year of a five-year medical program. Participants were videorecorded during structured classroom 'interactional skills' sessions, as well as clinical encounters with real patients and an OSCE station calling upon communication skills. The Calgary Cambridge Observational Guides was used to evaluate students at different settings.

RESULT

This study observed 28 students and findings revealed that while in the classroom students were able to practise a broad range of communication skills, in contrast in the clinical environment, information-gathering and relationship-building with patients became the focus of their encounters with patients. In the OSCEs, limited time and high-pressure scenarios caused the students to rush to complete the task which focussed solely on information-gathering and/or explanation, diminishing opportunity for rapport-building with the patient.

CONCLUSION

These findings indicate a poor alignment that can develop between the skills practiced across learning environments. Further research is needed to investigate the development and application of students' skills over the long term to understand supports for and barriers to effective teaching and learning of communication skills in different learning environments.

Peer assessment in medical communication skills training in programmatic assessment: A qualitative study examining faculty and student perceptions

INTRODUCTION

Current literature recommends assessment of communication skills in medical education combining different settings and multiple observers. There is still a gap in understanding about whether and how peers assessment facilitates learning in communication skills training.

METHODS

We designed a qualitative study using focus group interviews and thematic analysis, in a medical course in the Netherlands. We aimed to explore medical students' and teachers' experiences, perceptions, and perspectives about challenges and facilitating factors in PACST (Peer assessment in medical communication skills training).

RESULTS

Most of the participants reported that peer feedback was a valuable experience when learning communication skills. The major challenges for the quality and credibility of PACST reported by the participants are the question whether peer feedback is critical enough for learning and the difficulty of actually engaging students in the assessment process.

CONCLUSION

Teachers reviewing students' peer assessments may improve the quality and their credibility and the reviewed assessments can best be used for learning purposes. We suggest to pay sufficient attention to teachers' roles in PACST, ensuring a safe and trustworthy environment and additionally helping students to internalize the value of being vulnerable during the evaluation process.

Does it make sense to use written instruments to assess communication skills? Systematic review on the concurrent and predictive value of written assessment for performance

OBJECTIVES

To evaluate possible associations between learners' results in written and performance-based assessments of communication skills (CS), either in concurrent or predictive study designs.

METHODS

Search included four databases for peer-reviewed studies containing both written and performance-based CS assessment. Eleven studies met the inclusion criteria.

RESULTS

Included studies predominantly assessed undergraduate medical students. Studies reported mainly low to medium correlations between written and performance-based assessment results (Objective Structured Clinical Examinations or encounters with simulated patients), and gave correlation coefficients ranging from 0.13 to 0.53 (p < 0.05). Higher correlations were reported when specific CS, like motivational interviewing were assessed. Only a few studies gave sufficient reliability indicators of both assessment formats.

CONCLUSIONS

Written assessment scores seem to predict performance-based assessments to a limited extent but cannot replace them entirely. Reporting of assessment instruments' psychometric properties is essential to improve the interpretation of future findings and could possibly affect their predictive validity for performance.

PRACTICE IMPLICATIONS

Within longitudinal CS assessment programs, triangulation of assessment including written assessment is recommended, taking into consideration possible limitations. Written assessments with feedback can help students and trainers to elaborate on procedural knowledge as a strong support for the acquisition and transfer of CS to different contexts.

Developing communication curricula in healthcare education: An evidence-based guide

OBJECTIVE

To present a guide for communication curriculum development in healthcare professions for educators and curriculum planners.

METHODS

We collated a selection of theories, frameworks and approaches to communication curriculum development to provide a roadmap of the main factors to consider when developing or enhancing communication skills curricula.

RESULTS

We present an evidence-based guide for developing and enhancing communication curriculum that can be applied to undergraduate and postgraduate healthcare education. Recommended steps to consider during the communication curricula development process include thoughtful examination of current communication education, needs assessment, focused learning goals and objectives, incorporation of experiential educational strategies allowing for skills practice and feedback and use of formative and summative assessment methods. A longitudinal, developmental and helical implementation approach contributes to reinforcement and sustainment of learners' knowledge and skills.

CONCLUSION AND PRACTICE IMPLICATIONS

Drawing on best practices in developing communication curricula can be helpful in ensuring successful approaches to communication skills training for any level of learner or healthcare profession. This position paper provides a guide and identifies resources for new and established communication curriculum developers to reflect on strengths and opportunities in their own approaches to addressing the communication education needs of their learners.

What do we know about written assessment of health professionals' communication skills? A scoping review

OBJECTIVE

The aim of this scoping review was to investigate the published literature on written assessment of communication skills in health professionals' education.

METHODS

Pubmed, Embase, Cinahl and Psychnfo were screened for the period 1/1995-7/2020. Selection was conducted by four pairs of reviewers. Four reviewers extracted and analyzed the data regarding study, instrument, item, and psychometric characteristics.

RESULTS

From 20,456 assessed abstracts, 74 articles were included which described 70 different instruments. Two thirds of the studies used written assessment to measure training effects, the others focused on the development/validation of the instrument. Instruments were usually developed by the authors, often with little mention of the test development criteria. The type of knowledge assessed was rarely specified. Most instruments included clinical vignettes. Instrument properties and psychometric characteristics were seldom reported.

CONCLUSION

There are a number of written assessments available in the literature. However, the reporting of the development and psychometric properties of these instruments is often incomplete. Practice implications written assessment of communication skills is widely used in health professions education. Improvement in the reporting of instrument development, items and psychometrics may help communication skills teachers better identify when, how and for whom written assessment of communication should be used.

What does the structure of a medical consultation look like? A new method for visualising doctor-patient communication

OBJECTIVE

This project developed an innovative methodology for visualising consultation structure by categorising doctor-patient talk into the phases proposed by an established educational model of clinical communication.

METHOD

Consultation phases were identified from verbatim transcripts using the tasks and process skills of the Calgary-Cambridge Guide to the Medical Interview. Seventy-eight simulated consultations from a 'History-taking' station of a postgraduate examination for physicians were analysed by two independent raters. Transcripts were converted into diagrams comprising up to six phases: Initiating, Gathering information, Summary, Explanation, Planning and Closing.

RESULTS

The dominant phases were Gathering information, Planning and Explanation (66 %, 10 % and 12 % of talk respectively). While consultations broadly followed the expected chronological sequence, less than a third (23/78) contained all six phases, with Closing and Summary most frequently absent. Half of consultations (40/78) did not include phases in the predicted order, with intertwined phases commonly observed.

CONCLUSIONS

In this standardised setting, doctors created variable consultation structures, typically omitting phases involving consolidation and agreement of plans going forward.

PRACTICE IMPLICATIONS

The method enables visualisation and comparison of consultation structure. The findings pose questions about the alignment of practice with educational guidance and the opportunities afforded to patients to actively engage in consultations.

Cross-Cultural Adaptation of the Communication Assessment Tool for Use in a Simulated Clinical Setting

Construct: The Communication Assessment Tool (CAT) is a 14-item instrument developed in English to assess medical trainees' interpersonal communication skills from the patient's perspective in clinical settings. Background: Using validated instruments and simulated patients constitutes good practice in assessing doctor-patient communication. The CAT was designed for use in real practice, but has not yet been applied to assessing OB-GYN residents' delivery of bad news in Objective Structured Clinical Examination (OSCE) stations. This study aims to provide validity evidence for using the CAT to assess residents' interpersonal communication skills under difficult circumstances in a simulated clinical setting in Brazil. Approach: Cross-cultural adaptation comprised translation into Portuguese, synthesis of translations, and back-translation. Next, a committee of 10 external and independent experts rated the items for linguistic equivalence and relevance to the overall scale. Researchers used the expert ratings to produce a preliminary Brazilian-Portuguese version. This version was applied by four simulated patients to assess 28 OB-GYN residents completing two, 10-minute OSCE stations focused on delivering bad news. Item and scale content validity indices and internal-consistency reliability were calculated. Simulated patients were interviewed to clarify any doubt regarding the content and usability of the tool and their response process. Findings: Thirteen of the 14 items in the Brazilian-Portuguese version were considered "equivalent" by at least 70% of the experts. All items were considered relevant by 100% of the experts. The Item Content Validity Index ranged from .9 to 1, and the Scale Content Validity Index was .99. The instrument showed good reliability for both scenarios (Cronbach's alpha > .90). Simulated patients considered the CAT easy to understand and complete. Conclusions: This study provides validity evidence for using the Brazilian-Portuguese CAT in a simulated clinical environment to assess OB-GYN residents' delivery of bad news. Based on this study's findings, the OB-GYN Department organized an annual formative assessment for residents to improve their interpersonal communication skills. This version of the CAT may also be applicable to other specialties.

Improving the assessment of communication competencies in a national licensing OSCE: lessons learned from an experts' symposium

BACKGROUND

As the communication competencies of physicians are crucial for providing optimal patient care, their assessment in the context of the high-stakes Objective Structured Clinical Examination (OSCE) is of paramount importance. Despite abundant literature on the topic, evidence-based recommendations for the assessment of communication competencies in high stakes OSCEs are scarce. As part of a national project to improve communication-competencies assessments in the Swiss licensing exam, we held a symposium with national and international experts to derive corresponding guidelines.

METHODS

Experts were invited on account of their recognized expertise either in teaching or assessing communication competencies, or in conducting national high-stakes OSCEs. They were asked to propose concrete solutions related to four potential areas for improvement: the station design, the rating tool, the raters' training, and the role of standardized patients. Data gene.rated in the symposium was available for analysis and consisted of video recordings of plenary sessions, of the written summaries of group work, and the cards with participants' personal take-home messages. Data were analyzed using a thematic analysis approach.

RESULTS

Nine major suggestions for improving communication-competencies assessments emerged from the analysis and were classified into four categories, namely, the roles of the OSCE scenarios, rating tool, raters' training, and simulated patients.

CONCLUSION

In the absence of established evidence-based guidelines, an experts' symposium facilitated the identification of nine practical suggestions for improving the assessment of communication competencies in the context of high-stakes OSCEs. Further research is needed to test effectiveness of the suggestions and how they contribute to improvements in the quality of high-stakes communication-competencies assessment.