Clinical Psychology Students’ Experiences of a Pilot Objective Structured Clinical Examination

Examining Skills and Abilities During the Pandemic - Psychology Students' and Examiners' Perceptions of a Digital OSCE

Finding valid and reliable ways to assess complex clinical skills within psychology is a challenge. Recently, there have been some examples of applying Objective Structured Clinical Examinations (OSCEs) in psychology for making such assessments. The aim of this study was to examine students' and examiners' perceptions of a digital OSCE in psychology regarding quality and students' feelings about the OSCE. Participants were 51 students enrolled in the Programme for Master of Science in Clinical Psychology during two semesters and nine examiners assessing each OSCE occasion, at Umeå University, Sweden. Web-based questionnaires were used for data collection. Psychometric analyses indicated that the subscales in the student questionnaire had adequate or close to adequate levels of item and scale reliability. Both students and examiners felt that the digital OSCE was realistic, valid and well-aligned with professional practice. Although students perceived the digital OSCE as stressful, the results showed that they were focused and concentrated and found the OSCE to be a positive learning experience, implying that the stress did not affect performance to any significant extent. Based on the examiners' experiences, it can be concluded that there are both advantages and disadvantages which need to be considered when planning future digital OSCEs.

Lessons Learned and Preliminary Results from Implementing Simulation-Based Elements in a Clinical Psychology Programme

In a clinical psychology training context, there is a need to examine students’ theoretical knowledge as well as their professional competence. One promising method to assess students’ professional competence is the Objective and Structured Clinical Examination (OSCE). In this report, we describe and discuss the implementation of OSCE on a clinical psychology programme at a university in Sweden, including lesson learned regarding the structure and content for this examination. We also report on preliminary results, in which we explored students’ perceived competence and worries, and their supervisors’ reports regarding their clinical practicum, in relation to a new curriculum that includes more simulation-based elements (including the OSCE) than the old curriculum. Results showed that students on the new curriculum reported lower levels of perceived competence before the clinical practicum, but increased significantly more over time in comparison to students on the old curriculum. These results are discussed in relation to the potential role of OSCE in clinical psychology students’ development of professional competence. Due to methodological limitations, these results should be interpreted with caution and should be viewed as exploratory. All in all, this report can be viewed as a guideline for implementation of OSCE on similar programmes in psychology.

Einsatz von Simulationspersonen in der Lehre und Prüfung psychotherapeutischer Kompetenzen

Zusammenfassung. Theoretischer Hintergrund: In der Klinischen Psychologie und Psychotherapie werden bislang kaum Simulationspersonen (SPs) in Lehre und kompetenzorientierten Prüfungen eingesetzt. Deren geplante Nutzung etwa im Rahmen von Approbationsprüfungen nach dem neuen Psychotherapeutengesetz ist sinnvoll, jedoch mit inhaltlichen und methodischen Herausforderungen verknüpft. Fragestellung: Machbarkeit des Einsatzes von SPs in Prüfungsszenarien und Ableitung von relevanten therapeutischen Fertigkeiten, die in solchen Szenarien geprüft werden können. Methode: Quantitative Evaluation der Umsetzbarkeit einer SP-basierten Prüfung psychotherapeutischer Kompetenzen (Studie 1) und qualitative Erarbeitung therapeutischer Beziehungsfertigkeiten als Basis für Szenarien in kompetenzorientierten Prüfungen mit SPs (Studie 2). Ergebnisse: Die Evaluation der SP-basierten Prüfungen zeigte eine positive Bewertung hinsichtlich ihrer Machbarkeit durch Ausbildungsteilnehmende ( N = 48) und Prüfende ( N = 2), aber auch eine begrenzte Interrater-Reliabilität. In der zweiten Studie wurden aus Interviews mit Expertinnen und Experten ( N = 12 Psychotherapeutinnen und -therapeuten) Kompetenzen für Beziehungsaufbau und Aufrechterhaltung als Grundlage für SP-basierte Prüfungsszenarien aus den Bereichen Fertigkeiten, Haltung und Wissen abgeleitet. Schlussfolgerungen: SP-basierte Lehr- und Prüfungsformate in Psychotherapie und Klinischer Psychologie sind mit inhaltlichen und methodischen Anforderungen (u. a. Falldarstellung, Interrater-Reliabilität) verbunden, die bei der Implementierung berücksichtigt werden müssen.

Comparison of electronic versus conventional assessment methods in ophthalmology residents; a learner assessment scholarship study

BACKGROUND

Assessment is a necessary part of training postgraduate medical residents. The implementation of methods located at the "shows how" level of Miller's pyramid is believed to be more effective than previous conventional tools. In this study, we quantitatively compared electronic and conventional methods in assessing ophthalmology residents.

METHODS

In this retrospective study, eight different conventional methods of assessment including residents' attendance, logbook, scholarship and research skills, journal club, outpatient department participation, Multiple Choice Question (MCQ), Objective Structured Clinical Examination (OSCE), and professionalism/360-degree (as one complex) were used to assess 24 ophthalmology residents of all grades. Electronic media consisting of an online Patient Management Problem (e-PMP), and modified electronic OSCE (me-OSCE) tests performed 3 weeks later were also evaluated for each of the 24 residents. Quantitative analysis was then performed comparing the conventional and electronic assessment tools, statistically assessing the correlation between the two approaches.

RESULTS

Twenty-four ophthalmology residents of different grades were included in this study. In the electronic assessment, average e-PMP scores (48.01 ± 12.40) were much lower than me-OSCE (65.34 ± 17.11). The total average electronic score was 56.67 ± 11.28, while the total average conventional score was 80.74 ± 5.99. Female and male residents' average scores in the electronic and conventional method were (59.15 ± 12.32 versus 83.01 ± 4.95) and (55.19 ± 10.77 versus 79.38 ± 6.29), respectively. The correlation between modified electronic OSCE and all conventional methods was not statistically significant (P-value >0.05). Correlation between e-PMP and six conventional methods, consisting of professionalism/360-degree assessment tool, logbook, research skills, Multiple Choice Questions, Outpatient department participation, and Journal club active participation was statistically significant (P-value < 0.05). The overall correlation between conventional and electronic methods was significant (P-value = 0.017).

CONCLUSION

In this study, we conclude that electronic PMP can be used alongside all conventional tools, and overall, e-assessment methods could replace currently used conventional methods. Combined electronic PMP and me-OSCE can be used as a replacement for currently used gold-standard assessment methods, including 360-degree assessment.

Konzeption des longitudinalen DYNAMIK-Curriculums

Hintergrund

Durch die Einführung des neuen „Psychotherapiestudiums“ zum Herbst 2020 und die Wünsche bisheriger Ausbildungsteilnehmer an psychotherapeutischen Ausbildungsinstituten ist die Notwendigkeit gegeben, erfahrungsbasierte und kompetenzorientierte Lehre in das zukünftige Studium und die nachfolgende Weiterbildung werdender Psychotherapeuten zu integrieren.

Material und Methode

Orientiert am Kern-Zyklus, einem bedeutenden Modell zur Curriculumsentwicklung, werden die Entwicklung, Konzeptionalisierung und Implementierung eines longitudinalen Curriculums zur Vermittlung psychodynamischer Interventionskompetenzen (DYNAMIK-Curriculum) in die derzeitige Ausbildung zum psychologischen Psychotherapeuten vorgestellt.

Ergebnisse

Es wurde ein longitudinales Curriculum mit 8 Modulen zur Vermittlung allgemeiner und spezifischer psychodynamischer Interventionskompetenzen mithilfe von Peer-Rollenspielen und standardisierten Patienten entwickelt. Dieses stellt seit Mitte 2019 einen integralen Bestandteil der Ausbildung zum tiefenpsychologisch fundierten Psychotherapeuten am Heidelberger Institut für Psychotherapie (HIP) dar.

Schlussfolgerung

Das DYNAMIK-Curriculum ist der erste wichtige Schritt in Richtung der longitudinalen erfahrungsorientieren, praxisnahen Kompetenzorientierung in der derzeitigen Ausbildung und dem zukünftigen Studium mit nachfolgender Weiterbildung von psychologischen Psychotherapeuten. Es resultiert die Notwendigkeit zur Wirksamkeitsüberprüfung solcher Curricula und zur praktischen Prüfung von Interventionskompetenzen.

Standardized Patients in Clinical Psychology: From Research to Practice

Background: Within the pending reform of the German law of psychotherapy training, education in clinical psychology and the examination of psychotherapeutic competencies are established as more practice oriented. Students will acquire and demonstrate therapeutic skills through interactions with standardized patients (SPs). The aim of the current paper is to enhance evidence-based implementation of these new methods of education and examination by presenting the current evidence regarding the use of SPs and by pointing out areas for further research. Results: Results of recent studies demonstrate that SPs are able to present mental disorders authentically. Prerequisites are, among others, the selection of suitable SPs, detailed role scripts, specific training, feedback, and corrective training. Although some research questions, including the comparative effectiveness of SPs, remain unanswered, practice implications for using SPs in education, examination, and research can be drawn. These implications are illustrated schematically. Conclusions: The use of SPs has large potential for education in clinical psychology and for research on psychotherapy training. With a view to encouraging the widespread use of SPs, we provide exemplary materials (e.g., role script) within the online supplements (for all online suppl. material, see www.karger.com/doi/10.1159/000510049).

Einsatz standardisierter Patienten im Psychologiestudium: Von der Forschung in die Praxis

<b><i>Hintergrund:</i></b> Im Rahmen des reformierten Psychotherapeutengesetzes wird eine stärkere Praxisorientierung in der klinisch-psychologischen Lehre und in der Prüfung psychotherapeutischer Kompetenzen verankert. Hierbei sollen Studierende durch die Interaktion mit standardisierten Patient*innen (SP) therapeutische Kompetenzen erwerben und demonstrieren. <b><i>Fragestellung:</i></b> Das Ziel des vorliegenden Beitrags ist es, eine evidenzbasierte Umsetzung dieser neuen Lehr- und Prüfungsformate zu unterstützen, indem bisherige Forschungsbefunde zum Einsatz von SP dargestellt und Bereiche, in denen weitere Forschung notwendig ist, aufgezeigt werden. <b><i>Ergebnisse:</i></b> Empirische Befunde zeigen, dass SP psychische Störungen authentisch darstellen können. Voraussetzung dafür sind beispielsweise die Auswahl geeigneter SP, detaillierte Rollenanleitungen, spezifisches Training, Feedback und Nachschulungen. Auch wenn einige Forschungsfragen, wie zur vergleichenden Wirksamkeit des Einsatzes von SP, noch unbeantwortet sind, lassen sich praktische Implikationen für SP-Programme in Lehre, Prüfung und Forschung ableiten, die in einem Ablaufschema dargestellt werden. <b><i>Schlussfolgerungen:</i></b> Der Einsatz von SP bietet großes Potenzial für die klinisch-psychologische Lehre und Ausbildungsforschung. Um den Einsatz von SP an anderen Standorten zu unterstützen, werden Beispielmaterialien (z.B. Rollenanleitung) in den elektronischen Supplementen (siehe www.karger.com/doi/10.1159/000509249 für alle Supplemente) zum Artikel zur Verfügung gestellt.

Standardized patients in psychotherapy training and clinical supervision: study protocol for a randomized controlled trial

Background

Psychotherapy is highly effective and widely acknowledged for treating various mental disorders. Nevertheless, in terms of methods for teaching effective psychotherapeutic approaches and competencies, there has been a lack of investigation. Training and supervision are the main strategies for teaching therapist competencies, and standardized role-plays with simulated patients (i.e., trained individuals playing someone with a mental disorder) seem useful for evaluating training approaches. In medical education, this procedure is now internationally established. However, so far, little use has been made of standardized role-playing to evaluate training and supervision in the area of clinical psychology and psychotherapy.

Methods

In this study, standardized role-plays are used to evaluate methods for training and supervision. Central cognitive behavioral approaches for treating depression are taught in the training. The first experiment compares an active training approach (i.e., model learning) with a passive one (i.e., reading manual-based instructions). The second experiment compares a direct supervision technique (i.e., supervision based on video analysis) with an indirect one (i.e., supervision based on verbal reporting). In each experiment, 68 bachelor’s and master’s students of psychology will be randomly assigned to the experimental and control groups. Each student takes part in three role-plays (baseline, post and 3-month follow-up), which are all videotaped. Two independent raters assess therapist competencies in each role-play on the basis of a standardized competence scale.

Discussion

The research project aims to contribute to the development of specific training and supervision methods in order to improve psychotherapy training and patient care.

Trial registration

ISRCTN Registry, ISRCTN19173895. Registered on 10 December 2019.

Constructive Alignment in the Field of Educational Psychology: Development and Application of a Questionnaire for Assessing Constructive Alignment

Teaching of Psychology includes a great variety of topics, course formats, and assessment approaches. A central concept that incorporated the interface between teaching goals, instructional methods, and examination modalities is referred to as Constructive Alignment (CA). This model addresses possible designs of teaching to improve students’ learning outcomes as well as enhance their learning experiences, and claims to be applicable independent of disciplinary culture or content. Despite the importance of this approach from an instructional point of view, there is hardly any research, so far, that has been concerned with capturing the three dimensions of CA. As a consequence, the aim of our study was to create an instrument to assess the quality of CA within psychology classes. A questionnaire was designed and was additionally analyzed with regard to students’ judgements about overall course evaluation. The questionnaire was employed in two lectures within the field of educational psychology for teacher training students. Results reveal that overall course evaluation can be predicted by the match between course objectives and instructional methods whereas other course evaluation factors failed as predictors. With a high internal consistency, the instrument provides an alternative or a supplement for traditional course evaluation instruments.

State of the Art on the Use of Portable Digital Devices to Assess Stress in Humans

The aim of this study is to review the current tools for the assessment of stress response in humans, ranging from the use of psychological questionnaires to the latest tools involving portable digital devices. Practical implications in educational context are further discussed.

The sights and insights of examiners in objective structured clinical examinations

PURPOSE

The objective structured clinical examination (OSCE) is considered to be one of the most robust methods of clinical assessment. One of its strengths lies in its ability to minimise the effects of examiner bias due to the standardisation of items and tasks for each candidate. However, OSCE examiners' assessment scores are influenced by several factors that may jeopardise the assumed objectivity of OSCEs. To better understand this phenomenon, the current review aims to determine and describe important sources of examiner bias and the factors affecting examiners' assessments.

METHODS

We performed a narrative review of the medical literature using Medline. All articles meeting the selection criteria were reviewed, with salient points extracted and synthesised into a clear and comprehensive summary of the knowledge in this area.

RESULTS

OSCE examiners' assessment scores are influenced by factors belonging to 4 different domains: examination context, examinee characteristics, examinee-examiner interactions, and examiner characteristics. These domains are composed of several factors including halo, hawk/dove and OSCE contrast effects; the examiner's gender and ethnicity; training; lifetime experience in assessing; leadership and familiarity with students; station type; and site effects.

CONCLUSION

Several factors may influence the presumed objectivity of examiners' assessments, and these factors need to be addressed to ensure the objectivity of OSCEs. We offer insights into directions for future research to better understand and address the phenomenon of examiner bias.

Application of best practice guidelines for OSCEs-An Australian evaluation of their feasibility and value

BACKGROUND

Objective Structured Clinical Examinations (OSCEs) are widely used in health professional education and should be based on sound pedagogical foundations.

OBJECTIVES

The aim of this study is to evaluate the feasibility and utility of using Best Practice Guidelines (BPGs) within an OSCE format in a broad range of tertiary education settings with under-graduate and post-graduate nursing and midwifery students. We evaluated how feasible it was to apply the BPGs to modify OSCEs in a course; students' perspective of the OSCE; and finally, if the BPG-revised OSCEs better prepared students for clinical practice when compared with the original OSCEs.

DESIGN

A mixed method with surveys, focus groups and semi-structured interviews evaluated the BPGs within an OSCE.

SETTINGS

Four maximally different contexts across four sites in Australia were used.

PARTICIPANTS

Participants included lecturers and undergraduate nursing students in high and low fidelity simulation settings; under-graduate midwifery students; and post-graduate rural and remote area nursing students.

RESULTS

691 students participated in revised OSCEs. Surveys were completed by 557 students; 91 students gave further feedback through focus groups and 14 lecturers participated in interviews. At all sites the BPGs were successfully used to modify and implement OSCEs. Students valued the realistic nature of the modified OSCEs which contributed to students' confidence and preparation for clinical practice. The lecturers considered the revised OSCEs enhanced student preparedness for their clinical placements.

DISCUSSION AND CONCLUSIONS

The BPGs have a broad applicability to OSCEs in a wide range of educational contexts with improved student outcomes. Students and lecturers identified the revised OSCEs enhanced student preparation for clinical practice. Subsequent examination of the BPGs saw further refinement to a set of eight BPGs that provide a sequential guide to their application in a way that is consistent with best practice curriculum design principles.

Using an Objective Structured Clinical Examination for Bachelor of Midwifery students' preparation for practice

BACKGROUND

Contemporary midwifery practice needs a rigorous and standardised assessment of practical skills, and knowledge to ensure that safety is maintained for both women and neonates before, during and after childbirth.

AIM

To evaluate the use of Best Practice Guidelines (BPG) for Objective Structured Clinical Examinations (OSCE) as a standardised tool to develop clinical competence of Bachelor of Midwifery students.

METHOD

A pragmatic mixed method approach with surveys, focus groups and interviews was used to evaluate the OSCEs for first year students. Quantitative and qualitative data were combined to understand student and academic perceptions of students' confidence for clinical practice following the OSCE.

FINDINGS

Thirty-four students responded to surveys (response rate 94%); and 13 participated in focus groups. Two academic lecturers participated in an interview (100%). Two main themes emerged (1) the OSCEs improved student confidence (2) the OSCEs were relevant and prepared students for practice. Most students indicated that they practised for the OSCE using an integrated approach (70%), and that this assisted them in their approach to the assessment of the neonate or post-partum mother.

CONCLUSION

The use of BPGs to ensure that OSCEs focus on important aspects of knowledge and practice helped students to learn and to perform well. Students' confidence in their ability for the imminent professional experience placement was high. OSCEs designed with the BPGs should be implemented broadly across midwifery education to enhance students' competence and provide rigorous meaningful assessment.

The viva voce as an authentic assessment for clinical psychology students

Mental health consumers in Australia have a basic right of access to high-quality mental health assessment and treatment. Given that universities are the entry point to the training process, they have a responsibility to train high-quality clinicians. Clinical psychology is the sub-discipline of psychology with specialist training in mental health assessment, diagnosis and treatment. There has been a call for clinical psychology programs to assessment away from a largely theoretical base to one that integrates theory with practice. In 2008, the University of Southern Queensland developed a viva voce for a course on adult psychopathology. The viva voce is an Objective Structured Clinical Examination, modeled closely on the one used by the Royal College of Australian and New Zealand Psychiatrists. This paper outlines the development of this innovation to assessment and gives directions for future research and evaluation.