Quality of standardised patient research reports in the medical education literature: review and recommendations

Training simulated participants for role portrayal and feedback practices in communication skills training: A BEME scoping review: BEME Guide No. 86

BACKGROUND

Providing feedback is a key aspect of simulated participants' (SPs) educational work. In teaching contexts, the ability to provide feedback to learners is central to their role. Suboptimal feedback practices may deny learners the valuable feedback they need to learn and improve. This scoping review systematically maps the evidence related to SPs' role as educators and identifies how SPs prepare for their role and feedback practices.

METHODS

The authors conducted a scoping review and included a group of international stakeholders with experience and expertise in SP methodology. Five online databases were systematically searched and ERIC, MedEdPortal and MedEdPublish were hand searched to identify relevant studies. Inclusion/exclusion criteria were developed. Data screening and subsequently data charting were performed in pairs. The results of data charting were thematically analysed including categories relating to the Association of SP Educators (ASPE) Standards of Best Practice (SOBP).

RESULTS

From 8179 articles identified for the title and abstract screening, 98 studies were included. Studies reported the benefit of SPs' authentic role portrayal and feedback interactions for learners and on the reported learning outcomes. Data was heterogeneous with a notable lack of consistency in the detail regarding the scenario formats for communication skills training interventions, SP characteristics, and approaches to training for feedback and role portrayal.

CONCLUSIONS

The published literature has considerable heterogeneity in reporting how SPs are prepared for role portrayal and feedback interactions. Additionally, our work has identified gaps in the implementation of the ASPE SOBP, which promotes effective SP-learner feedback interactions. Further research is required to identify effective applications of SP methodology to prepare SPs for their role as educators.

Current State and Proposal of Best Practices of Standardized Patients for Spanish-Language Objective Structured Clinical Encounters in US Medical Schools

INTRODUCTION

Medical Spanish programs commonly engage Spanish-speaking standardized patients (SPs) for communication skills assessment, yet no studies address SP recruitment, selection, or training.

METHODS

We sent questionnaires to medical Spanish faculty at 20 US medical schools to gauge their practices in recruiting and selecting Spanish-language SPs. We invited faculty to distribute a separate questionnaire to Spanish-language SPs to gather SP language abilities, training, and experience. We analyzed data using descriptive statistics and qualitative content analysis. When available, we reviewed SP video encounters to formally assess participating SPs' linguistic performance using the SP Oral Language Observation Matrix, a rating tool adapted from the Physician Oral Language Observation Matrix to assess oral medical Spanish proficiency.

RESULTS

Eighty percent of faculty (16/20) responded. Standardized patient recruitment sources included institutional English-language SPs, Hispanic student groups and professional organizations, communities, and language professionals. Faculty-reported strategies to determine language readiness included interviewing SP candidates in Spanish and asking them to self-rate language skills using a validated scale. Fifteen SPs (54%, 15/28) from 5 schools responded to the SP questionnaire, and one third (5/15) reported that their Spanish was not assessed before being selected as an SP. In addition, one third (5/15) did not receive any initial training before performing a medical Spanish case. Raters assessed 11 different SPs using the SP Oral Language Observation Matrix, and 6 were rated as linguistically "ready" for the SP role.

CONCLUSIONS

Current approaches to recruitment, training, and language assessment of SPs vary. We propose strategies to ensure that medical Spanish encounters authentically reflect Spanish-speaking patients.

The Real Practice Prescribing Antibiotics in Outpatients: A Failed Control Case Assessed through the Simulated Patient Method

The first level of medical care provides the largest number of consultations for the most frequent diseases at the community level, including acute pharyngitis (AP), acute diarrhoea (AD) and uncomplicated acute urinary tract infections (UAUTIs). The inappropriate use of antibiotics in these diseases represents a high risk for the generation of antimicrobial resistance (AMR) in bacteria causing community infections. To evaluate the patterns of medical prescription for these diseases in medical offices adjacent to pharmacies, we used an adult simulated patient (SP) method representing the three diseases, AP, AD and UAUTI. Each person played a role in one of the three diseases, with the signs and symptoms described in the national clinical practice guidelines (CPGs). Diagnostic accuracy and therapeutic management were assessed. Information from 280 consultations in the Mexico City area was obtained. For the 101 AP consultations, in 90 cases (89.1%), one or more antibiotics or antivirals were prescribed; for the 127 AD, in 104 cases (81.8%), one or more antiparasitic drugs or intestinal antiseptics were prescribed; for the scenarios involving UAUTIs in adult women, in 51 of 52 cases (98.1%) one antibiotic was prescribed. The antibiotic group with the highest prescription pattern for AP, AD and UAUTIs was aminopenicillins and benzylpenicillins [27/90 (30%)], co-trimoxazole [35/104 (27.6%)] and quinolones [38/51 (73.1%)], respectively. Our findings reveal the highly inappropriate use of antibiotics for AP and AD in a sector of the first level of health care, which could be a widespread phenomenon at the regional and national level and highlights the urgent need to update antibiotic prescriptions for UAUTIs according to local resistance patterns. Supervision of adherence to the CPGs is needed, as well as raising awareness about the rational use of antibiotics and the threat posed by AMR at the first level of care.

The DoCTRINE Guidelines: Defined Criteria To Report INnovations in Education

PURPOSE

Reporting guidelines assist authors in conducting and describing their research in alignment with evidence-based and expert-determined standards. However, published research-oriented guidelines do not capture all of the components that must be present in descriptions of educational innovations in health professions education. The authors aimed to create guidelines for educational innovations in curriculum development that would be easy for early-career educators to use, support reporting necessary details, and promote educational scholarship.

METHOD

Beginning in 2017, the authors systematically developed a reporting checklist for educational innovations in curriculum development, called Defined Criteria To Report INnovations in Education (DoCTRINE), and collected validity evidence for its use according to the 4 inferences of Kane's framework. They derived the items using a modified Delphi method, followed by pilot testing, cognitive interviewing, and interrater reliability testing. In May-November 2019, they implemented DoCTRINE for authors submitting to MedEdPORTAL, half of whom were randomized to receive the checklist (intervention group). The authors scored manuscripts using DoCTRINE while blinded to group assignment, and they collected data on final editorial decisions.

RESULTS

The final DoCTRINE checklist consists of 19 items, categorized into 5 components: introduction, curriculum development, curriculum implementation, results, and discussion. The overall interrater agreement was 0.91. Among the 108 manuscripts submitted to MedEdPORTAL during the study period, the mean (SD) total score was higher for accepted than rejected submissions (16.9 [1.73] vs 15.7 [2.24], P = .006). There were no significant differences in DoCTRINE scores between the intervention group, who received the checklist, and the control group, who did not.

CONCLUSIONS

The authors developed DoCTRINE, using systematic approaches, for the scholarly reporting of educational innovations in curriculum development. This checklist may be a useful tool for supporting the publishing efforts of early-career faculty.

Online learning developments in undergraduate medical education in response to the COVID-19 pandemic: A BEME systematic review: BEME Guide No. 69

BACKGROUND

The COVID-19 pandemic spurred an abrupt transition away from in-person educational activities. This systematic review investigated the pivot to online learning for nonclinical undergraduate medical education (UGME) activities and explored descriptions of educational offerings deployed, their impact, and lessons learned.

METHODS

The authors systematically searched four online databases and conducted a manual electronic search of MedEdPublish up to December 21, 2020. Two authors independently screened titles, abstracts and full texts, performed data extraction and assessed risk of bias. A third author resolved discrepancies. Findings were reported in accordance with the STORIES (STructured apprOach to the Reporting in healthcare education of Evidence Synthesis) statement and BEME guidance.

RESULTS

Fifty-six articles were included. The majority (n = 41) described the rapid transition of existing offerings to online formats, whereas fewer (n = 15) described novel activities. The majority (n = 27) included a combination of synchronous and asynchronous components. Didactics (n = 40) and small groups (n = 26) were the most common instructional methods. Teachers largely integrated technology to replace and amplify rather than transform learning, though learner engagement was often interactive. Thematic analysis revealed unique challenges of online learning, as well as exemplary practices. The quality of study designs and reporting was modest, with underpinning theory at highest risk of bias. Virtually all studies (n = 54) assessed reaction/satisfaction, fewer than half (n = 23) assessed changes in attitudes, knowledge or skills, and none assessed behavioral, organizational or patient outcomes.

CONCLUSIONS

UGME educators successfully transitioned face-to-face instructional methods online and implemented novel solutions during the COVID-19 pandemic. Although technology's potential to transform teaching is not yet fully realized, the use of synchronous and asynchronous formats encouraged virtual engagement, while offering flexible, self-directed learning. As we transition from emergency remote learning to a post-pandemic world, educators must underpin new developments with theory, report additional outcomes and provide details that support replication.

Evaluating the Reliability of the Needs in Recovery Assessment (NiRA) with Simulated Patients

The Needs in Recovery Assessment (NiRA) is a tool designed to support recovery-oriented and person-centred approaches in mental health services through facilitating the identification and prioritisation of needs. The aim of this study was to evaluate the interrater reliability of the NiRA. Method: Ten mental health clinicians from various professional backgrounds used the NiRA to facilitate assessment interviews with Simulated Patients. Completed and semi-completed NiRA forms, questionnaires, and audio-visual recordings of assessment interviews were collected for analysis. The interrater reliability of the NiRA was calculated using percent agreement and Gwet's Agreement Coefficient (AC)¹. Results: Percent agreement across all items of the finalised tool was 0.84 (item range: 0.55 to 1.0). Overall interrater reliability (Gwet's AC¹) was 0.70 (95% CI 0.64-0.76) with items ranging from -0.08 to 1.0. Conclusion: The NiRA is a reliable tool and is ready to be trialled in a feasibility study in clinical settings. It is anticipated that the NiRA will facilitate a deeper understanding of service users' needs and a more targeted approach to meeting unmet needs.

Simulated Patient-Based Communication Skills Training for Undergraduate Medical Students at a University in Ethiopia

BACKGROUND

Educational strategies following interactive methods of teaching enhance communication skill acquisition of medical students. However, which method of interactivity results in the most compelling communication skill improvement is the prime question yet unanswered.

MATERIALS AND METHODS

In this study, experimental study design was employed among pre-clerkship II medical students of Jimma University, in the 2017/18 academic year. Accordingly, study participants were grouped in to two arms randomly. Arm 1 students were trained using simulated patients in a skill development lab and arm 2 students were trained using case-based role-plays. Both forms of training lasted for 20 hours and focused on 7 important communication skill dimensions key to patient care tasks, as adapted from the Kalamazoo II Report. An objective structured clinical examination (OSCE) form of assessment with a standardized checklist was utilized to investigate communication skill acquisition.

RESULTS

In this study a total of 60 medical students was assessed for communication skills via 12 SP-based OSCE stations. This study discerned that, out of the total students, 21 (35.0%) were found to be poor at listening effectively. Similarly, 19 (31.7%), 18 (30.0%) and 16 (26.7%) were rated as poor at demonstrating caring and respectful behaviors, building and maintaining a therapeutic relationship and making informed judgments based on client data and preferences, respectively. Furthermore, the modalities of communication skill training are associated with key communication skill competencies except for eliciting information with effective questioning skills and counseling and educating patients. Thus, the SP-based training approach was found to be significantly associated with communication skill acquisition as compared to the case-based role-play approach, with AOR 21.696 at a p-value of <0.001.

CONCLUSION

In this study, it was concluded that an SP-based communication skill training approach is superior to a case-based role-play approach.

"Who Am I and Why Am I Here?" A Scoping Review Exploring the Templates and Protocols That Direct Actors in Their Roles as Simulated (Standardized) Patients

SUMMARY STATEMENT

Simulated patients (SPs) are increasingly used in health education and research. The aim of this article was to investigate templates and protocols that enable SPs to accurately and consistently adopt these roles. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for scoping reviews guided the search strategy for articles that detailed such templates or protocols. Embase Classic + Embase, ProQuest ERIC, Ovid MEDLINE, Ovid EMCare, psycINFO, and Scopus were searched, and 17 articles were included in the review. The templates and protocols that were located differed in structure, length, and depth and were developed or used in medical, nursing, allied health, and veterinary medicine disciplines. The validity, reliability, and replicability of studies are explored, and the quality of reporting is evaluated using the Simulation Research Rubric. Recommendations for increasing the rigor of programs and the reporting of research where SPs are adopted are considered.

Assessing the validity of an OSCE developed to assess rare, emergent or complex clinical conditions in endocrinology & metabolism

BACKGROUND

Assessment of emergent, rare or complex medical conditions in Endocrinology and Metabolism (E&M) is an integral component of training. However, data is lacking on how this could be best achieved. The purpose of this study was to develop and administer an Objective Structured Clinical Examination (OSCE) for E&M residents, and to gather validity evidence for its use.

METHODS

A needs assessment survey was distributed to all Canadian E&M Program Directors and recent graduates to determine which topics to include in the OSCE. The top 5 topics were selected using a modified Delphi technique. OSCE cases based on these topics were subsequently developed. Five E&M residents (PGY4-5) and five junior Internal Medicine (IM) residents participated in the OSCE. Performance of E&M and IM residents was compared and results were analyzed using a Generalizability study. Examiners and candidates completed a survey following the OSCE to evaluate their experiences.

RESULTS

The mean score of IM and E&M residents was 41.7 and 69.3 % (p < 0.001), respectively, with a large effect size (partial η² = 0.75). Overall reliability of the OSCE was 0.74. Standard setting using a borderline regression method resulted in a pass rate of 100 % of E&M residents and 0 % of IM residents. All residents felt the OSCE had high value for learning as a formative exam.

CONCLUSIONS

The E&M OSCE is a feasible method for assessing emergent, rare and complex medical conditions and this study provides validity evidence to support its use in a competency-based curriculum.

Implementation and utilization of gynecological teaching associate and male urogenital teaching associate programs: a scoping review

BACKGROUND

Gynecological Teaching Associates (GTAs) and Male Urogenital Teaching Associates (MUTAs) are individuals trained to instruct health professional learners with their own body to conduct accurate, patient-centered breast, pelvic, urogenital, rectal, and/or prostate examinations. Evidence indicates that this results in improvements in technical competence and communication skills, but there is wide variability to how such programs are implemented and engaged within the curriculum. In this scoping review, we mapped evidence regarding (1) how GTA/MUTA programs are utilized with health professional learners, (2) how GTA/MUTA programs are implemented using the Association of Standardized Patient Educators (ASPE) Standards of Best Practice (SOBP) as a framework, and (3) what broad outcomes are addressed in publications.

METHODS

PubMed, ERIC, PsychINFO, CINAHL, and Sociological Abstracts were searched for all publications addressing instruction of physical examinations with a GTA/MUTA and/or administration of GTA/MUTA programs. Studies were charted in tandem until consensus was identified and then charted individually, using an iterative process. The scoping review protocol was registered prospectively.

RESULTS

One hundred and one articles were identified, and nearly all highlighted positive results regarding GTA/MUTA programs. Most studies addressed medical students within the USA and Europe. During instructional sessions, three (SD=1.4) learners worked with each GTA/MUTA and an average of 32 min (SD=17) was allocated per learner. GTAs/MUTA instructed both independently (n=33) and in pairs (n=51). Thirty-eight articles provided detailed information consistent with one or more of the Domains of the ASPE SOBP, with six providing specific information regarding safe work environments.

CONCLUSIONS

While studies demonstrate consistently positive outcomes for learners, there is wide variability in implementation patterns. This variability may impact learning outcomes and impact both physical and psychological safety for GTAs/MUTAs and learners. Terminology used to refer to GTAs/MUTAs is inconsistent and may obscure relevant publications. Additional research is indicated to explore the pedagogical variables that result in positive learning outcomes and examine methods to ensure physical and psychological safety of GTAs/MUTAs and learners.

TRIAL REGISTRATION

https://osf.io/x9w2u/ .

Training and standardization of simulated patients for multicentre studies in clinical pharmacy education

Objective:

To evaluate the training and standardization methods of multiple simulated patients (SPs) performing a single scenario in a multicenter study.

Methods:

A prospective quasi-experimental study, using a multicenter approach, evaluated the performance of five different individuals with the same biotype during a simulation session in a high-fidelity environment. The SPs training and standardization process consisted of four steps and six web or face-to-face mediated: Step 1: simulation scenario design and pilot test. Step 2: SPs selection, recruitment and beginning training (Session 1: performance instructions and memorization request.) Session 2: check the SPs’ performances and adjustments). Step 3 and session 3: training role-play and performance’s evaluation. Step 4: SPs’ standardization and performances’ evaluation (Sessions 4 and 5: first and second rounds of SPs’ standardization assessment. Session 6: Global training and standardization evaluation. SPs performance consistency was estimated using Cronbach’s alpha and ICC.

Results:

In the evaluation of training results, the Maastricht Simulated Patient Assessment dimensions of SPs performances “It seems authentic”, “Can be a real patient” and “Answered questions naturally”, presented “moderate or complete agreement” of all evaluators. The dimensions “Seems to retain information unnecessarily”, “Remains in his/her role all the time”, “Challenges/tests the student”, and “Simulates physical complaints in an unrealistic way” presented “moderate or complete disagreement” in all evaluations. The SPs “Appearance fits the role” showed “moderate or complete agreement” in most evaluations. In the second round of evaluations, the SPs had better performance than the first ones. This could indicate the training process’s had good influence on SPs performances. The Cronbach’s alpha in the second assessment was better than the first (varied from 0.699 to 0.978). The same improvement occurred in the second round of intraclass correlation coefficient that was between 0.424 and 0.978. The SPs were satisfied with the training method and standardization process. They could perceive improvement on their role-play authenticity.

Conclusions:

The SPs training and standardization process revealed good SPs reliability and simulation reproducibility, demonstrating to be a feasible method for SPs standardization in multicenter studies. The Maastricht Simulated Patient Assessment was regarded as missing the assessment of the information consistency between the simulation script and the SPs provision.

Evaluation of the reporting quality of observational studies in master of public health dissertations in China

Backgrounds

Master of public health (MPH) plays an important role in Chinese medical education, and the dissertations is an important part of MPH education. In MPH dissertations, most are observational studies. Compared with randomized controlled trial (RCT), observational studies are more prone to information bias. So, the reporting of the observational studies should be transparent and standard. But, no research on evaluating the reporting quality of the MPH dissertation has been found.

Methods

A systematic literature search was performed in the Wanfang database from January 1, 2014 to May 31, 2019. The Strengthening the Reporting of Observation Studies in Epidemiology (STROBE) statement was adopted to evaluate the reporting quality of the selected studies. Articles that met the following criteria were selected: (1) observational studies, including cross-sectional studies, case-control studies, and cohort studies; (2) original articles; (3) studies on humans, including both adults and children.

Results

The Median of compliance to individual STROBE items was 74.79%. The mean (standard deviation) of STROBE score was 14.29 (1.84). Five items/sub-items were 100% reported (“reported” and “partly reported” were combined): background, objectives, study design, report numbers of individuals at each stage, and key result. Fifteen items/sub-items were reported by 75% or more. Reporting of methods and results was often omitted: missing data (6.67%), sensitivity analyses (3.63%), flow diagram (15.15%), and absolute risk (0%). Logistic regression analysis indicated that cohort studies (OR = 3.41, 95% CI = 1.27–9.16), funding support (OR = 4.37, 95% CI = 1.27–9.16) and more published papers during postgraduate period (OR = 3.46, 95% CI = 1.40–8.60) were related to high reporting quality.

Conclusion

In short, the reporting quality of observational studies in MPH’s dissertations in China is suboptimal. However, it’s necessary to improve the reporting of method and results sections. We recommend that authors should be stricter to adhere STROBE statement when conducting observational studies.

Criteria for describing and evaluating training interventions in healthcare professions - CRe-DEPTH

BACKGROUND

Descriptions of training intervention in healthcare include a lot of vagueness, leaving the reader in doubt about the intervention and the true value of the reported effects. A set of reporting criteria to improve the transparency and comprehensiveness in reporting training interventions in healthcare is recommended.

OBJECTIVE

This study aimed to develop and content validate a set of reporting criteria for the development and evaluation of training interventions for healthcare professionals.

METHOD

The set of criteria was developed along three stages: (1) a literature review, (2) the design of the criteria, and (3) a Delphi and consensus meeting process with ten experts from different professional disciplines.

RESULTS

The Criteria for Reporting on Development and Evaluation of Professional Training interventions in Healthcare (CRe-DEPTH) set was systematically developed and consisted of 12 items representing four categories: (1) development of the training, (2) characteristics of the training, (3) characteristics of the providers, and (4) assessment of the training outcomes. Each item has a detailed description to support its unambiguous interpretation.

CONCLUSIONS

The developed CRe-DEPTH reporting criteria guide high quality development, evaluation and reporting for complex training interventions, while preserving the flexibility to incorporate various teaching content or teaching methods. The Cre-DEPTH criteria will allow for detailed reporting on the training, its developmental process, as well as on the rationale for underlying choices, and will avoid non-reporting of training details.

Hybrid Simulation in Teaching Clinical Breast Examination to Medical Students

Clinical breast examination (CBE) is traditionally taught to third-year medical students using a lecture and a tabletop breast model. The opportunity to clinically practice CBE depends on patient availability and willingness to be examined by students, especially in culturally sensitive environments. We propose the use of a hybrid simulation model consisting of a standardized patient (SP) wearing a silicone breast simulator jacket and hypothesize that this, compared to traditional teaching methods, would result in improved learning. Consenting third-year medical students (N = 82) at a university-affiliated tertiary care center were cluster-randomized into two groups: hybrid simulation (breast jacket + SP) and control (tabletop breast model). Students received the standard lecture by instructors blinded to the randomization, followed by randomization group-based learning and practice sessions. Two weeks later, participants were assessed in an Objective Structured Clinical Examination (OSCE), which included three stations with SPs blinded to the intervention. The SPs graded the students on CBE completeness, and students completed a self-assessment of their performance and confidence during the examination. CBE completeness scores did not differ between the two groups (p = 0.889). Hybrid simulation improved lesion identification grades (p < 0.001) without increasing false positives. Hybrid simulation relieved the fear of missing a lesion on CBE (p = 0.043) and increased satisfaction with the teaching method among students (p = 0.002). As a novel educational tool, hybrid simulation improves the sensitivity of CBE performed by medical students without affecting its specificity. Hybrid simulation may play a role in increasing the confidence of medical students during CBE.

Virtual Standardized Patients for Interactive Conversational Training

The USC Standard Patient is a virtual human-based conversational agent serving in the role of a simulated medical patient, also known as a virtual standardized patient (VSP). This research identified deficiencies of extant VSP systems, defined a robust set of requirements, and successfully achieved nearly all of them. Markedly impressive advancements were made in virtual human technology, techniques to apply natural language processing, automated assessment artificial intelligence, and pedagogical design. The effort succeeded with performance parameters of high conversational performance, accurate assessment, and strongly demonstrated user training effect. Although working well within its confined are of expertise, the ability for computers to create authentic mixed initiative conversations remains elusive. This effort leaves behind many lessons for interactive serious games, clinical virtual humans, and conversational virtual human training applications.

Perceptions of a longitudinal standardized patient experience by standardized patients, medical students, and faculty

BACKGROUND

Longitudinal standardized patient (LSP) experiences mimic clinical practice by allowing students to interact with standardized patients (SPs) over time. LSP cases facilitate practice, assessment, and feedback in clinical skills and foster an appreciation for the continuum of care.

OBJECTIVE

We sought to characterize the nature of relationship-building, feedback, and continuity among all stakeholders participating in a single LSP program.

DESIGN

We developed and implemented a novel LSP program. Students encountered two LSP characters six times each during the first 2 years of medical school, though continuity pairings of students, SPs, and faculty were frequently not possible. Focus groups were held with second-year medical students (N = 15), core faculty who coached these students in LSP encounters (N = 8), and SPs who had played the role of either LSP character (N = 10) participated. Results were analyzed thematically using a template analysis approach.

RESULTS

The longitudinal nature of the experience reinforced the importance of student growth over time, the key role of faculty and SPs in providing feedback, and the tension between feedback and assessment. Students reported that LSP cases encouraged practice and feedback. SPs felt wedded to the longitudinal characters. Continuity pairings were recommended by all stakeholders to increase authenticity and promote relationship-building.

CONCLUSION

Stakeholders observed that the LSP cases brought some sense of continuity missing in other clinical skills encounters which helped prepare students for patient care. Continuity pairings of students, faculty, and SPs were recommended to enhance relationship-building and feedback.

Simulated patient training: Using inter-rater reliability to evaluate simulated patient consistency in nursing education

BACKGROUND

Simulated patients (SPs) are frequently used for training nursing students in communication skills. An acknowledged benefit of using SPs is the opportunity to provide a standardized approach by which participants can demonstrate and develop communication skills. However, relatively little evidence is available on how to best facilitate and evaluate the reliability and accuracy of SPs' performances.

AIM

The aim of this study is to investigate the effectiveness of an evidenced based SP training framework to ensure standardization of SPs. The training framework was employed to improve inter-rater reliability of SPs.

METHODS

A quasi-experimental study was employed to assess SP post-training understanding of simulation scenario parameters using inter-rater reliability agreement indices. Two phases of data collection took place. Initially a trial phase including audio-visual (AV) recordings of two undergraduate nursing students completing a simulation scenario is rated by eight SPs using the Interpersonal Communication Assessments Scale (ICAS) and Quality of Discharge Teaching Scale (QDTS). In phase 2, eight SP raters and four nursing faculty raters independently evaluated students' (N=42) communication practices using the QDTS.

RESULTS

Intraclass correlation coefficients (ICC) were >0.80 for both stages of the study in clinical communication skills.

CONCLUSION

The results support the premise that if trained appropriately, SPs have a high degree of reliability and validity to both facilitate and evaluate student performance in nurse education.

Assessing the Believability of Standardized Patients Trained to Portray Communication Disorders

PURPOSE

The purpose of this study was to evaluate the believability of standardized patients portraying individuals with communication disorders as part of a larger study in which standardized patients help train medical and allied health students about communication disorders.

METHOD

Two women portrayed persons with aphasia, and 2 men depicted persons with dysarthria associated with Parkinson's disease. Two stakeholder groups rated believability. Speech-language pathologists rated believability of videos online. Persons with aphasia rated aphasia videos during in-person sessions with the researchers.

RESULTS

Targeted believability was 80 or higher (0-100 scale; 0 = not at all believable, 100 = very believable). For speech-language pathologist raters, average ratings met the target for the portrayals of the aphasia characteristics of word-finding problems, agrammaticism, nonverbal communication, and overall portrayal but not for auditory comprehension problems. Targets for the portrayals were met for the dysarthria characteristics of reduced speech movements, reduced loudness, reduced intonation, flat affect, and overall portrayal but not for speech rate. Ratings for different standardized patients portraying the same case were not significantly different from each other on most characteristics. Ratings from persons with aphasia were highly variable.

CONCLUSION

Standardized patients who do not have communication disorders can portray disorder characteristics in a believable manner.

Breaking bad news to patients with cancer: A randomized control trial of a brief communication skills training module incorporating the stories and preferences of actual patients

OBJECTIVE

This study tested the effectiveness of a brief, learner-centered, breaking bad news (BBN) communication skills training module using objective evaluation measures.

METHODS

This randomized control study (N=66) compared intervention and control groups of students (n=28) and residents' (n=38) objective structured clinical examination (OSCE) performance of communication skills using Common Ground Assessment and Breaking Bad News measures.

RESULTS

Follow-up performance scores of intervention group students improved significantly regarding BBN (colon cancer (CC), p=0.007, r=-0.47; breast cancer (BC), p=0.003, r=-0.53), attention to patient responses after BBN (CC, p<0.001, r=-0.74; BC, p=0.001, r=-0.65), and addressing feelings (BC, p=0.006, r=-0.48). At CC follow-up assessment, performance scores of intervention group residents improved significantly regarding BBN (p=0.004, r=-0.43), communication related to emotions (p=0.034, r=-0.30), determining patient's readiness to proceed after BBN and communication preferences (p=0.041, r=-0.28), active listening (p=0.011, r=-0.37), addressing feelings (p<0.001, r=-0.65), and global interview performance (p=0.001, r=-0.51).

CONCLUSION

This brief BBN training module is an effective method of improving BBN communication skills among medical students and residents.

PRACTICE IMPLICATIONS

Implementation of this brief individualized training module within health education programs could lead to improved communication skills and patient care.

Simulation and mental health outcomes: a scoping review

BACKGROUND

A scoping review was conducted in order to map and determine the gaps in literature on the impact of simulation as an educational approach to improve mental health care outcomes. As it became apparent that no literature existed on this topic, the study aimed to examine the educational impact of simulation on mental health education.

METHODS

An established five-stage scoping methodology was used: (1) identification of the research question, (2) identification of relevant studies, (3) study selection, (4) charting the data and (5) collation, summarising and reporting of results. CINAHL, ProQuest, PubMed, MEDLINE, EMBASE and PsychINFO databases were searched. These databases were deemed to represent a majority of the literature while accommodating for the particular search strategy used for this review. Websites that provide grey literature were also searched for articles of relevance.

RESULTS

A total of 48 articles were included in this review, with a considerable portion of studies conducted in the USA and UK. Others were conducted in an array of locations including Australia, Canada, Iran and Taiwan. Of the included articles, seven groups of simulation methods (including standardised patients, virtual reality and manikins as patients) were evident, with standardised patients being most prominent.

CONCLUSIONS

Literature is lacking to evidence the benefit of simulation on mental health patient outcomes. However, the available literature suggests a variety of simulation-based education, and training methods are currently being used within mental healthcare education. The findings do suggest some methods of simulation, such as the use of standardised patients, are more commonly used in education and have been deemed as effective to assist in mental health education. As no article specifically examining the mental health outcomes of patients treated by health professionals taught by simulation was identified, the educational outcomes outlined in this paper may be used to inform further research, incorporating mental health patient outcomes.

Use of simulated patients to develop communication skills in nursing education: An integrative review

BACKGROUND

Registered nurses are expected to communicate effectively with patients. To improve on this skill education programmes in both hospital and tertiary settings are increasingly turning to simulation modalities when training undergraduate and registered nurses. The roles simulated patients (SPs) assume can vary according to training purposes and approach.

AIMS

The first aim is to analyse how SPs are used in nursing education to develop communication skills. The second aim is to evaluate the evidence that is available to support the efficacy of using SPs for training nurses in communication skills and finally to review the SP recruitment and training procedure.

DESIGN

An Integrative review.

DATA SOURCES

A search was conducted on CINAHL, Psych-info, PubMed, Google Scholar, Scopus, Ovid, Medline, and ProQuest databases. Keywords and inclusion/exclusion criteria were determined and applied to the search strategy.

REVIEW METHODS

The integrative review included Nineteen studies from 2006-2016. Critical Appraisal Skills Program (CASP) method of evaluation was utilised. Emergent themes were extracted with similar and divergent perspectives.

RESULTS

Analysis identified seven clinical contexts for communication skills training (CST) and two SP roles from the eighteen studies. SPs were either directly involved in the teaching of communication (active role) or used in the evaluation of the effectiveness of a communication skills program (passive role). A majority of studies utilised faculty-designed measurement instruments.

CONCLUSION

The evidence presented in the 19 articles indicates that the use of SPs to teach nurse-patient communication skills targets more challenging clinical interactions. Engaging SPs in both CST program facilitation and course evaluation provides nurse educators with a strong foundation to develop further pedagogical and research capacity. Expanding the utilisation of SPs to augment nurses' communication skills and ability to engage with patients in a broader range of clinical contexts with increased methodological rigor is recommended.

Development and validation of the guideline for reporting evidence-based practice educational interventions and teaching (GREET)

BACKGROUND

The majority of reporting guidelines assist researchers to report consistent information concerning study design, however, they contain limited information for describing study interventions. Using a three-stage development process, the Guideline for Reporting Evidence-based practice Educational interventions and Teaching (GREET) checklist and accompanying explanatory paper were developed to provide guidance for the reporting of educational interventions for evidence-based practice (EBP). The aim of this study was to complete the final development for the GREET checklist, incorporating psychometric testing to determine inter-rater reliability and criterion validity.

METHODS

The final development for the GREET checklist incorporated the results of a prior systematic review and Delphi survey. Thirty-nine items, including all items from the prior systematic review, were proposed for inclusion in the GREET checklist. These 39 items were considered over a series of consensus discussions to determine the inclusion of items in the GREET checklist. The GREET checklist and explanatory paper were then developed and underwent psychometric testing with tertiary health professional students who evaluated the completeness of the reporting in a published study using the GREET checklist. For each GREET checklist item, consistency (%) of agreement both between participants and the consensus criterion reference measure were calculated. Criterion validity and inter-rater reliability were analysed using intra-class correlation coefficients (ICC).

RESULTS

Three consensus discussions were undertaken, with 14 items identified for inclusion in the GREET checklist. Following further expert review by the Delphi panelists, three items were added and minor wording changes were completed, resulting in 17 checklist items. Psychometric testing for the updated GREET checklist was completed by 31 participants (n = 11 undergraduate, n = 20 postgraduate). The consistency of agreement between the participant ratings for completeness of reporting with the consensus criterion ratings ranged from 19 % for item 4 Steps of EBP, to 94 % for item 16 Planned delivery. The overall consistency of agreement, for criterion validity (ICC 0.73) and inter-rater reliability (ICC 0.96), was good to almost perfect.

CONCLUSION

The final GREET checklist comprises 17 items which are recommended for reporting EBP educational interventions. Further validation of the GREET checklist with experts in EBP research and education is recommended.

Can we rely on simulated patients' satisfaction with their consultation for assessing medical students' communication skills? A cross-sectional study

BACKGROUND

In medical education, teaching methods offering intensive practice without high utilization of faculty resources are needed. We investigated whether simulated patients' (SPs') satisfaction with a consultation could predict professional observers' assessment of young doctors' communication skills.

METHODS

This was a comparative cross-sectional study of 62 videotaped consultations in a general practice setting with young doctors who were finishing their internship. The SPs played a female patient who had observed blood when using the toilet, which had prompted a fear of cancer. Immediately afterwards, the SP rated her level of satisfaction with the consultation, and the scores were dichotomized into satisfaction or dissatisfaction. Professional observers viewed the videotapes and assessed the doctors' communication skills using the Arizona Communication Interview Rating Scale (ACIR). Their ratings of communication skills were dichotomized into acceptable versus unacceptable levels of competence.

RESULTS

The SPs' satisfaction showed a predictive power of 0.74 for the observers' assessment of the young doctors and whether they reached an acceptable level of communication skills. The SPs' dissatisfaction had a predictive power of 0.71 for the observers' assessment of an unacceptable communication level. The two assessment methods differed in 26% of the consultations. When SPs felt relief about their cancer concern after the consultation, they assessed the doctors' skills as satisfactory independent of the observers' assessment.

CONCLUSIONS

Accordance between the dichotomized SPs' satisfaction score and communication skills assessed by observers (using the ACIR) was in the acceptable range. These findings suggest that SPs' satisfaction scores may provide a reliable source for assessing communication skills in educational programs for medical trainees (students and young doctors). Awareness of the patient's concerns seems to be of vital importance to patient satisfaction.

Cost-effectiveness of peer role play and standardized patients in undergraduate communication training

BACKGROUND

The few studies directly comparing the methodological approach of peer role play (RP) and standardized patients (SP) for the delivery of communication skills all suggest that both methods are effective. In this study we calculated the costs of both methods (given comparable outcomes) and are the first to generate a differential cost-effectiveness analysis of both methods.

METHODS

Medical students in their prefinal year were randomly assigned to one of two groups receiving communication training in Pediatrics either with RP (N = 34) or 19 individually trained SP (N = 35). In an OSCE with standardized patients using the Calgary-Cambridge Referenced Observation Guide both groups achieved comparable high scores (results published). In this study, corresponding costs were assessed as man-hours resulting from hours of work of SP and tutors. A cost-effectiveness analysis was performed.

RESULTS

Cost-effectiveness analysis revealed a major advantage for RP as compared to SP (112 vs. 172 man hours; cost effectiveness ratio .74 vs. .45) at comparable performance levels after training with both methods.

CONCLUSIONS

While both peer role play and training with standardized patients have their value in medical curricula, RP has a major advantage in terms of cost-effectiveness. This could be taken into account in future decisions.

'I wouldn't get that feedback from anywhere else': learning partnerships and the use of high school students as simulated patients to enhance medical students' communication skills

BACKGROUND

This article evaluates whether the use of high school students as simulated patients who provide formative feedback enhances the capacity of medical students in their fifth year of training to initiate screening conversations and communicate effectively with adolescents about sensitive health issues.

METHODS

Focus group interviews with medical students (n = 52) and school students aged 15-16 (n = 107) were conducted prior to and following involvement in Learning Partnerships workshops. Prior to workshops focus groups with school students asked about attitudes to help-seeking in relation to sensitive health issues, and following workshops asked whether the workshop had made a difference to their concerns. Prior to workshops focus groups with medical students asked about their needs in relation to initiating conversations with adolescents about sensitive health issues, and following workshops asked whether the workshop had made a difference to their concerns. Surveys were also completed by 164 medical students and 66 school students following the workshops. This survey featured 19 items asking participants to rank the usefulness of the workshops out of 10 (1 = not at all useful, 10 = extremely useful) across areas such as skills and understanding, value of learning activities and overall value of the workshop. SPSS software was used to obtain mean plus standard deviation scores for each item on the survey.

RESULTS

The Learning Partnerships workshops assisted medical students to improve their skills and confidence in communicating with adolescents about sensitive health issues such as mental health, sexual health and drug and alcohol use. They also assisted young people to perceive doctors as more likely potential sources for help.

CONCLUSIONS

These findings suggest that the innovative methods included in Learning Partnerships may assist in broader education programs training doctors to be more effective helping agents and aid the promotion of adolescent friendly health care. This research provides evidence that a new way of teaching may contribute to enhancing doctors' capacity and willingness to initiate screening conversations and enhance adolescents' preparedness to seek help. This has implications for educational design, content and communication style within adolescent health.

Effectiveness of a simulated patient training programme based on trainee response accuracy and appropriateness of feedback

INTRODUCTION

Simulated patients (SPs) need education and training in required skills to be effective resources in education. This study was conducted to examine the effectiveness of an SP training programme based on the accuracy of trainee responses and the appropriateness of their feedback.

METHODS

Thirty-two applicants to the training programme and 35 experienced SPs were included in this study. The experienced SPs served as a reference group. The rate of accurate responses and the rate of appropriate feedback were assessed with pre- and post-training tests, and these two outcome measures were compared with those of the experienced SPs.

RESULTS

No significant differences were found in trainee response accuracy or appropriateness of feedback between pre- and post-training tests. The response accuracy rate of the trainees on the pre-training test was significantly lower than that of SPs with 1-2 years of experience, whilst there was no significant difference between these SPs and the trainees on the post-training test.

CONCLUSIONS

Although our study suggests that more training is needed to improve the skills of SPs, the training programme may contribute to helping trainees reach a novice level in the skill of providing accurate responses. SP training should be encouraged to contribute to the effectiveness of such teaching and to establish the validity of the assessment.

The educational value of improvisational actors to teach communication and relational skills: perspectives of interprofessional learners, faculty, and actors

OBJECTIVE

To assess the educational value of improvisational actors in difficult conversation simulations to teach communication and relational skills to interprofessional learners.

METHODS

Surveys of 192 interprofessional health care professionals, and 33 teaching faculty, and semi-structured interviews of 10 actors. Descriptive statistics, Fisher's exact test and chi-square test were used for quantitative analyses, and the Crabtree and Miller approach was used for qualitative analyses.

RESULTS

191/192 (99.5%) interprofessional learners (L), and 31/33 (94%) teaching faculty (F) responded to surveys. All 10/10 actors completed interviews. Nearly all participants found the actors realistic (98%L, 96%F), and valuable to the learning (97%L, 100%F). Most felt that role-play with another clinician would not have been as valuable as learning with actors (80%L, 97%F). There were no statistically significant differences in perceived value between learners who participated in the simulations (47%) versus those who observed (53%), or between doctors, nurses, or psychosocial professionals. Qualitative assessment yielded five actor value themes: Realism, Actor Feedback, Layperson Perspective, Depth of Emotion, and Role of Improvisation in Education. Actors independently identified similar themes as goals of their work.

CONCLUSIONS

The value attributed to actors was nearly universal among interprofessional learners and faculty, and independent of enactment participation versus observation. Authenticity, feedback from actors, patient/family perspectives, emotion, and improvisation were key educational elements.

A Delphi survey to determine how educational interventions for evidence-based practice should be reported: stage 2 of the development of a reporting guideline

BACKGROUND

Undertaking a Delphi exercise is recommended during the second stage in the development process for a reporting guideline. To continue the development for the Guideline for Reporting Evidence-based practice Educational interventions and Teaching (GREET) a Delphi survey was undertaken to determine the consensus opinion of researchers, journal editors and educators in evidence-based practice (EBP) regarding the information items that should be reported when describing an educational intervention for EBP.

METHODS

A four round online Delphi survey was conducted from October 2012 to March 2013. The Delphi panel comprised international researchers, educators and journal editors in EBP. Commencing with an open-ended question, participants were invited to volunteer information considered important when reporting educational interventions for EBP. Over three subsequent rounds participants were invited to rate the importance of each of the Delphi items using an 11 point Likert rating scale (low 0 to 4, moderate 5 to 6, high 7 to 8 and very high >8). Consensus agreement was set a priori as at least 80 per cent participant agreement. Consensus agreement was initially calculated within the four categories of importance (low to very high), prior to these four categories being merged into two (<7 and ≥7). Descriptive statistics for each item were computed including the mean Likert scores, standard deviation (SD), range and median participant scores. Mean absolute deviation from the median (MAD-M) was also calculated as a measure of participant disagreement.

RESULTS

Thirty-six experts agreed to participate and 27 (79%) participants completed all four rounds. A total of 76 information items were generated across the four survey rounds. Thirty-nine items (51%) were specific to describing the intervention (as opposed to other elements of study design) and consensus agreement was achieved for two of these items (5%). When the four rating categories were merged into two (<7 and ≥7), 18 intervention items achieved consensus agreement.

CONCLUSION

This Delphi survey has identified 39 items for describing an educational intervention for EBP. These Delphi intervention items will provide the groundwork for the subsequent consensus discussion to determine the final inclusion of items in the GREET, the first reporting guideline for educational interventions in EBP.

A systematic review of how studies describe educational interventions for evidence-based practice: stage 1 of the development of a reporting guideline

BACKGROUND

The aim of this systematic review was to identify which information is included when reporting educational interventions used to facilitate foundational skills and knowledge of evidence-based practice (EBP) training for health professionals. This systematic review comprised the first stage in the three stage development process for a reporting guideline for educational interventions for EBP.

METHODS

The review question was 'What information has been reported when describing educational interventions targeting foundational evidence-based practice knowledge and skills?'MEDLINE, Academic Search Premier, ERIC, CINAHL, Scopus, Embase, Informit health, Cochrane Library and Web of Science databases were searched from inception until October - December 2011. Randomised and non-randomised controlled trials reporting original data on educational interventions specific to developing foundational knowledge and skills of evidence-based practice were included.Studies were not appraised for methodological bias, however, reporting frequency and item commonality were compared between a random selection of studies included in the systematic review and a random selection of studies excluded as they were not controlled trials. Twenty-five data items were extracted by two independent reviewers (consistency > 90%).

RESULTS

Sixty-one studies met the inclusion criteria (n = 29 randomised, n = 32 non-randomised). The most consistently reported items were the learner's stage of training, professional discipline and the evaluation methods used (100%). The least consistently reported items were the instructor(s) previous teaching experience (n = 8, 13%), and student effort outside face to face contact (n = 1, 2%).

CONCLUSION

This systematic review demonstrates inconsistencies in describing educational interventions for EBP in randomised and non-randomised trials. To enable educational interventions to be replicable and comparable, improvements in the reporting for educational interventions for EBP are required. In the absence of a specific reporting guideline, there are a range of items which are reported with variable frequency. Identifying the important items for describing educational interventions for facilitating foundational knowledge and skills in EBP remains to be determined. The findings of this systematic review will be used to inform the next stage in the development of a reporting guideline for educational interventions for EBP.

Relation of completeness of reporting of health research to journals' endorsement of reporting guidelines: systematic review

OBJECTIVE

To assess whether the completeness of reporting of health research is related to journals' endorsement of reporting guidelines.

DESIGN

Systematic review.

DATA SOURCES

Reporting guidelines from a published systematic review and the EQUATOR Network (October 2011). Studies assessing the completeness of reporting by using an included reporting guideline (termed "evaluations") (1990 to October 2011; addendum searches in January 2012) from searches of either Medline, Embase, and the Cochrane Methodology Register or Scopus, depending on reporting guideline name.

STUDY SELECTION

English language reporting guidelines that provided explicit guidance for reporting, described the guidance development process, and indicated use of a consensus development process were included. The CONSORT statement was excluded, as evaluations of adherence to CONSORT had previously been reviewed. English or French language evaluations of included reporting guidelines were eligible if they assessed the completeness of reporting of studies as a primary intent and those included studies enabled the comparisons of interest (that is, after versus before journal endorsement and/or endorsing versus non-endorsing journals).

DATA EXTRACTION

Potentially eligible evaluations of included guidelines were screened initially by title and abstract and then as full text reports. If eligibility was unclear, authors of evaluations were contacted; journals' websites were consulted for endorsement information where needed. The completeness of reporting of reporting guidelines was analyzed in relation to endorsement by item and, where consistent with the authors' analysis, a mean summed score.

RESULTS

101 reporting guidelines were included. Of 15,249 records retrieved from the search for evaluations, 26 evaluations that assessed completeness of reporting in relation to endorsement for nine reporting guidelines were identified. Of those, 13 evaluations assessing seven reporting guidelines (BMJ economic checklist, CONSORT for harms, PRISMA, QUOROM, STARD, STRICTA, and STROBE) could be analyzed. Reporting guideline items were assessed by few evaluations.

CONCLUSIONS

The completeness of reporting of only nine of 101 health research reporting guidelines (excluding CONSORT) has been evaluated in relation to journals' endorsement. Items from seven reporting guidelines were quantitatively analyzed, by few evaluations each. Insufficient evidence exists to determine the relation between journals' endorsement of reporting guidelines and the completeness of reporting of published health research reports. Journal editors and researchers should consider collaborative prospectively designed, controlled studies to provide more robust evidence.

SYSTEMATIC REVIEW REGISTRATION

Not registered; no known register currently accepts protocols for methodology systematic reviews.

Influence of standardized patient body habitus on undergraduate student performance in an Objective Structured Clinical Examination

PURPOSE

Previous studies have shown that the standardized patient's (SP) gender may affect student performance in an Objective Structured Clinical Examination (OSCE). The aim of this study is to investigate the influence of the SPs' body habitus on students' performance in an OSCE counseling station.

METHODS

Four equally trained female SPs, with either a normal or an obese BMI participated in an OSCE counseling station for cardiovascular risk factors. Ninety-two, second year medical students were randomly assigned to one of the SPs. Station scores were compared and student behavior and opinion regarding the influence of their SP's body habitus on their performance was assessed.

RESULTS

There was no difference in mean exam scores for students interacting with SPs with a normal BMI versus increased BMI (14.9 ± 2.2 versus 14.01 ± 2.2/20 respectively, p = 0.06). Additionally, almost all students gave advice about healthy diets (93.5% versus 95.7%) with no specificity regarding the BMI of the SP.

CONCLUSIONS

The body habitus of the SP did not significantly affect students' performance in an undergraduate OSCE about cardiovascular risk factors, suggesting that students at that level may primarily focus on gaining points the diagnostic checklist without considering SPs as real patients.

Do communication training programs improve students' communication skills?--a follow-up study

Background

Although it is taken for granted that history-taking and communication skills are learnable, this learning process should be confirmed by rigorous studies, such as randomized pre- and post-comparisons. The purpose of this paper is to analyse whether a communication course measurably improves the communicative competence of third-year medical students at a German medical school and whether technical or emotional aspects of communication changed differently.

Method

A sample of 32 randomly selected students performed an interview with a simulated patient before the communication course (pre-intervention) and a second interview after the course (post-intervention), using the Calgary-Cambridge Observation Guide (CCOG) to assess history taking ability.

Results

On average, the students improved in all of the 28 items of the CCOG. The 6 more technically-orientated communication items improved on average from 3.4 for the first interview to 2.6 in the second interview (p < 0.0001), the 6 emotional items from 2.7 to 2.3 (p = 0.023). The overall score for women improved from 3.2 to 2.5 (p = 0.0019); male students improved from 3.0 to 2.7 (n.s.). The mean interview time significantly increased from the first to the second interview, but the increase in the interview duration and the change of the overall score for the students’ communication skills were not correlated (Pearson’s r = 0.03; n.s.).

Conclusions

Our communication course measurably improved communication skills, especially for female students. These improvements did not depend predominantly on an extension of the interview time. Obviously, “technical” aspects of communication can be taught better than “emotional” communication skills.

The effect of using standardized patients or peer role play on ratings of undergraduate communication training: a randomized controlled trial

OBJECTIVES

Considering the expense of standardized patients (SP) for training communication skills and the convenience of peer role playing (RP) there is a surprising lack of studies directly comparing the two methods.

METHODS

Fifth year medical students (N=103) were assigned to three groups receiving a training in counseling parents of sick children with RP (N=34) or SP (N=35) or to a control group (CG, N=34). We assessed self-efficacy, as well as objective performance in parent-physician communication using questionnaires and the Calgary-Cambridge-Observation-Guide Checklist in a six-station OSCE, respectively.

RESULTS

The training led to an increase in self-efficacy ratings and in the post-intervention OSCE score after RP (p<.021 and p<.001 respectively) and SP-training (p<.007 and p<.006 respectively) compared to controls. Surprisingly, this benefit was higher after RP than after SP-training (p<.021) due to significantly higher performance in the domain understanding of parents'perspective (p<.001).

CONCLUSION

Both RP and SP are valuable tools for training specific communication skills. RP offer a methodological advantage in fostering empathy for patient perspectives.

PRACTICE IMPLICATIONS

Both peer-role-play and standardized patients hold specific benefits for communication training. Peer-role-play seems to foster a more empathic approach towards patients' concerns justifying its prominent role in medical curricula.

Reporting standards for studies of tailored interventions

Message tailoring is a promising innovative approach to persuasive communication that involves designing messages to meet a person's psychological, behavioral and/or demographic characteristics. Although the tailored intervention literature has many strengths, a weakness is inconsistency in reporting information related to intervention development, implementation and evaluation. The objective of this manuscript is to report recommendations for studies of tailored interventions. As part of ongoing original empirical and meta-analytical research, we reviewed the tailoring literature and identified inconsistencies in reporting. We compared these inconsistencies with existing reporting standards and developed recommendations specific to tailored interventions. An advisory board of preeminent tailoring researchers provided feedback on draft and final recommendations. This paper offers the resulting seven recommendations for reporting studies of tailored interventions. If we are to build a cumulative science of tailoring, both for theory development and research translation, then we should establish standards in the conduct and reporting of the science.

Predictors of primary care physicians' self-reported intention to conduct suicide risk assessments

Primary care physicians play a significant role in depression care, suicide assessment, and suicide prevention. However, little is known about what factors relate to and predict quality of depression care (assessment, diagnosis, and treatment), including suicide assessment. The authors explored the extent to which select patient and physician factors increase the probability of one element of quality of care: namely, intention to conduct suicide assessment. Data were collected from 404 randomly selected primary care physicians after their interaction with CD-ROM vignettes of actors portraying major depression with moderate levels of severity. The authors examined which patient factors and physician factors increase the likelihood of physicians' intention to conduct a suicide assessment. Data from the study revealed that physician-participants inquired about suicide 36% of the time. A random effects logistic model indicated that several factors were predictive of physicians' intention to conduct a suicide assessment: patient's comorbidity status (odds ratio (OR) = 0.61; 95% confidence interval (CI) = 0.37-1.00), physicians' age (OR = 0.67; 95% CI = 0.49-0.92), physicians' race (OR = 1.84; 95% CI = 1.08-3.13), and how depressed the physician perceived the virtual patient to be (OR = 0.58; 95% CI = 0.39-0.87). A substantial number of primary care physicians in this study indicated they would not assess for suicide, even though most physicians perceived the virtual patient to be depressed or very depressed. Further study is needed to establish factors that may be modified and targeted to increase the likelihood of physicians' providing one element of quality of care--suicide assessment--for depressed patients.

Six formats in simulated and standardized patients use, based on experiences of 13 undergraduate medical curricula in Belgium and the Netherlands

BACKGROUND

In Belgium and the Netherlands, 13 medical schools collaborate with regard to the use of simulated patients (SPs) and standardized patients in their undergraduate curricula. All schools use SPs in their curriculum but schools differ in (a) the timing or placement of the SPs and (b) the way they are used.

AIM

To give an overview of the formats used most in undergraduate medical education with SPs, including a description of the impact of these formats on the different aspects of SPs.

METHODS

Representatives of all medical schools in Belgium and The Netherlands all provided a structured overview of their use of SPs. We then collectively made a description of the main working formats. For each format we identified salient consequences on the SP level, including whether the format requires simulated or standardized patients or patient-actors, what is the impact of a format on the selection of SPs, content and duration of SP training, whether or not to use checklists in role training, feedback training or not, debriefing of training, impact on the case (role) description (e.g. free or structured), the number of SPs needed, and the selection criteria for SPs.

CONCLUSION

The working format with SPs more or less determines the whole process of selection, training, performance, and logistics of SPs.

Teaching the rectal examination with simulations: effects on knowledge acquisition and inhibition

CONTEXT

Undergraduate medical curricula are often deficient in teaching physical examinations in intimate zones, such as the rectal examination. Student inhibition is assumed to substantially hamper both the acquisition of knowledge and the performance of these examinations in practice.

OBJECTIVES

The two present studies examined the effects of low-fidelity (LFS) and high-fidelity (HFS) simulation on the acquisition of the necessary knowledge and inhibition about carrying out the rectal examination. In addition, we investigated the effects of the different sequencing of the two simulations (HFS-LFS versus LFS-HFS).

METHODS

A manikin for the rectal examination was used to implement the LFS. Standardised patients (SPs) were used to implement the HFS. Study samples consisted of 41 (Study 1) and 188 (Study 2) female and male undergraduate medical students. Each student participated in two individual sessions of 30 minutes each. Half the students participated first in the HFS and then in the LFS and the other half participated in the simulations in the opposite order. Outcome measures were self-rated inhibition and knowledge tests.

RESULTS

In both studies, HFS was found to reduce inhibition significantly more than LFS. Furthermore, in Study 2, a marginal effect of the sequence of simulation was found. In both studies, both types of simulation were found to facilitate the acquisition of knowledge. There was no sequence effect for the acquisition of knowledge.

CONCLUSIONS

Teaching the rectal examination with the help of SPs, who represent an HFS, can help medical undergraduate students to overcome inhibition about this examination. Standardised patient simulation is far more effective than that achieved using a manikin, which represents an LFS. Both types of simulation support the acquisition of knowledge.

Method and reporting quality in health professions education research: a systematic review

CONTEXT

Studies evaluating reporting quality in health professions education (HPE) research have demonstrated deficiencies, but none have used comprehensive reporting standards. Additionally, the relationship between study methods and effect size (ES) in HPE research is unknown.

OBJECTIVES

This review aimed to evaluate, in a sample of experimental studies of Internet-based instruction, the quality of reporting, the relationship between reporting and methodological quality, and associations between ES and study methods.

METHODS

We conducted a systematic search of databases including MEDLINE, Scopus, CINAHL, EMBASE and ERIC, for articles published during 1990-2008. Studies (in any language) quantifying the effect of Internet-based instruction in HPE compared with no intervention or other instruction were included. Working independently and in duplicate, we coded reporting quality using the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement, and coded study methods using a modified Newcastle-Ottawa Scale (m-NOS), the Medical Education Research Study Quality Instrument (MERSQI), and the Best Evidence in Medical Education (BEME) global scale.

RESULTS

For reporting quality, articles scored a mean±standard deviation (SD) of 51±25% of STROBE elements for the Introduction, 58±20% for the Methods, 50±18% for the Results and 41±26% for the Discussion sections. We found positive associations (all p<0.0001) between reporting quality and MERSQI (ρ=0.64), m-NOS (ρ=0.57) and BEME (ρ=0.58) scores. We explored associations between study methods and knowledge ES by subtracting each study's ES from the pooled ES for studies using that method and comparing these differences between subgroups. Effect sizes in single-group pretest/post-test studies differed from the pooled estimate more than ESs in two-group studies (p=0.013). No difference was found between other study methods (yes/no: representative sample, comparison group from same community, randomised, allocation concealed, participants blinded, assessor blinded, objective assessment, high follow-up).

CONCLUSIONS

Information is missing from all sections of reports of HPE experiments. Single-group pre-/post-test studies may overestimate ES compared with two-group designs. Other methodological variations did not bias study results in this sample.

Outcome of parent-physician communication skills training for pediatric residents

OBJECTIVE

communication skills represent an essential component of clinical competence. In the field of pediatrics, communication between physicians and patients' parents is characterized by particular difficulties. To investigate the effects of a parent-physician communication skills training program on OSCE performance and self-efficacy in a group control design.

METHODS

parallel to their daily work in the outpatient department, intervention-group experienced clinicians in practice (n=14) participated in a communication training with standardized parents. Control-group physicians (n=14) did not receive any training beyond their daily work. Performance was assessed by independent video ratings of an OSCE. Both groups rated their self-efficacy prior to and following training.

RESULTS

regarding OSCE performance, the intervention group demonstrated superior skills in building relationships with parents (p<.024) and tended to perform better in exploring parents' problems (p<.081). The communication training program led to significant improvement in self-efficacy with respect to the specific training objectives in the intervention group (p<.046).

CONCLUSION

even in physicians with considerable experience, structured communication training with standardized parents leads to significant improvement in OSCE performance and self-efficacy. PRACTISE IMPLICATIONS: briefness and tight structure make the presented communication training program applicable even for experienced physicians in daily clinical practice.

Assessment of clinical competence of medical students using the objective structured clinical examination: first 2 years' experience in Taipei Veterans General Hospital

BACKGROUND

Competence-oriented education is currently the mainstream method of teaching clinical medical education. The objective structured clinical examination (OSCE) is a widely employed and accepted tool to measure the clinical competence of medical students. We describe the first 2 years' experience of OSCE in Taipei Veterans General Hospital.

METHODS

At Taipei Veterans General Hospital, every 7(th)-year medical student has taken the OSCE since 2006. There were 15 stations in the first 2 years' OSCEs. In years 1 and 2, 133 and 132 students were assessed by the OSCE, respectively. The content of the OSCE included internal medicine, surgery, pediatrics, obstetrics and gynecology, communication, and emergency training. All categories and results of examinees' evaluation at each station were recorded inclusively and compared statistically.

RESULTS

The average scores of students from the 15 stations ranged from 47.7 ± 16.4 to 93.7 ± 8.5 in 2007. The score for communication skills was the lowest, whereas the score for Micro-Sim was the highest. Communication skills and electrocardiography interpretation were the 2 categories in which most of the students failed. A reliability analysis was conducted of the 2007 OSCE questions. The overall score and reliability (Cronbach's reliability) was 0.641. The difference between the impacts on reliability after deleting a test item ranged from 0.59 to 0.65 for all stations. This meant that every station had a similar impact on reliability after being deleted. The squared multiple correlation, R(2), of the reliability of each item was between 0.12 and 0.49, with chest X-ray interpretation being the lowest. The item-total correlation was between 0.10 and 0.41, with interactive case being the lowest.

CONCLUSION

The OSCE is an effective method for assessing the clinical competence of medical students. The OSCE could be improved further by modifying the examination questions and promoting effective training for standardized patients and examiners.

Using simulated patients to develop doctors' skills in facilitating behaviour change: addressing childhood obesity

OBJECTIVES

This study aimed to describe the application, feasibility and outcomes of using simulated patients (SPs) to increase the skills of general practitioners (GPs) delivering a behavioural intervention to reduce childhood overweight and mild obesity.

METHODS

Five female actors were trained as SPs. A total of 67 GPs from 46 general practices in Melbourne, Victoria, Australia, conducted two simulated consultation visits regarding healthy lifestyle family behaviour change, during which they practised their skills and received formative feedback. The GPs and SPs rated GP performance immediately after each consultation. Subsequently, 139 parents of overweight or obese 5-9-year-old children rated GP performance during real-life consultations. Other measures included child body mass index (BMI) Z-scores (at baseline and at a 9-month follow-up) and GP-reported levels of comfort and competence and the perceived value of SP visits.

RESULTS

Simulated patient ratings, but not GP self-ratings, of GP performance predicted both parent ratings of real-life consultations (Spearman's rho 0.39 for correlation with SP rating at Visit 1) and subsequent reductions in BMI Z-scores between baseline and follow-up (Visit 1, rho-0.45; Visit 2, rho-0.46). GP levels of comfort and competence were maintained during and after the SP visits. A total of 95% of GPs rated simulated consultations as useful, although only 18% said they would pay for them.

CONCLUSIONS

Simulated patient assessment may predict real patient feedback and clinical outcomes, helping to identify doctors who require further training in behaviour change techniques. Randomised controlled trials may establish whether SPs actually raise skills or improve outcomes.

Undergraduate students as standardized patients to assess advanced practice nursing student competencies

Evaluating clinical skills of students in an online distance education program can be challenging because of the diverse location of students. The authors describe a unique and cost-efficient method of using standardized patients to evaluate these skills. The project involved undergraduate students representing standardized patients for graduate advanced practice nurse students.