Evaluation of a Situational Judgement Test to Develop Non-Academic Skills in Pharmacy Students

Grit, academic resilience, and mindset of nursing students: A cross-sectional study

Background

Nursing students engage with the complex health system for competence development. These students are exposed to nerve-wrecking experience in addition to their everyday personal and social challenges. Non-cognitive attributes, namely grit, resilience and mindset can influence students' ability to overcome complexities as they become nurses resulting in academic success and well-being. Insights into the state of non-cognitive attributes among undergraduate nursing students are essential in developing tailor-made educational programmes to enhance their grit, resilience, and mindset.

Objectives

To describe the undergraduate nursing students' grit, academic resilience, and mindset at a university in South Africa.

Design

A quantitative descriptive cross-sectional design.

Setting

A School of Nursing at a public multi-campus university in South Africa.

Participants

All students (N = 315) registered for the undergraduate nursing programme were invited to participate, and 70 % (N = 221) chose to participate.

Methods

Data were collected using self-administered questionnaires that included a 7-item demographic survey, the 30-item Academic Resilience Scale (ARS-30) that measures affective, cognitive, and behavioural responses in an educational context, the 16-item Dweck Mindset Scale (DMI) that measures a personal belief about whether intelligence and talent are fixed or amenable to change, and the 8-item Grit-S scale measuring passion for long term goals and perseverance. The collected quantitative data were analysed statistically through the Statistical Analysis Software Version 9.4 computer programme.

Findings

The findings revealed that the participants have normal grit and a growth mindset, but low academic resilience.

Conclusions

Tailor-made educational programmes that target non-cognitive attributes must integrate interventions that are focused on enhancing academic resilience for undergraduate nursing students.

The effects of language proficiency and awareness of time limit in animated vs. text-based situational judgment tests

BACKGROUND

Situational Judgment Tests (SJTs) are commonly used in medical school admissions. However, it has been consistently found that native speakers tend to score higher on SJTs than non-native speakers, which can be particularly problematic in the admission context due to the potential risk of limited fairness. Besides type of SJT, awareness of time limit may play a role in subgroup differences in the context of cognitive load theory. This study examined the influence of SJT type and awareness of time limit against the background of language proficiency in a quasi high-stakes setting.

METHODS

Participants (N = 875), applicants and students in healthcare-related study programs, completed an online study that involved two SJTs: one with a text-based stimulus and response format (HAM-SJT) and another with a video-animated stimulus and media-supported response format (Social Shapes Test, SST). They were randomly assigned to a test condition in which they were either informed about a time limit or not. In a multilevel model analysis, we examined the main effects and interactions of the predictors (test type, language proficiency and awareness of time limit) on test performance (overall, response percentage).

RESULTS

There were significant main effects on overall test performance for language proficiency in favor of native speakers and for awareness of time limit in favor of being aware of the time limit. Furthermore, an interaction between language proficiency and test type was found, indicating that subgroup differences are smaller for the animated SJT than for the text-based SJT. No interaction effects on overall test performance were found that included awareness of time limit.

CONCLUSION

A SJT with video-animated stimuli and a media-supported response format can reduce subgroup differences in overall test performance between native and non-native speakers in a quasi high-stakes setting. Awareness of time limit is equally important for high and low performance, regardless of language proficiency or test type.

Assessing professionalism in mental health clinicians: development and validation of a situational judgement test

BACKGROUND

Situational judgement test (SJT) scores have been observed to predict actual workplace performance. They are commonly used to assess non-academic attributes as part of selection into many healthcare roles. However, no validated SJT yet exists for recruiting into mental health services.

AIMS

To develop and validate an SJT that can evaluate procedural knowledge of professionalism in applicants to clinical roles in mental health services.

METHOD

SJT item content was generated through interviews and focus groups with 56 professionals, patients and carers related to a large National Health Service mental health trust in England. These subject matter experts informed the content of the final items for the SJT. The SJT was completed by 73 registered nurses and 36 allied health professionals (AHPs). The primary outcome measure was supervisor ratings of professionalism and effectiveness on a relative percentile rating scale and was present for 69 of the participating nurses and AHPs. Personality assessment scores were reported as a secondary outcome.

RESULTS

SJT scores statistically significantly predicted ratings of professionalism (β = 0.31, P = 0.01) and effectiveness (β = 0.32, P = 0.01). The scores demonstrated statistically significant incremental predictive validity over the personality assessment scores for predicting supervisor ratings of professionalism (β = 0.26, P = 0.03).

CONCLUSIONS

These findings demonstrate that a carefully designed SJT can validly assess important personal attributes in clinicians working in mental health services. Such assessments are likely to represent evidence based, cost-effective tools that can support values-based recruitment to mental health service roles.

Opening the Black Box: Agreement and Reliability of a Situational Judgment Test Across Multiple Institutions

OBJECTIVE

The primary objective of this study was to examine the levels of agreement and reliability of a situational judgment test (SJT) using a diverse pool of pharmacy practice faculty as subject matter experts. Secondary aims included analyses to build support for test validity and fairness.

METHODS

An SJT containing 18 scenarios and 118 responses assessing empathy, integrity, and teamwork was developed and delivered to pharmacy practice faculty at 5 schools of pharmacy across the United States. Reliability was assessed by examining internal consistency, inter-rater reliability, and split-half reliability. Only responses which attained an inter-rater agreement>0.7 were included in the final version of the SJT. All responses were scored using a near-miss system, allowing higher scores for answers more closely aligned with the key, which was determined by the faculty who completed the SJT. Test fairness was reported using descriptive statistics.

RESULTS

Thirty-nine faculty across the 5 participating institutions completed the SJT. The final version of the SJT included 105 responses, achieving an inter-rater agreement of>0.7 (inter-rater reliability of 0.98). Split-half reliability was 0.72. The average score was 85.7%, and no differences in performance were observed based on demographic characteristics.

CONCLUSION

An SJT designed to assess empathy, integrity, and teamwork achieved reasonable levels of reliability among pharmacy practice faculty across the United States, and the results provided initial support for test validity and fairness. These results support a pilot to assess this SJT among students representing multiple institutions.

Perspectives of Professionalism, Self-Awareness, Leadership, and Innovation and Entrepreneurship From Experiential Education Preceptors

OBJECTIVE

The objectives of this study were to (1) quantify the perceived importance of and comfort with the Accreditation Council for Pharmacy Education Standard 4 (ACPE-S4) key elements, (2) describe how preceptors promote and assess key elements, and (3) describe preceptor perceptions of student key element skill demonstration.

METHODS

This study used a cross-sectional electronic survey to assess experiential education preceptors' perceptions of ACPE-S4 key elements. The survey consisted of 2 item categories, with 5 and 15 items included to collect information on preceptor demographics and ACPE-S4 key elements, respectively. Surveys were distributed to 451 introductory and advanced rotation preceptors. Frequencies and means were reported for quantitative survey items, open-ended survey responses were analyzed using a general interpretivist approach.

RESULTS

In total, 242 surveys were completed for a response rate of 54%. Most respondents identified as female (54.5%) and precepted community pharmacy rotations (28.5%). ACPE-S4 key elements, in order of most to least important, were professionalism, self-awareness, leadership, and innovation and entrepreneurship. Respondents varied in their conceptualization of key elements, with multiple activities used for skill demonstration and assessments. Qualitative analysis identified 2 themes: "Variation in preceptor fluency with S4 key elements" and "Communication as a means to demonstrate and evaluate ACPE-S4 key element skills."

CONCLUSION

Variation among preceptors in conceptualization, learning activities, and comfort in assessment for ACPE-S4 key elements highlights the need for support and resources to assist preceptors in developing activities and assessments for skills reflective of ACPE-S4 key elements. Preceptors commonly relied on vulnerable communication with students to assess key element skills, identifying a potential target for assessment and preceptor development. Pharmacy programs, in addition to the Academy, could benefit from using assessments already offered by preceptors. Finally, explicit evaluation criteria may help to guide skill acquisition and facilitate consistent assessment of key element skills.

Examining Validity for the Pharmacy Affective Domain Situational Judgment Test

Objective. With the integration of the affective domain in the pharmacy accreditation standards, it is important for programs to have methods for formatively assessing student competency in these areas. The objective of this study was to examine the validity evidence for a recently developed situational judgment test to assess the affective domain contained in the Accreditation Council for Pharmacy Education (ACPE) Standards 2016.Methods. After pilot testing items in multiple pharmacy schools, a revised 15-item situational judgment test instrument was developed measuring the affective domain as it relates to the pharmacy profession. The resulting Pharmacy Affective Domain Situational Judgment Test (PAD-S) was administered to students at three institutions and internal structure validity evidence was examined using item descriptives, Cronbach alpha, and results from a Rasch measurement model.Results. There were 559 valid responses included in the study. The items were of varying difficulty, with Rasch results confirming the hierarchy of items and suggesting that items were contributing to the measure. The Cronbach alpha was 0.70, suggesting acceptable reliability. However, the reliability items from the Rasch model were lower (0.65 and 0.66), indicating that the ranking of ability was limited, which may be due to fewer items of high difficulty.Conclusion. Overall, the PAD-S performed well as a measure of the affective domain. The PAD-S may be a useful formative instrument as part of a comprehensive assessment plan and may be less resource intensive than other measures.

Preceptor Perceptions of Pharmacy Student Performance Before and After a Curriculum Transformation

Objective. To explore preceptors' perceptions about the performance of undergraduate pharmacy students during experiential placements in Australia, before and after curricular transformation.Methods. Using a semi-structured approach, we interviewed 26 preceptors who had recently supervised students who took part in the transformed curriculum and students from the previous curriculum. A directed content analysis approach was used to analyze the transcripts.Results. Preceptors described students from the transformed curriculum as having improved professional skills, behaviors, and attitudes and as having an increased ability to perform clinical activities compared to students of the previous curriculum. Preceptors also perceived that students in the transformed curriculum had improved clinical knowledge and knowledge application. They less frequently expressed that students in the transformed curriculum had lower-than-expected knowledge levels.Conclusion. The results of this study suggest that curricular transformation with a focus on skill-based and active learning can improve the performance of pharmacy students in terms of their professional behaviors and attitudes, skills, knowledge, and clinical abilities, as perceived by preceptors.

Use of situational judgment tests for assessing non-cognitive attributes of final year dental students

OBJECTIVE

Situational judgment tests (SJT) have been previously used in the admission process of medical and dental undergraduate programs, but their utilization as an assessment tool is fairly new. The objective of this study was to explore essential non-cognitive attributes of dental graduates. We also aimed to assess the attributes of final year dental students utilizing SJT and later collected participants' reactions on the test.

METHODS

A mixed-method exploratory sequential study was conducted. One-on-one and group discussions were done with subject matter experts (SMEs) to explore the essential attributes of fresh dental graduates. Based on the thematic analysis, 10 SJT items per domain were constructed by one group of SMEs and expert validated by a different group of SMEs and vice versa. The final dental-SJT was piloted online on final year dental students and their perceptions were collected using a five-point Likert scale questionnaire. Descriptive statistics and Cronbach's alpha were calculated for the test.

RESULTS

The consultation with SMEs generated three main themes, namely, Conventional skills, Tribal skills, and Occupational skills. The 70-item dental-SJT piloted on 150 final year dental students revealed data were normally distributed. The internal reliability of test was good (0.738). Students scored highest in "management skills" (>70%) and lowest in "professional ethics" (56%) and "diverse experiences" (58%). The overall student perception about the test was good.

CONCLUSION

SJT is an effective tool to assess non-cognitive attributes of dental students. It has an overall positive perception about perceived predictive validity, perceived fairness, face and content validity.

Development and Implementation of Situational Judgment Tests as an Evaluation Method for Training Oncology Physicians: Application in the KOKON-KTO Training

Situational judgment tests (SJTs) are often used in aptitude testing and present practice-specific challenges. Their implementation into online training programs provides the opportunity to assess learning progress and improve training quality. In this study, text-based SJTs for oncology physicians were developed, validated, and implemented into the KOKON-KTO training which uses a blended learning training format to teach oncology physicians how to consult cancer patients on complementary and integrative medicine (CIM). The SJT was implemented to measure the e-learning results. In the development and validation phase, a total of 15 SJTs (each SJT including 1 best choice answer based on training content and 4 distractors; 9 SJTs for oncologists and 6 SJTs for oncology gynecologists only) were developed by an interprofessional team (n=5) using real-case vignettes and applying an in-depth review process. Best answers were validated by experts (oncologists and oncology gynecologists) with experience in advising cancer patients on CIM. In the implementation and evaluation phase, SJTs were answered by KOKON-KTO training participants (n=19) pre- and post e-learning. Results were analyzed using descriptive measurements, item difficulties, and Cohen's d for effect size pre- and post-training. The experts (n=12, 49.8% gynecologists) agreed with best choice answers (69.4% for oncology gynecology; 81.5% for oncology) in 12 out of 15 SJTs. Comparing pre- and post-training scores, KOKON-KTO training participants were able to improve knowledge substantially (effect sizes for oncologists d=1.7; oncology gynecologists d= .71). Future studies need to increase the number of experts and SJTs in order to apply further psychometric measurements. As part of the KOKON-KTO study, this project is registered as DRKS00012704 on the "German Clinical Trials Register" (Date of registration: 28.08.2017).

Use of Situational Judgment Tests to Teach Empathy, Assertiveness, Communication, and Ethics

Objective. The objectives of this study were to develop situational judgment test (SJT) scenarios for use in teaching empathy, assertiveness, and interprofessional communication in second-year pharmacy (P2) students and ethics in third-year pharmacy (P3) students, determine whether the SJTs developed were effective at measuring students' communication skills and ethical judgment, compare the performance of individual third-year students to the performance of teams of third-year students on the ethics SJT, and evaluate student feedback about SJTs as a teaching tool.Methods. Pharmacy faculty developed five SJT scenarios related to communications and five SJT scenarios related to ethics and piloted the scenarios with P2 and P3 students, respectively. Second-year students completed SJTs individually, while P3 students completed SJTs individually and in teams. Scenarios and responses were discussed with faculty after completion of the SJTs, and students completed a questionnaire to provide feedback on the SJTs.Results. The communications SJT was completed by 59 P2 students with a mean score of 67.5%. The ethics SJT was completed by 57 P3 students with a mean score of 80.1%. The ethics SJT was also completed by 10 teams of P3 students resulting in a mean score of 93.2%. Students indicated the SJT content was realistic and the tests provided the opportunity to reflect on how to approach challenging situations.Conclusion. Situational judgment tests were useful for teaching empathy, assertiveness, interprofessional communication, and ethics in pharmacy students. Future research should focus on predictive validity of SJTs for these content areas.

Situational judgment using ethical reasoning in Saudi undergraduate pharmacy students

Introduction

There is a paramount need for moral development for pharmacists and pharmacy students to practice the patient-centered profession. We aimed to explore the current situational judgment utilizing ethical reasoning among undergraduate pharmacy students.

Methods

A set of ten ethical dilemmas, representing potential real-life situations that the students come across in the university and may face in the future as a pharmacist were developed by a team of students, academic staff, and stakeholders. These ethical dilemmas were validated, checked for accuracy, and piloted. An online questionnaire was created consisting of these ten scenarios as open-ended questions and administered to fourth year and fifth year PharmD students in a public university located at the city of Mecca, Saudi Arabia, asking them how they would react in that situation. Responses of the participants were analyzed using thematic analysis independently by four researchers and inter-rater agreement were achieved through consensus.

Results

Out of 205, 186 students completed the questionnaire with a response rate of 90.7%. Analysis and resulted in the generation of 32 codes, which were then categorized into seven overarching themes: student engagement, social and professional responsibility, academic integrity, legal obligation, moral obligation, signposting, and moral engagement and patient safety.

Conclusions

Undergraduate pharmacy students experience complex state of mind in connection with ethical reasoning. The participants’ situational judgment were driven by cultural norm, authority, and responsibility. Student engagement is also affected by the state of mind and feelings of mutual trust, perceived cultural influence and peer pressure. The students were prone to seek help from university administrators or teachers when faced with situations in which they were helpless.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12910-022-00768-x.

A Situational Judgment Test to Assess Students' Achievement of Accreditation Council for Pharmacy Education Standards 3 and 4

Objective. For pharmacy students to successfully meet competencies related to Accreditation Council for Pharmacy Education Standards 3 and 4, it is essential for pharmacy programs to assess student progression in the affective domain. The purpose of this study was to develop and assess the validity of a situational judgment test (SJT) for evaluating student mastery of Standards 3 and 4.Methods. A multi-institutional faculty team developed an 18-item SJT that consisted of scenarios asking the respondent to rank the effectiveness of four response options mapped to Standards 3 and 4. The research team systematically reviewed the literature, created items, and deliberated until consensus was achieved. Subject matter experts (SMEs) reviewed and provided feedback on the instrument. Students from two institutions were recruited to participate in cognitive interviews about the finalized instrument. Cognitive interview data were analyzed to identify themes.Results. After edits were made to the instrument based on SME feedback, students (n=18) in the cognitive interviews identified item length as a concern and commented on item/response clarity and comprehensiveness. Data from the cognitive interviews were used to modify the SJT to reduce the length and clarify items. The result was two shorter versions of the instrument, both with similar mapping to all elements in Standards 3 and 4.Conclusion. Early steps in validating the SJT suggested that the instrument may be a promising tool to assess student progression in the affective domain. The SJT instrument is intended to provide evidence of student pharmacist development that occurs in the didactic, experiential, and co-curricular portions of pharmacy education. The instrument can serve as one part of a comprehensive assessment plan.

Development of a Vertically Integrated Pharmacy Degree

Whilst curriculum revision is commonplace, whole degree transformation is less so. In this paper we discuss the rationale, design and implementation of a unique pharmacy program by a research-intensive faculty. The new Monash pharmacy curriculum, which had its first intake in 2017, was built using a range of key innovations that aimed to produce graduates that demonstrate key conceptual understanding and all the skills required to deliver world-best patient outcomes. The key elements of the re-design are outlined and include the process and principles developed, as well as key features such as a student-centred individualised program of development arranged around specific, authentic tasks for each skill and earlier enhanced experiential placements where students become proficient in entrustable professional activities. It is hoped the dissemination of this process, as well as the lessons learnt in the process, will be useful to others looking to transform a health curriculum.

Exploring the Role of Item Scenario Features on Situational Judgment Test Response Selections

Objective. To explore pharmacists' and pharmacy students' perceptions regarding the significance of changing the features of test item scenario (eg, switching from a health care to a non-health care context) on their situational judgment test (SJT) responses.Methods. Fifteen Doctor of Pharmacy students and 15 pharmacists completed a 12-item SJT intended to measure empathy. The test included six scenarios in a health care context and six scenarios in a non-health care context; participants had to rank potential response options in order of appropriateness and no two items could be of equal rank. Qualitative data were collected individually from participants using think-aloud and cognitive interview techniques. During the cognitive interview, participants were asked how they selected their final responses for each item and whether they would have changed their answer if features of the scenario were switched (eg, changed to a non-health care context if the original item was in a health care context). Interviews were transcribed and a thematic analysis was conducted to identify the features of the scenario for each item that were perceived to impact response selections.Results. Participants stated that they would have changed their responses on average 51.3% of the time (range 20%-100%) if the features of the scenario for an item were changed. Qualitative analysis identified four pertinent scenario features that may influence response selections, which included information about the examinee, the actors in the scenario, the relationship between examinee and actors, and details about the situation. There was no discernible pattern linking scenario features to the component of empathy being measured or participant type.Conclusion. Results from this study suggest that the features of the scenario described in an SJT item could influence response selections. These features should be considered in the SJT design process and require further research to determine the extent of their impact on SJT performance.

Exploring Multiple Perspectives on Pharmacy Students' Readiness for Advanced Pharmacy Practice Experiences

Objective. This study aimed to enrich the Academy's understanding of pharmacy students' readiness for advanced pharmacy practice experiences (APPEs) by exploring the perspectives of three primary stakeholders: APPE students, APPE preceptors, and APPE faculty site directors.Methods. A descriptive qualitative study of APPE readiness was conducted using workplace learning as a guiding conceptual framework. Data were collected between March and September 2019 through semi-structured focus groups and interviews with students (five groups), preceptors (four groups), and faculty site directors (one group, two individual interviews). The data were analyzed using directed content analysis.Results. Participants described APPE readiness as a multifaceted construct comprised of four themes: learner characteristics, participation in workplace activities, relationship-building, and workplace practices to orient and support students. While all participants addressed each category, faculty site directors and preceptors tended to focus on learner characteristics, while students emphasized their participation in the workplace and relationship building.Conclusion. Knowledge is widely recognized as a requirement for APPE readiness. This study identified learner characteristics, workplace participation, and relational skills as additional requirements. Some of these criteria are challenging to assess prior to APPEs, which makes orienting students both prior to and at the start of APPEs particularly important to support readiness. Thus, a comprehensive review of APPE readiness might also include assessing the readiness of workplaces, administrators, and preceptors for APPE students.

A scoping review of admission criteria and selection methods in nursing education

BACKGROUND

Nursing education institutions are required to select and train applicants who have appropriate characteristics for delivering effective healthcare. Unlike other healthcare professions and despite the need to attract and select a competent workforce, there has been no comprehensive analysis of the selection criteria and methods used to recruit nursing students. As there is relatively limited prior research available, we conducted a scoping review to explore and synthesise the existing evidence regarding admission criteria and selection methods of nursing students and for the purpose of identifying an agenda for future research in this field.

METHODS

Our scoping review follows the Arksey and O'Malley five-step proposition including identifying the research question and relevant studies, study selection, tabulation of data, and summarizing and reporting the results. Seven databases (PubMed, CINAHL, Scopus, ERIC, SID, Irandoc and PsycINFO) were searched systematically using relevant keywords. Articles on admission of undergraduate nursing students published in both English and/or Persian from 2006 to 2019 were retrieved.

RESULTS

Existing research evidence suggests that nursing students are largely selected on the basis of two criteria - "cognitive-academic abilities" and "non-cognitive abilities." Cognitive-academic abilities were assessed in four main dimensions of mathematics, language, natural sciences and reasoning skills mainly through standardized tests and academic records. Our review shows a wide range of non-cognitive characteristics are evaluated in nursing applicants including: morality, interpersonal communication skills and psychological strength. The selection method most commonly used to assess characteristics was through interviews (panel interviews or multiple mini interviews). Other methods included references, personal statements and personality assessment tools.

CONCLUSIONS

This is the first scoping review of literature regarding nursing education selection and recruitment. Results can be used to inform nursing education policymakers and institutions in the design of their selection practices. Future research should concentrate on the evaluation and improvement methods of student selection including content and predictive validity analysis of multiple mini interview and standardized tests, development of cost-effective selection methods and job analysis studies to identify specific non-cognitive characteristics for nursing.

Predictors of Pharmacy Student Performance on Written and Clinical Examinations in a Flipped Classroom Curriculum

Objective. To examine the effects of student demographics, prior academic performance, course engagement, and time management on pharmacy students' performance on course examinations and objective structured clinical examinations (OSCEs).Methods. Study participants were one cohort of pharmacy students enrolled in a five-year combined Bachelor and Master of Pharmacy degree program at one institution. Variables included student demographics, baseline factors (language assessment and situational judgement test scores), prior academic performance (high school admission rank), course engagement, and student time management of pre-class online activities. Data were collected from course, learning management system, and institutional databases. Data were analyzed for univariate, bivariate, and multivariate associations (four linear regression models) between explanatory factors and outcome variables.Results. Three years of data on 159 pharmacy students were obtained and entered in the dataset. Significant positive predictors of OSCE communication performance included domestic (ie, Australian) student designation, higher baseline written English proficiency, and pre-class online activity completion. Positive predictors of OSCE problem-solving included workshop attendance and low empathy as measured by a baseline situational judgment test (SJT). Positive predictors of performance on year 2 end-of-course examinations included the Australian Tertiary Academic Rank, completing pre-class online activities prior to lectures, and high integrity as measured by an SJT.Conclusion. Several explanatory factors predicted pharmacy students' examination and OSCE performance in the regression models. Future research should continue to study additional contexts, explanatory factors, and outcome variables.

Development and Validation of a Situational Judgement Test to Assess Professionalism

Objective. The purpose of this study was to develop, pilot, and validate a situational judgement test (SJT) to assess professionalism in Doctor of Pharmacy (PharmD) students. Methods. Test specifications were developed and faculty members were educated on best practices in item writing for SJTs. The faculty members then developed 75 pilot scenarios. From those, two versions of the SJT, each containing 25 scenarios, were created. The pilot population for the SJT was student pharmacists in their third professional year, just prior to starting their advanced pharmacy practice experiences. The students completed the two versions of the test on different days, approximately 48 hours apart, with 50 minutes allowed to complete each. Subsequently, students completed a questionnaire regarding the SJT at the conclusion of the second test. Results. Version 1 of the SJT was completed by 228 students, and version 2 was completed by 225 students. Mean scores were 390 (SD=20, range 318-429) and 342 (SD=21, range 263-387) on test versions 1 and 2, respectively. The reliability of the tests was appropriate (test version 1, α=0.77; test version 2, α=0.79). Students felt that the content of the tests was realistic with respect to pharmacy practice (90.1%), and that the tests gave them an opportunity to reflect on how to approach challenging situations (82.6%). Conclusion. We developed a reliable SJT to assess professionalism in PharmD students. Future research should focus on creating a personalized learning plan for students who do not meet minimum performance standards on this SJT.