Work-based assessment: qualitative perspectives of novice nutrition and dietetics educators

Shifting the narrative and practice of assessing professionalism in dietetics education: An Australasian qualitative study

AIM

We aimed to explore current approaches to assessing professionalism in dietetics education in Australia and New Zealand, and asked the questions what is working well and what needs to improve?

METHOD

We employed a qualitative interpretive approach and conducted interviews with academic and practitioner (workplace-based) educators (total sample n = 78) with a key stake in dietetics education across Australia and New Zealand. Data were analysed using team-based, framework analysis.

RESULTS

Our findings suggest significant shifts in dietetics education in the area of professionalism assessment. Professionalism assessment is embedded in formal curricula of dietetics programs and is occurring in university and placement settings. In particular, advances have been demonstrated in those programs assessing professionalism as part of the programmatic assessment. Progress has been enabled by philosophical and curricula shifts; clearer articulation and shared understandings of professionalism standards; enhanced learner agency and reduced power distance; early identification and intervention of professionalism lapses; and increased confidence and capabilities of educators.

CONCLUSIONS

These findings suggest there have been considerable advances in professionalism assessment in recent years with shifts in practice in approaching professionalism through a more interpretivist lens, holistically and more student-centred. Professionalism assessment in dietetics education is a shared responsibility and requires further development and transformation to more fully embed and strengthen curricula approaches across programs. Further work should investigate strategies to build safer learning cultures and capacity for professionalism conversations and in strengthening approaches to remediation.

Implementation and evaluation of a preceptor educational development programme involving the health cluster colleges at Qatar University

BACKGROUND

'The Practice Educators' Academy Programme' is an innovative educational intervention developed following a preceptor-focussed needs assessment. The primary aim of this study was to evaluate the programme's effect on self-efficacy and knowledge amongst multi-disciplinary clinical preceptors who precept students across the Health Cluster in Qatar University. The secondary aim was to assess the preceptors' satisfaction with the programme's comprehensiveness, appropriateness, and relevance.

METHODS

This is a longitudinal study, with the same participants followed at different time points (i.e., pretest-posttest interventional design). The sample size was calculated to represent 10% of the sample expected for the main study, utilising a stratified convenience sampling technique. Preceptor self-efficacy was assessed using the Preceptor Self-Efficacy Questionnaire, a validated 21-item questionnaire. Preceptor knowledge was assessed through a 25-item multiple-choice question test. Satisfaction with programme content and delivery was assessed through a 14-item questionnaire with open comments.

RESULTS

Thirty pretest-posttest respondents for the PSEQ assessment, and 26 pretest-posttest respondents for the knowledge assessment were appropriately matched and analysed. Participation in the self-efficacy questionnaire resulted in a statistically significant increase in their posttest median score (pretest-to-posttest: 3.3-to-3.6, p = 0.001). Participation in the knowledge assessment resulted in a statistically significant increased posttest mean score (pretest-to-posttest: 10.2-to-15.7, p < 0.001). Participants indicated high levels of satisfaction with the programme (average score = 4.42/5).

CONCLUSIONS

Our findings suggest the programme is effective as demonstrated through a significant improvement in preceptors' self-efficacy and knowledge. Recommendations for future iterations include placing greater focus on active learning strategies, and inter-disciplinary interactions.

Assessment of Educational Needs and Design of a Preceptor Development Program for Health Professional Education Programs in Qatar

INTRODUCTION

Experiential learning is the backbone of many health care professional education programs; however, the quality of learning is profoundly dependent on the skills and experiences of clinical preceptors. This study was conducted at Qatar University Health Cluster (Colleges of Pharmacy, Medicine, and Health Sciences) with the primary objective of identifying the educational needs of preceptors to design and review an educational professional development program.

METHODS

This study adopted a mixed-methods approach and was conducted in three stages: (1) assessment of preceptor educational needs, (2) designing of the Practice Educators' Academy program, and (3) revision and refinement of the designed program. The needs' assessment was conducted at all the three colleges through a validated survey and focus groups comprising of preceptors, students, and clinical faculty members. The sample included 209 survey respondents and 11 focus group sessions.

RESULTS

The results yielded five key themes and a variety of individual preferences, which were used to design a five-module face-to-face two-day interactive workshop. For the revision of the designed program, the syllabus was shared purposively with selected scholars and experts in the area of health professions education, and their feedback was collected and critically examined. Furthermore, the refinement of the program was performed on the basis of this feedback, resulting in the revised and representative program being ready for piloting.

DISCUSSION

A preceptor development program on experiential teaching and learning skills was successfully designed and revised with the needs of the clinical preceptors at its core. Preceptors' skills development can advance health care outcomes by preparing competent health professional graduates.

Competency-based assessment in nutrition education: A systematic literature review

Background

A suitably prepared and qualified nutrition and dietetics workforce is part of the solution to combating the burden of disease. Competency‐based assessment is a key part of the education of future workforces. Although there has been recent attention on competency‐based assessment in dietetics, there is little exploration of competency‐based education for the preparation of nutritionists. The present study aimed to understand how competency‐based assessment is implemented and evaluated in nutrition education.

Methods

A systematic literature review was carried out according to PRISMA guidelines. Four databases were initially searched in February 2020 using key words related to competenc* in combination with nutrition or dietetic and their synonyms. An updated search was completed again in March 2021. Studies that met eligibility criteria where the focus was on nutrition and involved a method of competency‐based assessment were synthesised narratively.

Results

From a total of 6262 titles and abstracts, six studies on competency assessment in nutrition education were identified. The assessments focused on the development of key skills, including motivational interviewing and nutrition assessment, changes to knowledge and attitudes on food and culture, and self‐perceived development of communication, collaboration, management, advocacy, scholarship and professional capabilities. No studies were found that assessed promotion of health and wellbeing or the food chain competencies.

Conclusions

The lack of research in competency‐based assessment must be addressed to ensure we are effectively preparing future nutritionists for work such that they can impact health outcomes.

Preceptors' perceptions of assessing clinical associate students at district hospital sites

BACKGROUND

Preceptors are key stakeholders in distributed health professions' education. They supervise students in the clinical setting to enable them to have a practical experience with patients, and they assess students' skills at the highest tier of clinical assessment. The university where this study was done conducts a distributed Bachelor of Clinical Medical Practice course on a distributed platform which is dependent on preceptors at the training sites. Understanding the perceptions of preceptors, as major stakeholders, regarding the student assessment they do will assist the faculty to provide better support and development that might be needed and assist in maximising the benefits of distributed training.

AIM

The aim of this study was to explore the perceptions of preceptors regarding assessing clinical associate students at district hospitals in the Bachelor of Clinical Medical Practice programme.

SETTING

The study was conducted at a rural university in the Eastern Cape province of South Africa.

METHODS

This was a qualitative study involving nine preceptors who were purposively selected from three district hospital training sites based on their involvement in assessing clinical associate students. Semi-structured interviews were conducted, recorded, transcribed and thematically analysed.

RESULTS

Four themes emerged from thematic analysis: assessment issues, preceptor issues, student issues and university support issues. Preceptors are committed and enthusiastic in training and assessing the clinical associate students but require input from the university in terms of training and ongoing support.

CONCLUSION

Lack of training threatens the validity of preceptor assessment. Academic institutions should train and support preceptors to enable them better to fulfil their roles.

The Value of Programmatic Assessment in Supporting Educators and Students to Succeed: A Qualitative Evaluation

BACKGROUND

Programmatic assessment has been proposed as the way forward for competency-based assessment, yet there is a dearth of literature describing the implementation and evaluation of programmatic assessment approaches.

OBJECTIVE

To evaluate the implementation of a programmatic assessment and explore its ability to support students and assessors.

DESIGN

A qualitative evaluation of programmatic assessment was employed.

PARTICIPANTS/SETTING

Interviews with graduates (n = 8) and preceptors (n = 12) together with focus groups with faculty assessors (n = 9) from the one Australian university explored experiences of the programmatic approach, role of assessment in learning, and defensibility of assessment decisions in determining competence.

ANALYSIS PERFORMED

Data were analyzed into key themes using framework analysis.

RESULTS

The programmatic assessment increased confidence in defensibility of assessment decisions, reduced emotional burden of assessment, increased value of assessment, and identified and remediated at-risk students earlier when philosophical and practice shifts in approaches to assessment were embraced.

CONCLUSIONS

Programmatic assessment supports a holistic approach to competency development and assessment and has multiple benefits for learners and assessors.

E-portfolios and Entrustable Professional Activities to support competency-based education in dietetics

The concept of Entrustable Professional Activities, recently pioneered in medical education, has emerged to support the implementation of competency-based education. Although competency-based frameworks are widely used in healthcare professional education to develop outcomes-based curricula, assessment of student competency in professional placement settings remains challenging. The novel concept of Entrustable Professional Activities together with established methods of competency assessment, namely e-portfolios and self-assessment, was implemented in the "[La Trobe University Dietetic program in 2015-2016. This study aimed to appraise the e-portfolio and evaluate the use of Entrustable Professional Activities to assess competence. A mixed-methods evaluation, using qualitative and quantitative surveys with follow-up structured consultations, was conducted with final year dietetics students and their supervisors. Dietetics students were comfortable with Entrustable Professional Activities and competency-based assessment, whereas supervisors preferred Entrustable Professional Activity based assessment. All stakeholders valued student self-assessment and the ongoing use of structured e-portfolios to develop and document competency. The use of structured e-portfolios, student self-assessment, and the emerging concept of Entrustable Professional Activities are useful tools to support dietetics student education in professional placement settings.

A self-determination theory analysis of reflective debrief themes about dietetic student placement experiences in hospital: implications for education

BACKGROUND

Healthcare placements in dietetics education contribute significantly to student learning. Exploring students' self-conceptualisation of placement experiences may provide insights to better support learning. Self-determination theory (SDT) has been used to seek insight into clinical and educational settings but has not yet been applied to dietetic placement learning. The present study investigated dietetics students' reflections of key influences on placement learning experiences and their alignment with an SDT framework.

METHODS

A post-placement two-stage critical incident debrief was conducted with seven successive cohorts (168 students) of dietetic undergraduate students on final placement. In debriefs, students' anonymous themes were collected and discussed, inductively analysed, and then mapped against an SDT framework of psychological and motivational constructs.

RESULTS

Nine key themes were identified that impacted upon placement experiences. Four themes related to framework constructs: (1) Supervisor (and Peer) Autonomy Support; (2) Perceived Competence; (3) Relatedness; and (4) Autonomy and Intrinsic Motivation. Non-SDT themes were also present, including: (5) Learning Environment and Experience; as well as themes about professional behaviours and identity: (6) Teamwork and Interactions; (7) Managing Emotions and Self-Care; (8) Dietetic Communications and Behaviours; and (9) Developing a Professional Identity.

CONCLUSIONS

Embedding a structured debrief in the curriculum and using a psychological motivational SDT framework to analyse themes arising can provide valuable information about the learning needs of students on placement with potential for wider application in dietetic learning and teaching and workforce employability. The current findings may have application in university curricula before and after professional placement.

Clinical educators' skills and qualities in allied health: a systematic review

BACKGROUND

The skills and qualities of effective clinical educators are linked to improved student learning and ultimately patient care; however, within allied health these have not yet been systematically summarised in the literature.

AIMS

To identify and synthesise the skills and qualities of clinical educators in allied health and their effect on student learning and patient care.

METHOD

A systematic search of the literature was conducted across five electronic databases in November 2017. Study identification, data extraction and quality appraisal were performed in duplicate. Qualitative and quantitative data were extracted separately but analysed together using a thematic analysis approach whereby items used in quantitative surveys and themes from qualitative approaches were interpreted together.

RESULTS

Data revealed seven educator skills and qualities: (i) intrinsic and personal attributes of clinical educators; (ii) provision of skillful feedback; (iii) teaching skills; (iv) fostering collaborative learning; (v) understanding expectations; (vi) organisation and planning; and (vii) clinical educators in their professional role. Across all themes was the concept of taking time to perform the clinical educator role. No studies used objective measurements as to how these skills and qualities affect learning or patient care.

DISCUSSION AND CONCLUSION

Despite much primary evidence of clinical educator skills and qualities, this review presents the first synthesis of this evidence in allied health. There is a need to examine clinical education from new perspectives to develop deeper understanding of how clinical educator qualities and skills influence student learning and patient care.

A dietetic clinical educator enhances the experience and assessment of clinical placement

AIM

The aim of this study was to evaluate the impact of a Clinical Educator model on the learning experience and environment for students, preceptors and managers.

METHODS

A Clinical Educator position was established for the 10-week dietetic clinical placement at Edith Cowan University. The Clinical Educator was responsible for overseeing the placement and assisting in the supervision of students. A qualitative descriptive approach using focus groups with purposive sampling explored the research question. Students (n = 10), preceptors (n = 21) and managers (n = 3) participated in separate focus groups. Data were thematically analysed with consideration given to participant and focus group commonalities and differences.

RESULTS

The findings revealed that the Clinical Educator (i) reduced the logistical burden of student placements and improved time efficiency; (ii) facilitated student assessment within a programme of assessment; (iii) was uniquely positioned to provide support and enhance student confidence; and (iv) enhanced capacity to manage underperforming and challenging students.

CONCLUSIONS

The Clinical Educator model increased student confidence, facilitated quality assessment and supported the management of underperforming students. This was achieved by reducing the burden of clinical placements, facilitating effective and timely communication between stakeholders and supporting the establishment of meaningful relationships which enriched learning. The results highlight the importance of the people involved in placement to facilitate a positive student learning environment and high quality assessment.

Challenges in preparing the dietetics workforce of the future: An exploration of dietetics educators' experiences

AIM

Dietetics educators represent a small but influential workforce group that has experienced significant change in recent years. The workforce development challenges faced by this group have been largely unexplored. The present study aimed to explore the experiences of, and challenges faced by, academic dietetics educators in preparing dietitians for the workforce.

METHODS

The approach taken in the present study was informed by qualitative description. Fifteen dietetics educators employed by 13 universities across Australia were purposively sampled. In-depth, semi-structured interviews conducted via telephone (n = 12) or face-to-face (n = 3) were digitally recorded then transcribed verbatim. Data were managed with NVivo and inductively analysed using open coding. Codes were condensed into themes through an iterative process involving multiple researchers.

RESULTS

The overarching theme of 'aiming for a moving target' was underpinned by the themes of: (i) striving for betterment; (ii) bridging dissonance and (iii) distressing impossibilities. Interviewees described how they were driven to enhance the preparation of dietitians but acknowledged disparity between what graduates are being prepared for and what they need to be prepared for. Heightened expectations of others, professional constraints and a lack of collegiality among the profession were among the challenges that manifested in a sense of frustration, concern and isolation.

CONCLUSIONS

Dietetics educators are motivated to shape and enhance the future profession. However, they face numerous challenges in their efforts to prepare graduates who are well-equipped for increasingly diverse dietetics practice. Strong leadership, academic collaboration and greater engagement of the broader workforce are required for the benefit of the entire profession.

A cross-sectional study exploring the different roles of individual and group assessment methods in assessing public health nutrition competence

BACKGROUND

Competency in the practice of public health is essential for dietitians, yet little is known about credible and dependable assessment in this field. The present study aimed to investigate the role of individual and group assessment tasks as elements of a public health nutrition competency-based assessment system.

METHODS

Assessment performance data from 158 dietetics students (three group tasks and one individual task) who had completed a practical placement learning experience in a public health nutrition setting were examined using nonparametric techniques. All 158 students were deemed individually 'competent' on completion of the placement.

RESULTS

The median mark was significantly lower for the individual compared to the group task, with a greater range of marks achieved in the individual assessment. There was a weak relationship between individual and group marks for the whole cohort (n = 158) (Spearman's rho correlation coefficient = 0.193, P = 0.015). Bland-Altman analysis showed that the mean (SD) agreement between the two assessment tasks was -5.9 (17.7) marks. Systematic bias between the two tasks was also demonstrated, indicating that students with the lowest average mark of the two assessments scored lower on the individual assessment task compared to their group task and those who had a higher average mark scored higher on the individual group assessment compared to their group task.

CONCLUSIONS

Student performance in public health differs between individual and group assessment. Individual assessment appears to differentiate between students, yet group work is essential for the development of teamwork skills. Both should be considered in the judgement of public health nutrition competency.