The Value of Programmatic Assessment in Supporting Educators and Students to Succeed: A Qualitative Evaluation

Oral interview in place of traditional objective structured clinical examinations for assessing placement readiness in nutrition and dietetics education

AIM

Objective structured clinical examinations have long been used in dietetics education. This observational study aims to describe the development, deployment, feasibility and validity of assessment using an oral interview in place of traditional objective structured clinical examinations, and to determine the ability of this assessment to identify students who are either not ready for placement or may require early support and/or remediation.

METHODS

Student assessment data were collected over a two-and-a-half-year period and used to test the predictive ability of an oral interview to determine dietetic placement outcomes and highlight a need for early remediation. Descriptive statistics as well as a between-group one-way ANOVA was used to describe results.

RESULTS

A total of 169 students participated in the oral interview and subsequent medical nutrition therapy placement over the study period. Significant differences in oral interview score were seen between students who passed placement and students who passed with remediation or those who failed. Oral interview performance was able to predict placement outcome, yet required less resources than traditional objective structured clinical examinations.

CONCLUSION

An oral interview may provide the same utility as the objective structured clinical examination in dietetics education .

Shifting the narrative and practice of assessing professionalism in dietetics education: An Australasian qualitative study

AIM

We aimed to explore current approaches to assessing professionalism in dietetics education in Australia and New Zealand, and asked the questions what is working well and what needs to improve?

METHOD

We employed a qualitative interpretive approach and conducted interviews with academic and practitioner (workplace-based) educators (total sample n = 78) with a key stake in dietetics education across Australia and New Zealand. Data were analysed using team-based, framework analysis.

RESULTS

Our findings suggest significant shifts in dietetics education in the area of professionalism assessment. Professionalism assessment is embedded in formal curricula of dietetics programs and is occurring in university and placement settings. In particular, advances have been demonstrated in those programs assessing professionalism as part of the programmatic assessment. Progress has been enabled by philosophical and curricula shifts; clearer articulation and shared understandings of professionalism standards; enhanced learner agency and reduced power distance; early identification and intervention of professionalism lapses; and increased confidence and capabilities of educators.

CONCLUSIONS

These findings suggest there have been considerable advances in professionalism assessment in recent years with shifts in practice in approaching professionalism through a more interpretivist lens, holistically and more student-centred. Professionalism assessment in dietetics education is a shared responsibility and requires further development and transformation to more fully embed and strengthen curricula approaches across programs. Further work should investigate strategies to build safer learning cultures and capacity for professionalism conversations and in strengthening approaches to remediation.

Teacher, Gatekeeper, or Team Member: supervisor positioning in programmatic assessment

Competency-based assessment is undergoing an evolution with the popularisation of programmatic assessment. Fundamental to programmatic assessment are the attributes and buy-in of the people participating in the system. Our previous research revealed unspoken, yet influential, cultural and relationship dynamics that interact with programmatic assessment to influence success. Pulling at this thread, we conducted secondary analysis of focus groups and interviews (n = 44 supervisors) using the critical lens of Positioning Theory to explore how workplace supervisors experienced and perceived their positioning within programmatic assessment. We found that supervisors positioned themselves in two of three ways. First, supervisors universally positioned themselves as a Teacher, describing an inherent duty to educate students. Enactment of this position was dichotomous, with some supervisors ascribing a passive and disempowered position onto students while others empowered students by cultivating an egalitarian teaching relationship. Second, two mutually exclusive positions were described-either Gatekeeper or Team Member. Supervisors positioning themselves as Gatekeepers had a duty to protect the community and were vigilant to the detection of inadequate student performance. Programmatic assessment challenged this positioning by reorientating supervisor rights and duties which diminished their perceived authority and led to frustration and resistance. In contrast, Team Members enacted a right to make a valuable contribution to programmatic assessment and felt liberated from the burden of assessment, enabling them to assent power shifts towards students and the university. Identifying supervisor positions revealed how programmatic assessment challenged traditional structures and ideologies, impeding success, and provides insights into supporting supervisors in programmatic assessment.

An intentional approach to the development and implementation of meaningful assessment in advanced radiation therapy practice curricula

•

Intentional assessment design is essential for advanced practitioner courses.

•

Using programmatic assessment enables learner agency and individualised learning.

•

Aligning assessment longitudinally facilitates transparent achievement of advanced practitioner capabilities.

Creating meaningful assessment for advanced radiation therapy practice training programs is a challenge. This is because it requires a balance of formative and summative assessments, which meet the academic and professional needs of the practitioner, as well as the requirements of local service delivery, educational and professional standards. This paper discusses educational strategies and models used to integrate assessment into theoretical and clinical curricula, allowing practitioners to demonstrate higher order cognitive knowledge, advanced level clinical performance and attitudes/values associated with advanced practice. The discussion draws upon concepts of constructive alignment and programmatic approaches to assessment, which use Bloom’s taxonomy, Benner’s beginner to competent model of skill development, and Miller’s pyramid of clinical competence. These models are analysed with respect to an advanced practice program in adaptive radiation therapy to provide context.

Ottawa 2020 consensus statement for programmatic assessment - 1. Agreement on the principles

INTRODUCTION

In the Ottawa 2018 Consensus framework for good assessment, a set of criteria was presented for systems of assessment. Currently, programmatic assessment is being established in an increasing number of programmes. In this Ottawa 2020 consensus statement for programmatic assessment insights from practice and research are used to define the principles of programmatic assessment.

METHODS

For fifteen programmes in health professions education affiliated with members of an expert group (n = 20), an inventory was completed for the perceived components, rationale, and importance of a programmatic assessment design. Input from attendees of a programmatic assessment workshop and symposium at the 2020 Ottawa conference was included. The outcome is discussed in concurrence with current theory and research.

RESULTS AND DISCUSSION

Twelve principles are presented that are considered as important and recognisable facets of programmatic assessment. Overall these principles were used in the curriculum and assessment design, albeit with a range of approaches and rigor, suggesting that programmatic assessment is an achievable education and assessment model, embedded both in practice and research. Knowledge on and sharing how programmatic assessment is being operationalized may help support educators charting their own implementation journey of programmatic assessment in their respective programmes.