The midwifery miniCEX--a valuable clinical assessment tool for midwifery education

Development of a tool for assessing the clinical competency of Chinese master's nursing students based on the mini-CEX: a Delphi method study

OBJECTIVE

To construct a scientific and systematic competency evaluation tool for master of nursing specialists (MNS) and to provide a reference for the training, assessment and competency evaluation of MNS.

METHODS

A first draft of the indicators for assessing MNS core competencies was developed on the basis of published research and group discussions. Between June and December 2020, the indicators were revised using two rounds of the Delphi expert consultation method, with questionnaires completed by 16 experts from five provinces in China.

RESULTS

The valid retrieval rate of the two questionnaires was 100.00%, and the coefficient of expert authority was 0.931. The Kendall's concordance coefficients of the two rounds of questionnaires were 0.136 (p<0.05) and 0.147 (p<0.05), respectively. Consensus was reached on the seven dimensions and 52 items of the MNS competency assessment instrument. The instrument dimensions included nurse‒patient communication (9 items), health assessment (7 items), clinical decision-making (8 items), operational skills (7 items), health promotion (6 items), humanistic care (9 items) and organisational effectiveness (6 items).

CONCLUSIONS

The MNS competency assessment tool constructed in this study is focused and highly credible. The findings can be used as a guide for the training, assessment and competence evaluation of MNS in the future.

Developing the Occupational Therapy-Specific Mini-Clinical Evaluation Exercise (Mini-CEX) for Evaluating Interns' Clinical Skills and Attitudes in Pediatric Occupational Therapy

IMPORTANCE

The Mini-Clinical Evaluation Exercise (Mini-CEX) is highly recommended for assessing interns' performance.

OBJECTIVE

To develop a pediatric occupational therapy-specific Mini-CEX and examine its psychometrics.

DESIGN

Stage 1 had a retrospective design; Stage 2 had a prospective design.

SETTING

Pediatric occupational therapy unit in a hospital in Taiwan.

PARTICIPANTS

Thirty-four occupational therapy interns were evaluated with the Mini-CEX (physician version), and 57 were evaluated with the occupational therapy-specific Mini-CEX.

OUTCOMES AND MEASURES

The occupational therapy-specific Mini-CEX was developed with seven items on a 9-point scale categorized into three levels (unsatisfactory, satisfactory, highly satisfactory).

RESULTS

In Stage 1, the frequency of Mini-CEX (physician version) items receiving a rating of not applicable ranged from 1.9% to 88.1%. In Stage 2, the frequency of occupational therapy-specific Mini-CEX items receiving a rating of not applicable ranged from 3.5% to 31.6%. With the theme of evaluation taken into consideration, the frequency of not-applicable ratings was 0% to 8.8%. For the occupational therapy-specific Mini-CEX, content validity (item-level content validity index = 1, scale-level content validity index = 1) and internal consistency (Cronbach's α = .93) were excellent. The interns' scores on the second evaluation were significantly higher than those on their first evaluation, indicating good discriminant validity.

CONCLUSIONS AND RELEVANCE

The occupational therapy-specific Mini-CEX appears to be reliable and valid, and it is appropriate for evaluating interns' skills and attitudes in pediatric occupational therapy practice. What This Article Adds: The results support the development of the occupational therapy-specific Mini-CEX and its application in pediatric internship training.

Reliability and validity of new online selection tests for midwifery students

BACKGROUND AND OBJECTIVE

Newly created midwifery bachelor programmes attract an increasing number of applicants. Hence, educators have moved away from traditional non-standardised selection methods and started developing new tools for the admission of students. Researchers have expressed concerns around the validity of such selection measures and their ability to identify the appropriate candidates. The objective of this study was to develop and implement selection procedures for a new midwifery Bachelor of Science degree in Hamburg, Germany, and assess their psychometric properties.

DESIGN

This is a mixed-methods longitudinal study on the reliability, validity and acceptability of newly developed selection criteria.

SETTING

The newly established midwifery bachelor of science in Hamburg, Germany.

PARTICIPANTS

Upon completion of their online application, all midwifery applicants were invited by email to participate in our research project and thereby informed of its purpose and voluntary nature of their participation. The total number of candidates who took the test was 366, of which 309 agreed to take part in this study. Sixty-five applicants were eventually admitted to the midwifery Bachelor programme, of which 59 were included in the study.

MEASUREMENTS AND FINDINGS

We developed two tests for cognitive ability (HAM-Mid I and II) and adapted one test for personal and professional characteristics (Casper - Computer-based Assessment for Sampling Personal Characteristics). Due to the Covid pandemic, we implemented the tests online instead of on site as initially planned. HAM-Mid I had the lowest (Cronbach's Alpha = 0.38) whereas Casper had the highest internal consistency (Cronbach's Alpha = 0.77) of all three tests. Age, previous academic achievement and native tongue were significantly associated with applicants' performance on HAM-Mid II and Casper admission tests. HAM-Mid II was associated with students' performance in the first year midwifery exam (r = 0.31, p < 0.05). Overall evaluation of HAM-Mid II and Casper was good while HAM-Mid I received a more negative feedback.

KEY CONCLUSIONS

Evidence on the reliability and predictive validity of the newly developed tests suggests that the applied admission criteria are appropriate for an objective selection of bachelor midwifery students.

IMPLICATIONS FOR PRACTICE

The interests of university and clinical stakeholders of a midwifery bachelor degree can be represented through the development of tests for the assessment of cognitive ability and personal and professional characteristics.

Development of a nursing-specific Mini-CEX and evaluation of the core competencies of new nurses in postgraduate year training programs in Taiwan

BACKGROUND

Modern nursing requires a broad set of academic and practical skills, and an effective nurse must integrate these skills in a wide range of healthcare contexts. Cultivation of core competencies has recently become a key issue globally in the development of nursing education. To assess the performance of new nurses, this study developed a nursing-specific Mini-Clinical Evaluation Exercise (Mini-CEX) to evaluate the effect of postgraduate year (PGY) nurse training programs in Taiwan.

METHODS

A nursing-specific Mini-CEX was developed based on the required core competencies of nurses. Reliability and validity were confirmed in evaluator workshops carried out prior to the administration of the pilot test and final test. Thirty-two PYG trainees were recruited with a supervisor-to-trainee ratio of 1:1.94. Data were collected from February to June 2012 and analyzed using the Kruskal-Wallis test.

RESULTS

The 32 PGY trainees scored highest in the "nursing professionalism" dimension and the lowest in the "physical examination" dimension. The overall competency score was satisfactory. The trainee nurses with 19-24 months of experience scored higher than the other two groups in overall performance.

CONCLUSION

The results of this research indicate the feasibility of using our Mini-CEX tool to evaluate the competencies of PGY trainees.

The educational impact of Mini-Clinical Evaluation Exercise (Mini-CEX) and Direct Observation of Procedural Skills (DOPS) and its association with implementation: A systematic review and meta-analysis

Introduction

Mini Clinical Evaluation Exercise (Mini-CEX) and Direct Observation of Procedural Skills (DOPS) are used as formative assessments worldwide. Since an up-to-date comprehensive synthesis of the educational impact of Mini-CEX and DOPS is lacking, we performed a systematic review. Moreover, as the educational impact might be influenced by characteristics of the setting in which Mini-CEX and DOPS take place or their implementation status, we additionally investigated these potential influences.

Methods

We searched Scopus, Web of Science, and Ovid, including All Ovid Journals, Embase, ERIC, Ovid MEDLINE(R), and PsycINFO, for original research articles investigating the educational impact of Mini-CEX and DOPS on undergraduate and postgraduate trainees from all health professions, published in English or German from 1995 to 2016. Educational impact was operationalized and classified using Barr’s adaptation of Kirkpatrick’s four-level model. Where applicable, outcomes were pooled in meta-analyses, separately for Mini-CEX and DOPS. To examine potential influences, we used Fisher’s exact test for count data.

Results

We identified 26 articles demonstrating heterogeneous effects of Mini-CEX and DOPS on learners’ reactions (Kirkpatrick Level 1) and positive effects of Mini-CEX and DOPS on trainees’ performance (Kirkpatrick Level 2b; Mini-CEX: standardized mean difference (SMD) = 0.26, p = 0.014; DOPS: SMD = 3.33, p<0.001). No studies were found on higher Kirkpatrick levels. Regarding potential influences, we found two implementation characteristics, “quality” and “participant responsiveness”, to be associated with the educational impact.

Conclusions

Despite the limited evidence, the meta-analyses demonstrated positive effects of Mini-CEX and DOPS on trainee performance. Additionally, we revealed implementation characteristics to be associated with the educational impact. Hence, we assume that considering implementation characteristics could increase the educational impact of Mini-CEX and DOPS.

Integrating learning assessment and supervision in a competency framework for clinical workplace education

Although competency-based education is well established in health care education, research shows that the competencies do not always match the reality of clinical workplaces. Therefore, there is a need to design feasible and evidence-based competency frameworks that fit the workplace reality. This theoretical paper outlines a competency-based framework, designed to facilitate learning, assessment and supervision in clinical workplace education. Integration is the cornerstone of this holistic competency framework.

A framework to facilitate self-directed learning, assessment and supervision in midwifery practice: a qualitative study of supervisors' perceptions

BACKGROUND

Self-directed learning is an educational concept that has received increasing attention. The recent workplace literature, however, reports problems with the facilitation of self-directed learning in clinical practice. We developed the Midwifery Assessment and Feedback Instrument (MAFI) as a framework to facilitate self-directed learning. In the present study, we sought clinical supervisors' perceptions of the usefulness of MAFI.

METHODS

Interviews with fifteen clinical supervisors were audio taped, transcribed verbatim and analysed thematically using Atlas-Ti software for qualitative data analysis.

RESULTS

Four themes emerged from the analysis. (1) The competency-based educational structure promotes the setting of realistic learning outcomes and a focus on competency development, (2) instructing students to write reflections facilitates student-centred supervision, (3) creating a feedback culture is necessary to achieve continuity in supervision and (4) integrating feedback and assessment might facilitate competency development under the condition that evidence is discussed during assessment meetings. Supervisors stressed the need for direct observation, and instruction how to facilitate a self-directed learning process.

CONCLUSION

The MAFI appears to be a useful framework to promote self-directed learning in clinical practice. The effect can be advanced by creating a feedback and assessment culture where learners and supervisors share the responsibility for developing self-directed learning.