Evaluation of usefulness of smart device-based testing: a survey study of Korean medical students

Development and application of a mobile-based multimedia nursing competency evaluation system for nursing students: A mixed-method randomized controlled study

AIM

To develop a mobile-based multimedia Nursing Competency Evaluation (NCE) system based on the Attention, Relevance, Confidence, Satisfaction model and verify its effectiveness.

BACKGROUND

In education, mobile devices can enable the delivery of learning content without time and spatial constraints. Mobile-based test is emerging as a novel method using technologies to appraise students' performance on practicum. This mobile-based test go beyond the simple evaluation of memorised knowledge, a limitation common to paper-based tests. They are useful because they can include multimedia items such as videos, animations and pictures to comprehensively evaluate students' clinical competencies.

METHODS

This study was conducted in a nursing university in Seoul, South Korea in September 2021. A mixed method randomised controlled design was employed to evaluate its usability. The participants in the experimental group used the Nursing Competency Evaluation system and joined in focus group interviews for verifying the effects of the Nursing Competency Evaluation system qualitatively. Those in the control group responded to the mobile-based test, but which has only text-based test items. The system usability, effectiveness and learning satisfaction in both groups were measured after the mobile-based test experience. Quantitative and qualitative data were analysed using t-tests and thematic analysis using focus group interviews, respectively.

RESULTS

Sixty nursing students participated, with 30 each in the experimental and control groups. There were no significant differences in nursing competency scores between the two groups. However, average scores for effectiveness and learning satisfaction were significantly higher in the experimental than in the control group. Nineteen experimental group participants partook in the interviews, with many describing that the Nursing Competency Evaluation system allowed them to experience new learning contents and efficiently learn practical nursing skills that can be useful in clinical settings.

CONCLUSIONS

The Nursing Competency Evaluation system is a promising method because it used mobile technologies and multimedia to appraise students' performance on nursing practicum. It was found that the Nursing Competency Evaluation system with multimedia items is more realistic, interactive and satisfactory compared with text-based mobile test. Thus, we expect it to be used in future nursing curriculums to improve students' nursing competencies.

Evaluation of Student Satisfaction with Ubiquitous-Based Tests in Women's Health Nursing Course

Learning evaluation using ubiquitous-based tests may be essential during a public health crisis, such as the COVID-19 pandemic, during which theoretical classes and clinical practice are conducted online. However, students may not be as familiar with ubiquitous-based tests as they are with paper-based tests. This survey study aimed to evaluate students' satisfaction with ubiquitous-based tests and compare the evaluation results of a paper-based test with that of a ubiquitous-based test in nursing education. For the midterm exam of the Women's Health Nursing course, a paper-based test was conducted, while a ubiquitous-based test using a tablet computer was used for the final exam. The Ubiquitous-Based Test Usefulness and Satisfaction tool, which has a five-point Likert-type response scale, was employed to evaluate the post-test usefulness and satisfaction scores of the ubiquitous-based test. The mean score of the ubiquitous-based test usefulness was 4.01 ± 0.67. There was a significant difference in satisfaction levels between the ubiquitous-based and the paper-based test (t = -3.36, p = 0.001). Specifically, the evaluation scores were not affected by different evaluation methods. Study participants deemed the ubiquitous-based test highly useful and satisfactory, suggesting that such tests may be a future-oriented evaluation method, potentially replacing paper-based tests.