Development of an instrument to assess individual student performance in small group tutorials

The collaborative learning development exercise (CLeD-EX): an educational instrument to promote key collaborative learning behaviours in medical students

BACKGROUND

Modern clinical practice increasingly relies on collaborative and team-based approaches to care. Regulatory bodies in medical education emphasise the need to develop collaboration and teamwork competencies and highlight the need to do so from an early stage of medical training. In undergraduate medical education, the focus is usually on collaborative learning, associated with feedback and reflection on this learning This article describes a novel educational instrument, the Collaborative Learning Development Exercise (CLeD-EX), which aims to foster the development of key collaborative learning competencies in medical students. In this article we report on the effectiveness, feasibility and educational impact of the CLeD-EX.

METHODS

In this study, the "educational design research" framework was used to develop, implement and evaluate the CLeD-EX. This involved adopting a systematic approach towards designing a creative and innovative instrument which would help solve a real-world challenge in developing collaborative learning skills. The systematic approach involved a qualitative exploration of key collaborative learning behaviours which are influential in effective collaborative learning contexts. The identified competencies were employed in the design of the CLeD-EX. The design of the CLeD-EX included features to facilitate structured feedback by tutors to students, complemented by self-evaluation and reflection. The CLeD-EX was field-tested with volunteer junior medical students, using a controlled pre-test post-test design. Analysis of the completed CLeD-EX forms, self-perception surveys (i.e. pre-test and post-test surveys) and analyses of reflective reports were used to explore the educational impact of CLeD-EX, as well as its utility and practicality.

RESULTS

After using the CLeD-EX, students showed a significant improvement in critical thinking and group process as measured by a previously validated instrument. Both students and tutors recognised CLeD-EX as an effective instrument, especially as a structured basis for giving and receiving feedback and for completing the feedback loop. CLeD-EX was also found to be feasible, practical and focused, while promoting learning and effective interactions in small group learning.

CONCLUSION

The findings of this study support the introduction of an effective and feasible educational instrument such as the CLeD-EX, to facilitate the development of students' skills in collaborative learning.

Assessing Attitudes of Nursing Pedagogy: An Instrument Based on the Midrange Theory of Empowered Holistic Nursing Education

BACKGROUND AND PURPOSE

The Empowered Holistic Nursing Education (EHNE) midrange theory provides a structure for a pedagogy for nursing, bringing the core values of nursing into the classroom. There are five principles of EHNE: Self-Care, Interconnectedness, Prior Knowledge, Contextual Teaching-Learning, and Meet Them Where They Are.

METHODS

The Thurstone method was used in the development of this scale, resulting in two subscales representing unique phenomena, each with 11 relational items that represent the spectrum of attitudes for the phenomenon.

RESULTS

With a CVI score of 0.92 (n = 23) for subscale 1 and 0.89 (n = 23) for subscale 2, the scale was found to have high inter-rater reliability with an agreement of K = 0.814 (99% confidence interval). Analysis revealed a significant, positive association between the two subscales (r = .96 and sig/2 tailed = .003).

CONCLUSIONS

This instrument development study as a whole allows for further testing of the midrange theory and advancement to implementation and intervention research.

Multiple tutorial-based assessments: a generalizability study

BACKGROUND

Tutorial-based assessment commonly used in problem-based learning (PBL) is thought to provide information about students which is different from that gathered with traditional assessment strategies such as multiple-choice questions or short-answer questions. Although multiple-observations within units in an undergraduate medical education curriculum foster more reliable scores, that evaluation design is not always practically feasible. Thus, this study investigated the overall reliability of a tutorial-based program of assessment, namely the Tutotest-Lite.

METHODS

More specifically, scores from multiple units were used to profile clinical domains for the first two years of a system-based PBL curriculum.

RESULTS

G-Study analysis revealed an acceptable level of generalizability, with g-coefficients of 0.84 and 0.83 for Years 1 and 2, respectively. Interestingly, D-Studies suggested that as few as five observations over one year would yield sufficiently reliable scores.

CONCLUSIONS

Overall, the results from this study support the use of the Tutotest-Lite to judge clinical domains over different PBL units.

Building life-long learning capacity in undergraduate nursing freshmen within an integrative and small group learning context

BACKGROUND

Life-long learning involves the development of skills in critical thinking (CT), effective group process (GP), and self-directedness (SDL). Recent studies have shown that small group learning with active interactions is effective in enabling students to develop themselves as independent learners beyond graduation. With a view to integrative learning, the purpose of this study was to evaluate life-long learning outcomes through the work of small group teaching and learning for a class of undergraduate nursing freshmen during one academic year.

METHODS

A mixed-methods approach was used to evaluate the CT, GP and SDL of 99 freshmen with a self-assessment questionnaire before and after their learning activities in three nursing courses, and to identify themes from a total of six focus group interviews with the students and teachers.

RESULTS

The CT, GP and SDL results obtained from self-assessment did not indicate significant differences. Four themes emerged from the qualitative analysis.

DISCUSSION

Many factors contributed to the results on life-long learning skill development of students in this study. The qualitative analysis provided good insights for future teaching and learning development.

CONCLUSIONS

With a developmental perspective, life-long learning may be better developed and evaluated over a longer period of time in the nursing program.

Students generating questions for their own written examinations

Assessment partnerships between staff and students are considered a vital component of the student-centred educational process. To enhance the development of this partnership in a problem-based learning curriculum, all first-year students were involved in generating a bank of formative assessment questions with answers, some of which were included in their final written examination. Important principles to guide development of a sound methodology for such an assessment partnership have been described. These include organisational issues as well as matters pertaining to participation, education and motivation of students and teaching staff.

Problem-based learning: an educational method for nurses in clinical practice

To enhance the professional development of nurses, a problem-based learning program was developed and implemented for nurses at Joseph Brant Memorial Hospital, a Southern Ontario community hospital. The author outlines the key concepts associated with the educational methodology of problem-based learning. In addition, a detailed narrative illustrates the processes involved in the development and implementation of this program specific to the Joseph Brant Memorial Hospital experience, including exemplars from nurses participating in the initiative.

Multiple rubric-based assessments of student case presentations

OBJECTIVES

To evaluate a rubric-based method of assessing pharmacy students' case presentations in the recitation component of a therapeutics course.

METHODS

A rubric was developed to assess knowledge, skills, and professional behavior. The rubric was used for instructor, student peer, and student self-assessment of case presentations. Rubric-based composite scores were compared to the previous dichotomous checklist-based scores.

RESULTS

Rubric-based instructor scores were significantly lower and had a broader score distribution than those resulting from the checklist method. Spring 2007 rubric-based composite scores from instructors and peers were significantly lower than those from the pilot study results, but self-assessment composite scores were not significantly different.

CONCLUSIONS

Successful development and implementation of a grading rubric facilitated evaluation of knowledge, skills, and professional behavior from the viewpoints of instructor, peer, and self in a didactic course.

Critical thinking as an outcome of context-based learning among post RN students: a literature review

Increasingly complex needs and expanding roles in the delivery of health care require professional nurses to be capable critical thinkers and self-directed learners. Problem-based learning (PBL) is promoted as a mean to facilitate critical thinking (CT) in nursing students attending generic and post RN baccalaureate programs. The authors summarize and analyze nursing research and theoretical literature (1992-2005) related to four key topic areas: critical thinking, nursing education, PBL, and post RN education, to determine what is known about the impact of PBL on CT among post RN students. CINAHL, ERIC, Medline and PsychINFO databases were searched and based on the literature retrieved, there is no strong research evidence to suggest changes occur in baccalaureate nursing students' CT during their educational programs, including those using PBL. Similar results were found among post RN students although the literature on this topic is sparse. There is a need for additional research and ongoing development of valid and reliable instruments to measure CT in nurses to guide teaching and learning strategies that effectively facilitate CT among nursing students.

Using a sampling strategy to address psychometric challenges in tutorial-based assessments

INTRODUCTION

Tutorial-based assessment, despite providing a good match with the philosophy adopted by educational programmes that emphasize small group learning, remains one of the greatest challenges for educators working in this context. The current study was performed in an attempt to assess the psychometric characteristics of tutorial-based evaluation upon adopting a multiple sampling approach that requires minimal recording of observations.

METHOD

After reviewing the literature, a simple 3-item evaluation form was created. The items were "Professional Behaviour," "Contribution to Group Process," and "Contribution to Group Content." Explicit definition of these items was provided on an evaluation form. Twenty five tutors in five different programmes were asked to use the form to evaluate their students (N=169) after every tutorial over the course of an academic unit. Each item was rated using a 10-point scale.

RESULTS

Cronbach's alpha revealed an appropriate internal consistency in all five programmes. Test-retest reliability of any single rating was low, but the reliability of the average rating was at least 0.75 in all cases. The construct validity of the tool was supported by the observation of increasing ratings over the course of the academic unit and by the finding that more senior students received higher ratings than more junior students.

CONCLUSION

Consistent with the context specificity phenomenon, the adoption of a "minimal observations often" approach to tutorial-based assessment appears to maintain better psychometric characteristics than do attempts to assess tutorial performance using more comprehensive measurement tools.