A natural language intelligent tutoring system for training pathologists: implementation and evaluation

INTRODUCTION

We developed and evaluated a Natural Language Interface (NLI) for an Intelligent Tutoring System (ITS) in Diagnostic Pathology. The system teaches residents to examine pathologic slides and write accurate pathology reports while providing immediate feedback on errors they make in their slide review and diagnostic reports. Residents can ask for help at any point in the case, and will receive context-specific feedback.

RESEARCH QUESTIONS

We evaluated (1) the performance of our natural language system, (2) the effect of the system on learning (3) the effect of feedback timing on learning gains and (4) the effect of ReportTutor on performance to self-assessment correlations.

METHODS

The study uses a crossover 2 x 2 factorial design. We recruited 20 subjects from 4 academic programs. Subjects were randomly assigned to one of the four conditions--two conditions for the immediate interface, and two for the delayed interface. An expert dermatopathologist created a reference standard and 2 board certified AP/CP pathology fellows manually coded the residents' assessment reports. Subjects were given the opportunity to self grade their performance and we used a survey to determine student response to both interfaces.

RESULTS

Our results show a highly significant improvement in report writing after one tutoring session with 4-fold increase in the learning gains with both interfaces but no effect of feedback timing on performance gains. Residents who used the immediate feedback interface first experienced a feature learning gain that is correlated with the number of cases they viewed. There was no correlation between performance and self-assessment in either condition.

Student evaluation of a standardized comprehensive testing program

Computer-based, standardized, comprehensive testing is becoming a popular assessment tool in nursing education.This study sought to determine student response and satisfaction regarding such testing at a large state nursing school. Surveys that reviewed the entire testing process were provided to all students taking the computerized testing. Student feedback led to revisions for future testing.

Satisfaction with web-based training in an integrated healthcare delivery network: do age, education, computer skills and attitudes matter?

BACKGROUND

Healthcare institutions spend enormous time and effort to train their workforce. Web-based training can potentially streamline this process. However the deployment of web-based training in a large-scale setting with a diverse healthcare workforce has not been evaluated. The aim of this study was to evaluate the satisfaction of healthcare professionals with web-based training and to determine the predictors of such satisfaction including age, education status and computer proficiency.

METHODS

Observational, cross-sectional survey of healthcare professionals from six hospital systems in an integrated delivery network. We measured overall satisfaction to web-based training and response to survey items measuring Website Usability, Course Usefulness, Instructional Design Effectiveness, Computer Proficiency and Self-learning Attitude.

RESULTS

A total of 17,891 healthcare professionals completed the web-based training on HIPAA Privacy Rule; and of these, 13,537 completed the survey (response rate 75.6%). Overall course satisfaction was good (median, 4; scale, 1 to 5) with more than 75% of the respondents satisfied with the training (rating 4 or 5) and 65% preferring web-based training over traditional instructor-led training (rating 4 or 5). Multivariable ordinal regression revealed 3 key predictors of satisfaction with web-based training: Instructional Design Effectiveness, Website Usability and Course Usefulness. Demographic predictors such as gender, age and education did not have an effect on satisfaction.

CONCLUSION

The study shows that web-based training when tailored to learners' background, is perceived as a satisfactory mode of learning by an interdisciplinary group of healthcare professionals, irrespective of age, education level or prior computer experience. Future studies should aim to measure the long-term outcomes of web-based training.

Fostering diagnostic knowledge through computer-supported, case-based worked examples: effects of erroneous examples and feedback

OBJECTIVES

A case-based, worked example approach was realised in a computer-based learning environment with the intention of facilitating medical students' diagnostic knowledge. In order to enhance the effectiveness of the approach, two additional measures were implemented: erroneous examples and elaborated feedback. In the context of an experimental study, the two measures were varied experimentally.

METHODS

A total of 153 medical students were randomly assigned to four experimental conditions of a 2 x 2-factor design (errors versus no errors, elaborated feedback versus knowledge of correct result [KCR]). In order to verify the sustainability of the effects, a subgroup of subjects (n = 52) was compared with a control group of students who did not participate in the experiment (n = 145) on a regular multiple-choice question (MCQ) test.

RESULTS

Results show that the acquisition of diagnostic knowledge is mainly supported by providing erroneous examples in combination with elaborated feedback. These effects were independent from differences in time-on-task and prior knowledge. Furthermore, the effects of the learning environment proved sustainable.

CONCLUSIONS

Our results demonstrate that the case-based, worked example approach is effective and efficient.

Collaborative action learning: A professional development model for educational innovation in nursing

The paper describes the processes and outcomes of a major curriculum innovation which was conducted by a collaborative multi-disciplinary team (nurse academics, educational developers and software developers). The paper argues that collaborative professional development in pedagogical innovation in nursing can be successfully supported by action learning as a framework for practice. In presenting this argument the paper draws on the experience of the School of Nursing and Midwifery (SNM) at the University of Tasmania in integrating high-fidelity simulation-based learning into an existing undergraduate case-based learning curriculum in the three year Bachelor of Nursing (BN).

Innovations in integrative healthcare education: a healing journey for cancer patients in ontario and an online interdisciplinary CAM course for allied health students in Kentucky

Content on integrative healthcare and complementary and alternative medicine (CAM) is being taught in hundreds of educational programs across the country. Nursing, medical, osteopathic, chiropractic, acupuncture, naturopathic, and other programs are finding creative and innovative ways to include these approaches in new models of education and practice. This column spotlights such innovations in integrative healthcare and CAM education and presents readers with specific educational interventions they can adapt into new or ongoing educational efforts at their institution or programs. We invite readers to submit brief descriptions of efforts in their institutions that reflect the creativity, diversity, and interdisciplinary nature of the field.

The perceptions of undergraduate student nurses of high-fidelity simulation-based learning: a case report from the University of Tasmania

This paper reports a qualitatively informed curriculum research project in the three-year Bachelor of Nursing (BN) at the School of Nursing and Midwifery (SNM) at the University of Tasmania. The project investigated the perceptions of second year undergraduate nurses and their academic teachers of their experiences of high-fidelity simulation using the Laerdal Vital Sim Nursing Kelly and Nursing Anne technology(2) as part of their preparation for clinical practice. An associated curriculum benchmarking audit was also undertaken. A voluntary purposeful sample of students enrolled in a clinically-based practise unit(3) participated in the research, along with a small cohort of academic teachers. The results indicated that undergraduate nursing students value the opportunity to practice nursing activities in a safe environment prior to clinical placement. Students believed that simulation is an innovative strategy that promotes active learning and has great potential for developing clinical competence and increasing confidence prior to practise. The academic staff reported a similar belief about the potential of high-fidelity simulation in a case-based curriculum. The associated curriculum benchmarking audit provided evidence to support further integration of high-fidelity simulation in the undergraduate nursing program.

Computer-assisted instruction for novice nurses in critical care

The current worldwide nursing shortage is having the greatest effect on the critical care areas as novice nurses enter these specialty areas in unprecedented numbers. Massive healthcare restructuring, a rapidly aging nursing work force, and high nursing turnover and absenteeism rates have all contributed to a healthcare environment that is struggling to care for critically ill patients. Computer-assisted instruction in the form of interactive computer programs may be used as an adjunct educational resource for these novice practitioners. These programs have the potential to bridge the gap between theory and practice, which may assist the novice nurse in achieving safe, competent clinical practice.

HIT plants SEEDS in healthcare education

By incorporating a clinical information system in the education curriculum as a teaching platform, the University of Kansas School of Nursing teaches nurses and other health professional students how to assess, plan, document and manage care in an electronic medium that develops healthcare informatics competencies. The outcomes of this integrated technology curriculum brings hope for transforming health professional education for 21st century practice and graduating a workforce with the leadership and competencies for improving quality and safety in patient care. It results in IT savvy healthcare providers who will cross the quality chasm.

Virtual reality and brain anatomy: a randomised trial of e-learning instructional designs

CONTEXT

Computer-aided instruction is used increasingly in medical education and anatomy instruction with limited research evidence to guide its design and deployment.

OBJECTIVES

To determine the effects of (a) learner control over the e-learning environment and (b) key views of the brain versus multiple views in the learning of brain surface anatomy.

DESIGN

Randomised trial with 2 phases of study. Participants Volunteer sample of 1st-year psychology students (phase 1, n = 120; phase 2, n = 120). Interventions Phase 1: computer-based instruction in brain surface anatomy with 4 conditions: (1) learner control/multiple views (LMV); (2) learner control/key views (LKV); (3) programme control/multiple views (PMV); (4) programme control/key views (PKV). Phase 2: 2 conditions: low learner control/key views (PKV) versus no learner control/key views (SKV). All participants performed a pre-test, post-test and test of visuospatial ability.

MAIN OUTCOME MEASURES

A 30-item post-test of brain surface anatomy structure identification.

RESULTS

The PKV group attained the best post-test score (57.7%) and the PMV group received the worst (42.2%), with the 2 high learner control groups performing in between. For students with low spatial ability, estimated scores are 20% lower for those who saw multiple views during learning. In phase 2, students with the most static condition and no learner control (SKV) performed similarly to those students in the PKV group.

CONCLUSIONS

Multiple views may impede learning, particularly for those with relatively poor spatial ability. High degrees of learner control may reduce effectiveness of learning.

Peer reviewing e-learning: opportunities, challenges, and solutions

Peer review is the foundation of academic publication and a necessary step in the scrutiny of any scholarly work. Simply defined, peer review is the attentive, unbiased assessment of any scholarly work that is submitted for formal scrutiny. Although medical school faculty increasingly use technology in clinical teaching, e-learning materials are often not subjected to a rigorous peer review process. The authors contrast peer review of e-learning materials with that of print materials, describe peer review issues regarding e-learning materials, propose approaches to address the challenges of peer review of e-learning materials, and outline directions for refinement of the e-learning peer review process. At its core, the peer review of e-learning materials should not differ substantially from that of traditional manuscripts. However, e-learning introduces new demands that impel reviewers to consider aspects that are unique to educational technology, including pedagogy, format, usability, navigation, interactivity, delivery, ease of updating, distribution, and access. Four approaches are offered to ease the burden and improve the quality of e-learning peer review: develop peer review training, embrace multidisciplinary peer review, develop guidelines, and provide incentives and compensation. The authors conclude with suggestions about peer review research.

A tale of two studies: the importance of setting, subjects and context in two randomized, controlled trials of a web-based decision support for perimenopausal and postmenopausal health decisions

OBJECTIVE

Knowledge relevant to women's peri- and postmenopausal health decisions has been evolving rapidly. Web-based decision supports can be rapidly updated and have the potential to improve the quality of patients' decisions. We developed and tested a web-based decision support for peri- and postmenopausal health decisionmaking.

METHODS

We recruited 409 women aged 45-75 for one randomized, controlled trial and 54 women with an upcoming clinic appointment for a subsequent trial. Women were randomized to use the web-based decision support versus a printed brochure (first trial) and usual care (second trial). Outcomes were changes in decisional satisfaction, decisional conflict, and knowledge, both within each trial and compared across the trials.

RESULTS

Intervention subjects had greater increases in decisional satisfaction in the second trial and knowledge in both trials. A high dropout rate among women randomized to the website in the first trial effectively negated benefits in that trial, but not in the second.

CONCLUSIONS

The utility of this web-based decision support in two trials depended on a number of factors that appear related to the urgency of making a decision.

PRACTICE IMPLICATIONS

Decision aids should be targeted to patients actively trying to make a decision.

Quality control of epidemiological lectures online: scientific evaluation of peer review

AIM

To examine the feasibility of using peer review for the quality control of online materials.

METHODS

We analyzed the inter-rater agreement on the quality of epidemiological lectures online, based on the Global Health Network Supercourse lecture library. We examined the agreement among reviewers by looking at kappa statistics and intraclass correlations. Seven expert reviewers examined and rated a random sample of 100 Supercourse lectures. Their reviews were compared with the reviews of the lay Supercourse reviewers.

RESULTS

Both expert and non-expert reviewers rated lectures very highly, with a mean overall score of 4 out of 5. Kappa (Kappa) statistic and intraclass correlations indicated that inter-rater agreement for experts and non-experts was surprisingly low (below 0.4).

CONCLUSIONS

To our knowledge, this was the first time that poor inter-rater agreement was demonstrated for the Internet lectures. Future research studies need to evaluate the alternatives to the peer review system, especially for online materials.

Design guidelines for the development of digital nutrigenomics learning material for heterogeneous target groups

Nutritional genomics, or nutrigenomics, can be considered as the combination of molecular nutrition and genomics. Students who attend courses in nutrigenomics differ with respect to their prior knowledge. This study describes digital nutrigenomics learning material suitable for students from various backgrounds and provides design guidelines for the development of the learning material. These design guidelines, derived from theories on cognitive science and instructional design, describe the selection of interaction types for learning tasks and the timing of information presentation. The learning material supports two learning goals: 1) the formulation of meaningful research questions in the field of nutrigenomics and 2) the development of feasible experiments to answer these questions. The learning material consists of two cases built around important nutrigenomics topics: 1) personalized diets and 2) the role of free fatty acids in the regulation of hepatic gene transcription. Each case consists of several activities to promote active learning by the student. Evaluation of the cases in a realistic academic educational setting indicates that the cases were useful.

Incorporating Geriatric Content into an Undergraduate Parent–Child Nursing Course—An Innovative Approach

The purpose of this article is to describe the development of an innovative educational strategy for incorporating geriatric content into an undergraduate parent-child nursing course. Faculty supervised the development of an interactive media-based module that met content objectives for the parent-child course and included the functional assessment of the geriatric client living in a home setting. Students found this module to be a creative, user-friendly, and practical application of theory to a realistic clinical situation. This strategy successfully infused geriatric content into a pediatric course to enhance students' abilities to care for older adults.

Evaluation of Online and On-site Options for Master's Degree and Post-master's Certificate Programs

Measures related to program evaluation were selected to compare on-site and online options of a master's and post-master's nursing program. Although some differences were found in student-related outcomes, the decision to continue the online options was based on program and organizational effectiveness. The decline in the number of on-site students has been offset by the expanding enrollments of online students and their national distribution. Increased costs were associated with offering online courses.