Quality Improvement Amid a Global Pandemic: A Virtual Curriculum for Medical Students in the Time of COVID-19

INTRODUCTION

The COVID-19 pandemic in March of 2020 necessitated the removal of medical students from direct patient care activities to prevent disease spread and to conserve personal protective equipment. In order for medical student education to continue, virtual and online electives were designed and implemented expeditiously. We created a virtual curriculum that taught quality improvement (QI) skills within the context of the global pandemic.

METHODS

This 4-week curriculum enrolled 16 students. Students completed the revised QI knowledge application tool (QIKAT-R) before and after the course to assess QI knowledge. Students completed prereading, online modules, and received lectures on QI and incident command systems. Each group designed their own QI project related to our hospital system's response to the pandemic. Finally, groups presented their projects at a peer symposium and completed peer evaluations.

RESULTS

Students' QIKAT-R scores improved throughout the course from a mean of 5.5 (SD = 1.3) to a mean of 7.5 (SD = 1.1; p < 0.001). Students reported that the virtual learning experience delivered the material effectively, and all students agreed that they would participate in QI work in the future.

DISCUSSION

Patient safety and QI topics are content areas for multiple medical licensing examinations. Virtual learning is an effective way to deliver QI content to medical students and residents, especially when projects are trainee-led, QI-trained faculty serve as mentors, and the projects harmonize with institutional goals. Our virtual pandemic-focused curriculum has demonstrated efficacy in increasing medical student QI knowledge.

[Evaluation of the use of a simulation software in the learning of cardiopulmonary auscultation in undergraduate medical students]

INTRODUCTION

Medsounds™ software allows to create an auscultation learning platform, by providing real pre-recorded cardiopulmonary sounds on virtual chests. The study aimed at comparing the skills in cardiopulmonary auscultation between students who benefited from this platform and students who did not have access to it.

METHODS

A controlled trial was conducted with 2nd year medical students randomised into three groups. Groups A, B and C received 10 h of cardiopulmonary clinical training. In addition, group B benefited from an online access to the educative platform, and group C had a demonstration of the platform during their clinical training, then an online access. The main outcome was a 3-point multiple-choice questionnaire based on 2 original case vignettes about the description of cardiopulmonary sounds. The secondary outcome was the faculty exam on high-fidelity cardiopulmonary simulator.

RESULTS

Groups A and B included 127 students, and group C 117. Students in group C had a significantly higher score than those in group A (1.72/3 versus 1.48/3; p = 0.02), without difference between the groups B and C. Students who actually had a demonstration of the platform and used it at home had a higher score than those who did not use it (1.87 versus 1.51; p = 0.01). Students who had a demonstration of the platform before using it performed a better pulmonary examination on high-fidelity simulators.

CONCLUSION

The supervised use of an online auscultation simulation software in addition to the traditional clinical training seems to improve the auscultation performances of undergraduated medical students.

Evaluating critical success factors in implementing E-learning system using multi-criteria decision-making

Learning using the Internet or training through E-Learning is growing rapidly and is increasingly favored over the traditional methods of learning and teaching. This radical shift is directly linked to the revolution in digital computer technology. The revolution propelled by innovation in computer technology has widened the scope of E-Learning and teaching, whereby the process of exchanging information has been made simple, transparent, and effective. The E-Learning system depends on different success factors from diverse points of view such as system, support from the institution, instructor, and student. Thus, the effect of critical success factors (CSFs) on the E-Learning system must be critically analyzed to make it more effective and successful. This current paper employed the analytic hierarchy process (AHP) with group decision-making (GDM) and Fuzzy AHP (FAHP) to study the diversified factors from different dimensions of the web-based E-Learning system. The present paper quantified the CSFs along with its dimensions. Five different dimensions and 25 factors associated with the web-based E-Learning system were revealed through the literature review and were analyzed further. Furthermore, the influence of each factor was derived successfully. Knowing the impact of each E-Learning factor will help stakeholders to construct education policies, manage the E-Learning system, perform asset management, and keep pace with global changes in knowledge acquisition and management.

Ten simple rules for making training materials FAIR

Everything we do today is becoming more and more reliant on the use of computers. The field of biology is no exception; but most biologists receive little or no formal preparation for the increasingly computational aspects of their discipline. In consequence, informal training courses are often needed to plug the gaps; and the demand for such training is growing worldwide. To meet this demand, some training programs are being expanded, and new ones are being developed. Key to both scenarios is the creation of new course materials. Rather than starting from scratch, however, it's sometimes possible to repurpose materials that already exist. Yet finding suitable materials online can be difficult: They're often widely scattered across the internet or hidden in their home institutions, with no systematic way to find them. This is a common problem for all digital objects. The scientific community has attempted to address this issue by developing a set of rules (which have been called the Findable, Accessible, Interoperable and Reusable [FAIR] principles) to make such objects more findable and reusable. Here, we show how to apply these rules to help make training materials easier to find, (re)use, and adapt, for the benefit of all.

The evaluation of e-learning resources as an adjunct to otolaryngology teaching: a pilot study

BACKGROUND

The concept of e-Learning has been rapidly accepted as an important component of medical education and is especially adept at teaching clinical skills. However, their impact on learning, particularly in Otolaryngology Head and Neck Surgery (OHNS) medical school curriculum, has yet to be adequately explored. The aim of this pilot study is to develop interactive e-Learning resources and evaluate their impact in enhancing OHNS teaching in medical school.

METHODS

This pilot study is a randomized controlled trial assessing the effectiveness of e-Learning resources in enhancing the current traditional lecture and tutorial-based teaching of OHNS in medical school. Nineteen final-year medical students from the University of Sydney were recruited for this study, who were randomly allocated into intervention group with additional e-Learning resources (Group A) and control group (Group B). Student knowledge was assessed through objective structured clinical examinations (OSCE) with use of standardized forms for objective scoring. Assessors were blinded to student randomization status. A post-study questionnaire was distributed to assess student feedback on the e-Learning resources.

RESULTS

Eight students were allocated to Group A and 11 students to Group B. Group A performed significantly better than Group B in the overall examination scores (78.50 ± 13.88 v. 55.82 ± 8.23; P = < 0.01). With the minimum pass mark of 65%, the majority of students in Group A was able to pass the OSCE assessments, while the majority of students in Group B failed (87.50% v. 9.10%; P = 0.01). The post-test questionnaire on the e-Learning resources showed very favorable feedback from the students' perspective.

CONCLUSION

Results from our pilot study suggests that the use of interactive online e-Learning resources can be a valuable adjunct in supplementing OHNS teaching in medical school, as they are readily accessible and allow flexible on-demand learning. Future studies involving large numbers of medical students are needed to validate these results.

From Modules to MOOCs: Application of the Six-Step Approach to Online Curriculum Development for Medical Education

Online curricula can make high-quality health professions education accessible in virtually any setting. They can enhance teaching and learning by both standardizing curricular resources and individualizing curricular experiences. Despite growing demand for and institutional interest in online curricula for medical education, many medical educators lack a framework for online curriculum development. Without rigorous and thoughtful development, online curricula can waste opportunity and resources by leading to education that is inferior to traditional methods. In this article, the authors describe a systematic approach to online curriculum development based on the Six-Step Approach for Curriculum Development for Medical Education, a widely used method that has led to successful implementation of a variety of traditional and online curricula. In each step, special considerations for curricula with larger and more diverse learner audiences-characteristic of many online curricula-are highlighted. Four common online curricular formats are also discussed: blended curricula, instructor-led fully online curricula, self-paced modules, and massive open online courses (MOOCs). The authors emphasize factors that differentiate one online format from another, including the budgetary, technical, and human resource requirements for each. The article concludes by urging medical educators to pursue opportunities to study and disseminate online curricular work.

Expert reading coaching via technology: Investigating the reading, writing, and spelling outcomes of students in grades K-8 experiencing significant reading learning disabilities

While qualitative research has shown great benefits for teachers who receive coaching, there is a paucity of experimental research examining students' academic outcomes after their teachers received ongoing support from a knowledgeable and experienced coach. Thus, a quasi-experimental design investigated the literacy outcomes of 452 students experiencing reading learning disabilities in grades K-8th whose special education and/or resource room teachers (n = 44) received student data-focused coaching support through on-site coaching, on-demand coaching (teachers could request support if needed), or through technology-based coaching. Specifically, researchers wanted to investigate if technology-based coaching was as effective as in-classroom support for increasing teachers' knowledge and implementation of research-based reading instructional routines and ultimately, improving the reading, writing, and spelling outcomes of students with reading learning disabilities. Results yielded positive student academic growth for all three methods of coaching; however, coaching via technology, a more efficient, less time-consuming method of giving teachers ongoing professional development, produced larger statistically significant Cohen's d effect sizes than the other two forms of coaching ranging from 0.22 to 1.01 in areas of phonemic awareness, decoding, comprehension, fluency, writing, and spelling. Other findings as well as the educational implications of implementing coaching via technology are also included.

Online decision aids for primary cardiovascular disease prevention: systematic search, evaluation of quality and suitability for low health literacy patients

OBJECTIVES

Recent guideline changes for cardiovascular disease (CVD) prevention medication have resulted in calls to implement shared decision-making rather than arbitrary treatment thresholds. Less attention has been paid to existing tools that could facilitate this. Decision aids are well-established tools that enable shared decision-making and have been shown to improve CVD prevention adherence. However, it is unknown how many CVD decision aids are publicly available for patients online, what their quality is like and whether they are suitable for patients with lower health literacy, for whom the burden of CVD is greatest. This study aimed to identify and evaluate all English language, publicly available online CVD prevention decision aids.

DESIGN

Systematic review of public websites in August to November 2016 using an environmental scan methodology, with updated evaluation in April 2018. The decision aids were evaluated based on: (1) suitability for low health literacy populations (understandability, actionability and readability); and (2) International Patient Decision Aids Standards (IPDAS).

PRIMARY OUTCOME MEASURES

Understandability and actionability using the validated Patient Education Materials Assessment Tool for Printed Materials (PEMAT-P scale), readability using Gunning-Fog and Flesch-Kincaid indices and quality using IPDAS V.3 and V.4.

RESULTS

A total of 25 unique decision aids were identified. On the PEMAT-P scale, the decision aids scored well on understandability (mean 87%) but not on actionability (mean 61%). Readability was also higher than recommended levels (mean Gunning-Fog index=10.1; suitable for grade 10 students). Four decision aids met criteria to be considered a decision aid (ie, met IPDAS qualifying criteria) and one sufficiently minimised major bias (ie, met IPDAS certification criteria).

CONCLUSIONS

Publicly available CVD prevention decision aids are not suitable for low literacy populations and only one met international standards for certification. Given that patients with lower health literacy are at increased risk of CVD, this urgently needs to be addressed.

Phys-MAPS: a programmatic physiology assessment for introductory and advanced undergraduates

We describe the development of a new, freely available, online, programmatic-level assessment tool, Measuring Achievement and Progress in Science in Physiology, or Phys-MAPS ( http://cperl.lassp.cornell.edu/bio-maps ). Aligned with the conceptual frameworks of Core Principles of Physiology, and Vision and Change Core Concepts, Phys-MAPS can be used to evaluate student learning of core physiology concepts at multiple time points in an undergraduate physiology program, providing a valuable longitudinal tool to gain insight into student thinking and aid in the data-driven reform of physiology curricula. Phys-MAPS questions have a modified multiple true/false design and were developed using an iterative process, including student interviews and physiology expert review to verify scientific accuracy, appropriateness for physiology majors, and clarity. The final version of Phys-MAPS was tested with 2,600 students across 13 universities, has evidence of reliability, and has no significant statement biases. Over 90% of the physiology experts surveyed agreed that each Phys-MAPS statement was scientifically accurate and relevant to a physiology major. When testing each statement for bias, differential item functioning analysis demonstrated only a small effect size (<0.008) of any tested demographic variable. Regarding student performance, Phys-MAPS can also distinguish between lower and upper division students, both across different institutions (average overall scores increase with each level of class standing; two-way ANOVA, P < 0.001) and within each of three sample institutions (each ANOVA, P ≤ 0.001). Furthermore, at the level of individual concepts, only evolution and homeostasis do not demonstrate the typical increase across class standing, suggesting these concepts likely present consistent conceptual challenges for physiology students.

Eradicating Barriers to Advancement From RN to BSN: An Exploratory Study

BACKGROUND

To improve the quality of patient care, the Institute of Medicine has called for eliminating barriers to academic progression for nurses. Despite this call, nurses who matriculate from institutions that lack national nursing-specific accreditation are denied admission to advanced degree programs.

METHOD

Within an online RN-to-baccalaureate nursing (BSN) program, the performance of students who matriculated from institutions lacking discipline-specific accreditation was compared with student performance representative of programs holding discipline-specific accreditation. The bases for comparison were grade point average (GPA), on-time completion rate, and academic withdrawal.

RESULTS

Students admitted from institutions lacking national nursing accreditation had a higher on-time completion rate and a lower GPA than comparison groups. Differences in GPA were statistically insignificant (p > .05). Evidence regarding withdrawals was insufficient for adequate comparisons.

CONCLUSION

Admission policies that deny education opportunities to candidates from institutions lacking national nursing accreditation may create unwarranted barriers to educational advancement. [J Contin Educ Nurs. 2019;50(1):15-19.].

Effectiveness of e-Learning in a Medical School 2.0 Model: Comparison of Item Analysis for Student-Generated vs. Faculty-Generated Multiple-Choice Questions

BACKGROUND

Early reports in the literature describe using student-generated questions as a method of student learning as well as augmenting question exam banks. Reports on the performance of student-generated questions versus faculty-generated questions, however, remain limited. This study aims to compare the question performance of student-generated versus faculty-generated multiple-choice questions (MCQ).

OBJECTIVES

To determine if student-generated questions using mobile audience response systems and online discussion boards have similar item discrimination scores as faculty-generated questions.

METHODS

A team-based learning session was used to create 113 student-generated multiple-choice questions (SGQs). A 20 question MCQ quiz was presented to a second year medical school class made of 10 randomly selected SGQs and 10 randomly selected faculty-generated multiple-choice questions (FGQs). Item analysis was performed on the test results.

RESULTS

The data showed no statistical difference in the point-biserial scores between the two groups (average point-biserial 0.31 students vs 0.36 faculty, p=0.14), with 90% of student-generated and 100% of faculty-generated questions meeting a cut-off of point-biserial score &gt;0.2. Interestingly, student-generated questions were statistically more difficult than the faculty-generated questions (Item Difficulty score 0.46 students vs 0.69 faculty, p=0.003).

CONCLUSIONS

This study suggests that student-generated compared to faculty-generated MCQs have similar item discrimination scores, but are perhaps more difficult questions.

Evaluation of a virtual neurophysiology laboratory as a new pedagogical tool for medical undergraduate students in China

This study compared the effect of a virtual laboratory, a living tissue laboratory, and a blended laboratory on student learning about the generation and conduction of neural action potentials and perceptions about life science. Sixty-three second-year medical students were randomly assigned to one of three groups (living tissue laboratory, virtual laboratory, and blended group). The students conducted the practical activity, and then they were given a postlaboratory quiz and an attitude survey. The blended group euthanized fewer animals and spent less time to finish the animal experiment than the living tissue group did. In the postlaboratory quiz, students who performed the virtual laboratory alone got significantly lower scores than students in the other two groups, and the blended group did not get better scores than the living tissue group. The attitude surveys showed that the virtual laboratory group had a lower perceived value of the science research and activity in which they participated than the other two groups did. Here, 77.8% of all students chose the blended style as the ideal teaching method for experiments. Our findings led us to believe that isolated use of the virtual laboratory in China is not the best practice: the virtual laboratory serves as an effective preparation tool, and the blended laboratories may become the best laboratory teaching practice, provided that the software design for the virtual laboratory is further improved.

Interactive Multimodal Curriculum on Use and Interpretation of Inpatient Telemetry

INTRODUCTION

Inpatient telemetry monitoring is a commonly used technology designed to detect and monitor life-threatening arrhythmias. However, residents are rarely educated in the proper use and interpretation of telemetry monitoring.

METHODS

We developed a training module containing an educational video, PowerPoint presentation, and hands-on interactive learning session with a telemetry expert. The module highlights proper use of telemetry monitoring, recognition of telemetry artifact, and interrogation of telemetry to identify clinically significant arrhythmias. Learners completed pre- and postcurriculum knowledge-based assessments and a postcurriculum survey on their experience with the module. In total, the educational curriculum had three 60-minute sessions.

RESULTS

Thirty-two residents participated in the training module. Residents scored higher on the posttest (77% ± 12%) than on the pretest (70% ± 12%), t(31) = -4.3, p < .001. Wilcoxon signed rank tests indicated PGY-3s performed better on the posttest (Mdn = 0.86) than on the pretest (Mdn = 0.72), z = -2.19, p = .031. PGY-2s also performed better on the posttest (Mdn = 0.86) than on the pretest (Mdn = 0.76), z = -2.04, p = .042. There was no difference between pretest (Mdn = 0.66) and posttest (Mdn = 0.71) scores for PGY-1s, z = -1.50, p = .142. The majority of residents reported that the telemetry curriculum boosted their self-confidence, helped prepare them to analyze telemetry on their patients, and should be a required component of the residency.

DISCUSSION

This module represents a new paradigm for teaching residents how to successfully and confidently interpret and use inpatient telemetry.

Self-regulated learning and critical reflection in an e-learning on patient safety for third-year medical students

OBJECTIVES

To explore the influence of critical thinking, self-regulated learning and system usability on the acceptance of e-learning on patient safety.

METHODS

A cross-sectional study was conducted, using a 32-question online survey. One hundred ninety-three (n=193 medical students participated in the study and were asked to rate levels of reflective thinking, self-regulated learning and attitudes towards patient safety using scales from the Questionnaire for Reflective Thinking, the Attitudes to Patient Safety Questionnaire and the System Usability Scale. Differences between reflection levels were calculated using paired t-tests, associations between critical thinking and self-regulated learning were calculated using linear correlations. We performed linear multiple regression analysis to identify predictors for student acceptance of the e-learning.

RESULTS

Students (n=193) engaged in intermediate levels of reflection (5-point Likert, M=3.62, SD=0.73) and significantly (t₍₁₄₃₎=15.15, p<0.001, d=1.57) lower levels (M=2.35, SD=0.87) of critical reflection. Most students showed high (≥ 4; 44.1%) or intermediate (<4 level > 2; 29.4 %) levels of self-regulated learning. A regression model indicated that 5 predictors (Reflection, critical reflection, self-regulated learning, relevance, usability) explained 65.3% of the variance (R²=0.653, F_{(5, 96)}=39.02, p<0.01) of perceived total quality.

CONCLUSIONS

This study shows that reflection and learning skills are important factors for e-learning acceptance, but perceived relevance and system usability play a more important role.  From a didactic perspective, it is indispensable to provide the students with sufficient examples and links to professional practice to enhance the perception of relevance and to improve system usability permanently.

Optional anatomy and physiology e-learning resources: student access, learning approaches, and academic outcomes

Anatomy and physiology interactive video clips were introduced into a blended learning environment, as an optional resource, and were accessed by ~50% of the cohort. Student feedback indicated that clips were engaging, assisted understanding of course content, and provided lecture support. Students could also access two other optional online resources, lecture capture recordings and an interactive atlas of anatomy, and individuals were tracked with respect to their access behavior, learning approach, and subject score. Deep learning was highest among those accessing the clips or atlas or those accessing more online resources, and thus self-regulatory skill development might be a useful approach to increase student access to optional online resources. Those who accessed clips, lecture capture recordings, or atlas achieved a significantly higher subject score than those who did not. When combinations of resources used were considered, we found an approximately linear relationship between number of resources accessed and subject score, with a 16% difference in score between those who accessed none or all of the resources. However, the low resource access rate suggests that academic advantage may not be simply due to the learning support offered by the resources. As students accessing the optional resources tended to be more self-regulated, it may be that it was the extra effort made with respect to other subject resources, rather than just the access to the online resources, that contributed to higher subject score. Further studies are required to establish the relationships between academic performance, optional online resource access, and deep learning.

[Development of a French-language online health policy course: an international collaboration]

OBJECTIVE

To present the process and challenges of developing an online competency-based course on public health policy using a collaborative international approach.

METHODS

Five public health experts, supported by an expert in educational technology, adopted a rigorous approach to the development of the course: a needs analysis, identification of objectives and competencies, development of a pedagogical scenario for each module and target, choice of teaching methods and learning activities, material to be identified or developed, and the responsibilities and tasks involved.

RESULTS

The 2-credit (90-hour) graduate course consists of six modules including an integration module. The modules start with a variety of case studies: tobacco law (neutral packaging), supervised injection sites, housing, integrated services for the frail elderly, a prevention programme for mothers from disadvantaged backgrounds, and the obligatory use of bicycle helmets. In modules 1, 3, 4 and 5, students learn about different stages of the public policy development process: emergence, formulation and adoption, implementation and evaluation. Module 2 focuses on the importance of values and ideologies in public policy. The integration module allows the students to apply the knowledge learned and addresses the role of experts in public policy and ethical considerations.

CONCLUSION

The course has been integrated into the graduate programmes of the participating universities and allows students to follow, at a distance, an innovative training programme.

Use of e-learning in clinical clerkships: effects on acquisition of dermatological knowledge and learning processes

OBJECTIVES

To obtain a deeper understanding of how the e-learning program, Education in Dermatology (ED), affects the acquisition of dermatological knowledge and the underlying learning processes of medical students in their clinical phase.

METHODS

The study used a mixed method design with a convergent parallel collection of data. Medical students (n=62) from Maastricht University (The Netherlands) were randomized to either a conventional teaching group (control group n=30) or conventional teaching plus the e-learning program (application on smartphone) group (e-learning group n=32). Pre- and post-intervention knowledge test results were analysed using an independent t-test. Individual semi-structured interviews (n=9) were conducted and verbatim-transcribed recordings were analysed using King's template analysis.

RESULTS

The e-learning program positively influenced students' level of knowledge and their process of learning. A significant difference was found in the post-test scores for the control group (M=51.4, SD=6.43) and the e-learning group (M=73.09, SD=5.12); t(60)=-14.75, p<0.000). Interview data showed that the e-learning program stimulated students' learning as the application promoted the identification and recognition of skin disorders, the use of references, creation of documents and sharing information with colleagues.

CONCLUSIONS

This study demonstrated that use of the e-learning program led to a significant improvement in basic dermatological knowledge. The underlying learning processes indicated that e-learning programs in dermatology filled a vital gap in the understanding of clinical reasoning in dermatology. These results might be useful when developing (clinical) teaching formats with a special focus on visual disciplines.

Computerized versus hand-scored health literacy tools: a comparison of Simple Measure of Gobbledygook (SMOG) and Flesch-Kincaid in printed patient education materials

Objective

The research compared and contrasted hand-scoring and computerized methods of evaluating the grade level of patient education materials that are distributed at an academic medical center in east Tennessee and sought to determine if these materials adhered to the American Medical Association’s (AMA’s) recommended reading level of sixth grade.

Methods

Librarians at an academic medical center located in the heart of Appalachian Tennessee initiated the assessment of 150 of the most used printed patient education materials. Based on the Flesch-Kincaid (F-K) scoring rubric, 2 of the 150 documents were excluded from statistical comparisons due to the absence of text (images only). Researchers assessed the remaining 148 documents using the hand-scored Simple Measure of Gobbledygook (SMOG) method and the computerized F-K grade level method. For SMOG, 3 independent reviewers hand-scored each of the 150 documents. For F-K, documents were analyzed using Microsoft Word. Reading grade levels scores were entered into a database for statistical analysis. Inter-rater reliability was calculated using intra-class correlation coefficients (ICC). Paired t-tests were used to compare readability means.

Results

Acceptable inter-rater reliability was found for SMOG (ICC=0.95). For the 148 documents assessed, SMOG produced a significantly higher mean reading grade level (M=9.6, SD=1.3) than F-K (M=6.5, SD=1.3; p<0.001). Additionally, when using the SMOG method of assessment, 147 of the 148 documents (99.3%) scored above the AMA’s recommended reading level of sixth grade.

Conclusions

Computerized health literacy assessment tools, used by many national patient education material providers, might not be representative of the actual reading grade levels of patient education materials. This is problematic in regions like Appalachia because materials may not be comprehensible to the area’s low-literacy patients. Medical librarians have the potential to advance their role in patient education to better serve their patient populations.

An Evidence-Based Approach to Teaching Obesity Management to Medical Students

Introduction

The need for education of future and current providers in evidence-based management of obesity and the release of new treatment guidelines prompted the development of a resource for use in the education of medical students and residents.

Methods

A self-contained module was developed to provide an overview of recent guidelines for obesity management utilizing evidence-based medicine while debunking popular myths associated with available weight-loss strategies. The module was delivered over 15 months to six groups of learners (N = 180) and was continuously improved through feedback from content experts and the learners. After completion of the module, one subset of learners responded to a three-question survey using a 5-point Likert scale (1 = strongly disagree, 5 = strongly agree).

Results

Formal evaluation of the module was completed by a subset of the learners (N = 32, 64% response rate). The majority agreed or strongly agreed with these survey statements: "Overall this module was valuable as an educational tool" (97%, Mdn = 4); "After completion of this module, I am confident of my knowledge on how to manage obesity in adult patients" (84%, Mdn = 4); and "It was easy to navigate the module" (94%, Mdn = 5).

Discussion

This module could be implemented as is at other institutions that strive to educate medical students or residents on the most recent guidelines and evidence-based medicine regarding obesity management.

Design and Evaluation of Video Podcasts for Providing Online Feedback on Formative Pharmaceutical Calculations Assessments

Objective. To evaluate worked example video podcasts as a method of providing feedback to pharmacy interns for an online and formative pharmaceutical calculations assessment. Methods. A theory-informed approach based on multimedia learning theory was used to design video podcasts as feedback on a calculations examination. A mixed-methods evaluation completed by pharmacy interns enrolled in Ireland's National Pharmacy Internship Programme was used to establish cognitive and affective attitudes toward video podcasts compared with conventional written solutions. Results. The majority of students found video podcasts were clear, helpful for learning, easy to understand, and useful as a method of feedback. Majority reported that they felt positively about standard written solutions. The evaluation suggested distinct benefits for each kind of feedback, something that has not been previously reported. Thematic analysis of qualitative data indicated useful features of video podcasts, including clear explanation, step-by-step approach, and synchronization of audio and visual information. Conclusion. Respondents reported positive cognitive and affective attitudes toward video podcasts as online feedback. Video podcasts are a helpful and novel way of providing feedback on pharmaceutical calculations. A similar opinion of traditional written solutions suggests that students may benefit from both forms of feedback. Further study is required to identify the particular benefits associated with both kinds.

Newborn Care Curriculum: Palliative Care for the Newborn

Introduction

Because pediatric hospitalists have increasing responsibilities in newborn hospitalization, training in perinatal palliative care is beneficial. A 2015 needs assessment revealed 68% of surveyed pediatric hospitalists were interested in more education on this topic. Thus, this learning module was designed to provide a concise, easy-to-use introduction to palliative care for the newborn.

Methods

This module was developed as part of the computer-based Newborn Care Curriculum to fill a gap in educational resources on perinatal palliative care. The primary tool in this learning module is a PowerPoint slide show with a script in the notes section. Using the presenter mode to view the PowerPoint slide show allows the learner to simultaneously view the slides and read the script for instruction. This module was tested by members of the pediatric hospitalist division at Children's National Health System in Washington, DC. Participants completed a pretest, posttest, and module evaluation. While tested as a self-study tool, the module may also be used in a small-group teaching setting.

Results

The module was well received during the trial. The average posttest score was 96%, compared to pretest scores of 90%. Learners' comments were overwhelmingly positive, and constructive feedback has been addressed.

Discussion

This module provides pediatric hospitalists and others who care for newborns with a well-received introduction to perinatal palliative care. The computer-based format of the module adds to its uniqueness and utility.

Concept and benefits of the Inverted Classroom method for a competency-based biochemistry course in the pre-clinical stage of a human medicine course of studies

Background: Medical students often have a problem recognising the relevance of basic science subjects for their later professional work in the pre-clinical stage of their studies. This can lead to a lower motivation to learn biochemical content and dissatisfaction in the courses amongst the students. Alternative teaching methods such as the Inverted Classroom (IC) method can address this deficiency. The goal of this study was: to analyse the motivation and satisfaction of the students in a biochemistry seminar through the use of the e-learning-based IC method, to investigate the acceptance against the IC teaching method in biochemistry, and to compare the learning success achieved using the IC approach with that of a traditional course. We also investigated how a biochemistry course in the pre-clinical stage of a human medicine course of studies can be successfully organised according to the IC method. Furthermore, we examined the benefits of the IC method over conventional teaching formats. Method: The IC method was implemented in accordance with the guidelines of the GMA committee "New Media" [30] in a biochemistry seminar for two student IC intervention groups with 42 students. A part of the factual knowledge from the on-site phase in the form of teaching videos together with self-learning control tasks were provided online before the seminar for both IC intervention groups. Exporting content to the self-learning phase creates new free time in the on-site phase, during which the content can be critically considered and processed and additional competency-based learning objectives can be taught. Identical biochemistry teaching content was taught in parallel control groups (14 student groups with n=299 students), but no material was handed out beforehand for a self-learning phase. These students only received the materials after the on-site phase. Motivation and satisfaction as well as the acceptance for the teaching methods were recorded by questionnaires, the acquisition of knowledge by MC exams. Results: On a Likert scale from 1 (strongly disagree) to 6 (strongly agree), the students in the IC intervention groups could be seen to be much more motivated (5.53) than students in the control group (4.01). Students in the IC intervention groups also recognised the relevance of the learning content much more clearly (5.44) than students in the control group (4.01). Furthermore, the IC group also observed that additional competencies were trained in addition to the biochemistry content. In addition, the IC intervention group award the event a school grade of 1.53, the traditional control group a grade of 2.96. The teaching videos were rated very positively by both groups with an average school grade of 1.3 in each case. A qualitative analysis showed that the motivation and a positive attitude of the lecturers played a decisive role in the successful implementation of the IC method. Discussion and conclusion: Pre-clinical students display a high acceptance of the e-learning-based IC method. Teaching communication competencies in a biochemistry seminar was also rated very positively by the students. The quality of the teaching video and the motivation of the lecturers were shown to be a critical parameter for the successful performance of the IC method. What's more, the IC method can contribute to implementing a competence orientation in medical studies.

Overview and current management of computerized adaptive testing in licensing/certification examinations

Computerized adaptive testing (CAT) has been implemented in high-stakes examinations such as the National Council Licensure Examination-Registered Nurses in the United States since 1994. Subsequently, the National Registry of Emergency Medical Technicians in the United States adopted CAT for certifying emergency medical technicians in 2007. This was done with the goal of introducing the implementation of CAT for medical health licensing examinations. Most implementations of CAT are based on item response theory, which hypothesizes that both the examinee and items have their own characteristics that do not change. There are 5 steps for implementing CAT: first, determining whether the CAT approach is feasible for a given testing program; second, establishing an item bank; third, pretesting, calibrating, and linking item parameters via statistical analysis; fourth, determining the specification for the final CAT related to the 5 components of the CAT algorithm; and finally, deploying the final CAT after specifying all the necessary components. The 5 components of the CAT algorithm are as follows: item bank, starting item, item selection rule, scoring procedure, and termination criterion. CAT management includes content balancing, item analysis, item scoring, standard setting, practice analysis, and item bank updates. Remaining issues include the cost of constructing CAT platforms and deploying the computer technology required to build an item bank. In conclusion, in order to ensure more accurate estimations of examinees' ability, CAT may be a good option for national licensing examinations. Measurement theory can support its implementation for high-stakes examinations.

Investigating Interrater Reliability in an Online RN-to-BSN Program: Disparate Conclusions

BACKGROUND

Interrater reliability of student-written assignments within a nursing education program is a desirable characteristic to ensure uniform quality of academic work. As online programs that advance RNs to baccalaureate-prepared nurses (i.e., RN-to-BSN programs) grow in popularity, the challenges of quality control increase.

METHOD

Grading data representing mean, median, mode, and range from 19 course sections in an online RN-to-BSN program were submitted. Data were analyzed for measures of central tendency with a query of interrater reliability.

RESULTS

The data indicate that instructors assess student performance at a uniformly high level.

CONCLUSION

The data lead to two disparate conclusions: student performance is high and is reliably assessed as such, or the online program suffers from grade inflation. Such disparate conclusions are uniquely endemic to any program that serves the RN-to-BSN population, typically high-performing students who have established prior mastery on licensure examinations. [J Nurs Educ. 2017;56(6):360-363.].

Evidence-Based Principles for Using Technology-Enhanced Learning in the Continuing Professional Development of Health Professionals

Increasingly, health professional training involves the use of educational technologies through what is broadly termed "Technology-Enhanced Learning" (TEL). TEL includes hardware, such as computers and mobile devices, and software, such as software applications (apps), learning management systems, and discussion boards. For many years, TEL has formed an integral part of health professional programs and is growing in acceptance, if not expectation, in postgraduate training and continuing education. TEL generally aims to be flexible, engaging, learner focused and interactive, and may involve collaboration and communication. It offers many benefits for learning and teaching, whether used on its own or in conjunction with face-to-face teaching through blended learning. The ubiquity of mobile devices in clinical settings means TEL is ideal for busy clinicians, both as learners and teachers. TEL enables participants to learn at a time and place that is convenient to them, so learners living in geographically dispersed locations can access standardized courses. To realize these potential benefits, we recommend that those developing TEL programs for health professionals take a systematic approach to planning, development, implementation, and evaluation. To that end, we propose 10 principles: clarify purpose and conduct a needs assessment; allocate adequate time and technology; incorporate proven approaches to improve learning; consider the need for a skills component; enable interaction between learners and with others; create different resources for different groups; pilot before implementing; incorporate measures to retain learners; provide opportunities for revision to aid retention; and evaluate learning outcomes, not just satisfaction.

A holistic model for evaluating the impact of individual technology-enhanced learning resources

BACKGROUND

The use of technology within education has now crossed the Rubicon; student expectations, the increasing availability of both hardware and software and the push to fully blended learning environments mean that educational institutions cannot afford to turn their backs on technology-enhanced learning (TEL). The ability to meaningfully evaluate the impact of TEL resources nevertheless remains problematic.

AIMS

This paper aims to establish a robust means of evaluating individual resources and meaningfully measure their impact upon learning within the context of the program in which they are used.

METHODS

Based upon the experience of developing and evaluating a range of mobile and desktop based TEL resources, this paper outlines a new four-stage evaluation process, taking into account learner satisfaction, learner gain, and the impact of a resource on both the individual and the institution in which it has been adapted.

RESULTS

A new multi-level model of TEL resource evaluation is proposed, which includes a preliminary evaluation of need, learner satisfaction and gain, learner impact and institutional impact. Each of these levels are discussed in detail, and in relation to existing TEL evaluation frameworks.

CONCLUSIONS

This paper details a holistic, meaningful evaluation model for individual TEL resources within the specific context in which they are used. It is proposed that this model is adopted to ensure that TEL resources are evaluated in a more meaningful and robust manner than is currently undertaken.

Genetics and Type 1 Diabetes: Online Resources for Patients

PURPOSE

This Web-based review was undertaken to compile online resources for diabetes educators on genetics--specifically, the genetics of type 1 diabetes--and to provide helpful and accurate information for the public.

METHODS

Keyword searches were performed to identify Web sites for genetics education for the lay public and for sites specifically geared toward children/young adults. Web sites were critiqued based on credibility (source, currency, relevance/utility), content (accuracy), and design (accessibility, logical organization). Additional keyword searches were conducted to find sites describing the genetics of type 1 diabetes, which were evaluated for content validity.

RESULTS

The Web sites selected for general genetics education contain accessible, credible, and accurate information about basic genetics in an easy-to-follow format with both text and visual aides. Although these sites adequately educate the public about genetics, only diabetes-specific Web sites discussed the relationship between genetics and risk for type 1 diabetes associated with high-risk HLA alleles.

CONCLUSIONS

In this genomic age, it is important for healthcare professionals to provide genetics information. Educational tools that specifically address the genetics of type 1 diabetes are urgently needed to fill the current information gaps on the Internet.

Quality specifications in postgraduate medical e-learning: an integrative literature review leading to a postgraduate medical e-learning model

BACKGROUND

E-learning is driving major shifts in medical education. Prioritizing learning theories and quality models improves the success of e-learning programs. Although many e-learning quality standards are available, few are focused on postgraduate medical education.

METHODS

We conducted an integrative review of the current postgraduate medical e-learning literature to identify quality specifications. The literature was thematically organized into a working model.

RESULTS

Unique quality specifications (n = 72) were consolidated and re-organized into a six-domain model that we called the Postgraduate Medical E-learning Model (Postgraduate ME Model). This model was partially based on the ISO-19796 standard, and drew on cognitive load multimedia principles. The domains of the model are preparation, software design and system specifications, communication, content, assessment, and maintenance.

CONCLUSION

This review clarified the current state of postgraduate medical e-learning standards and specifications. It also synthesized these specifications into a single working model. To validate our findings, the next-steps include testing the Postgraduate ME Model in controlled e-learning settings.

Physiotherapy students' perspectives of online e-learning for interdisciplinary management of chronic health conditions: a qualitative study

BACKGROUND

To qualitatively explore physiotherapy students' perceptions of online e-learning for chronic disease management using a previously developed, innovative and interactive, evidence-based, e-learning package: Rheumatoid Arthritis for Physiotherapists e-Learning (RAP-eL).

METHODS

Physiotherapy students participated in three focus groups in Perth, Western Australia. Purposive sampling was employed to ensure maximum heterogeneity across age, gender and educational background. To explore students' perspectives on the advantages and disadvantages of online e-learning, ways to enhance e-learning, and information/learning gaps in relation to interdisciplinary management of chronic health conditions, a semi-structured interview schedule was developed. Verbatim transcripts were analysed using inductive methods within a grounded theory approach to derive key themes.

RESULTS

Twenty-three students (78 % female; 39 % with previous tertiary qualification) of mean (SD) age 23 (3.6) years participated. Students expressed a preference for a combination of both online e-learning and lecture-style learning formats for chronic disease management, citing flexibility to work at one's own pace and time, and access to comprehensive information as advantages of e-learning learning. Personal interaction and ability to clarify information immediately were considered advantages of lecture-style formats. Perceived knowledge gaps included practical application of interdisciplinary approaches to chronic disease management and developing and implementing physiotherapy management plans for people with chronic health conditions.

CONCLUSIONS

Physiotherapy students preferred multi-modal and blended formats for learning about chronic disease management. This study highlights the need for further development of practically-oriented knowledge and skills related to interdisciplinary care for people with chronic conditions among physiotherapy students. While RAP-eL focuses on rheumatoid arthritis, the principles of learning apply to the broader context of chronic disease management.

Viability of a Web-Based Module for Teaching Electrocardiogram Reading Skills to Psychiatry Residents: Learning Outcomes and Trainee Interest

OBJECTIVE

Web-based instruction in post-graduate psychiatry training has shown comparable effectiveness to in-person instruction, but few topics have been addressed in this format. This study sought to evaluate the viability of a web-based curriculum in teaching electrocardiogram (EKG) reading skills to psychiatry residents. Interest in receiving educational materials in this format was also assessed.

METHODS

A web-based curriculum of 41 slides, including eight pre-test and eight post-test questions with emphasis on cardiac complications of psychotropic medications, was made available to all psychiatry residents via email.

RESULTS

Out of 57 residents, 30 initiated and 22 completed the module. Mean improvement from pre-test to post-test was 25 %, and all 22 completing participants indicated interest in future web-based instruction.

CONCLUSIONS

This pilot study suggests that web-based instruction is feasible and under-utilized as a means of teaching psychiatry residents. Potential uses of web-based instruction, such as tracking learning outcomes or patient care longitudinally, are also discussed.

Performance of undergraduate dental students on ICDAS clinical caries detection after different learning strategies

AIM

This study aimed to evaluate the effect of a digital learning tool on undergraduate dental students' performance in detecting dental caries using ICDAS.

METHODS

An experimental digital learning tool (DLT) was created using digital photographs of sound and carious teeth. Thirty-nine students were divided into three groups (n = 13) and each assessed 12 randomly allocated patients before and after learning strategies: G1, ICDAS e-learning program; G2, ICDAS e-learning program plus DLT; G3, no learning strategy. Students (n = 32) reassessed patients 2 weeks after training.

RESULTS

Comparing before and after the learning strategies, any difference in the values of specificity and area under the ROC curve for all groups was found. Sensitivity was statistically significantly higher for G1 and G2. Comparing the groups, G2 showed a significant increase in sensitivity at the D2 and D3 thresholds. Spearman's correlations with the gold standard before and after the learning strategy were 0.60 and 0.61 for G1, 0.57 and 0.63 for G2, and 0.54 and 0.54 for G3, respectively. The Wilcoxon test showed a statistically significant difference between the values obtained before and after learning strategies for G1 and G2.

CONCLUSIONS

Use of the DLT after the ICDAS e-learning program tended to increase the sensitivity of ICDAS used by undergraduate dental students. The DLT appeared to improve dental students' ability to use ICDAS.

Electronic learning in advanced resuscitation training: The perspective of the candidate

BACKGROUND

Studies have shown that blended approaches combining e-learning with face-to-face training reduces costs whilst maintaining similar learning outcomes. The preferences in learning approach for healthcare providers to this new style of learning have not been comprehensively studied. The aim of this study is to evaluate the acceptability of blended learning to advanced resuscitation training.

METHODS

Participants taking part in the traditional and blended electronic advanced life support (e-ALS) courses were invited to complete a written evaluation of the course. Participants' views were captured on a 6-point Likert scale and in free text written comments covering the content, delivery and organisation of the course. Proportional-odds cumulative logit models were used to compare quantitative responses. Thematic analysis was used to synthesise qualitative feedback.

RESULTS

2848 participants from 31 course centres took part in the study (2008-2010). Candidates consistently scored content delivered face-to-face over the same content delivered over the e-learning platform. Candidates valued practical hands on training which included simulation highly. Within the e-ALS group, a common theme was a feeling of "time pressure" and they "preferred the face-to-face teaching". However, others felt that e-ALS "suited their learning style", was "good for those recertifying", and allowed candidates to "use the learning materials at their own pace".

CONCLUSIONS

The e-ALS course was well received by most, but not all participants. The majority felt the e-learning module was beneficial. There was universal agreement that the face-to-face training was invaluable. Individual learning styles of the candidates affected their reaction to the course materials.

Accuracy and readability of cardiovascular entries on Wikipedia: are they reliable learning resources for medical students?

OBJECTIVE

To evaluate accuracy of content and readability level of English Wikipedia articles on cardiovascular diseases, using quality and readability tools.

METHODS

Wikipedia was searched on the 6 October 2013 for articles on cardiovascular diseases. Using a modified DISCERN (DISCERN is an instrument widely used in assessing online resources), articles were independently scored by three assessors. The readability was calculated using Flesch-Kincaid Grade Level. The inter-rater agreement between evaluators was calculated using the Fleiss κ scale.

RESULTS

This study was based on 47 English Wikipedia entries on cardiovascular diseases. The DISCERN scores had a median=33 (IQR=6). Four articles (8.5%) were of good quality (DISCERN score 40-50), 39 (83%) moderate (DISCERN 30-39) and 4 (8.5%) were poor (DISCERN 10-29). Although the entries covered the aetiology and the clinical picture, there were deficiencies in the pathophysiology of diseases, signs and symptoms, diagnostic approaches and treatment. The number of references varied from 1 to 127 references; 25.9±29.4 (mean±SD). Several problems were identified in the list of references and citations made in the articles. The readability of articles was 14.3±1.7 (mean±SD); consistent with the readability level for college students. In comparison, Harrison's Principles of Internal Medicine 18th edition had more tables, less references and no significant difference in number of graphs, images, illustrations or readability level. The overall agreement between the evaluators was good (Fleiss κ 0.718 (95% CI 0.57 to 0.83).

CONCLUSIONS

The Wikipedia entries are not aimed at a medical audience and should not be used as a substitute to recommended medical resources. Course designers and students should be aware that Wikipedia entries on cardiovascular diseases lack accuracy, predominantly due to errors of omission. Further improvement of the Wikipedia content of cardiovascular entries would be needed before they could be considered a supplementary resource.

Online technology use in physiotherapy teaching and learning: a systematic review of effectiveness and users' perceptions

BACKGROUND

The use of online technologies in health professionals' education, including physiotherapy, has been advocated as effective and well-accepted tools for enhancing student learning. The aim of this study was to critically review the effectiveness, and user perceptions of online technology for physiotherapy teaching and learning.

METHODS

Following databases were systematically searched on the 31(st) of August 2013 for articles describing implementation of online technologies into physiotherapy teaching and learning: ERIC, CINAHL, Web of Science, Academic search complete, ProQuest Nursing and Allied Health Source, Medline, Embase, and Scopus. No language, design or publishing date restrictions were imposed. Risk of bias was assessed using the 2011 Mixed Methods Appraisal Tool checklist (MMAT).

RESULTS

A total of 4133 articles were retrieved; 22 articles met the inclusion criteria and were accepted for final analysis: 15 on the effectiveness of technology, and 14 on users' perceptions. Included studies used three designs: case study (14 articles), controlled trial (3), and randomized controlled trial (5). Studies investigated both pre-registration physiotherapy students (1523) and physiotherapy professionals (171). The quality of studies ranged from 67 to 100 % on the MMAT checklist which can be considered moderate to excellent. More than half of the studies (68 %) received scores greater than 80 %. Studies typically investigated websites and discussion boards. The websites are effective in enhancing practical skills performance, and discussion boards in knowledge acquisition, as well as in development of critical and reflective thinking. Students' perceptions of the use of websites were mostly positive, providing students with entertaining, easy accessible resources. Perceived barriers to the use of websites included difficulties with internet connection, insufficiently interactive material, or personal preference for paper-based materials. Discussion boards were perceived as deepening students' thinking and facilitating reflection, allowing for learning from multiple perspectives, and providing easy communication and support.

CONCLUSIONS

The results of this review suggest that online technologies (i.e., websites and discussion boards) have many benefits to offer for physiotherapy teaching and learning; There was minimal evidence of barriers for the use of online technologies, however, addressing the identified ones could enhance adherence to use of online technologies in health professionals' education.

Improving consistency in large laboratory courses: a design for a standardized practical exam

Laboratory courses serve as important gateways to science, technology, engineering, and mathematics education. One of the challenges in assessing laboratory learning is to conduct meaningful and standardized practical exams, especially for large multisection laboratory courses. Laboratory practical exams in life sciences courses are frequently administered by asking students to move from station to station to answer questions, apply knowledge gained during laboratory experiments, interpret data, and identify various tissues and organs using various microscopic and gross specimens. This approach puts a stringent time limit on all questions regardless of the level of difficulty and also invariably increases the potential risk of cheating. To avoid potential cheating in laboratory courses with multiple sections, the setup for practical exams is often changed in some way between sections. In laboratory courses with multiple instructors or teaching assistants, practical exams may be handled inconsistently among different laboratory sections, due to differences in background knowledge, perceptions of the laboratory goals, or prior teaching experience. In this article, we describe a design for a laboratory practical exam that aims to align the assessment questions with well-defined laboratory learning objectives and improve the consistency among all laboratory sections.

Twelve tips for improving the effectiveness of web-based multimedia instruction for clinical learners

Using educational technology does not necessarily make medical education more effective. There are many different kinds of technology available to the contemporary medical teacher and what constitutes effective use may depend on the technology, the learning situation and many other factors. Web-based multimedia instruction (WBMI) provides learners with self-directed independent learning opportunities based on didactic material enhanced with multimedia features such as video and animations. WBMI may be used to replace other didactic events (e.g. lectures) or it may be provided in addition to other learning opportunities. Clinical educators looking to use WBMI need to make sure that it will meet both their learners' needs and the program's needs, and it has to align to the contexts in which it is used. The following 12 tips have been developed to help guide faculty through some of the key features of the effective use of WBMI in clinical teaching programs. These tips are based on more than a decade developing, using and appraising WBMI in support of surgical clerkship education across the USA and beyond and they are intended both to inform individual uses of WBMI in clinical training and to guide the strategic use of WBMI in clinical clerkship curricula.

Cost-effectiveness of electronic training in domestic violence risk assessment: ODARA 101

The need for domestic violence training has increased with the development of evidence-based risk assessment tools, which must be scored correctly for valid application. Emerging research indicates that training in domestic violence risk assessment can increase scoring accuracy, but despite the increasing popularity of electronic training, it is not yet known whether it can be an effective method of risk assessment training. In the present study, 87 assessors from various professions had training in the Ontario Domestic Assault Risk Assessment either face-to-face or using an electronic training program. The two conditions were equally effective, as measured by performance on a post-training skill acquisition test. Completion rates were 100% for face-to-face and 86% for electronic training, an improvement over a previously evaluated manual-only condition. The estimated per-trainee cost of electronic training was one third that of face-to-face training and expected to decrease. More rigorous evaluations of electronic training for risk assessment are recommended.

Evaluating technology-enhanced learning: A comprehensive framework

BACKGROUND

The absence of a standard, comprehensive approach to evaluating technology-enhanced learning (TEL) limits the utility of individual evaluations, and impedes the integration and synthesis of results across studies.

PURPOSE

To outline a comprehensive framework for approaching TEL evaluation in medical education, and to develop instruments for measuring the perceptions of TEL learners and instructors.

METHODS AND RESULTS

Using both theoretical constructs of inquiry in education and a synthesis of existing models and instruments, we outlined a general model for evaluation that links utility, principles, and practices. From this we derived a framework for TEL evaluation that identifies seven data collection activities: needs analysis; documentation of processes, decisions, and final product; usability testing; observation of implementation; assessment of participant experience; assessment of learning outcomes; and evaluation of cost, reusability, and sustainability. We then used existing quality standards and approaches to develop instruments for assessing the experiences of learners and instructors using TEL.

CONCLUSIONS

No single evaluation is likely to collect all of this information, nor would any single audience likely find all information elements equally useful. However, consistent use of a common evaluation framework across different courses and institutions would avoid duplication of effort and allow cross-course comparisons.

Effects of computer-based graphic organizers to solve one-step word problems for middle school students with mild intellectual disability: A preliminary study

The purpose of this study was to examine the effects of computer-based graphic organizers, using Kidspiration 3© software, to solve one-step word problems. Participants included three students with mild intellectual disability enrolled in a functional academic skills curriculum in a self-contained classroom. A multiple probe single-subject research design (Horner & Baer, 1978) was used to evaluate the effectiveness of computer-based graphic organizers to solving mathematical one-step word problems. During the baseline phase, the students completed a teacher-generated worksheet that consisted of nine functional word problems in a traditional format using a pencil, paper, and a calculator. In the intervention and maintenance phases, the students were instructed to complete the word problems using a computer-based graphic organizer. Results indicated that all three of the students improved in their ability to solve the one-step word problems using computer-based graphic organizers compared to traditional instructional practices. Limitations of the study and recommendations for future research directions are discussed.