Evaluation of Teaching and Learning Strategies

Analysing synthesis of evidence in a systematic review in health professions education: observations on struggling beyond Kirkpatrick

Background: Systematic reviews in health professions education may well under-report struggles to synthesize disparate evidence that defies standard quantitative approaches. This paper reports further process analysis in a previously reported systematic review about mobile devices on clinical placements.Objective: For a troublesome systematic review: (1) Analyse further the distribution and reliability of classifying the evidence to Maxwell quality dimensions (beyond 'Does it work?') and their overlap with Kirkpatrick K-levels. (2) Analyse how the abstracts represented those dimensions of the evidence-base. (3) Reflect on difficulties in synthesis and merits of Maxwell dimensions.Design: Following integrative synthesis of 45 K2-K4 primary studies (by combined content-thematic analysis in the pragmatism paradigm): (1) Hierarchical cluster analysis explored overlap between Maxwell dimensions and K-levels. Independent and consensus-coding to Maxwell dimensions compared (using: percentages; kappa; McNemar hypothesis-testing) pre- vs post-discussion and (2) article abstract vs main body. (3) Narrative summary captured process difficulties and merits.Results: (1) The largest cluster (five-cluster dendrogram) was acceptability-accessibility-K1-appropriateness-K3, with K1 and K4 widely separated. For article main bodies, independent coding agreed most for appropriateness (good; adjusted kappa = 0.78). Evidence increased significantly pre-post-discussion about acceptability (p = 0.008; 31/45→39/45), accessibility, and equity-ethics-professionalism. (2) Abstracts suggested efficiency significantly less than main bodies evidenced: 31.1% vs 44.4%, p = 0.031. 3) Challenges and merits emerged for before, during, and after the review.Conclusions: There should be more systematic reporting of process analysis about difficulties synthesizing suboptimal evidence-bases. In this example, Maxwell dimensions were a useful framework beyond K-levels for classifying and synthesizing the evidence-base.

Evidence-based medicine

It is no longer considered acceptable for medical practice to be based largely on clinical anecdotes and expert opinion. Rather, medical decisions should be based on best available evidence. Evidence-based medicine (EBM) is now recognized as a practice that should be learned and practiced by all radiologists, and indeed, all physicians. When practiced effectively, EBM integrates clinical expertise, patients' values, and best evidence, and promotes optimal patient care. The process of EBM has five components: (1) formulation of a clinical question based on a relevant and immediate clinical problem; (2) selection of appropriate resources and conduction of a literature search; (3) appraisal of the literature for its validity and applicability; (4) integration of the information with clinical expertise and unique patient needs; and (5) self-evaluation of one's performance with a specific patient. Each clinical problem is different, and the resources available to solve each problem vary. Radiologists must search for the best evidence but understand that in some circumstances there may be no good evidence to support clinical judgment.

The effectiveness of computer-aided learning in teaching orthodontics: A review of the literature

INTRODUCTION

The purposes of this review were to evaluate the effectiveness of computer-aided learning (CAL) in orthodontic education, to make evidence-based recommendations for the use of CAL in orthodontics, and to develop guidelines for conducting comparative trials to evaluate CAL as a mode of learning in orthodontic education.

METHODS

Medline, the Cochrane Library Database, ERIC, CINAHL, LISA, Psycinfo, and IPA were searched for randomized controlled trials evaluating the effectiveness of CAL in orthodontics. Outcome measures included objectively measured posttest scores on multiple choice, written, or oral tests; performance on a clinical procedure or clinical interview; time spent on CAL programs to learn the material presented; and responses to questionnaires conveying participants' attitudes toward various modes of learning.

RESULTS

Four randomized controlled trials comparing CAL with conventional teaching fulfilled the inclusion criteria and met the cutoff quality assessment checklist (QAC) score of > or = 8. Each study was assessed for quality by 2 independent reviewers. The validity and strength of the selected studies were assessed by using a QAC for an educational intervention.

CONCLUSIONS

The controlled trials of CAL in orthodontics that met our QAC cutoff score of 8 were split, with 2 showing that CAL enjoyed a significant advantage over conventional teaching, 1 showing no difference, and 1 showing that the conventional tutorial method was better. More high-quality trials evaluating the effectiveness of CAL in orthodontics are needed. CAL programs in orthodontics elicit mostly positive responses and attitudes from students toward learning.