The easy-to-hard training advantage with real-world medical images

Diagnostic errors during perceptual learning in dermatology: a prospective cohort study of Finnish undergraduate students

BACKGROUND

Perceptual learning modules (PLMs) have been shown to significantly improve learning outcomes in teaching dermatology.

OBJECTIVES

To investigate the quantity and quality of diagnostic errors made during undergraduate PLMs and their potential implications.

METHODS

The study data were acquired from 8 successive dermatology courses (2021-23) from 142 undergraduate medical students. Digital PLMs were held before, during and after the courses. We investigated the number and distribution of diagnostic errors, differences between specific skin conditions and classified the errors based on type.

RESULTS

Diagnostic errors were not randomly distributed. Some skin conditions were almost always correctly identified, whereas a significant number of errors were made for other diagnoses. Errors were classified into one of three groups: mostly systematic errors of relevant differential diagnoses ('similarity' errors); partly systematic errors ('mixed' errors); and 'random' errors. While a significant learning effect during the repeated measures was found in accuracy (P < 0.001, η²P = 0.64), confidence (P < 0.001, η²P = 0.60) and fluency (P < 0.001, η²P = 0.16), the three categories differed in all outcome measures (all P < 0.001, all η²P > 0.47). Visual learning was more difficult for diagnoses in the similarity category (all P < 0.001, all η²P > 0.12) than for those in the mixed and random categories.

CONCLUSIONS

Error analysis of PLMs provided relevant information about learning efficacy and progression, and systematic errors in tasks and more difficult-to-learn conditions. This information could be used in the development of adaptive, individual error-based PLMs to improve learning outcomes, both in dermatology and medical education in general.

An analytical theory of curriculum learning in teacher-student networks

In animals and humans, curriculum learning-presenting data in a curated order-is critical to rapid learning and effective pedagogy. A long history of experiments has demonstrated the impact of curricula in a variety of animals but, despite its ubiquitous presence, a theoretical understanding of the phenomenon is still lacking. Surprisingly, in contrast to animal learning, curricula strategies are not widely used in machine learning and recent simulation studies reach the conclusion that curricula are moderately effective or even ineffective in most cases. This stark difference in the importance of curriculum raises a fundamental theoretical question: when and why does curriculum learning help? In this work, we analyse a prototypical neural network model of curriculum learning in the high-dimensional limit, employing statistical physics methods. We study a task in which a sparse set of informative features are embedded amidst a large set of noisy features. We analytically derive average learning trajectories for simple neural networks on this task, which establish a clear speed benefit for curriculum learning in the online setting. However, when training experiences can be stored and replayed (for instance, during sleep), the advantage of curriculum in standard neural networks disappears, in line with observations from the deep learning literature. Inspired by synaptic consolidation techniques developed to combat catastrophic forgetting, we propose curriculum-aware algorithms that consolidate synapses at curriculum change points and investigate whether this can boost the benefits of curricula. We derive generalisation performance as a function of consolidation strength (implemented as an L 2 regularisation/elastic coupling connecting learning phases), and show that curriculum-aware algorithms can yield a large improvement in test performance. Our reduced analytical descriptions help reconcile apparently conflicting empirical results, trace regimes where curriculum learning yields the largest gains, and provide experimentally-accessible predictions for the impact of task parameters on curriculum benefits. More broadly, our results suggest that fully exploiting a curriculum may require explicit adjustments in the loss.

Teaching dermatology to medical students: a Scoping Review of published interventional studies

It is unclear how dermatology should be optimally taught to medical students. Therefore, this scoping review was conducted aiming to identify and structure all published interventional studies that investigated dermatological teaching approaches with medical students. The methodology of this scoping review followed the PRISMA Extension for Scoping Reviews. The databases Medline and Embase were searched without restriction until 30.06.2020. A categorization and a descriptive analysis of the studies published as full articles were performed. The database search yielded 36,627 hits. 114 studies met all inclusion criteria. These came from 19 countries, were mainly published since 2010 and were distributed across 64 different journals. 32 randomized controlled trials were identified. A wide variety of teaching approaches was found, including both E-learning and conventional teaching formats. The results of the studies are presented in structured tables. This scoping review documents a large number of studies published worldwide on teaching dermatology to medical students. The teaching of dermatology appears to be successful with numerous teaching approaches, whereby interventions that incorporate didactic principles were verifiably more successful. This literature review can serve as an aid for evidence-based teaching design in dermatology as well as a basis for future research approaches.

Perceptual learning modules in undergraduate dermatology teaching

BACKGROUND

Dermatologic diagnosis depends highly on visual skills and implicit non-analytical proficiency plays a key role.

OBJECTIVES

To investigate the effectiveness of digital perceptual learning modules (PLMs) in undergraduate teaching.

METHODS

The study cohort included 39 students of an undergraduate dermatology course. Online PLMs designed for dermatology were carried out three times: before, during and at the end of the course. The modules provided four outcome measures: percentage of correct responses, response/decision time, list of features that the decision was based on and confidence rating.

RESULTS

Diagnostic accuracy increased significantly from 66 to 94% (p< .001, η² _p =0.92), fluency improved, and response times decreased from 10 to 6 seconds (p< .001, η² _p =0.69), and self-perceived confidence increased from 2.5 to 4.3 (p< .001, η² _p =0.86) with subsequent PLMs and course duration. There was a diversification of recognized features, an increase in pattern recognition and attention to localization and contextual association. Based on student feedback, PLMs functioned well and enhanced motivation and learning.

CONCLUSION

PLMs increase diagnostic accuracy, had a positive effect on learning outcomes and were easily integrated side-by-side with clinical teachings. Considering our current era of digital technologies, we believe that there is potential for a wider use of PLMs to improve visual skills and strengthen implicit learning in dermatology.

Predicting the Ease of Human Category Learning Using Radial Basis Function Networks

Our goal is to understand and optimize human concept learning by predicting the ease of learning of a particular exemplar or category. We propose a method for estimating ease values, quantitative measures of ease of learning, as an alternative to conducting costly empirical training studies. Our method combines a psychological embedding of domain exemplars with a pragmatic categorization model. The two components are integrated using a radial basis function network (RBFN) that predicts ease values. The free parameters of the RBFN are fit using human similarity judgments, circumventing the need to collect human training data to fit more complex models of human categorization. We conduct two category-training experiments to validate predictions of the RBFN. We demonstrate that an instance-based RBFN outperforms both a prototype-based RBFN and an empirical approach using the raw data. Although the human data were collected across diverse experimental conditions, the predicted ease values strongly correlate with human learning performance. Training can be sequenced by (predicted) ease, achieving what is known as fading in the psychology literature and curriculum learning in the machine-learning literature, both of which have been shown to facilitate learning.

Reliable test of clinicians' mastery in skin cancer diagnostics

Differentiating between benign and malignant skin lesions can be very difficult and should only be done by sufficiently trained and skilled clinicians. To our knowledge there are no validated tests for reliable assessments of clinicians' ability to perform skin cancer diagnostics. To develop and gather validity evidence for a test in skin cancer diagnostics, a multiple-choice questionnaire (MCQ) was developed based on informal interviews with seven content experts from five skin cancer centers in Denmark. Validity evidence for the test was gathered from May until July 2019 using Messick's validity framework (content, response process, internal structure, relationship to other variables and consequences). Item content was revised through a Delphi-like review process and then piloted on 36 medical students and 136 doctors using a standardized response process. Results enabled an analysis of the internal structure and relationship to other variables of the test. Finally, the contrasting groups method was used to investigate the test's consequences (pass-fail standard). The initial 90-item MCQ was reduced to 40 items during the Delphi-like review process. Item analysis revealed that 25 of the 40 selected items were level I-III quality items with a high internal consistency (Cronbach's α = 0.83) and highly significant (P ≤ 0.0001) differences in test scores between participants with different occupations or levels of experience. A pass-fail standard of 12 (48%) correct answers was established using the contrasting groups' method. The skin cancer diagnostics MCQ developed in this study can be used for reliable assessments of clinicians' competencies.

Human factors in forensic science: The cognitive mechanisms that underlie forensic feature-comparison expertise

After a decade of critique from leading scientific bodies, forensic science research is at a crossroads. Whilst emerging research has shown that some forensic feature-comparison disciplines are not foundationally valid, others are moving towards establishing reliability and validity. Forensic examiners in fingerprint, face and handwriting comparison disciplines have skills and knowledge that distinguish them from novices. Yet our understanding of the basis of this expertise is only beginning to emerge. In this paper, we review evidence on the psychological mechanisms contributing to forensic feature-comparison expertise, with a focus on one mechanism: statistical learning, or the ability to learn how often things occur in the environment. Research is beginning to emphasise the importance of statistical learning in forensic feature-comparison expertise. Ultimately, this research and broader cognitive science research has an important role to play in informing the development of training programs and selection tools for forensic feature-comparison examiners.

First trial outcome but not training difficulty predicts performance in goldfish visual discrimination

The easy-to-hard effect in perceptual learning shows that training with easier examples can facilitate initially difficult or impossible distinctions between very similar stimuli. This effect has been reported in humans and other species. We tested whether easy-to-hard training could facilitate visual discrimination in common goldfish (Carassius auratus). Fish (n = 6) performed a two-alternative forced choice discrimination task, which consisted of simultaneously presenting two striped patterns at a constant distance away on the outside of the tank. Fish were required to approach and bite a porthole corresponding to one of the stimuli for a food reward. Half of the fish were randomly assigned to a training schedule where stimuli became more similar as training progressed. The rest were trained only on the most difficult to distinguish version of the stimuli. All fish received a similar total amount of training regardless of the assigned schedule. We also examined whether performance on the first training trial for a given day was related to overall performance. Contrary to our hypothesis, fish in the easy-to-hard group did not perform significantly better than those in the constant-hard group. However, performance was found to be significantly higher on days when the first trial was correct compared to days on which it was incorrect, regardless of the type of training schedule. The current results contribute to understanding individual differences in perceptual learning in fish, and are consistent with research in humans, and other species, reporting better learning after initial reward.

Obtaining psychological embeddings through joint kernel and metric learning

Psychological embeddings provide a powerful formalism for characterizing human-perceived similarity among members of a stimulus set. Obtaining high-quality embeddings can be costly due to algorithm design, software deployment, and participant compensation. This work aims to advance state-of-the-art embedding techniques and provide a comprehensive software package that makes obtaining high-quality psychological embeddings both easy and relatively efficient. Contributions are made on four fronts. First, the embedding procedure allows multiple trial configurations (e.g., triplets) to be used for collecting similarity judgments from participants. For example, trials can be configured to collect triplet comparisons or to sort items into groups. Second, a likelihood model is provided for three classes of similarity kernels allowing users to easily infer the parameters of their preferred model using gradient descent. Third, an active selection algorithm is provided that makes data collection more efficient by proposing comparisons that provide the strongest constraints on the embedding. Fourth, the likelihood model allows the specification of group-specific attention weight parameters. A series of experiments are included to highlight each of these contributions and their impact on converging to a high-quality embedding. Collectively, these incremental improvements provide a powerful and complete set of tools for inferring psychological embeddings. The relevant tools are available as the Python package PsiZ, which can be cloned from GitHub ( https://github.com/roads/psiz ).

A review of eye tracking for understanding and improving diagnostic interpretation

Inspecting digital imaging for primary diagnosis introduces perceptual and cognitive demands for physicians tasked with interpreting visual medical information and arriving at appropriate diagnoses and treatment decisions. The process of medical interpretation and diagnosis involves a complex interplay between visual perception and multiple cognitive processes, including memory retrieval, problem-solving, and decision-making. Eye-tracking technologies are becoming increasingly available in the consumer and research markets and provide novel opportunities to learn more about the interpretive process, including differences between novices and experts, how heuristics and biases shape visual perception and decision-making, and the mechanisms underlying misinterpretation and misdiagnosis. The present review provides an overview of eye-tracking technology, the perceptual and cognitive processes involved in medical interpretation, how eye tracking has been employed to understand medical interpretation and promote medical education and training, and some of the promises and challenges for future applications of this technology.