Are Measures of Transfer Effects Missing the Target?

Does it work? Using a Meta-Impact score to examine global effects in quasi-experimental intervention studies

The evaluation of applied psychological interventions in the workplace or elsewhere is challenging. Randomisation and matching are difficult to achieve and this often results in substantial heterogeneity within intervention and control groups. As a result, traditional comparison of group means using null hypothesis significance testing may mask effects experienced by some participants. Using longitudinal studies of coaching interventions designed to provide support for dyslexic employees, this study describes and evaluates a different approach using a Meta-Impact score. We offer a conceptual rationale for our method, illustrate how this score is calculated and analysed, and show how it highlights person-specific variations in how participants react and respond to interventions. We argue that Meta-Impact is an incremental supplement to traditional variable-centric group-wise comparisons and can more accurately demonstrate in practice the extent to which an intervention worked. Such methods are needed for applied research, where personalized intervention protocols may require impact analysis for policy, legal and ethical purposes, despite modest sample sizes.

Context matters: A review to formulate a conceptual framework for coaching as a disability accommodation

Although dyslexia affects 5-8% of the workforce this developmental disorder has not been sufficiently researched in adult populations. Yet a diagnosis confers legal protections as employers must provide disability 'accommodations' to assist work functioning and performance. The implementation of such accommodations, including coaching, lacks theoretical framing and evaluations of impact in practice. Recognizing a need for conceptual work, we undertook a narrative, systematic scoping review from a realist pragmatic epistemology, taking an iterative approach to define and address the review question: 'to what extent, and under what conditions, can face-to-face learning interventions improve Working Memory (WM) and Self-Efficacy (SE) and can these lead to functional improvements related to work performance?' Informed by expert and stakeholder consultation and user data, our review extracted and synthesized 25 studies from eleven countries to identify potentially applicable learning intervention theories, their effects upon WM and SE but also functional outcomes such as comprehension. We suggest that intervention protocols informed by Social Cognitive Learning Theory can improve SE, as would be expected, and more surprisingly also WM. The development of metacognition, stress management and fidelity to Goal Setting Theory were identified as valuable intervention features. We propose that coaching activities may provide a more contextualized environment for transfer of learning from WM to functional skills such as comprehension, when compared to computerized training interventions. We call for theoretically underpinned, primary studies to evaluate interventions with adult dyslexic populations to further our understanding of disability accommodations.