Unintended consequences of technology in competency-based education: a qualitative study of lessons learned in an OtoHNS program

BACKGROUND

Formative feedback and entrustment ratings on assessments of entrustable professional activities (EPAs) are intended to support learner self-regulation and inform entrustment decisions in competency-based medical education. Technology platforms have been developed to facilitate these goals, but little is known about their effects on these new assessment practices. This study investigates how users interacted with an e-portfolio in an OtoHNS surgery program transitioning to a Canadian approach to competency-based assessment, Competence by Design.

METHODS

We employed a sociomaterial perspective on technology and grounded theory methods of iterative data collection and analysis to study this OtoHNS program's use of an e-portfolio for assessment purposes. All residents (n = 14) and competency committee members (n = 7) participated in the study; data included feedback in resident portfolios, observation of use of the e-portfolio in a competency committee meeting, and a focus group with residents to explore how they used the e-portfolio and visualize interfaces that would better meet their needs.

RESULTS

Use of the e-portfolio to document, access, and interpret assessment data was problematic for both residents and faculty, but the residents faced more challenges. While faculty were slowed in making entrustment decisions, formative assessments were not actionable for residents. Workarounds to these barriers resulted in a "numbers game" residents played to acquire EPAs. Themes prioritized needs for searchable, contextual, visual, and mobile aspects of technology design to support use of assessment data for resident learning.

CONCLUSION

Best practices of technology design begin by understanding user needs. Insights from this study support recommendations for improved technology design centred on learner needs to provide OtoHNS residents a more formative experience of competency-based training.

Social norms in digital spaces: Conflict reports and implications for technology design in the teleworking context

With the increase of telework during the COVID-19 pandemic, employees' interactions with colleagues have shifted almost exclusively to digital channels. Hence, it is critical to understand the underpinnings of successful collaboration and individual wellbeing in digital working structures. Combining qualitative experience reports and quantitative surveys from 145 teleworkers, this study sheds light on teleworking from a psychological perspective, taking social norms as a conceptual frame. The qualitative reports revealed five types of typical conflicts related to communication in the telework context, including both (1) technical problems (e.g., a bad connection) and psychological aspects such as (2) uncertainty or a lack of social feedback, (3) norm violations, (4) a lack of rules or meta-communication about appropriate behavior, and (5) digital communication barriers. Respondents' quantitative ratings of qualities of telework versus working on-site revealed benefits of telework regarding task fulfillment and efficiency, but lower levels of motivation, conflict management, leadership, team spirit, inspiration and creativity. Participants qualitative reports on perceived challenges in remote work conditions included feelings of loneliness and increased demands related to self-management, creating boundaries between private and working life, motivation and self-regulation. This paper connects these findings with theoretical concepts from psychology and human-computer interaction and discusses implications for leadership and technology design. Practical Relevance: This article discusses practical implications for leadership and technology design, e.g., interventions against conflicts in the context of digital work.

Aging Concerns, Challenges, and Everyday Solution Strategies (ACCESS) for adults aging with a long-term mobility disability

BACKGROUND

There are growing numbers of adults aging with long-term mobility disabilities. Very little is known about the challenges this population experiences with everyday activities, and such challenges are likely to be greater and more complex than those of older adults who experience mobility declines later in life.

OBJECTIVES

The current manuscript presents in-depth insights on the specific activity challenges experienced by older adults with long-term mobility disabilities, and the response strategies they employ to overcome those challenges.

METHODS

In-depth, structured interviews designed to assess challenges and response strategies for a range of daily activities were conducted with sixty older adult participants (ages 60-79) who self-identified as having a mobility disability for a minimum of 10 years. A coding scheme was developed to classify emerging themes. We identify the most common challenges and responses reported across all 6 broad categories as well as for a single activity category as an exemplar of an in-depth activity analysis.

RESULTS

A needs taxonomy revealed challenges and response themes in relation to the most difficult activities identified by the participants. Frequently reported challenges included physical limitations and accessibility challenges. Common response strategies (e.g., utilizing tools or technologies, obtaining assistance from others, or modifying the approach to the task) were successfully mapped onto the Selection, Optimization, and Compensation model.

CONCLUSIONS

Findings reveal the unique challenges individuals face when engaging in everyday activities, and the potential of affordable and effective supports to promote aging-in-place, functional independence, and community engagement for adults aging with long-term mobility disability.