Cognitive tasks and combined statistical methods to evaluate, model, and predict mental workload

Mental workload (MWL) is a concept that is used as a reference for assessing the mental cost of activities. In recent times, challenges related to user experience are determining the expected MWL value for a given activity and real-time adaptation of task complexity level to achieve or maintain desired MWL. As a consequence, it is important to have at least one task that can reliably predict the MWL level associated with a given complexity level. In this study, we used several cognitive tasks to meet this need, including the N-Back task, the commonly used reference test in the MWL literature, and the Corsi test. Tasks were adapted to generate different MWL classes measured via NASA-TLX and Workload Profile questionnaires. Our first objective was to identify which tasks had the most distinct MWL classes based on combined statistical methods. Our results indicated that the Corsi test satisfied our first objective, obtaining three distinct MWL classes associated with three complexity levels offering therefore a reliable model (about 80% accuracy) to predicted MWL classes. Our second objective was to achieve or maintain the desired MWL, which entailed the use of an algorithm to adapt the MWL class based on an accurate prediction model. This model needed to be based on an objective and real-time indicator of MWL. For this purpose, we identified different performance criteria for each task. The classification models obtained indicated that only the Corsi test would be a good candidate for this aim (more than 50% accuracy compared to a chance level of 33%) but performances were not sufficient to consider identifying and adapting the MWL class online with sufficient accuracy during a task. Thus, performance indicators require to be complemented by other types of measures like physiological ones. Our study also highlights the limitations of the N-back task in favor of the Corsi test which turned out to be the best candidate to model and predict the MWL among several cognitive tasks.

When Self-Driving Fails: Evaluating Social Media Posts Regarding Problems and Misconceptions about Tesla’s FSD Mode

With the recent deployment of the latest generation of Tesla’s Full Self-Driving (FSD) mode, consumers are using semi-autonomous vehicles in both highway and residential driving for the first time. As a result, drivers are facing complex and unanticipated situations with an unproven technology, which is a central challenge for cooperative cognition. One way to support cooperative cognition in such situations is to inform and educate the user about potential limitations. Because these limitations are not always easily discovered, users have turned to the internet and social media to document their experiences, seek answers to questions they have, provide advice on features to others, and assist other drivers with less FSD experience. In this paper, we explore a novel approach to supporting cooperative cognition: Using social media posts can help characterize the limitations of the automation in order to get information about the limitations of the system and explanations and workarounds for how to deal with these limitations. Ultimately, our goal is to determine the kinds of problems being reported via social media that might be useful in helping users anticipate and develop a better mental model of an AI system that they rely on. To do so, we examine a corpus of social media posts about FSD problems to identify (1) the typical problems reported, (2) the kinds of explanations or answers provided by users, and (3) the feasibility of using such user-generated information to provide training and assistance for new drivers. The results reveal a number of limitations of the FSD system (e.g., lane-keeping and phantom braking) that may be anticipated by drivers, enabling them to predict and avoid the problems, thus allowing better mental models of the system and supporting cooperative cognition of the human-AI system in more situations.