Time-dependant Bayesian knowledge tracing-Robots that model user skills over time

Creating an accurate model of a user's skills is an essential task for Intelligent Tutoring Systems (ITS) and robotic tutoring systems. This allows the system to provide personalized help based on the user's knowledge state. Most user skill modeling systems have focused on simpler tasks such as arithmetic or multiple-choice questions, where the user's model is only updated upon task completion. These tasks have a single correct answer and they generate an unambiguous observation of the user's answer. This is not the case for more complex tasks such as programming or engineering tasks, where the user completing the task creates a succession of noisy user observations as they work on different parts of the task. We create an algorithm called Time-Dependant Bayesian Knowledge Tracing (TD-BKT) that tracks users' skills throughout these more complex tasks. We show in simulation that it has a more accurate model of the user's skills and, therefore, can select better teaching actions than previous algorithms. Lastly, we show that a robot can use TD-BKT to model a user and teach electronic circuit tasks to participants during a user study. Our results show that participants significantly improved their skills when modeled using TD-BKT.

A panoramic view of personalization based on individual differences in persuasive and behavior change interventions

Persuasive technologies are designed to change human behavior or attitude using various persuasive strategies. Recent years have witnessed increasing evidence of the need to personalize and adapt persuasive interventions to various users and contextual factors because a persuasive strategy that works for one individual may rather demotivate others. As a result, several research studies have been conducted to investigate how to effectively personalize persuasive technologies. As research in this direction is gaining increasing attention, it becomes essential to conduct a systematic review to provide an overview of the current trends, challenges, approaches used for developing personalized persuasive technologies, and opportunities for future research in the area. To fill this need, we investigate approaches to personalize persuasive interventions by understanding user-related factors considered when personalizing persuasive technologies. Particularly, we conducted a systematic review of 72 research published in the last ten years in personalized and adaptive persuasive systems. The reviewed papers were evaluated based on different aspects, including metadata (e.g., year of publication and venue), technology, personalization dimension, personalization approaches, target outcome, individual differences, theories and scales, and evaluation approaches. Our results show (1) increased attention toward personalizing persuasive interventions, (2) personality trait is the most popular dimension of individual differences considered by existing research when tailoring their persuasive and behavior change systems, (3) students are among the most commonly targeted audience, and (4) education, health, and physical activity are the most considered domains in the surveyed papers. Based on our results, the paper provides insights and prospective future research directions.

Physiological Signals and Affect as Predictors of Advertising Engagement

This study investigated the use of affect and physiological signals of heart rate, electrodermal activity, pupil dilation, and skin temperature to classify advertising engagement. The ground truth for the affective and behavioral aspects of ad engagement was collected from 53 young adults using the User Engagement Scale. Three gradient-boosting classifiers, LightGBM (LGBM), HistGradientBoostingClassifier (HGBC), and XGBoost (XGB), were used along with signal fusion to evaluate the performance of different signal combinations as predictors of engagement. The classifiers trained on the fusion of skin temperature, valence, and tiredness (features n = 5) performed better than those trained on all signals (features n = 30). The average AUC ROC scores for the fusion set were XGB = 0.68 (0.10), LGBM = 0.69 (0.07), and HGBC = 0.70 (0.11), compared to the lower scores for the set of all signals (XGB = 0.65 (0.11), LGBM = 0.66 (0.11), HGBC = 0.64 (0.10)). The results also show that the signal fusion set based on skin temperature outperforms the fusion sets of the other three signals. The main finding of this study is the role of specific physiological signals and how their fusion aids in more effective modeling of ad engagement while reducing the number of features.

Personalized Gamification for Learning: A Reactive Chatbot Architecture Proposal

A key factor for successfully implementing gamified learning platforms is making students interact with the system from multiple digital platforms. Learning platforms that try to accomplish all their objectives by concentrating all the interactions from users with them are less effective than initially believed. Conversational bots are ideal solutions for cross-platform user interaction. In this paper, an open student-player model is presented. The model includes the use of machine learning techniques for online adaptation. Then, an architecture for the solution is described, including the open model. Finally, the chatbot design is addressed. The chatbot architecture ensures that its reactive nature fits into our defined architecture. The approach's implementation and validation aim to create a tool to encourage kids to practice multiplication tables playfully.

Users' Responsiveness to Persuasive Techniques in Recommender Systems

Understanding user's behavior and their interactions with artificial-intelligent-based systems is as important as analyzing the performance of the algorithms used in these systems. For instance, in the Recommender Systems domain, the accuracy of the recommendation algorithm was the ultimate goal for most systems designers. However, researchers and practitioners have realized that providing accurate recommendations is insufficient to enhance users' acceptance. A recommender system needs to focus on other factors that enhance its interactions with the users. Recent researches suggest augmenting these systems with persuasive capabilities. Persuasive features lead to increasing users' acceptance of the recommendations, which, in turn, enhances users' experience with these systems. Nonetheless, the literature still lacks a comprehensive view of the actual effect of persuasive principles on recommender users. To fill this gap, this study diagnoses how users of different characteristics get influenced by various persuasive principles that a recommender system uses. The study considers four users' aspects: age, gender, culture (continent), and personality traits. The paper also investigates the impact of the context (or application domain) on the influence of the persuasive principles. Two application domains (namely eCommerce and Movie recommendations) are considered. A within-subject user study was conducted. The analysis of (279) responses revealed that persuasive principles have the potential to enhance users' experience with recommender systems. The study also shows that, among the considered factors, culture, personality traits, and the domain of recommendations have a higher impact on the influence of persuasive principles than other factors. Based on the analysis of the results, the study provides insights and guidelines for recommender systems designers. These guidelines can be used as a reference for designing recommender systems with users' experience in mind. We suggest that considering the results presented in this paper could help to improve recommender-users interaction.

Lifelong Personalization via Gaussian Process Modeling for Long-Term HRI

Across a wide variety of domains, artificial agents that can adapt and personalize to users have potential to improve and transform how social services are provided. Because of the need for personalized interaction data to drive this process, long-term (or longitudinal) interactions between users and agents, which unfold over a series of distinct interaction sessions, have attracted substantial research interest. In recognition of the expanded scope and structure of a long-term interaction, researchers are also adjusting the personalization models and algorithms used, orienting toward "continual learning" methods, which do not assume a stationary modeling target and explicitly account for the temporal context of training data. In parallel, researchers have also studied the effect of "multitask personalization," an approach in which an agent interacts with users over multiple different tasks contexts throughout the course of a long-term interaction and learns personalized models of a user that are transferrable across these tasks. In this paper, we unite these two paradigms under the framework of "Lifelong Personalization," analyzing the effect of multitask personalization applied to dynamic, non-stationary targets. We extend the multi-task personalization approach to the more complex and realistic scenario of modeling dynamic learners over time, focusing in particular on interactive scenarios in which the modeling agent plays an active role in teaching the student whose knowledge the agent is simultaneously attempting to model. Inspired by the way in which agents use active learning to select new training data based on domain context, we augment a Gaussian Process-based multitask personalization model with a mechanism to actively and continually manage its own training data, allowing a modeling agent to remove or reduce the weight of observed data from its training set, based on interactive context cues. We evaluate this method in a series of simulation experiments comparing different approaches to continual and multitask learning on simulated student data. We expect this method to substantially improve learning in Gaussian Process models in dynamic domains, establishing Gaussian Processes as another flexible modeling tool for Long-term Human-Robot Interaction (HRI) Studies.

Listener Modeling and Context-Aware Music Recommendation Based on Country Archetypes

Music preferences are strongly shaped by the cultural and socio-economic background of the listener, which is reflected, to a considerable extent, in country-specific music listening profiles. Previous work has already identified several country-specific differences in the popularity distribution of music artists listened to. In particular, what constitutes the "music mainstream" strongly varies between countries. To complement and extend these results, the article at hand delivers the following major contributions: First, using state-of-the-art unsupervized learning techniques, we identify and thoroughly investigate (1) country profiles of music preferences on the fine-grained level of music tracks (in contrast to earlier work that relied on music preferences on the artist level) and (2) country archetypes that subsume countries sharing similar patterns of listening preferences. Second, we formulate four user models that leverage the user's country information on music preferences. Among others, we propose a user modeling approach to describe a music listener as a vector of similarities over the identified country clusters or archetypes. Third, we propose a context-aware music recommendation system that leverages implicit user feedback, where context is defined via the four user models. More precisely, it is a multi-layer generative model based on a variational autoencoder, in which contextual features can influence recommendations through a gating mechanism. Fourth, we thoroughly evaluate the proposed recommendation system and user models on a real-world corpus of more than one billion listening records of users around the world (out of which we use 369 million in our experiments) and show its merits vis-à-vis state-of-the-art algorithms that do not exploit this type of context information.

Feature Selection for Learning to Predict Outcomes of Compute Cluster Jobs with Application to Decision Support

We present a machine learning framework and a new test bed for data mining from the Slurm Workload Manager for high-performance computing (HPC) clusters. The focus was to find a method for selecting features to support decisions: helping users decide whether to resubmit failed jobs with boosted CPU and memory allocations or migrate them to a computing cloud. This task was cast as both supervised classification and regression learning, specifically, sequential problem solving suitable for reinforcement learning. Selecting relevant features can improve training accuracy, reduce training time, and produce a more comprehensible model, with an intelligent system that can explain predictions and inferences. We present a supervised learning model trained on a Simple Linux Utility for Resource Management (Slurm) data set of HPC jobs using three different techniques for selecting features: linear regression, lasso, and ridge regression. Our data set represented both HPC jobs that failed and those that succeeded, so our model was reliable, less likely to overfit, and generalizable. Our model achieved an R² of 95% with 99% accuracy. We identified five predictors for both CPU and memory properties.

User Interaction Modeling and Profile Extraction in Interactive Systems: A Groupware Application Case Study

A relevant goal in human-computer interaction is to produce applications that are easy to use and well-adjusted to their users' needs. To address this problem it is important to know how users interact with the system. This work constitutes a methodological contribution capable of identifying the context of use in which users perform interactions with a groupware application (synchronous or asynchronous) and provides, using machine learning techniques, generative models of how users behave. Additionally, these models are transformed into a text that describes in natural language the main characteristics of the interaction of the users with the system.

Perceptions of document relevance

This article presents a study of how humans perceive and judge the relevance of documents. Humans are adept at making reasonably robust and quick decisions about what information is relevant to them, despite the ever increasing complexity and volume of their surrounding information environment. The literature on document relevance has identified various dimensions of relevance (e.g., topicality, novelty, etc.), however little is understood about how these dimensions may interact. We performed a crowdsourced study of how human subjects judge two relevance dimensions in relation to document snippets retrieved from an internet search engine. The order of the judgment was controlled. For those judgments exhibiting an order effect, a q–test was performed to determine whether the order effects can be explained by a quantum decision model based on incompatible decision perspectives. Some evidence of incompatibility was found which suggests incompatible decision perspectives is appropriate for explaining interacting dimensions of relevance in such instances.

A Conjoint Analysis Framework for Evaluating User Preferences in Machine Translation

Despite much research on machine translation (MT) evaluation, there is surprisingly little work that directly measures users' intuitive or emotional preferences regarding different types of MT errors. However, the elicitation and modeling of user preferences is an important prerequisite for research on user adaptation and customization of MT engines. In this paper we explore the use of conjoint analysis as a formal quantitative framework to assess users' relative preferences for different types of translation errors. We apply our approach to the analysis of MT output from translating public health documents from English into Spanish. Our results indicate that word order errors are clearly the most dispreferred error type, followed by word sense, morphological, and function word errors. The conjoint analysis-based model is able to predict user preferences more accurately than a baseline model that chooses the translation with the fewest errors overall. Additionally we analyze the effect of using a crowd-sourced respondent population versus a sample of domain experts and observe that main preference effects are remarkably stable across the two samples.

Investigating the link between radiologists' gaze, diagnostic decision, and image content

OBJECTIVE

To investigate machine learning for linking image content, human perception, cognition, and error in the diagnostic interpretation of mammograms.

METHODS

Gaze data and diagnostic decisions were collected from three breast imaging radiologists and three radiology residents who reviewed 20 screening mammograms while wearing a head-mounted eye-tracker. Image analysis was performed in mammographic regions that attracted radiologists' attention and in all abnormal regions. Machine learning algorithms were investigated to develop predictive models that link: (i) image content with gaze, (ii) image content and gaze with cognition, and (iii) image content, gaze, and cognition with diagnostic error. Both group-based and individualized models were explored.

RESULTS

By pooling the data from all readers, machine learning produced highly accurate predictive models linking image content, gaze, and cognition. Potential linking of those with diagnostic error was also supported to some extent. Merging readers' gaze metrics and cognitive opinions with computer-extracted image features identified 59% of the readers' diagnostic errors while confirming 97.3% of their correct diagnoses. The readers' individual perceptual and cognitive behaviors could be adequately predicted by modeling the behavior of others. However, personalized tuning was in many cases beneficial for capturing more accurately individual behavior.

CONCLUSIONS

There is clearly an interaction between radiologists' gaze, diagnostic decision, and image content which can be modeled with machine learning algorithms.