A Multimodal Deep Log-Based User Experience (UX) Platform for UX Evaluation

A Hybrid Multimodal Emotion Recognition Framework for UX Evaluation Using Generalized Mixture Functions

Multimodal emotion recognition has gained much traction in the field of affective computing, human-computer interaction (HCI), artificial intelligence (AI), and user experience (UX). There is growing demand to automate analysis of user emotion towards HCI, AI, and UX evaluation applications for providing affective services. Emotions are increasingly being used, obtained through the videos, audio, text or physiological signals. This has led to process emotions from multiple modalities, usually combined through ensemble-based systems with static weights. Due to numerous limitations like missing modality data, inter-class variations, and intra-class similarities, an effective weighting scheme is thus required to improve the aforementioned discrimination between modalities. This article takes into account the importance of difference between multiple modalities and assigns dynamic weights to them by adapting a more efficient combination process with the application of generalized mixture (GM) functions. Therefore, we present a hybrid multimodal emotion recognition (H-MMER) framework using multi-view learning approach for unimodal emotion recognition and introducing multimodal feature fusion level, and decision level fusion using GM functions. In an experimental study, we evaluated the ability of our proposed framework to model a set of four different emotional states (Happiness, Neutral, Sadness, and Anger) and found that most of them can be modeled well with significantly high accuracy using GM functions. The experiment shows that the proposed framework can model emotional states with an average accuracy of 98.19% and indicates significant gain in terms of performance in contrast to traditional approaches. The overall evaluation results indicate that we can identify emotional states with high accuracy and increase the robustness of an emotion classification system required for UX measurement.

A Test Management System to Support Remote Usability Assessment of Web Applications

Nowadays, web designers are forced to have an even deeper perception of how users approach their products in terms of user experience and usability. Remote Usability Testing (RUT) is the most appropriate tool to assess the usability of web platforms by measuring the level of user attention, satisfaction, and productivity. RUT does not require the physical presence of users and evaluators, but for this very reason makes data collection more difficult. To simplify data collection and analysis and help RUT moderators collect and analyze user’s data in a non-intrusive manner, this research work proposes a low-cost comprehensive framework based on Deep Learning algorithms. The proposed framework, called Miora, employs facial expression recognition, gaze recognition, and analytics algorithms to capture data about other information of interest for in-depth usability analysis, such as interactions with the analyzed software. It uses a comprehensive evaluation methodology to elicit information about usability metrics and presents the results in a series of graphs and statistics so that the moderator can intuitively analyze the different trends related to the KPI used as usability indicators. To demonstrate how the proposed framework could facilitate the collection of large amounts of data and enable moderators to conduct both remote formative and summative tests in a more efficient way than traditional lab-based usability testing, two case studies have been presented: the analysis of an online shop and of a management platform. Obtained results suggest that this framework can be employed in remote usability testing to conduct both formative and summative tests.

User Experience Quantification Model from Online User Reviews

Due to the advancement in information technology and the boom of micro-blogging platforms, a growing number of online reviews are posted daily on product distributed platforms in the form of spontaneous and insightful user feedback, and these can be used as a significant data source to understand user experience (UX) and satisfaction. However, despite the vast amount of online reviews, the existing literature focuses on online ratings and ignores the real textual context in reviews. We proposed a three-step UX quantification model from online reviews to understand customer satisfaction using the effect-based Kano model. First, the relevant online reviews are selected using various filter mechanisms. Second, UX dimensions (UXDs) are extracted using a proposed method called UX word embedding Latent Dirichlet allocation (UXWE-LDA) and sentiment orientation using a transformer-based pipeline. Then, the casual relationships are identified for the extracted UXDs. Third, the UXDs are mapped on the customer satisfaction model (effect-based Kano) to understand the user perspective about the system, product, or services. Finally, the different parts of the proposed quantification model are evaluated to examine the performance of this method. We present different results of the proposed method in terms of accuracy, topic coherence (TC), Topic-wise performance, and expert-based evaluation for the proposed framework validation. For review quality filters, we achieved 98.49% accuracy for the spam detection classifier and 95% accuracy for the relatedness detection classifier. The results show that the proposed method for the topic extractor module always gives a higher TC value than other models such as WE-LDA and LDA. Regarding topic-wise performance measures, UXWE-LDA achieves a 3% improvement on average compared to LDA due to the incorporation of semantic domain knowledge. We also compute the Jaccard coefficient similarity between the extracted dimensions using UXWE-LDA and UX experts-based analysis for checking the mutual agreement, which is 0.3, 0.5, and 0.4, respectively. Based on the Kano model, the presented study has potential implications concerning issues and knowing the product’s strengths and weaknesses in product design.

Mining Early Life Risk and Resiliency Factors and Their Influences in Human Populations from PubMed: A Machine Learning Approach to Discover DOHaD Evidence

The Developmental Origins of Health and Disease (DOHaD) framework aims to understand how early life exposures shape lifecycle health. To date, no comprehensive list of these exposures and their interactions has been developed, which limits our ability to predict trajectories of risk and resiliency in humans. To address this gap, we developed a model that uses text-mining, machine learning, and natural language processing approaches to automate search, data extraction, and content analysis from DOHaD-related research articles available in PubMed. Our first model captured 2469 articles, which were subsequently categorised into topics based on word frequencies within the titles and abstracts. A manual screening validated 848 of these as relevant, which were used to develop a revised model that finally captured 2098 articles that largely fell under the most prominently researched domains related to our specific DOHaD focus. The articles were clustered according to latent topic extraction, and 23 experts in the field independently labelled the perceived topics. Consensus analysis on this labelling yielded mostly from fair to substantial agreement, which demonstrates that automated models can be developed to successfully retrieve and classify research literature, as a first step to gather evidence related to DOHaD risk and resilience factors that influence later life human health.

A Systematic Review for Cognitive State-Based QoE/UX Evaluation

Traditional evaluation of user experience is subjective by nature, for what is sought is to use data from physiological and behavioral sensors to interpret the relationship that the user's cognitive states have with the elements of a graphical interface and interaction mechanisms. This study presents the systematic review that was developed to determine the cognitive states that are being investigated in the context of Quality of Experience (QoE)/User Experience (UX) evaluation, as well as the signals and characteristics obtained, machine learning models used, evaluation architectures proposed, and the results achieved. Twenty-nine papers published in 2014-2019 were selected from eight online sources of information, of which 24% were related to the classification of cognitive states, 17% described evaluation architectures, and 41% presented correlations between different signals, cognitive states, and QoE/UX metrics, among others. The amount of identified studies was low in comparison with cognitive state research in other contexts, such as driving or other critical activities; however, this provides a starting point to analyze and interpret states such as mental workload, confusion, and mental stress from various human signals and propose more robust QoE/UX evaluation architectures.

UXmood—A Sentiment Analysis and Information Visualization Tool to Support the Evaluation of Usability and User Experience

This paper presents UXmood, a tool that provides quantitative and qualitative information to assist researchers and practitioners in the evaluation of user experience and usability. The tool uses and combines data from video, audio, interaction logs and eye trackers, presenting them in a configurable dashboard on the web. The UXmood works analogously to a media player, in which evaluators can review the entire user interaction process, fast-forwarding irrelevant sections and rewinding specific interactions to repeat them if necessary. Besides, sentiment analysis techniques are applied to video, audio and transcribed text content to obtain insights on the user experience of participants. The main motivations to develop UXmood are to support joint analysis of usability and user experience, to use sentiment analysis for supporting qualitative analysis, to synchronize different types of data in the same dashboard and to allow the analysis of user interactions from any device with a web browser. We conducted a user study to assess the data communication efficiency of the visualizations, which provided insights on how to improve the dashboard.

Exploring the dominant features of social media for depression detection

Recently, social media have been used by researchers to detect depressive symptoms in individuals using linguistic data from users’ posts. In this study, we propose a framework to identify social information as a significant predictor of depression. Using the proposed framework, we develop an application called the Socially Mediated Patient Portal (SMPP), which detects depression-related markers in Facebook users by applying a data-driven approach with machine learning classification techniques. We examined a data set of 4350 users who were evaluated for depression using the Center for Epidemiological Studies Depression (CES-D) scale. From this analysis, we identified a set of features that can distinguish between individuals with and without depression. Finally, we identified the dominant features that adequately assess individuals with and without depression on social media. The model trained on these features will be helpful to physicians in diagnosing mental diseases and psychiatrists in analysing patient behaviour.