A privacy-preserving approach to streaming eye-tracking data

Hyperparameter tuning using Lévy flight and interactive crossover-based reptile search algorithm for eye movement event classification

Introduction: Eye movement is one of the cues used in human-machine interface technologies for predicting the intention of users. The developing application in eye movement event detection is the creation of assistive technologies for paralyzed patients. However, developing an effective classifier is one of the main issues in eye movement event detection. Methods: In this paper, bidirectional long short-term memory (BILSTM) is proposed along with hyperparameter tuning for achieving effective eye movement event classification. The Lévy flight and interactive crossover-based reptile search algorithm (LICRSA) is used for optimizing the hyperparameters of BILSTM. The issues related to overfitting are avoided by using fuzzy data augmentation (FDA), and a deep neural network, namely, VGG-19, is used for extracting features from eye movements. Therefore, the optimization of hyperparameters using LICRSA enhances the classification of eye movement events using BILSTM. Results and Discussion: The proposed BILSTM-LICRSA is evaluated by using accuracy, precision, sensitivity, F1-score, area under the receiver operating characteristic (AUROC) curve measure, and area under the precision-recall curve (AUPRC) measure for four datasets, namely, Lund2013, collected dataset, GazeBaseR, and UTMultiView. The gazeNet, human manual classification (HMC), and multi-source information-embedded approach (MSIEA) are used for comparison with the BILSTM-LICRSA. The F1-score of BILSTM-LICRSA for the GazeBaseR dataset is 98.99%, which is higher than that of the MSIEA.

Privacy-Preserving Gaze Data Streaming in Immersive Interactive Virtual Reality: Robustness and User Experience

Eye tracking is routinely being incorporated into virtual reality (VR) systems. Prior research has shown that eye tracking data, if exposed, can be used for re-identification attacks [14]. The state of our knowledge about currently existing privacy mechanisms is limited to privacy-utility trade-off curves based on data-centric metrics of utility, such as prediction error, and black-box threat models. We propose that for interactive VR applications, it is essential to consider user-centric notions of utility and a variety of threat models. We develop a methodology to evaluate real-time privacy mechanisms for interactive VR applications that incorporate subjective user experience and task performance metrics. We evaluate selected privacy mechanisms using this methodology and find that re-identification accuracy can be decreased to as low as 14% while maintaining a high usability score and reasonable task performance. Finally, we elucidate three threat scenarios (black-box, black-box with exemplars, and white-box) and assess how well the different privacy mechanisms hold up to these adversarial scenarios. This work advances the state of the art in VR privacy by providing a methodology for end-to-end assessment of the risk of re-identification attacks and potential mitigating solutions. f.

Privacy-preserving datasets of eye-tracking samples with applications in XR

Virtual and mixed-reality (XR) technology has advanced significantly in the last few years and will enable the future of work, education, socialization, and entertainment. Eye-tracking data is required for supporting novel modes of interaction, animating virtual avatars, and implementing rendering or streaming optimizations. While eye tracking enables many beneficial applications in XR, it also introduces a risk to privacy by enabling re-identification of users. We applied privacy definitions of it-anonymity and plausible deniability (PD) to datasets of eye-tracking samples and evaluated them against the state-of-the-art differential privacy (DP) approach. Two VR datasets were processed to reduce identification rates while minimizing the impact on the performance of trained machine-learning models. Our results suggest that both PD and DP mechanisms produced practical privacy-utility trade-offs with respect to re-identification and activity classification accuracy, while k-anonymity performed best at retaining utility for gaze prediction.

Clear Aligners and Smart Eye Tracking Technology as a New Communication Strategy between Ethical and Legal Issues

Smart eye-tracking technology (SEET) that determines visual attention using smartphones can be used to determine the aesthetic perception of different types of clear aligners. Its value as a communication and comprehension tool, in addition to the ethical and legal concerns which it entails, can be assessed. One hundred subjects (50 F, 50 M; age range 15-70) were equally distributed in non-orthodontic (A) and orthodontic (B) groups. A smartphone-based SEET app assessed their knowledge of and opinions on aligners. Subjects evaluated images of smiles not wearing aligners, with/without attachments and with straight/scalloped gingival margins, as a guided calibration step which formed the image control group. Subsequently, the subjects rated the same smiles, this time wearing aligners (experimental images group). Questionnaire data and average values for each group of patients, and images relating to fixation times and overall star scores, were analyzed using these tests: chi-square, t-test, Mann-Whitney U, Spearman's rho, and Wilcoxon (p < 0.05). One-way ANOVA and related post-hoc tests were also applied. Orthodontic patients were found to be better informed than non-orthodontic patients. Aesthetic perception could be swayed by several factors. Attachments scored lower in aesthetic evaluation. Lips distracted attention from attachments and improved evaluations. Attachment-free aligners were better rated overall. A more thorough understanding as to the opinions, expectations and aesthetic perception of aligners can improve communication with patients. Mobile SEET is remarkably promising, although it does require a careful medicolegal risk-benefit assessments for responsible and professional use.

An investigation of privacy preservation in deep learning-based eye-tracking

Background

The expanding usage of complex machine learning methods such as deep learning has led to an explosion in human activity recognition, particularly applied to health. However, complex models which handle private and sometimes protected data, raise concerns about the potential leak of identifiable data. In this work, we focus on the case of a deep network model trained on images of individual faces.

Materials and methods

A previously published deep learning model, trained to estimate the gaze from full-face image sequences was stress tested for personal information leakage by a white box inference attack. Full-face video recordings taken from 493 individuals undergoing an eye-tracking- based evaluation of neurological function were used. Outputs, gradients, intermediate layer outputs, loss, and labels were used as inputs for a deep network with an added support vector machine emission layer to recognize membership in the training data.

Results

The inference attack method and associated mathematical analysis indicate that there is a low likelihood of unintended memorization of facial features in the deep learning model.

Conclusions

In this study, it is showed that the named model preserves the integrity of training data with reasonable confidence. The same process can be implemented in similar conditions for different models.

Differential privacy for eye tracking with temporal correlations

New generation head-mounted displays, such as VR and AR glasses, are coming into the market with already integrated eye tracking and are expected to enable novel ways of human-computer interaction in numerous applications. However, since eye movement properties contain biometric information, privacy concerns have to be handled properly. Privacy-preservation techniques such as differential privacy mechanisms have recently been applied to eye movement data obtained from such displays. Standard differential privacy mechanisms; however, are vulnerable due to temporal correlations between the eye movement observations. In this work, we propose a novel transform-coding based differential privacy mechanism to further adapt it to the statistics of eye movement feature data and compare various low-complexity methods. We extend the Fourier perturbation algorithm, which is a differential privacy mechanism, and correct a scaling mistake in its proof. Furthermore, we illustrate significant reductions in sample correlations in addition to query sensitivities, which provide the best utility-privacy trade-off in the eye tracking literature. Our results provide significantly high privacy without any essential loss in classification accuracies while hiding personal identifiers.