Subliminal salience search illustrated: EEG identity and deception detection on the fringe of awareness

Is predictive coding falsifiable?

Predictive-coding has justifiably become a highly influential theory in Neuroscience. However, the possibility of its unfalsifiability has been raised. We argue that if predictive-coding were unfalsifiable, it would be a problem, but there are patterns of behavioural and neuroimaging data that would stand against predictive-coding. Contra (vanilla) predictive patterns are those in which the more expected stimulus generates the largest evoked-response. However, basic formulations of predictive-coding mandate that an expected stimulus should generate little, if any, prediction error and thus little, if any, evoked-response. It has, though, been argued that contra (vanilla) predictive patterns can be obtained if precision is higher for expected stimuli. Certainly, using precision, one can increase the amplitude of an evoked-response, turning a predictive into a contra (vanilla) predictive pattern. We demonstrate that, while this is true, it does not present an absolute barrier to falsification. This is because increasing precision also reduces latency and increases the frequency of the response. These properties can be used to determine whether precision-weighting in predictive-coding justifiably explains a contra (vanilla) predictive pattern, ensuring that predictive-coding is falsifiable.

Concealed identity information detection with pupillometry in rapid serial visual presentation

The concealed information test (CIT) relies on bodily reactions to stimuli that are hidden in mind. However, people can use countermeasures, such as purposely focusing on irrelevant things, to confound the CIT. A new method designed to prevent countermeasures uses rapid serial visual presentation (RSVP) to present stimuli on the fringe of awareness. Previous studies that used RSVP in combination with electroencephalography (EEG) showed that participants exhibit a clear reaction to their real first name, even when they try to prevent such a reaction (i.e., when their name is concealed information). Because EEG is not easily applicable outside the laboratory, we investigated here whether pupil size, which is easier to measure, can also be used to detect concealed identity information. In our first study, participants adopted a fake name, and searched for this name in an RSVP task, while their pupil sizes were recorded. Apart from this fake name, their real name and a control name also appeared in the task. We found pupil dilation in response to the task-irrelevant real name, as compared to control names. However, while most participants showed this effect qualitatively, it was not statistically significant for most participants individually. In a second study, we preregistered the proof-of-concept methodology and replicated the original findings. Taken together, our results show that the current RSVP task with pupillometry can detect concealed identity information at a group level. Further development of the method is needed to create a valid and reliable concealed identity information detector at the individual level.

Aging reduces EEG markers of recognition despite intact performance: Implications for forensic memory detection

ERP-based forensic memory detection is based on the logic that guilty suspects will hold incriminating knowledge about crimes they have committed, and therefore should show parietal ERP positivities related to recognition when presented with reminders of their crimes. We predicted that such forensic memory detection might however be inaccurate in older adults, because of changes to recognition-related brain activity that occurs with aging. We measured both ERPs and EEG oscillations associated with episodic old/new recognition and forensic memory detection in 30 younger (age < 30) and 30 older (age > 65) adults. EEG oscillations were included as a complementary measure which is less sensitive to temporal variability and component overlap than ERPs. In line with predictions, recognition-related parietal ERP positivities were significantly reduced in the older compared to younger group in both tasks, despite highly similar behavioural performance. We also observed aging-related reductions in oscillatory markers of recognition in the forensic memory detection test, while the oscillatory effects associated with episodic recognition were similar across age groups. This pattern of results suggests that while both forensic memory detection and episodic recognition are accompanied by aging-induced reductions in parietal ERP positivities, these reductions may be caused by non-overlapping mechanisms across the two tasks. Our findings suggest that EEG-based forensic memory detection tests are less valid in older than younger populations, limiting their practical applications.

Breakthrough percepts of famous names

Studies have shown that presenting own-name stimuli on the fringe of awareness in Rapid Serial Visual Presentation (RSVP) generates a P3 component and provides an accurate and countermeasure resistant method for detecting identity deception (Bowman et al., 2013, 2014). The current study investigates how effective this Fringe-P3 method is at detecting recognition of familiar name stimuli with lower salience (i.e., famous names) than own-name stimuli, as well as its accuracy with multi-item stimuli (i.e., first and second name pairs presented sequentially). The results demonstrated a highly significant ERP difference between famous and non-famous names at the group level and a detectable P3 for famous names for 86% of participants at the individual level. This demonstrates that the Fringe-P3 method can be used for detecting name stimuli other than own-names and for multi-item stimuli, thus further supporting the method's potential usefulness in forensic applications such as in detecting recognition of accomplices.

Breakthrough percepts of online identity: Detecting recognition of email addresses on the fringe of awareness

A key issue facing cybercrime investigations is connecting online identities to real-world identities. This paper shows that by combining the Fringe-P3 method with a concealed information test, we can detect a participant's familiarity with their own email address, thus connecting their real-world identity to their online one. Participants were shown Rapid Serial Visual Presentation (RSVP) streams of email addresses, some including their own email address (probe) or a target email address. Familiarity with the probe was accurately detected with significant results at the group level and for 7 of 11 participants at the individual level. These promising results demonstrate that the method can be successfully used to detect online identities. Factors that may affect how well an email address probe stands out in the RSVP streams are also discussed.

Breaking the circularity in circular analyses: Simulations and formal treatment of the flattened average approach

There has been considerable debate and concern as to whether there is a replication crisis in the scientific literature. A likely cause of poor replication is the multiple comparisons problem. An important way in which this problem can manifest in the M/EEG context is through post hoc tailoring of analysis windows (a.k.a. regions-of-interest, ROIs) to landmarks in the collected data. Post hoc tailoring of ROIs is used because it allows researchers to adapt to inter-experiment variability and discover novel differences that fall outside of windows defined by prior precedent, thereby reducing Type II errors. However, this approach can dramatically inflate Type I error rates. One way to avoid this problem is to tailor windows according to a contrast that is orthogonal (strictly parametrically orthogonal) to the contrast being tested. A key approach of this kind is to identify windows on a fully flattened average. On the basis of simulations, this approach has been argued to be safe for post hoc tailoring of analysis windows under many conditions. Here, we present further simulations and mathematical proofs to show exactly why the Fully Flattened Average approach is unbiased, providing a formal grounding to the approach, clarifying the limits of its applicability and resolving published misconceptions about the method. We also provide a statistical power analysis, which shows that, in specific contexts, the fully flattened average approach provides higher statistical power than Fieldtrip cluster inference. This suggests that the Fully Flattened Average approach will enable researchers to identify more effects from their data without incurring an inflation of the false positive rate.

Event-related potentials reflect prediction errors and pop-out during comprehension of degraded speech

Comprehension of degraded speech requires higher-order expectations informed by prior knowledge. Accurate top-down expectations of incoming degraded speech cause a subjective semantic 'pop-out' or conscious breakthrough experience. Indeed, the same stimulus can be perceived as meaningless when no expectations are made in advance. We investigated the event-related potential (ERP) correlates of these top-down expectations, their error signals and the subjective pop-out experience in healthy participants. We manipulated expectations in a word-pair priming degraded (noise-vocoded) speech task and investigated the role of top-down expectation with a between-groups attention manipulation. Consistent with the role of expectations in comprehension, repetition priming significantly enhanced perceptual intelligibility of the noise-vocoded degraded targets for attentive participants. An early ERP was larger for mismatched (i.e. unexpected) targets than matched targets, indicative of an initial error signal not reliant on top-down expectations. Subsequently, a P3a-like ERP was larger to matched targets than mismatched targets only for attending participants-i.e. a pop-out effect-while a later ERP was larger for mismatched targets and did not significantly interact with attention. Rather than relying on complex post hoc interactions between prediction error and precision to explain this apredictive pattern, we consider our data to be consistent with prediction error minimization accounts for early stages of processing followed by Global Neuronal Workspace-like breakthrough and processing in service of task goals.

Concealed information revealed by involuntary eye movements on the fringe of awareness in a mock terror experiment

Involuntary eye movements during fixation are typically inhibited following stimulus onset (Oculomotor Inhibition, OMI), depending on the stimulus saliency and attention, with an earlier and longer OMI for barely visible familiar faces. However, it is still unclear whether OMI regarding familiarities and perceptual saliencies differ enough to allow a reliable OMI-based concealed information test (CIT). In a “mock terror” experiment with 25 volunteers, 13 made a concealed choice of a “terror-target” (one of eight), associated with 3 probes (face, name, and residence), which they learned watching text and videos, whereas 12 “innocents” pre-learned nothing. All participants then watched ~ 25 min of repeated brief presentations of barely visible (masked) stimuli that included the 8 potential probes, as well as a universally familiar face as a reference, while their eye movements were monitored. We found prolonged and deviant OMI regarding the probes. Incorporated with the individual pattern of responses to the reference, our analysis correctly identified 100% of the terror targets, and was 95% correct in discriminating “terrorists” from “innocents”. Our results provide a “proof of concept” for a novel approach to CIT, based on involuntary oculomotor responses to barely visible stimuli, individually tailored, and with high accuracy and theoretical resistance to countermeasures.

Item Roles Explored in a Modified P300-Based CTP Concealed Information Test

In this study, we introduced familiarity-related inducer items (expressions referring to the participant’s self-related, familiar details: “mine,” “familiar”; and expressions referring to other, unfamiliar details, e.g., “other,” “irrelevant”) to the Complex Trial Protocol version of the P300-based Concealed Information Test (CIT), at the same time using different item categories with various levels of personal importance to the participants (forenames, birthdays, favorite animals). The inclusion of inducers did not significantly improve the overall efficiency of the method as we would have expected considering that these inducers should increase awareness of the denial of the recognition of the probes (the true details of the participants), and hence the subjective saliency of the items (Lukács in J Appl Res Mem Cognit, 6:283–284, 2017a). This may be explained by the visual similarity of inducers to the probe and irrelevant items and the consequent distracting influence of inducers on probe-task performance. On the other hand, the CIT effect (probe-irrelevant P300 differences) was always lower for less personally important (low-salient) and higher for more personally important (high-salient) items.

The effect of mental countermeasures on neuroimaging-based concealed information tests

During the last decade and a half, functional magnetic resonance imaging (fMRI) has been used to determine whether it is possible to detect concealed knowledge by examining brain activation patterns, with mixed results. Concealed information tests rely on the logic that a familiar item (probe) elicits a stronger response than unfamiliar, but otherwise comparable items (irrelevants). Previous work has shown that physical countermeasures can artificially modulate neural responses in concealed information tests, decreasing the accuracy of these methods. However, the question remains as to whether purely mental countermeasures, which are much more difficult to detect than physical ones, can also be effective. An fMRI study was conducted to address this question by assessing the effect of attentional countermeasures on the accuracy of the classification between knowledge and no-knowledge cases using both univariate and multivariate analyses. Results replicate previous work and show reliable group activation differences between the probe and the irrelevants in fronto-parietal networks. Critically, classification accuracy was generally reduced by the mental countermeasures, but only significantly so with region of interest analyses (both univariate and multivariate). For whole-brain analyses, classification accuracy was relatively low, but it was not significantly reduced by the countermeasures. These results indicate that mental countermeasure need to be addressed before these paradigms can be used in applied settings and that methods to defeat countermeasures, or at least to detect their use, need to be developed. HIGHLIGHTS: FMRI-based concealed information tests are vulnerable to mental countermeasures Measures based on regions of interest are affected by mental countermeasures Whole-brain analyses may be more robust than region of interest ones Methods to detect mental countermeasure use are needed for forensic applications.

Familiarity revealed by involuntary eye movements on the fringe of awareness

Involuntary eye movements during fixation of gaze are typically transiently inhibited following stimulus onset. This oculomotor inhibition (OMI), which includes microsaccades and spontaneous eye blinks, is modulated by stimulus saliency and anticipation, but it is currently unknown whether it is sensitive to familiarity. To investigate this, we measured the OMI while observers passively viewed a slideshow of one familiar and 7 unfamiliar facial images presented briefly at 1 Hz in random order. Since the initial experiments indicated that OMI was occasionally insensitive to familiarity when the facial images were highly visible, and to prevent top-down strategies and potential biases, we limited visibility by backward masking making the faces barely visible or at the fringe of awareness. Under these conditions, we found prolonged inhibition of both microsaccades and eye-blinks, as well as earlier onset of microsaccade inhibition with familiarity. These findings demonstrate, for the first time, the sensitivity of OMI to familiarity. Because this is based on involuntary eye movements and can be measured on the fringe of awareness and in passive viewing, our results provide direct evidence that OMI can be used as a novel physiological measure for studying hidden memories with potential implications for health, legal, and security purposes.

Broadening the Use of the Concealed Information Test in the Field

Japan is the only country where the polygraph with the concealed information test (CIT) is widely applied to criminal investigations. The CIT can reveal whether an examinee has knowledge of specific details of a crime. Furthermore, the CIT can extract crime-relevant information that investigative organizations have not yet uncovered. This article introduces how Japanese polygraphers take advantage of the CIT in criminal investigations. We also describe how polygraphs with the CIT are currently used in court. Then we propose statistical discrimination methods that can be easily applied to CIT interpretation in the field. Appropriate application of the statistical values is discussed. We hope that this article will facilitate more active use of the CIT outside Japan.

An Inflatable and Wearable Wireless System for Making 32-Channel Electroencephalogram Measurements

Potable electroencephalography (EEG) devices have become critical for important research. They have various applications, such as in brain-computer interfaces (BCI). Numerous recent investigations have focused on the development of dry sensors, but few concern the simultaneous attachment of high-density dry sensors to different regions of the scalp to receive qualified EEG signals from hairy sites. An inflatable and wearable wireless 32-channel EEG device was designed, prototyped, and experimentally validated for making EEG signal measurements; it incorporates spring-loaded dry sensors and a novel gasbag design to solve the problem of interference by hair. The cap is ventilated and incorporates a circuit board and battery with a high-tolerance wireless (Bluetooth) protocol and low power consumption characteristics. The proposed system provides a 500/250 Hz sampling rate, and 24 bit EEG data to meet the BCI system data requirement. Experimental results prove that the proposed EEG system is effective in measuring audio event-related potential, measuring visual event-related potential, and rapid serial visual presentation. Results of this work demonstrate that the proposed EEG cap system performs well in making EEG measurements and is feasible for practical applications.

Latency as a region contrast: Measuring ERP latency differences with Dynamic Time Warping

Methods for measuring onset latency contrasts are evaluated against a new method utilizing the dynamic time warping (DTW) algorithm. This new method allows latency to be measured across a region instead of single point. We use computer simulations to compare the methods' power and Type I error rates under different scenarios. We perform per-participant analysis for different signal-to-noise ratios and two sizes of window (broad vs. narrow). In addition, the methods are tested in combination with single-participant and jackknife average waveforms for different effect sizes, at the group level. DTW performs better than the other methods, being less sensitive to noise as well as to placement and width of the window selected.

Identifying the location of a concealed object through unintentional eye movements

In some investigative and interrogative contexts, the investigator is seeking to identify the location of an object (e.g., implanted bomb) which is known to a given subject (e.g., a terrorist). In this paper, we present a non-intrusive methodology for uncovering the loci of a concealed object by analyzing the subject's eye movements. Using a combination of eye tracking, psychological manipulation and a search algorithm, we have performed two experiments. In the first experiment, we have gained 58% hit rate in identifying the location of the concealed object and in the second experiment 56% hit rate. The pros and cons of the methodology for forensic investigation are discussed.

Countering countermeasures: detecting identity lies by detecting conscious breakthrough

One major drawback of deception detection is its vulnerability to countermeasures, whereby participants wilfully modulate their physiological or neurophysiological response to critical guilt-determining stimuli. One reason for this vulnerability is that stimuli are usually presented slowly. This allows enough time to consciously apply countermeasures, once the role of stimuli is determined. However, by increasing presentation speed, stimuli can be placed on the fringe of awareness, rendering it hard to perceive those that have not been previously identified, hindering the possibility to employ countermeasures. We tested an identity deception detector by presenting first names in Rapid Serial Visual Presentation and instructing participants to lie about their own identity. We also instructed participants to apply a series of countermeasures. The method proved resilient, remaining effective at detecting deception under all countermeasures.

The cost of space independence in P300-BCI spellers

Background

Though non-invasive EEG-based Brain Computer Interfaces (BCI) have been researched extensively over the last two decades, most designs require control of spatial attention and/or gaze on the part of the user.

Methods

In healthy adults, we compared the offline performance of a space-independent P300-based BCI for spelling words using Rapid Serial Visual Presentation (RSVP), to the well-known space-dependent Matrix P300 speller.

Results

EEG classifiability with the RSVP speller was as good as with the Matrix speller. While the Matrix speller’s performance was significantly reliant on early, gaze-dependent Visual Evoked Potentials (VEPs), the RSVP speller depended only on the space-independent P300b. However, there was a cost to true spatial independence: the RSVP speller was less efficient in terms of spelling speed.

Conclusions

The advantage of space independence in the RSVP speller was concomitant with a marked reduction in spelling efficiency. Nevertheless, with key improvements to the RSVP design, truly space-independent BCIs could approach efficiencies on par with the Matrix speller. With sufficiently high letter spelling rates fused with predictive language modelling, they would be viable for potential applications with patients unable to direct overt visual gaze or covert attentional focus.