An interleaved tracking procedure to monitor unstable psychometric functions

Trail-Traced Threshold Test (T4) With a Weighted Binomial Distribution for a Psychophysical Test

Clinical visual field testing is performed with commercial perimetric devices and employs psychophysical techniques to obtain thresholds of the differential light sensitivity (DLS) at multiple retinal locations. Current thresholding algorithms are relatively inefficient and tough to get satisfied test accuracy, stability concurrently. Thus, we propose a novel Bayesian perimetric threshold method called the Trail-Traced Threshold Test (T4), which can better address the dependence of the initial threshold estimation and achieve significant improvement in the test accuracy and variability while also decreasing the number of presentations compared with Zippy Estimation by Sequential Testing (ZEST) and FT. This study compares T4 with ZEST and FT regarding presentation number, mean absolute difference (MAD between the real Visual field result and the simulate result), and measurement variability. T4 uses the complete response sequence with the spatially weighted neighbor responses to achieve better accuracy and precision than ZEST, FT, SWeLZ, and with significantly fewer stimulus presentations. T4 is also more robust to inaccurate initial threshold estimation than other methods, which is an advantage in subjective methods, such as in clinical perimetry. This method also has the potential for using in other psychophysical tests.

Effects of Gap Position on Perceptual Gap Detection Across Late Childhood and Adolescence

The ability to detect a silent gap within a sound is critical for accurate speech perception, and gap detection has been shown to have an extended developmental trajectory. In certain conditions, the detectability of the gap decreases as the gap is placed closer to the beginning of the signal. Early in development, the detection of gaps shortly after signal onset may be especially difficult due to immaturities in the encoding and perception of rapidly changing sounds. The present study explored the development of gap detection from age 8 to 19 years, specifically when the temporal placement of the gap varied. Performance improved with age for all temporal placements of the gap, demonstrating a gradual maturation of gap detection abilities throughout adolescence. Younger adolescents did not benefit from increasing gap onset times, while older adolescents' thresholds gradually improved as gap onset time lengthened. Regardless of age, listeners learned between the two testing days but did not improve within days. Younger adolescents had poorer thresholds for the last block of testing on the second day, returning to baseline performance despite learning between days. These data support earlier studies showing that gaps are harder to detect near stimulus onset and confirm that gap detection abilities continue to mature into adolescence. The data also suggest that younger adolescents do not receive the same benefit of increasing gap onset time and respond differently to repeated testing than older adolescents and young adults.

Nonparametric tests for equality of psychometric functions

Many empirical studies measure psychometric functions (curves describing how observers' performance varies with stimulus magnitude) because these functions capture the effects of experimental conditions. To assess these effects, parametric curves are often fitted to the data and comparisons are carried out by testing for equality of mean parameter estimates across conditions. This approach is parametric and, thus, vulnerable to violations of the implied assumptions. Furthermore, testing for equality of means of parameters may be misleading: Psychometric functions may vary meaningfully across conditions on an observer-by-observer basis with no effect on the mean values of the estimated parameters. Alternative approaches to assess equality of psychometric functions per se are thus needed. This paper compares three nonparametric tests that are applicable in all situations of interest: The existing generalized Mantel-Haenszel test, a generalization of the Berry-Mielke test that was developed here, and a split variant of the generalized Mantel-Haenszel test also developed here. Their statistical properties (accuracy and power) are studied via simulation and the results show that all tests are indistinguishable as to accuracy but they differ non-uniformly as to power. Empirical use of the tests is illustrated via analyses of published data sets and practical recommendations are given. The computer code in MATLAB and R to conduct these tests is available as Electronic Supplemental Material.

Enhancing simulations with intra-subject variability for improved psychophysical assessments

Psychometric properties of perceptual assessments, like reliability, depend on stochastic properties of psychophysical sampling procedures resulting in method variability, as well as inter- and intra-subject variability. Method variability is commonly minimized by optimizing sampling procedures through computer simulations. Inter-subject variability is inherent to the population of interest and cannot be influenced. Intra-subject variability introduced by confounds (e.g., inattention or lack of motivation) cannot be simply quantified from experimental data, as these data also include method variability. Therefore, this aspect is generally neglected when developing assessments. Yet, comparing method variability and intra-subject variability could give insights on whether effort should be invested in optimizing the sampling procedure, or in addressing potential confounds instead. We propose a new approach to estimate intra-subject variability of psychometric functions by combining computer simulations and behavioral data, and to account for it when simulating experiments. The approach was illustrated in a real-world scenario of proprioceptive difference threshold assessments. The behavioral study revealed a test-retest reliability of r = 0.212. Computer simulations without considering intra-subject variability predicted a reliability of r = 0.768, whereas the new approach including an intra-subject variability model lead to a realistic estimate of reliability (r = 0.207). Such a model also allows computing the theoretically maximally attainable reliability (r = 0.552) assuming an ideal sampling procedure. Comparing the reliability estimates when exclusively accounting for method variability versus intra-subject variability reveals that intra-subject variability should be reduced by addressing confounds and that only optimizing the sampling procedure may be insufficient to achieve a high reliability. This new approach allows computing the intra-subject variability with only two measurements per subject, and predicting the reliability for a larger number of subjects and retests based on simulations, without requiring additional experiments. Such a tool of predictive value is especially valuable for target populations where time is scarce, e.g., for assessments in clinical settings.

Performance metrics for an application-driven selection and optimization of psychophysical sampling procedures

When estimating psychometric functions with sampling procedures, psychophysical assessments should be precise and accurate while being as efficient as possible to reduce assessment duration. The estimation performance of sampling procedures is commonly evaluated in computer simulations for single psychometric functions and reported using metrics as a function of number of trials. However, the estimation performance of a sampling procedure may vary for different psychometric functions. Therefore, the results of these type of evaluations may not be generalizable to a heterogeneous population of interest. In addition, the maximum number of trials is often imposed by time restrictions, especially in clinical applications, making trial-based metrics suboptimal. Hence, the benefit of these simulations to select and tune an ideal sampling procedure for a specific application is limited. We suggest to evaluate the estimation performance of sampling procedures in simulations covering the entire range of psychometric functions found in a population of interest, and propose a comprehensive set of performance metrics for a detailed analysis. To illustrate the information gained from these metrics in an application example, six sampling procedures were evaluated in a computer simulation based on prior knowledge on the population distribution and requirements from proprioceptive assessments. The metrics revealed limitations of the sampling procedures, such as inhomogeneous or systematically decreasing performance depending on the psychometric functions, which can inform the tuning process of a sampling procedure. More advanced metrics allowed directly comparing overall performances of different sampling procedures and select the best-suited sampling procedure for the example application. The proposed analysis metrics can be used for any sampling procedure and the estimation of any parameter of a psychometric function, independent of the shape of the psychometric function and of how such a parameter was estimated. This framework should help to accelerate the development process of psychophysical assessments.

Psychophysics with children: Investigating the effects of attentional lapses on threshold estimates

When assessing the perceptual abilities of children, researchers tend to use psychophysical techniques designed for use with adults. However, children's poorer attentiveness might bias the threshold estimates obtained by these methods. Here, we obtained speed discrimination threshold estimates in 6- to 7-year-old children in UK Key Stage 1 (KS1), 7- to 9-year-old children in Key Stage 2 (KS2), and adults using three psychophysical procedures: QUEST, a 1-up 2-down Levitt staircase, and Method of Constant Stimuli (MCS). We estimated inattentiveness using responses to "easy" catch trials. As expected, children had higher threshold estimates and made more errors on catch trials than adults. Lower threshold estimates were obtained from psychometric functions fit to the data in the QUEST condition than the MCS and Levitt staircases, and the threshold estimates obtained when fitting a psychometric function to the QUEST data were also lower than when using the QUEST mode. This suggests that threshold estimates cannot be compared directly across methods. Differences between the procedures did not vary significantly with age group. Simulations indicated that inattentiveness biased threshold estimates particularly when threshold estimates were computed as the QUEST mode or the average of staircase reversals. In contrast, thresholds estimated by post-hoc psychometric function fitting were less biased by attentional lapses. Our results suggest that some psychophysical methods are more robust to attentiveness, which has important implications for assessing the perception of children and clinical groups.

Algorithm for improving psychophysical threshold estimates by detecting sustained inattention in experiments using PEST

Psychophysical procedures are applied in various fields to assess sensory thresholds. During experiments, sampled psychometric functions are usually assumed to be stationary. However, perception can be altered, for example by loss of attention to the presentation of stimuli, leading to biased data, which results in poor threshold estimates. The few existing approaches attempting to identify non-stationarities either detect only whether there was a change in perception, or are not suitable for experiments with a relatively small number of trials (e.g., [Formula: see text] 300). We present a method to detect inattention periods on a trial-by-trial basis with the aim of improving threshold estimates in psychophysical experiments using the adaptive sampling procedure Parameter Estimation by Sequential Testing (PEST). The performance of the algorithm was evaluated in computer simulations modeling inattention, and tested in a behavioral experiment on proprioceptive difference threshold assessment in 20 stroke patients, a population where attention deficits are likely to be present. Simulations showed that estimation errors could be reduced by up to 77% for inattentive subjects, even in sequences with less than 100 trials. In the behavioral data, inattention was detected in 14% of assessments, and applying the proposed algorithm resulted in reduced test-retest variability in 73% of these corrected assessments pairs. The novel algorithm complements existing approaches and, besides being applicable post hoc, could also be used online to prevent collection of biased data. This could have important implications in assessment practice by shortening experiments and improving estimates, especially for clinical settings.

Sequential dependencies in pitch judgments

Studies that measure pitch discrimination relate a subject's response on each trial to the stimuli presented on that trial, but there is evidence that behavior depends also on earlier stimulation. Here, listeners heard a sequence of tones and reported after each tone whether it was higher or lower in pitch than the previous tone. Frequencies were determined by an adaptive staircase targeting 75% correct, with interleaved tracks to ensure independence between consecutive frequency changes. Responses for this specific task were predicted by a model that took into account the frequency interval on the current trial, as well as the interval and response on the previous trial. This model was superior to simpler models. The dependence on the previous interval was positive (assimilative) for all subjects, consistent with persistence of the sensory trace. The dependence on the previous response was either positive or negative, depending on the subject, consistent with a subject-specific suboptimal response strategy. It is argued that a full stimulus + response model is necessary to account for effects of stimulus history and obtain an accurate estimate of sensory noise.

A sliding two-alternative forced-choice paradigm for pitch discrimination

Studies that measure frequency discrimination often use 2, 3, or 4 tones per trial. This paper shows an investigation of a two-alternative forced choice (2AFC) task in which each tone of a series is judged relative to the previous tone ("sliding 2AFC"). Potential advantages are a greater yield (number of responses per unit time), and a more uniform history of stimulation for the study of context effects, or to relate time-varying performance to cortical activity. The new task was evaluated relative to a classic 2-tone-per-trial 2AFC task with similar stimulus parameters. For each task, conditions with different stimulus parameters were compared. The main results were as follows: (1) thresholds did not differ significantly between tasks when similar parameters were used. (2) Thresholds did differ between conditions for the new task, showing a deleterious effect of inserting relatively large steps in the frequency sequence. (3) Thresholds also differed between conditions for the classic task, showing an advantage for a fixed frequency standard. There was no indication that results were more variable with either task, and no reason was found not to use the new sliding 2AFC task in lieu of the classic 2-tone-per-trial 2AFC task.

Psychophysical measurements in children: challenges, pitfalls, and considerations

Measuring sensory sensitivity is important in studying development and developmental disorders. However, with children, there is a need to balance reliable but lengthy sensory tasks with the child’s ability to maintain motivation and vigilance. We used simulations to explore the problems associated with shortening adaptive psychophysical procedures, and suggest how these problems might be addressed. We quantify how adaptive procedures with too few reversals can over-estimate thresholds, introduce substantial measurement error, and make estimates of individual thresholds less reliable. The associated measurement error also obscures group differences. Adaptive procedures with children should therefore use as many reversals as possible, to reduce the effects of both Type 1 and Type 2 errors. Differences in response consistency, resulting from lapses in attention, further increase the over-estimation of threshold. Comparisons between data from individuals who may differ in lapse rate are therefore problematic, but measures to estimate and account for lapse rates in analyses may mitigate this problem.

Observation of time-dependent psychophysical functions and accounting for threshold drifts

Methods to obtain estimates of psychophysical functions are used in numerous fields, such as audiology, vision, and pain. Neurophysiological and psychological processes underlying this function are assumed to remain stationary throughout a psychophysical experiment. However, violation of this assumption (e.g., due to habituation or changing decisional factors) likely affects the estimates of psychophysical parameters. We used computer simulations to study how non-stationary processes, resulting in a time-dependent psychophysical function, affect threshold and slope estimates. Moreover, we propose methods to improve the estimation quality when stationarity is violated. A psychophysical detection experiment was modeled as a stochastic process ruled by a logistic psychophysical function. The threshold was modeled to drift over time and was defined as either a linear or nonlinear function. Threshold and slope estimates were obtained by using three estimation procedures: a static procedure assuming stationarity, a relaxed procedure accounting for linear effects of time, and a threshold tracking paradigm. For illustrative purposes, data acquired from two human subjects were used to estimate their thresholds and slopes using all estimation procedures. Threshold estimates obtained by all estimations procedures were similar to the mean true threshold. However, due to threshold drift, the slope was underestimated by the static procedure. The relaxed procedure only underestimated the slope when the threshold drifted nonlinearly over time. The tracking paradigm performed best and therefore, we recommend using the tracking paradigm in human psychophysical detection experiments to obtain estimates of the threshold and slope and to identify the mode of non-stationarity.

A user-operated audiometry method based on the maximum likelihood principle and the two-alternative forced-choice paradigm

OBJECTIVE

To create a user-operated pure-tone audiometry method based on the method of maximum likelihood (MML) and the two-alternative forced-choice (2AFC) paradigm with high test-retest reliability without the need of an external operator and with minimal influence of subjects' fluctuating response criteria. User-operated audiometry was developed as an alternative to traditional audiometry for research purposes among musicians.

DESIGN

Test-retest reliability of the user-operated audiometry system was evaluated and the user-operated audiometry system was compared with traditional audiometry.

STUDY SAMPLE

Test-retest reliability of user-operated 2AFC audiometry was tested with 38 naïve listeners. User-operated 2AFC audiometry was compared to traditional audiometry in 41 subjects.

RESULTS

The repeatability of user-operated 2AFC audiometry was comparable to traditional audiometry with standard deviation of differences from 3.9 dB to 5.2 dB in the frequency range of 250-8000 Hz. User-operated 2AFC audiometry gave thresholds 1-2 dB lower at most frequencies compared to traditional audiometry.

CONCLUSIONS

User-operated 2AFC audiometry does not require specific operating skills and the repeatability is acceptable and similar to traditional audiometry. User operated 2AFC audiometry is a reliable alternative to traditional audiometry.

Psychometric functions for pure tone intensity discrimination: slope differences in school-aged children and adults

Previous work on pure tone intensity discrimination in school-aged children concluded that children might have higher levels of internal noise than adults for this task [J. Acoust. Soc. Am. 120, 2777-2788 (2006)]. If true, this would imply that psychometric function slopes are shallower for children than adults, a prediction that was tested in the present experiment. Normal hearing children (5-9 yr) and adults were tested in a two-stage protocol. The first stage used a tracking procedure to estimate 71% correct for intensity discrimination with a gated 500 Hz pure tone and a 65 dB sound pressure level standard level. The mean and standard deviation of these tracks were used to identify a set of five signal levels for each observer. In the second stage of the experiment percent correct was estimated at these five levels. Psychometric functions fitted to these data were significantly shallower for children than adults, as predicted by the internal noise hypothesis. Data from both stages of testing are consistent with a model wherein performance is based on a stable psychometric function, with sensitivity limited by psychometric function slope. Across observers the relationship between slope and threshold conformed closely to predictions of a simple signal detection model.

Effect of masker type and age on speech intelligibility and spatial release from masking in children and adults

Speech recognition in noisy environments improves when the speech signal is spatially separated from the interfering sound. This effect, known as spatial release from masking (SRM), was recently shown in young children. The present study compared SRM in children of ages 5-7 with adults for interferers introducing energetic, informational, and/or linguistic components. Three types of interferers were used: speech, reversed speech, and modulated white noise. Two female voices with different long-term spectra were also used. Speech reception thresholds (SRTs) were compared for: Quiet (target 0 degrees front, no interferer), Front (target and interferer both 0 degrees front), and Right (interferer 90 degrees right, target 0 degrees front). Children had higher SRTs and greater masking than adults. When spatial cues were not available, adults, but not children, were able to use differences in interferer type to separate the target from the interferer. Both children and adults showed SRM. Children, unlike adults, demonstrated large amounts of SRM for a time-reversed speech interferer. In conclusion, masking and SRM vary with the type of interfering sound, and this variation interacts with age; SRM may not depend on the spectral peculiarities of a particular type of voice when the target speech and interfering speech are different sex talkers.

Operant methods for mouse psychoacoustics

Tone detection thresholds for a 10-kHz tone in NMRI mice were determined in psychoacoustic experiments using both a constant-stimuli procedure and a two-down/one-up adaptive-tracking procedure in the same subjects and applying identical threshold criteria (70.7% response probability). Constant-stimuli thresholds were on average 24 dB lower than adaptive-tracking thresholds, and there was a trend indicating that constant-stimuli thresholds were less variable than adaptive-tracking thresholds. Furthermore, in the constant-stimuli procedure the number of trials constituting the psychometric function could be reduced from 100 to 50 trials without a large loss of accuracy of threshold determination. In the constant-stimuli procedure, the threshold value was affected by the threshold criteria. The lowest and least variable constant-stimuli thresholds were obtained by applying signal detection theory and a criterion of d' = 1. Thus, the constant-stimuli procedure in combination with signal detection theory appears to be better suited than the adaptive-tracking procedure to determine auditory sensory thresholds.

Measuring, estimating, and understanding the psychometric function: a commentary

The psychometric function, relating the subject's response to the physical stimulus, is fundamental to psychophysics. This paper examines various psychometric function topics, many inspired by this special symposium issue of Perception & Psychophysics: What are the relative merits of objective yes/no versus forced choice tasks (including threshold variance)? What are the relative merits of adaptive versus constant stimuli methods? What are the relative merits of likelihood versus up-down staircase adaptive methods? Is 2AFC free of substantial bias? Is there no efficient adaptive method for objective yes/no tasks? Should adaptive methods aim for 90% correct? Can adding more responses to forced choice and objective yes/no tasks reduce the threshold variance? What is the best way to deal with lapses? How is the Weibull function intimately related to the d' function? What causes bias in the likelihood goodness-of-fit? What causes bias in slope estimates from adaptive methods? How good are nonparametric methods for estimating psychometric function parameters? Of what value is the psychometric function slope? How are various psychometric functions related to each other? The resolution of many of these issues is surprising.

Adaptive procedures in psychophysical research

As research on sensation and perception has grown more sophisticated during the last century, new adaptive methodologies have been developed to increase efficiency and reliability of measurement. An experimental procedure is said to be adaptive if the physical characteristics of the stimuli on each trial are determined by the stimuli and responses that occurred in the previous trial or sequence of trials. In this paper, the general development of adaptive procedures is described, and three commonly used methods are reviewed. Typically, a threshold value is measured using these methods, and, in some cases, other characteristics of the psychometric function underlying perceptual performance, such as slope, may be developed. Results of simulations and experiments with human subjects are reviewed to evaluate the utility of these adaptive procedures and the special circumstances under which one might be superior to another.

A letter-recognition task to assess lingual tactile acuity

PURPOSE

This study evaluated a novel neurosensory test based on letter recognition developed for the assessment of spatial acuity of the tongue.

PATIENTS AND METHODS

An up-down tracking procedure was used to estimate the threshold height for the recognition of embossed letters of the alphabet examined by the tongue tip. The 48 stimuli consisted of acrylic strips, one side of which bore a letter (A, I, J, L, O, T, U, or W) of 3, 4, 5, 6, 7, or 8 mm in height. Twenty neurologically normal young adults were tested.

RESULTS

Stable estimates of the threshold height were obtained after delivery of only 15 trials. Threshold height averaged 5.1 mm (range, 3.7 to 6.6 mm) and was unaffected by gender (P>.88). Although the letters were identified correctly 54% of the time, on average, the tangibility of individual letters varied from 5% correct (for W) to 82% correct (for T). The letter W was never identified by 16 of 20 subjects; T, O, and U were identified by all subjects. Analysis of errors confirmed that subjects relied on spatial cues to make the discriminations; 58% of the incorrect responses were made to 1 or 2 letters with spatial features similar to those of the stimulus letter.

CONCLUSIONS

The threshold height for letter recognition can be obtained easily and rapidly, exhibits low among-subject variability, and reflects the capacity to extract and process spatial information. Letters of similar legibility on the tongue should be used to minimize underestimation of subjects' true acuities.

Efficient and unbiased modifications of the QUEST threshold method: theory, simulations, experimental evaluation and practical implementation

QUEST [Watson and Pelli, Perception and Psychophysics, 13, 113-120 (1983)] is an efficient method of measuring thresholds which is based on three steps: (1) Specification of prior knowledge and assumptions, including an initial probability density function (p.d.f.) of threshold (i.e. relative probability of different thresholds in the population). (2) A method for choosing the stimulus intensity of any trial. (3) A method for choosing the final threshold estimate. QUEST introduced a Bayesian framework for combining prior knowledge with the results of previous trials to calculate a current p.d.f.; this is then used to implement Steps 2 and 3. While maintaining this Bayesian approach, this paper evaluates whether modifications of the QUEST method (particularly Step 2, but also Steps 1 and 3) can lead to greater precision and reduced bias. Four variations of the QUEST method (differing in Step 2) were evaluated by computer simulations. In addition to the standard method of setting the stimulus intensity to the mode of the current p.d.f. of threshold, the alternatives of using the mean and the median were evaluated. In the fourth variation--the Minimum Variance Method--the next stimulus intensity is chosen to minimize the expected variance at the end of the next trial. An exact enumeration technique with up to 20 trials was used for both yes-no and two-alternative forced-choice (2AFC) experiments. In all cases, using the mean (here called ZEST) provided better precision than using the median which in turn was better than using the mode. The Minimum Variance Method provided slightly better precision than ZEST. The usual threshold criterion--based on the "ideal sweat factor"--may not provide optimum precision; efficiency can generally be improved by optimizing the threshold criterion. We therefore recommend either using ZEST with the optimum threshold criterion or the more complex Minimum Variance Method. A distinction is made between "measurement bias", which is derived from the mean of repeated threshold estimates for a single real threshold, and "interpretation bias", which is derived from the mean of real thresholds yielding a single threshold estimate. If their assumptions are correct, the current methods have no interpretation bias, but they do have measurement bias. Interpretation bias caused by errors in the assumptions used by ZEST is evaluated. The precisions and merits of yes-no and 2AFC techniques are compared.(ABSTRACT TRUNCATED AT 400 WORDS)