The effect of serial day on the measurement of positivity and emotional complexity in diary studies

Comparisons of Self-Report With Objective Measurements Suggest Faster Responding but Little Change in Response Quality Over Time in Ecological Momentary Assessment Studies

Response times (RTs) to ecological momentary assessment (EMA) items often decrease after repeated EMA administration, but whether this is accompanied by lower response quality requires investigation. We examined the relationship between EMA item RTs and EMA response quality. In one data set, declining response quality was operationalized as decreasing correspondence over time between subjective and objective measures of blood glucose taken at the same time. In a second EMA study data set, declining response quality was operationalized as decreasing correspondence between subjective ratings of memory test performance and objective memory test scores. We assumed that measurement error in the objective measures did not increase across time, meaning that decreasing correspondence across days within a person could be attributed to lower response quality. RTs to EMA items decreased across study days, while no decrements in the mean response quality were observed. Decreasing EMA item RTs across study days did not appear problematic overall.

Bias in Self-Reports: An Initial Elevation Phenomenon

Researchers have long worried about a phenomenon where study participants give higher ratings on self-report scales the first time they take a survey compared to subsequent times, particularly for negative subjective experiences. Recent experimental evidence, using samples of U.S. college students, suggests that this initial elevation phenomenon is due to an upward bias in people’s initial responses. Such bias potentially undermines the validity of many research findings. However, more recent studies have found little evidence in support of the phenomenon. To investigate the robustness of the initial elevation phenomenon, we conducted the largest experiments to date in diverse online samples ( N = 5,285 across three studies, from Prolific.co). We observed an initial elevation on self-reports of negative subjective experiences such as mood and mental and physical health symptoms. Our findings show that the threats to validity posed by the phenomenon are real and need to be reckoned with.