Uncontrolled Web-based administration of surveys on factual health-related knowledge: a randomized study of untimed versus timed quizzing

Creating web applications for online psychological experiments: A hands-on technical guide including a template

The present tutorial provides a technical overview of how to create web applications for online psychological experiments from scratch via the HTML/CSS/JavaScript framework. This approach allows virtually unlimited flexibility in accomplishing anything in an online experiment that a regular computer (or smartphone, etc.) is capable of. Apart from offering a fast introduction for complete beginners, this tutorial may also serve as a helpful guideline for more experienced programmers and researchers. Connected to the tutorial, a specific implementation is also given via the free and open-source template project at https://github.com/gasparl/expapp , intended to be improved by the community to always follow the latest technological advancements and general good practices.

Self-initiated versus instructed cheating in the physiological Concealed Information Test

The validity of the Concealed Information Test (CIT) to detect recognition of critical details has been demonstrated in hundreds of laboratory studies. These studies, however, lack the factor of deliberate intent to deceive. This disparity between research and practice may affect the generalizability of laboratory based CIT findings. In the current study, 65 out of 174 participants cheated on their own initiative in a trivia quiz. These self-initiated cheaters were compared to 68 participants who were explicitly requested to cheat. Skin conductance, heart rate, and respiration were found to detect concealed information related to cheating. No significant differences emerged between self-initiated and instructed cheaters, supported by Bayesian statistics showing substantial evidence for the null hypothesis. The data demonstrate that the validity of the CIT is not restricted to instructed deception. This finding is encouraging from an ecological validity perspective and may pave the way for further field implementation of memory detection.

Memory-Based Deception Detection: Extending the Cognitive Signature of Lying From Instructed to Self-Initiated Cheating

From a cognitive perspective, lying can be regarded as a complex cognitive process requiring the interplay of several executive functions. Meta‐analytic research on 114 studies encompassing 3,307 participants (Suchotzki, Verschuere, Van Bockstaele, Ben‐Shakhar, & Crombez, 2017) suggests that computerized paradigms can reliably assess the cognitive burden of lying, with large reaction time differences between lying and truth telling. These studies, however, lack a key ingredient of real‐life deception, namely self‐initiated behavior. Research participants have typically been instructed to commit a mock crime and conceal critical information, whereas in real life, people freely choose whether or not to engage in antisocial behavior. In this study, participants (n = 433) engaged in a trivia quiz and were provided with a monetary incentive for high accuracy performance. Participants were randomly allocated to either a condition where they were instructed to cheat on the quiz (mimicking the typical laboratory set‐up) or to a condition in which they were provided with the opportunity to cheat, yet without explicit instructions to do so. Assessments of their response times in a subsequent Concealed Information Test (CIT) revealed that both instructed cheaters (n = 107) and self‐initiated cheaters (n = 142) showed the expected RT‐slowing for concealed information. The data indicate that the cognitive signature of lying is not restricted to explicitly instructed cheating, but it can also be observed for self‐initiated cheating. These findings are highly encouraging from an ecological validity perspective.

Geven, Ben‐Shakhar, Kindt and Verschuere point out that research on deception detection usually employs instructed cheating. They experimentally demonstrate that participants show slower reaction times for concealed information than for other information, regardless of whether they are explicitly instructed to cheat or whether they can freely choose to cheat or not. Finding this ‘cognitive signature of lying’ with self‐initiated cheating too is argued by the authors to strengthen the external validity of deception detection research. [75]

Impact of different scoring algorithms applied to multiple-mark survey items on outcome assessment: an in-field study on health-related knowledge

INTRODUCTION

Health-related knowledge is often assessed through multiple-choice tests. Among the different types of formats, researchers may opt to use multiple-mark items, i.e. with more than one correct answer. Although multiple-mark items have long been used in the academic setting - sometimes with scant or inconclusive results - little is known about the implementation of this format in research on in-field health education and promotion.

METHODS

A study population of secondary school students completed a survey on nutrition-related knowledge, followed by a single- lecture intervention. Answers were scored by means of eight different scoring algorithms and analyzed from the perspective of classical test theory. The same survey was re-administered to a sample of the students in order to evaluate the short-term change in their knowledge.

RESULTS

In all, 286 questionnaires were analyzed. Partial scoring algorithms displayed better psychometric characteristics than the dichotomous rule. In particular, the algorithm proposed by Ripkey and the balanced rule showed greater internal consistency and relative efficiency in scoring multiple-mark items. A penalizing algorithm in which the proportion of marked distracters was subtracted from that of marked correct answers was the only one that highlighted a significant difference in performance between natives and immigrants, probably owing to its slightly better discriminatory ability. This algorithm was also associated with the largest effect size in the pre-/post-intervention score change.

DISCUSSION

The choice of an appropriate rule for scoring multiple- mark items in research on health education and promotion should consider not only the psychometric properties of single algorithms but also the study aims and outcomes, since scoring rules differ in terms of biasness, reliability, difficulty, sensitivity to guessing and discrimination.