Online psychophysics: reaction time effects in cognitive experiments
Behav Res
Online psychophysics: reaction time effects in cognitive experiments
Kilian Semmelmann 0
Sarah Weigelt 0
0 Developmental Neuropsychology, Department of Psychology, Ruhr-University Bochum , Universitätsstr. 150, 44801 Bochum , Germany
1 Kilian Semmelmann
Using the Internet to acquire behavioral data is currently on the rise. However, very basic questions regarding the feasibility of online psychophysics are still open. Here, we aimed to replicate five well-known paradigms in experimental psychology (Stroop, Flanker, visual search, masked priming, attentional blink) in three settings (classical Blab^, Bweb-inlab^, Bweb^) to account for possible changes in technology and environment. Lab and web-in-lab data were both acquired in an in-lab setting with lab using BGold Standard^ methods, while web-in-lab used web technology. This allowed for a direct comparison of potential differences in acquisition software. To account for additional environmental differences, the web technology experiments were published online to participate from home (setting web), thereby keeping the software and experimental design identical and only changing the environmental setting. Our main results are: First, we found an expected fixed additive timing offset when using web technology (M = 37 ms, SD = 8.14) and recording online (M = 87 ms, SD = 16.04) in comparison to lab data. Second, all taskspecific effects were reproduced except for the priming paradigm, which couldn't be replicated in any setting. Third, there were no differences in error rates, which are independent of the timing offset. This finding further supports the assumption of data equality over all settings. Fourth, we found that browser type might be influencing absolute reaction times. Together, these results contribute to the slowly but steadily growing
Psychophysics; Web technology; Online study; Replication; Cognitive psychology; Reaction time
-
literature that online psychophysics is a suitable complement
– or even substitute – to lab data acquisition.
The Internet has a huge impact on every aspect of today’s life.
In experimental psychology, the introduction and
establishment of online surveys has allowed researchers to reach more
people from a broader background more easily, efficiently,
and faster than ever before
(Reips, 2000)
. Thanks to the higher
heterogeneity of the participants
(age, gender, origin, social
status; see Birnbaum, 2004)
online studies allow us to tackle
one of the biggest open questions in psychology: To what
extent can the results of a single study be mapped to the whole
population? Furthermore, through the worldwide availability
of online experiments, scientists have easier access to
hard-toreach populations, benefit from a double-blind situation, and
are able to conduct parallel experiments independent of
equipment or available experimenters
(Gosling, Vazire, Srivastava,
& John, 2000; Reips, 2000)
. For example Joinson (2001)
found through the results of three studies that participants in
an anonymous, computer-mediated research environment
disclose significantly more information than in classical research
settings. A work by
Reimers (2007)
collected data from over
255,000 participants in a study about sex differences in Britain
– one of the largest up to date. Collecting such large samples
will help scientists to battle the issue of insufficient power,
which often renders studies irreplicable. A much more
selective, but still numerous, sample was collected by Cohen,
Collins, Darkes, and Gwartney (2007), who used online
message boards to get nearly 2,000 responses from a very specific
population of non-medical anabolic steroid users in the USA.
These and more advantages have been covered in excessive
exploration and validation studies
(Birnbaum, 2000; Gosling
et al., 2000; Skitka & Sargis, 2006; amongst others)
.
However, experimental psychological research is more than
survey data.
Especially in cognitive psychology, the study of mental
processes in human beings, one of the most common
approaches is to measure reactions to stimuli that are usually
presented on a computer screen, thereby gaining an external
measurement of internal processes. The two main variables of
such psychophysical measurements are error rates (ERs) and
reaction times (RTs). In a typical cognitive experiment, a
participant is presented with a display of different conditions and
reacts to this display via a keypress. Usually, this is done by
inviting participants into the lab, seating them in a testing
chamber, and instructing them to press a mouse or keyboard
button when appropriate. Through this approach researchers
can control for external factors like hardware (computer
system, keyboard, and mouse), software (operating system,
programming language, versions), and the environment (e.g.,
noise, lightning, distractions). On the other hand, conducting
cognitive experiments in a classical in-lab setting requires a
high amount of resources, i (...truncated)