How to fit a response time distribution

TRISHA VAN ZANDT 0 1 0 Portions of this article were presented at the 31st Annual Meeting of the Society for Mathematical Psychology , August 1998, Vanderbilt University , and at the 39th Annual Meeting of the Psychonomic Soci ety, November 1998, Dallas . The project was funded by NSF Grant SBR-9702291. The author gratefully acknowledges the many contri butions to this project by Steven Yantis. Thanks also are due Howard Egeth, Andrew Heathcote, and John Wixted for many helpful com article should be addressed to T. Van Zandt, Department of Psychology, Ohio State University , 1885 Neil Avenue Mall, Columbus, OH 43210 1222 1 Johns Hopkins University , Baltimore, Maryland Among the most valuable tools in behavioral science is statistically fitting mathematical models of cognition to data-response time distributions, in particular. However, techniques for fitting distributions vary widely, and little is known about the efficacy of different techniques. In this article, we assess several fitting techniques by simulating six widely cited models of response time and using the fitting procedures to recover model parameters. The techniques include the maximization of likelihood and least squares fits of the theoretical distributions to different empirical estimates of the simulated distributions. Arunning example is used to illustrate the different estimation and fitting procedures. The simulation studies reveal that empirical density estimates are biased even for very large sample sizes. Some fitting techniques yield more accurate and less variable parameter estimates than do others. Methods that involve least squares fits to density estimates generally yield very poor parameter estimates. - The importance of considering the entire response time (RT) distribution in testing formal models ofcognition is now widely appreciated. Fitting a model to mean RT alone can mask important details of the data that examination of the entire distribution would reveal, such as the behav ior of fast and slow responses across the conditions of an experiment (e.g., Heathcote, Popiel, & Mewhort, 1991), the extent of facilitation between perceptual channels (Miller, 1982), and the effects ofpractice on RT quantiles (Logan, 1992). Techniques for testing hypotheses based on the RT distribution have been developed (Townsend, 1990). In addition, the RT distribution provides an impor tant meeting ground between theory and data; the ability ofa model to predict the observed shape of the RT distri bution is seen as a critical test of that model (Luce, 1986). Many models state explicitly the characteristics ofRT by specifying it as a random variable. All of the informa tion about a random variable is contained in its probabil ity density function (density, for short) or cumulative dis tribution function (CDF).l The density represents the likelihood that an RT is observed within some arbitrarily small window of time, whereas the CDF represents the probability that an RT is less than or equal to some spe cific time. Most models ofRT predict CDFs that are ogi val: monotonic, nondecreasing S-shaped functions that begin at zero and asymptote at one. The RT densities pre dicted by most models are, in contrast, bell shaped: nonmonotonic, usually positively skewed, and unimodal. Despite their differences, the density and CDF are math ematically equivalent ways of stating the properties of a random variable. The shape ofthe density often provides clues to the kind of random variable and, therefore, potentially to the can didate processes underlying the execution of a particular RT task. To investigate density shape from a sample of RTs requires that those RTs be used as the basis of an es timate ofthe density. As we will demonstrate in this paper, some density estimation techniques more accurately re cover the true shape of the density than do others. In ad dition, it is often a goal of analysis to fit or estimate the parameters ofa model, which also may require an estimate of the density or the CDF. The model-fitting strategy used will determine the accuracy of the estimated param eter values. Although a number of computer programs are now available to assist researchers in plotting and per forming fits of models to RT distributions (Cousineau & Larochelle, 1997; Dawson, 1988; Heathcote, 1996), no detailed assessment of the techniques most commonly used in density and parameter estimation currently exists in the psychological literature, especially in the context of the most likely models to be examined in such estima tions. The purpose of this article is to provide such an evaluation. Two separate but related estimation issues must to be addressed. The first is the purely descriptive problem of accurately estimating the shape of the distribution from which a sample was derived, without making assump tions about the functional form of the population. This problem has been the focus of most of the work in the sta tistical area of nonparametric density estimation (Dev roye, 1987; Silverman, 1986; Tapia & Thompson, 1978). The second issue is the analytical problem of accurately estimating the parameters of a presumed distribution or, simply, fitting a model to the sample. To estimate these parameters, we may need to make use of a density or a CDF estimate, the same estimate used to investigate the shape of the distribution, as we will describe in some de tail below. In what follows, we first discuss the different ways to estimate the shape of an RT distribution (either the den sity or the CDF) and the properties of each estimator. Us ing a small sample, we illustrate the techniques by which densities and CDFs are estimated. We then present a simulation study, in which the quality of each estimator is explored. We explain how models are fit to data and then fit models to the simulated data to test the accuracy of different parameter estimation techniques. The study had two goals: first, to determine which of the nonpara metric density estimators most popular in psychological research are the most accurate, and second, to determine how best to estimate the parameters of a model, using these density estimates. We also explore the effects of sample size on the accuracy of the density and parameter estimates. The results demonstrate that certain density estimators, including the popular Vincent estimator (Rat cliff, 1979), are highly biased estimators of the true den sity function and that model fits to density estimates (rather than maximum-likelihood estimates or fits to es timates of the CDF) often fail to recover accurate param eter values. DENSITY AND CUMULATIVE DISTRIBUTION FUNCTION ESTIMATORS There is a large statistical literature concerning various parametric and nonparametric density function estimators (see, e.g., Tapia & Thompson, 1978, for a historical re view). Parametric density estimators make assumptions about the functional form of the empirical density, and the estimate is constructed by fin (...truncated)