Prediction Accuracy in Multivariate Repeated-Measures Bayesian Forecasting Models with Examples Drawn from Research on Sleep and Circadian Rhythms
Hindawi Publishing Corporation
Computational and Mathematical Methods in Medicine
Volume 2016, Article ID 4724395, 23 pages
http://dx.doi.org/10.1155/2016/4724395
Research Article
Prediction Accuracy in Multivariate Repeated-Measures
Bayesian Forecasting Models with Examples Drawn from
Research on Sleep and Circadian Rhythms
Clark Kogan,1 Leonid Kalachev,2 and Hans P. A. Van Dongen1,3
1
Sleep and Performance Research Center, Washington State University, Spokane, WA 99210, USA
Department of Mathematical Sciences, University of Montana, Missoula, MT 59812, USA
3
Elson S. Floyd College of Medicine, Washington State University, Spokane, WA 99210, USA
2
Correspondence should be addressed to Hans P. A. Van Dongen;
Received 23 May 2015; Accepted 27 August 2015
Academic Editor: Chung-Min Liao
Copyright © 2016 Clark Kogan et al. This is an open access article distributed under the Creative Commons Attribution License,
which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
In study designs with repeated measures for multiple subjects, population models capturing within- and between-subjects variances
enable efficient individualized prediction of outcome measures (response variables) by incorporating individuals response data
through Bayesian forecasting. When measurement constraints preclude reasonable levels of prediction accuracy, additional
(secondary) response variables measured alongside the primary response may help to increase prediction accuracy. We investigate
this for the case of substantial between-subjects correlation between primary and secondary response variables, assuming negligible
within-subjects correlation. We show how to determine the accuracy of primary response predictions as a function of secondary
response observations. Given measurement costs for primary and secondary variables, we determine the number of observations
that produces, with minimal cost, a fixed average prediction accuracy for a model of subject means. We illustrate this with estimation
of subject-specific sleep parameters using polysomnography and wrist actigraphy. We also consider prediction accuracy in an
example time-dependent, linear model and derive equations for the optimal timing of measurements to achieve, on average, the
best prediction accuracy. Finally, we examine an example involving a circadian rhythm model and show numerically that secondary
variables can improve individualized predictions in this time-dependent nonlinear model as well.
1. Introduction
Significant steps forward in the analysis of repeated-measures
data were made with the introduction of linear and nonlinear mixed-effects models [1–3], which distinguish withinsubjects variance (from multiple measurements in each subject) versus between-subjects variance (from multiple subjects being measured). Distinguishing these types of variance
can also be thought of as explicitly modeling random error in
the data. This can be useful in understanding how different
individuals are from one another as compared to how
different multiple measurements are for given individuals. In
research on sleep and sleepiness, for example, breakthroughs
made possible by mixed-effects models include elucidation
of the dose-response effects of sustained sleep restriction on
sleep architecture and neurobehavioral impairment [4, 5]
and demonstration of the trait characteristics of individual
differences in vulnerability to sleep loss [6]. In recent years,
mixed-effects model approaches to statistical regression and
analysis of variance have become widely available in statistical
software packages. They are nowadays the methodology of
choice for many repeated-measures investigations in sleep
research and other fields of study.
A further advance was the introduction of a model
individualization technique called Bayesian posterior distribution estimation or Bayesian forecasting. This technique
was first used in sleep research to overcome a shortcoming
of biomathematical models of fatigue and performance.
Existing models did not account for individual differences in
sleep regulation and vulnerability to sleep loss and therefore
2
Computational and Mathematical Methods in Medicine
50
50
40
t = 19.5
(time 19:30)
30
t − t0 = 12.0
(awake 12 h)
20
PVT lapses
t − t0 = 4.0
(awake 4 h)
PVT lapses
40
t = 11.5
(time 11:30)
10
0
0
12
24
36
48
Time awake (h)
60
72
50
t − t0 = 28.0
(awake 28 h)
20
PVT lapses
PVT lapses
30
60
72
0
12
24
36
48
Time awake (h)
60
72
0
12
24
36
48
Time awake (h)
60
72
30
20
0
0
12
24
36
48
Time awake (h)
60
72
50
50
40
t = 51.5
(time 51:30)
30
t − t0 = 44.0
(awake 44 h)
20
PVT lapses
40
PVT lapses
24
36
48
Time awake (h)
10
0
t − t0 = 36.0
(awake 36 h)
12
40
t = 35.5
(time 35:30)
10
t = 43.5
(time 43:30)
0
50
40
t − t0 = 20.0
(awake 20 h)
20
10
0
t = 27.5
(time 27:30)
30
30
20
10
10
0
0
0
12
24
36
48
Time awake (h)
60
72
Figure 1: Lapses of attention on a psychomotor vigilance test (PVT) for a subject in a study involving 88 h of total sleep deprivation under
controlled laboratory conditions. In each of the six plots, different amounts of subject data are assumed known (black dots), and the Bayesian
forecasting procedure is applied to the known data to construct predictions of PVT number of lapses for a 24 h interval immediately following
the most recent collected data point at time 𝑡. For the 24 h interval, 95% prediction intervals (vertical bars) are shown, along with the remainder
of the data for comparison (gray dots). The beginning of the 88 h sleep deprivation period, 𝑡0 , was at 07:30. Graphs taken from [7] with
permission.
did not accurately predict performance for given individuals.
Bayesian forecasting addressed this shortcoming by utilizing
the separation of within- and between-subjects variance in
model parameters as enabled by mixed-effects modeling
[2]. In Bayesian forecasting, the between-subjects variance
of model parameters serves as Bayesian prior information.
Measurements from a new individual, not previously studied,
are combined with the prior information to efficiently derive
model parameters tailored to the new individual, thereby
yielding a subject-specific mathematical model [2, 7, 8].
As a bonus, the Bayesian forecasting technique also yields
quantitative estimates of the accuracy of individualized predictions made with the subject-specific mathematical model
[9].
In a published example, Bayesian forecasting was implemented for the two-process model of sleep regulation [10, 11]
to predict performance impairment of selected individuals
undergoing a period of total sleep deprivation (see Figure 1).
Comparisons with the individuals’ actual data revealed that
the model parameters converged efficiently to those that
best characterized each individual, and the response predictions were significantly more accurate than could have been
Computational and Mathemati (...truncated)
