A General Statistic for Testing the Validity of a Model’s Forecasts
Ann. Data. Sci. (2015) 2(2):131–144
DOI 10.1007/s40745-015-0037-9
A General Statistic for Testing the Validity of a Model’s
Forecasts
Hadi Alizadeh Noughabi1
Received: 9 May 2015 / Revised: 19 June 2015 / Accepted: 22 June 2015 / Published online: 9 July 2015
© Springer-Verlag Berlin Heidelberg 2015
Abstract The paper introduces a new approach for testing the validity of a model’s
forecasts and it is established that the proposed test is exact for location–scale family.
The results are used to introduce goodness-of-fit tests for the normal, exponential,
uniform and Laplace distributions. Through Monte Carlo simulation, the power values
of the proposed test under various alternatives are compared with the other tests. The
proposed test has higher power than the competing tests against some alternatives.
The use of the proposed test is shown in real examples.
Keywords Goodness of fit tests · Model validity · Location–scale family · Exact
test · Power comparison · Nonparametric kernel density estimation
1 Introduction
In reliability studies, engineering and management sciences, it is important to test
whether the underlying distribution has a particular form. Most statistical methods
assume an underlying distribution in the derivation of their results and inferences.
Therefore approaches for determining the underlying distribution, i.e. goodness of fit
test, is necessary.
Some general goodness of fit tests can be found in the literature, namely,
Kolmogorov–Smirnov, Cramér–von Mises, Kuiper and Anderson–Darling. These tests
are common and famous tests which are widely used in practice. For further study about
these tests, see [17].
B Hadi Alizadeh Noughabi
1
Department of Statistics, University of Birjand, Birjand, Iran
123
132
Ann. Data. Sci. (2015) 2(2):131–144
Different methods for goodness of fit tests are introduced by researchers such as
goodness of fit tests based on the empirical distribution function, empirical characteristic function, entropy and Kullback–Leibler information, maximum correlations, and
divergences.
Comparison of goodness of fit tests has received attention in the literature. Stephens
[35] by Monte Carlo simulation presented comparisons for some tests for normality.
Moreover, some authors compared the power of goodness of fit tests for other distributions.
The goodness of fit tests has been discussed by many authors including, [1–9,13–
18,20,22,23,26–28,31–33,37]. Moreover, some tests for censored data are proposed
by authors; see, for example, [10,11,21,25,29,30].
In Sect. 2, we introduce an exact goodness of fit test by using the kernel density
estimation. Also, properties of proposed test are discussed. Section 3 discusses the
special case of location–scale family and we used the results for introducing tests for
normal, exponential, uniform and Laplace distributions. In Sect. 4, by simulation, the
power values of the proposed test are compared with the power values of the competitor
tests. Section 5 contains the use of the proposed test in real examples.
2 The Test Statistic
Let X 1 , . . . , X n be a random sample from an unknown distribution F with a probability density function f (x). Let F0 (x; θ) be a parametric family of distributions with
probability density function f 0 (x; θ). The hypothesis of interest is
H0 : f (x) = f 0 (x; θ),
f or some θ ∈ ,
and the alternative to H0 is
H1 : f (x) = f 0 (x; θ),
f or any θ ∈ .
To discriminate between the two hypotheses H0 and H1 , we apply the following
method.
We generate the following data (Y(i) ’s) under the null distribution F0 as
Y(i) = F0−1
i
,θ ,
n+1
i = 1, 2, . . . , n.
If θ be unknown, we will estimate
itby a reasonable estimator.
i
−1
In order hand, X (i) = F
n+1 , where X (1) ≤ X (2) ≤ · · · ≤ X (n) are the order
statistics of sample.
123
Ann. Data. Sci. (2015) 2(2):131–144
133
Let β be a constant. We have under H0 ,
∀i X (i) = Y(i) ⇔ ∀i F(X (i) ) = F0 (Y(i) ) ⇔ ∀i f (X (i) ) = f 0 (Y(i) )
f (X (i) ) β
⇔ ∀i
−1=0
f 0 (Y(i) )
2
n
f (X (i) ) β
1
⇔ Tβ ( f, f 0 ) =
− 1 = 0.
n
f 0 (Y(i) )
i=1
And under H1 ,
∃i X (i) = Y(i) ⇔ ∃i F(X (i) ) = F0 (Y(i) ) ⇔ ∃i f (X (i) ) = f 0 (Y(i) )
f (X (i) ) β
⇔ ∃i
− 1 = 0
f 0 (Y(i) )
2
n
f (X (i) ) β
1
⇔ Tβ ( f, f 0 ) =
− 1 > 0.
n
f 0 (Y(i) )
i=1
It is obvious that Tβ ( f, f 0 ) ≥ 0 and the equality holds if and only if f (x) = f 0 (x)
almost everywhere. Therefore, Tβ ( f, f 0 ) measures the discrepancy between F and
F0 . Under H0 , Tβ ( f, f 0 ) = 0, and Tβ ( f, f 0 ) > 0 if H0 be false.
We consider the kernel density estimator for estimating f (xi ) and f 0 (yi ). The
kernel density function estimator is defined by
1
fˆ(X i ) =
nh
n
Xi − X j
,
k
h
j=1
where h is a bandwidth and k is a kernel function which satisfies
∞
−∞
k(x)d x = 1.
Usually, k will be a symmetric probability density function.
Therefore, we propose the following test statistic for goodness of fit test.
⎡
n
1 ⎣
Tβ ( f, f 0 ) =
n
i=1
fˆ(X (i) )
fˆ0 (Y(i) ; θ̂)
β
⎤2
− 1⎦ ,
where
1
fˆ(x(i) ) =
k
nh x
n
j=1
x(i) − x j
,
hx
1
fˆ0 (y(i) ) =
k
nh y
n
j=1
y(i) − y j
,
hy
123
134
Ann. Data. Sci. (2015) 2(2):131–144
and the kernel function is chosen to be the standard normal density function and the
1
bandwidth h is chosen to be the normal optimal smoothing formula, h = 1.06sn − 5 ,
where s is the sample standard deviation.
Since large values of Tβ ( f, f 0 ) favor the alternative hypothesis to H0 , we reject H0
if Tβ ( f, f 0 ) is large, i.e., reject H0 if Tβ ( f, f 0 ) ≥ Cα for some critical value Cα .
Since
p.
fˆ(x) −→ f (x)
as n → ∞,
[34] and, by of LLN,
⎡
n
1 ⎣
n
i=1
β
fˆ(x(i) )
fˆ0 (y(i) ; θ̂)
⎧
2 ⎫
⎨ f (x) β
⎬
− 1⎦ → E f
−1
,
⎩
⎭
f 0 (y; θ)
⎤2
the test based on Tβ ( f, f 0 ) is consistent.
Let G be the group of transformations, we have
∀g ∈ G, Tβ (g(x)) =
n
1
n
⎡
⎣
i=1
fˆ(g(x(i) ))
fˆ0 (g(y(i) ); θ̂)
⎤2
β
− 1⎦ ,
where
fˆ(g(x(i) )) =
2
Sg(x)
=
fˆ(g(y(i) )) =
2
Sg(y)
=
1
k
nh g(x)
n
1
n−1
1
nh g(y)
1
n−1
j=1
n
i=1
n
2
g(xi ) − g(x) ,
k
j=1
n
g(x(i) ) − g(x j )
,
h g(x)
1
h g(x) = 1.06Sg(x) n − 5 ,
1
g(xi ),
n
n
g(x) =
i=1
g(y(i) ) − g(y j )
,
h g(y)
2
g(yi ) − g(yi ) ,
i=1
1
h g(y) = 1.06Sg(y) n − 5 ,
1
g(y) =
g(yi ).
n
n
i=1
Generally, the proposed test is not exact, because
fˆ(g(x(i) ))
fˆ0 (g(y(i) ); θ̂)
=
fˆ(x(i) )
fˆ0 (y(i) ; θ̂)
.
However, if g(x) = x +d, g(x) = cx or g(x) = cx +d, which is the case for location,
scale and location–scale families respectively, the test is exact. Because we have
123
Ann. Data. Sci. (2015) 2(2):131–144
135
1
k
nch x
n
(cx(i) + d) − (cx j + d)
ch x
1 ˆ
f (x(i) ),
c
h g(x) = ch x ,
fˆ(g(x(i) )) =
h g(y) = ch y ,
n
(cy(i) + d) − (cy j + d)
1
1
ˆ
= fˆ(y(i) ),
f (g(y(i) )) =
k
nch y
ch y
c
j=1
=
j=1
fˆ(g(x(i) ))
fˆ0 (g(y(i) ); θ̂)
fˆ(x(i) )
=
fˆ0 (y(i) ; θ̂)
⇒ Tβ (g(x)) = Tβ (x).
Several examples of scale and location–scale families are considered in S (...truncated)
