#### An Orthogonal-Polynomial Approach to First-Hitting Times of Birth–Death Processes

J Theor Probab
An Orthogonal-Polynomial Approach to First-Hitting Times of Birth-Death Processes
Erik A. van Doorn 0
0 Department of Applied Mathematics, University of Twente , P.O. Box 217, 7500, AE, Enschede , The Netherlands
In a recent paper in this journal, Gong, Mao and Zhang, using the theory of Dirichlet forms, extended Karlin and McGregor's classical results on first-hitting times of a birth-death process on the nonnegative integers by establishing a representation for the Laplace transform E[esTi j ] of the first-hitting time Ti j for any pair of states i and j , as well as asymptotics for E[esTi j ] when either i or j tends to infinity. It will be shown here that these results may also be obtained by employing tools from the orthogonal-polynomial toolbox used by Karlin and McGregor, in particular associated polynomials and Markov's theorem.
Birth-death process; First-hitting time; Orthogonal polynomials; Associated polynomials; Markov's theorem
1 Introduction
A birth–death process is a continuous-time Markov chain X := { X (t ), t ≥ 0} taking
values in S := {0, 1, 2, . . .} with q-matrix Q := (qi j , i, j ∈ S) given by
qi,i+1 = λi ,
qi+1,i = μi+1,
qii = −(λi + μi ),
qi j = 0,
|i − j | > 1,
where λi > 0 for i ≥ 0, μi > 0 for i ≥ 1 and μ0 ≥ 0. Positivity of μ0 entails that
the process may evanesce by escaping from S, via state 0, to an absorbing state −1.
Throughout this paper, we will assume that the transition probabilities
Pi j (t ) := P(X (t ) = j | X (0) = i ), i, j ∈ S,
satisfy both the backward and forward Kolmogorov equations, and mostly also that
they are uniquely determined by the birth rates λi and death rates μi . Karlin and
McGregor [14] have shown that the latter is equivalent to assuming
∞
n=0
1
πn + λnπn
= ∞,
where the πn are constants given by
π0 := 1 and πn := λ0λ1 · · · λn−1 , n > 0.
μ1μ2 · · · μn
and
we have the well-known result
0
0
We note that condition (
1
) does not exclude the possibility of explosion, escape from
S, via all states larger than the initial state, to an absorbing state ∞.
We denote by Ti j the (possibly defective) first-hitting time of state j , starting in
state i = j . Then, writing
Pˆi j (s) :=
∞ est Pi j (t )dt, s < 0,
Fˆi j (s) := E[esTi j ] =
∞ est dP(Ti j ≤ t ), i = j, s < 0,
(
1
)
(
2
)
(
3
)
(
4
)
Fˆi j (s) =
Pˆi j (s)
Pˆ j j (s)
, i = j
Qi (s)
Fˆi j (s) = Q j (s) , 0 ≤ i < j,
(see, for example, [15, Equation (1.3)]). Karlin and McGregor give in [14,
Equation (3.21)] a representation for Pˆi j (s), which upon substitution in (
2
) yields
where Qn, n = 0, 1, . . . , are the birth–death polynomials associated with the process
X , that is, the Qn satisfy the recurrence relation
λn Qn+1(x ) = (λn + μn − x )Qn(x ) − μn Qn−1(x ), n > 0,
λ0 Q1(x ) = λ0 + μ0 − x ,
Q0(x ) = 1.
The representation (
3
) was observed explicitly for the first time by Karlin and
McGregor themselves in [16, p 378]. Since then several authors have rediscovered the result
or provided alternative proofs (see Diaconis and Miclo [4] for some references).
In a recent paper in this journal, Gong et al. [12], using the theory of Dirichlet
forms, extended Karlin and McGregor’s result by establishing a representation for the
Laplace transform of the first-hitting time Ti j for any pair of states i = j , as well
as asymptotics when either i or j tends to infinity. It will be shown here that these
results may also be obtained by exploiting Karlin and McGregor’s toolbox, which is
the theory of orthogonal polynomials.
Our findings, which are actually somewhat more general than those of Gong et al.
are presented in Sect. 3 and proven in Sect. 4. In the next section, we introduce some
further notation, terminology and preliminary results. Since a path between two states
in a birth–death process has to hit all intermediate states, we obviously have
So for notational simplicity—and without loss of generality—we will restrict ourselves
to an analysis of T0n and Tn0 for n > 0.
2 Preliminaries
We will use the shorthand notation
and, following Anderson [1, Chapter 8],
n
i=0
n
i=0
Kn :=
πi , Ln :=
(λi πi )−1, 0 ≤ n ≤ ∞,
C :=
∞
n=0
(λnπn)−1 Kn,
D :=
(λnπn)−1(K∞ − Kn).
∞
n=0
We have K∞ + L∞ = ∞ by our assumption (
1
), while, obviously,
K∞ = ∞
⇒ D = ∞, L∞ = ∞
⇒ C = ∞.
Also, C + D = K∞ L∞, so (
1
) is actually equivalent to C + D = ∞. Whether the
quantities C and D are infinite or not determines the type of the boundary at infinity
(see, for example, Anderson [1, Section 8.1]), but also, as we shall see, the asymptotic
behavior of the polynomials Qn of (
4
).
Since the birth–death polynomials Qn satisfy the three-terms recurrence relation
(
4
), they are orthogonal with respect to a positive Borel measure on the nonnegative
real axis, and have positive and simple zeros. The orthogonalizing measure for the
polynomials Qn (normalized to be a probability measure) is not necessarily uniquely
determined by the birth and death rates, but there exists, in any case, a unique natural
measure ψ , characterized by the fact that the minimum of its support is maximal. We
refer to Chihara’s book [3] for properties of orthogonal polynomials in general, and
to Karlin and McGregor’s papers [14,15] for results on birth–death polynomials in
particular (see also [10, Section 3.1] for a concise overview). For our purposes, the
following properties of birth–death polynomials are furthermore relevant.
With xn1 < xn2 < · · · < xnn denoting the n zeros of Qn(x ), there is the classical
separation result
0 < xn+1,i < xni < xn+1,i+1, i = 1, 2, · · · , n, n ≥ 1,
so that the limits
exist. We further let
ξi := nl→im∞ xni , i = 1, 2, . . . .
σ := il→im∞ ξi
(possibly infinity). The numbers ξi may be defined alternatively as
ξ1 := inf supp(ψ ) and ξi+1 := inf{supp(ψ ) ∩ (ξi , ∞)}, i ≥ 1,
where supp stands for support. So knowledge of the (natural) orthogonalizing measure
for the polynomials Qn implies knowledge of the numbers ξi . It is clear from the
definition of ξi that
(
7
)
(
8
)
Moreover, we have, for all i ≥ 1,
0 ≤ ξi ≤ ξi+1 ≤ σ, i ≥ 1.
ξi+1 = ξi
⇐⇒
ξi = σ,
as is evident from the alternative definition of ξi . By suitably interpreting [6,
Equations (2.6) and (2.11)], it follows that
∞
i=1
ξi−1
1
= nl→im∞ 1 + μ0 Ln j=0
n
(λ j π j )−1
j
i=0
πi (1 + μ0 Li−1),
where the left-hand side should be interpreted as infinity if ξ1 = 0. In particular,
and
μ0 > 0 :
Given a sequence of birth–death polynomials {Qn}, we obtain the sequence {Q(nl)}
of associated polynomials of order l ≥ 0 by replacing Qn by Q(nl), λn by λn+l and μn
by μn+l in the recurrence relation (
4
). Evidently, the polynomials Q(nl) are birth–death
polynomials again, so Q(nl)(x ) has simple, positive zeros xn(l1) < xn(l2) < · · · < xn(ln) and
we can write
Q(nl)(x ) = Q(nl)(0)
, n, l ≥ 0,
n
i=1
1 −
x
xn(li)
while it follows by induction that
Q(nl)(0) = 1 + μl πl (Ln+l−1 − Ll−1), n, l ≥ 0,
where L−1 := 0. Note that Q(n0)(0) = Qn(0) = 1 for all n if μ0 = 0.
Defining the quantities ξi(l) and σ (l) in analogy to (
7
) and (
8
), we have, by [3,
Theorem III.4.2],
ξi(l) ≤ ξi(l+1) ≤ ξi(+l)1, l ≥ 0, i ≥ 1,
Moreover, [5, Theorem 1] tells us that
σ (l) = σ, l ≥ 0.
ll→im∞ ξi(l) = σ, i ≥ 1.
Since the polynomials Q(nl) are birth–death polynomials, they are orthogonal with
respect to a unique natural (probability) measure ψ (l) on the nonnegative real axis. A
key ingredient in our analysis is Markov’s theorem, which relates the Stieltjes transform
of the measure ψ (l) to the polynomials Q(nl) and Q(nl+1), namely
We note that ψ (l) is not necessarily the only orthogonalizing measure for the
polynomials Q(nl), a setting usually not covered in statements of Markov’s theorem in the
literature (see, for example, [3, p 89]). However, an extension of the original theorem
that serves our needs can be found in Berg [2] (see in particular [2, Section 3], where
the measure μ(0) corresponds to our ψ (l)).
(
10
)
(
11
)
(
12
)
(
13
)
(
14
)
We will also have use for a classical result in the theory of continued fractions
relating the Stieltjes transforms of the measures ψ (l) and ψ (l+1), namely
x − s
=
λl + μl − s − λl μl+1
∞ ψ (l+1)(dx )
0
x − s
−1
,
(
15
)
Again we refer to Berg [2, Section 4] for statements of this result in the generality
required in our setting.
Our final preliminary results concern asymptotics for the polynomials Q(nl) as n →
∞, which may be obtained by suitably interpreting the results of [17] (which extend
those of [6]). We state the results in three propositions and give more details about
their derivations in Sect. 4. Recall that ξ0(l) = −∞ and Qn (0) = 1 if μ0 = 0.
Proposition 1 Let K∞ = L ∞ = ∞. Then C = D = ∞, σ = 0 and, for l ≥ 0,
an entire function with simple, positive zeros ξi(l), i ≥ 1.
Proposition 3 Let K∞ < ∞ and L ∞ = ∞. Then C = ∞ and,
(i) for l = 0 and μ0 > 0, or l ≥ 1,
(ii) if D = ∞, for l ≥ 0,
an entire function with simple zeros ξ1 = 0 and ξi+1 > 0, i ≥ 1;
(iv) if D < ∞, for l = 0 and μ0 > 0, or l ≥ 1,
lim
n→∞ Ln−1
Qn(x )
= −x K∞
∞
i=1
an entire function with simple, positive zeros ξi(l), i ≥ 1.
3 Results
Representations for E[esT0n I{T0n<∞}] and E[esTn0 I{Tn0<∞}] in terms of the polynomials
Q(nl) are collected in the first theorem.
Theorem 1 We have, for μ0 ≥ 0 and n ≥ 1,
1
E[esT0n I{T0n<∞}] = Qn(s) , s < xn,1,
and, if C + D = ∞,
E[esTn0 I{Tn0<∞}] = λnλπ0n Nl→im∞
Note that for s < 0, we have E[esT0n I{T0n<∞}] = E[esT0n ], so the representation
(
16
) reduces to Karlin and McGregor’s result (
3
). The explicit representation (
17
) is
new, but may be obtained by a limiting procedure from Gong et al. [12, Corollary 3.6],
where a finite state space is assumed.
By choosing s = 0 in (
16
) and (
17
) and using (
11
), we obtain expressions for
the probabilities P(T0n < ∞) and P(Tn0 < ∞) that are in accordance with [15,
an unnumbered formula on page 387 and Theorem 10]. For convenience, we state the
results as a corollary of Theorem 1, but remark that a proof of (19) on the basis of
(
17
) would require additional motivation in the case ξ1(
1
) = 0.
Corollary 1 ([15]) We have, for μ0 ≥ 0 and n ≥ 1,
P(Tn0 < ∞) = 1 − LLn∞−1 .
After a little algebra, (
17
) and (
11
) lead to
E[esTn0 I{Tn0<∞}] =
Ln−1
1 − L∞
lim
N →∞
Q(Nn−+n1)(s)/Q(Nn−+n1)(0)
Q(N1)(s)/Q(N1)(0)
(19)
where the infinite products are entire functions with simple, positive zeros ξi(n+1) and
Assuming a denumerable state space, but under the condition C = ∞ and D < ∞,
Gong et al. give in [12, Theorem 5.5 (a)] a representation for E[esTn0 ], s < 0, which
is encompassed by Corollary 2. Indeed, in this case we have L∞ = ∞, and hence, by
(19), P(Tn0 < ∞) = 1.
Asymptotic results for E[esT0n I{T0n<∞}] and E[esTn0 I{Tn0<∞}] as n → ∞ are
summarized in the second theorem.
Theorem 2 We have, for μ0 ≥ 0 and s < 0,
⎧
nl→im∞ E[esT0n I{T0n<∞}] = ⎨⎪
1
∞
ξi
1 + μ0 L∞ i=1 ξi − s
⎪⎩ 0
and
⎧ 0
nl→im∞ E[esTn0 I{Tn0<∞}] = ⎨⎪ ∞
The infinite products in (21) and (22) are reciprocals of entire functions with simple,
positive zeros ξi and ξi(
1
), i ≥ 1, respectively.
By (18), we have
so (21) implies
nl→im∞ E[esT0n | T0n < ∞] =
∞
ξi
i=1 ξi − s
if C < ∞, D = ∞,
which generalizes [12, Theorem 4.6] where μ0 = 0 is assumed. (At the end of [12,
Section 4], the authors remark that the case μ0 > 0 may be treated in a way analogous
to the case μ0 = 0, but no explicit result is given.) If C = ∞ and D < ∞, we must
have L∞ = ∞, and hence, by (19), P(Tn0 < ∞) = 1. So (22) implies
nl→im∞ E[esTn0 ] =
∞
4.1 Proofs of Propositions 1–3
The conclusions regarding C and D in the Propositions 1, 2 and 3 are given already
in (
6
), while the statements (i) in Propositions 2 and 3 are implied by (
11
). The
other statements follow from results in [17], where two cases—corresponding in the
setting at hand to μ0 = 0 and μ0 > 0—are considered simultaneously by means of a
duality relation involving polynomials Rn and Rn∗. The asymptotic results for Rn may
be translated into asymptotics for Qn if μ0 = 0, while the results for R∗, suitably
n
interpreted, give asymptotics for Qn if μ0 > 0, and for Q(nl) with l ≥ 1. Concretely, the
statements in Proposition 1, Proposition 2 (ii) and Proposition 3 (ii) regarding the case
x < 0 follow from [17, Lemma 2.4 and Theorems 3.1 and 3.3], while the results for x >
0 are implied by [17, Theorems 2.2, 3.6 and 3.8]. Proposition 2 (iii) follows from [17,
Theorem 3.1] for l = 0 and μ0 = 0, and from [17, Corollary 3.2] for l = 0 and μ0 > 0,
and for l ≥ 1. Proposition 3 (iii) is implied by [17, Theorems 2.2, 3.3 and 3.4 (ii)],
while Proposition 3 (iv) is a consequence of [17, Corollary 3.2]. Finally, the fact that
σ = 0 in the setting of Proposition 1 is stated, for example, in [17, Theorem 2.2 (iv)].
4.2 Proof of Theorem 1
As observed already, substitution in (
2
) of Karlin and McGregor’s formula for Pˆi j (s)
given on [14, Equation (3.21)] leads to (
3
) and hence, by analytic continuation, to (
16
).
To obtain (
17
) we note that [15, Equation (3.21)] also yields
1 1
Pˆ10(s) = − λ0 + Q1(s) Pˆ00(s) = λ0 (λ0 + μ0 − s) Pˆ00(s) − 1 ,
which upon substitution in (
2
) leads to
Moreover, by Karlin and McGregor’s representation formula for the transition
probabilities Pi j (t ) (see [14, Section III.6]) we have P00(t ) = 0∞ e−xt ψ (dx ), where ψ
is a (probability) measure with respect to which the polynomials Qn are orthogonal.
Since the condition C + D = ∞ is equivalent to (
1
), it ensures that the transition
probabilities are uniquely determined by the birth and death rates, whence ψ must be
the natural measure (see [14]). So we have
Subsequently applying (
15
) with l = 0, it follows that
0
0
Pˆ00(s) =
and, by analytic continuation,
Since Tn0 = Tn,n−1 + · · · + T10, while Tn,n−1, . . . , T10 are independent random
variables, Markov’s theorem (
14
) implies that we can write
E[esTn0 I{Tn0<∞}] = E[esTn,n−1 I{Tn,n−1<∞}] · · · E[esT10 I{T10<∞}]
μ1 · · · μn Q(Nn−+n1)(s)
= λ1 · · · λn · · ·
lim
N →∞ Q(Nn−)n+1(s)
lim
N →∞ Q(N1)(s)
Q(N2−)1(s)
λ0
lim
= λnπn N →∞
.
Recalling (
12
), we conclude that this expression holds for s < ξ1(
1
).
(23)
(24)
(25)
4.3 Proof of Theorem 2
Letting n → ∞ in (
16
) and applying the results of Propositions 1, 2 and 3 readily
yields the first statement of Theorem 2.
To prove the second statement, we employ Corollary 2. First note that, for a > 0
s
and s ≤ 0, we have 1 ≤ 1 − a ≤ e−s/a , so that, for l ≥ 0,
1 ≤
∞
i=1
provided ξ1(l) > 0. Defining C (l) and D(l) in analogy to (
5
) it is easily seen that
C < ∞
⇐⇒
C (l) < ∞,
D < ∞
⇐⇒
D(l) < ∞.
So, assuming C < ∞ or D < ∞, we have, by (
10
),
Hence σ (l) = σ = ∞, so that, by (
13
), ξi(l) → ∞ as l → ∞. As a consequence
∞ 1
i=1 ξi(l) < ∞, l ≥ 1.
∞
lim
l→∞ i=1
s
1 − ξ (l)
i
= 1, s ≤ 0,
and the result follows since L∞ < ∞ if C < ∞, whereas L∞ = ∞ if C = ∞ and
D < ∞.
5 Concluding Remarks
First we note that the result (24)—or rather a generalization of (24)—may be derived
directly from the Kolmogorov differential equations and Karlin and McGregor’s
representation formula for the transition probabilities Pi j (t ). The argument is given on
[8, p 508] (and essentially already on [15, p 385]) and yields
P(t < Tn0 < ∞) = μ1
0
∞ e−xt
x
Q(n1−)1(x )ψ (
1
)(dx ), n ≥ 1,
Fˆn0(s) = μ1
0
∞ Q(n1−)1(x ) ψ (
1
)(dx ), s < 0, n ≥ 1.
x − s
(26)
Note that as a consequence of (
17
) and (26), we have, for all m ≥ 0,
0
∞ Q(m1)(x ) ψ (
1
)(dx ) = λm+1πm+1 n→∞
π1 lim
x − s
which implies a partial extension of Markov’s theorem (
14
) to the effect that, for
m ≥ 0 and l ≥ 1 (and l = 0 if μ0 > 0),
0
∞ Q(ml)(x ) ψ (l)(dx ) = λm+l πm+l n→∞
πl lim
x − s
.
(27)
Using (
14
), (
15
), and the recurrence relation for the polynomials Q(nl), it may be
shown by induction that (27) is actually valid for all l ≥ 0, m ≥ 0 and Re(s) < ξ1(l).
Substitution s = 0 in (27) and (
11
) leads in particular to
0
∞ Qm (x )
x
1
ψ (dx ) = λm πm n→∞
lim
Q(nm−+m1−)1(0)
Qn(0)
L∞ − Lm−1 ,
= 1 + μ0 L∞
which is consistent with [15, Equations (9.9) and (9.14)] (also when ξ1 = 0).
If we do not impose the condition C + D = ∞, the birth and death rates do not
necessarily determine a birth–death process uniquely. However, as observed in [12],
several results remain valid if C + D < ∞, provided they are interpreted as properties
of the minimal process, which is the process with an absorbing boundary at infinity
(and which is always associated with the natural measure for the polynomials Qn,
see [7]). Concretely, if C + D < ∞ the arguments leading to Theorem 1, and hence
Theorem 1 itself and Corollary 1, remain valid. Moreover, the results in [17] imply
that, for l ≥ 0,
C + D < ∞
⇒ nl→im∞ Q(nl)(0) =
,
an entire function with simple, positive zeros ξi(l), i ≥ 1. (Note that this complements
Propositions 1—3.) Hence, also (20) remains valid. Finally, letting n → ∞ in (
16
)
and (20), we readily conclude that the results in Theorem 2 for C < ∞, D = ∞ are
actually valid for C + D < ∞ as well.
In the setting C + D < ∞, Gong et al. [12] pay attention also to the maximal
process, the process that is characterized by a reflecting barrier at infinity. In this
case, the measure featuring in the representation for P00(t ), and hence in (23), is not
the natural measure. Although, applying the results of [7], the relevant measure can
be identified and expressed in terms of a natural measure corresponding to a dual
birth–death process, application of Markov’s theorem does not seem feasible in this
case.
Our final remark is the following. Choosing l = 0, letting s ↑ ξ1 in (
15
), and using
the recurrence relation (
4
), we readily get
0
In fact, using (
15
) again, it is not difficult to generalize this result to
which upon substitution in (25) leads to
0
0
(Since, by [9, Theorem 3.1], the condition above is equivalent to ξ1-recurrence of the
process, this result may also be obtained by applying [13, Lemma 3.3.3 (iii)] to the
setting at hand, see [11, Lemma 3.2].) It now follows from (22) that
C = ∞, D < ∞
⇒
If μ0 = 0 then ξ1 = 0, so the result does not take us by surprise, but for μ0 > 0 we
regain an interesting extension of Proposition 3 (iv)—recently obtained by Gao and
Mao [11, Lemma 3.4]—since it has consequences for the existence of quasi-stationary
distributions.
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0
International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution,
and reproduction in any medium, provided you give appropriate credit to the original author(s) and the
source, provide a link to the Creative Commons license, and indicate if changes were made.
1. Anderson , W.J. : Continuous-Time Markov Chains. Springer, New York ( 1991 )
2. Berg , C. : Markov's theorem revisited . J. Approx. Theory 78 , 260 - 275 ( 1994 )
3. Chihara , T.S.: An Introduction to Orthogonal Polynomials . Gordon & Breach, New York ( 1978 )
4. Diaconis , P. , Miclo , L. : On times to quasi-stationarity for birth and death processes . J. Theor. Probab . 22 , 558 - 586 ( 2009 )
5. van Doorn, E.A. : On oscillation properties and the interval of orthogonality of orthogonal polynomials . SIAM J. Math. Anal . 15 , 1031 - 1042 ( 1984 )
6. van Doorn, E.A. : On orthogonal polynomials with positive zeros and the associated kernel polynomials . J. Math. Anal. Appl . 86 , 441 - 450 ( 1986 )
7. van Doorn, E.A. : The indeterminate rate problem for birth-death processes . Pac. J. Math. 130 , 379 - 393 ( 1987 )
8. van Doorn, E.A. : On associated polynomials and decay rates for birth-death processes . J. Math. Anal. Appl . 278 , 500 - 511 ( 2003 )
9. van Doorn, E.A. : On the α-classification of birth-death and quasi-birth-death processes . Stoch. Models 22 , 411 - 421 ( 2006 )
10. van Doorn , E.A. : Representations for the decay parameter of a birth-death process based on the Courant-Fischer theorem . J. Appl. Probab . 52 , 278 - 289 ( 2015 )
11. Gao , W.-J., Mao , Y.-H.: Quasi-stationary distribution for the birth-death process with exit boundary . J. Math. Anal. Appl . 427 , 114 - 125 ( 2015 )
12. Gong , Y. , Mao , Y.-H. , Zhang, C. : Hitting time distributions for denumerable birth and death processes . J. Theor. Probab . 25 , 950 - 980 ( 2012 )
13. Jacka , S.D. , Roberts , G.O. : Weak convergence of conditioned processes on a countable state space . J. Appl. Probab . 32 , 902 - 916 ( 1995 )
14. Karlin , S. , McGregor , J.L. : The differential equations of birth-and-death processes, and the Stieltjes moment problem . Trans. Am. Math. Soc . 86 , 489 - 546 ( 1957 )
15. Karlin , S. , McGregor , J.L. : The classification of birth and death processes . Trans. Am. Math. Soc . 86 , 366 - 400 ( 1957 )
16. Karlin , S. , McGregor , J.L.: A characterization of birth and death processes . Proc. Nat. Acad. Sci . 45 , 375 - 379 ( 1959 )
17. Kijima , M., van Doorn , E.A. : Weighted sums of orthogonal polynomials with positive zeros . J. Comput. Appl . Math. 65 , 195 - 206 ( 1995 )