New York Journal of Mathematics

New York J. Math. 1 (1995) 120–129.

Characterizations of Embeddable 3×3 Stochastic Matrices with a Negative Eigenvalue

Philippe Carette

Abstract. The problem of identifying a stochastic matrix as a transition matrix between two fixed times, say t = 0 and t = 1, of a continuous-time and finite-state Markov chain has been shown to have practical importance, especially in the area of stochastic models applied to social phenomena. The embedding problem of finite Markov chains, as it is called, comes down to investigating whether the stochastic matrix can be expressed as the exponential of some matrix with row sums equal to zero and nonnegative off-diagonal elements. The aim of this paper is to answer a question left open by S. Johansen (1974), i.e., to characterize those stochastic matrices of order three with an eigenvalue λ < 0 of multiplicity 2.

Contents

1. Introduction 120 2. Notations and preliminaries 122 3. Characterizations of embeddability 123 4. Illustration of the results 128 References 129

1. Introduction

Let the stochastic process (X(t))_{t≥0} on a probability triple (Ω, A, ℙ) be a continuous time Markov chain with a finite number N of states and time-homogeneous transition probabilities

P_{ij}(t) = ℙ[ X(t + u) = j | X(u) = i], i, j = 1, …, N,

i.e., which are independent of the time u ≥ 0. Under the assumption

lim_{t→^{>}0} P_{ii}(t) = 1, i = 1, …, N,

Received February 20, 1995. Mathematics Subject Classification. 60J27,15A51. Key words and phrases. Continuous-time Markov chain, stochastic matrix, matrix exponential,

embedding problem.

\begin{matrix}c \\ ◯ \end{matrix}1995 State University of New York

ISSN 1076-9803/95

the transition probabilities P_{ij}(t) are gathered into stochastic N × N matrices P(t) , t ≥ 0, which satisfy

(1) P(0) = I, the identity matrix,

P(t + s) = P(t)P(s), t, s ≥ 0. (2)

It is a well-known result (see e.g., [Fre71], p. 148) that in this case

(3)  \frac{d}{dt}P(0) = R and P(t) = exp (Rt),  t ≥ 0,

where R is a N × N matrix satisfying

R_{ij} ≥ 0,  for i ≠ j,
(4)  ∑_{j=1}^{N} R_{ij} = 0,  i = 1, …, N.

The (i, j)-th element of the matrix R represents the transition intensity from state i to state j. In general, any matrix R that satisfies (4) will be called an intensity matrix. Thus, the Markov chain is completely determined by its intensity matrix.

Now, an interesting problem emerges as follows. Suppose (Y (t))_{t=0,1,2,...} is a discrete time Markov chain with N states and time-homogeneous transition probability matrix P, i.e., the elements of P

P_{ij} = ℙ[ Y (u + 1) = j | Y (u) = i], i, j = 1, …, N (5)

are independent of the time u = 0, 1, 2, . . .. Then, one can ask the question whether or not this process is a discrete manifestation of an underlying time-homogeneous and continuous N-state Markov chain (X(s))_{s≥0}, or equivalently, under what circumstances P has the form

P = exp R (6)

for some intensity matrix R. If this occurs, we say that {Y (t) | t = 0, 1, 2, … } is embeddable into a continuous and time-homogeneous Markov chain, or briefly that P is embeddable and that R generates P.

This problem has been acknowledged to have practical relevance when mathematical modelling of social phenomena is concerned (see [Bar82, SS77]).

From a purely mathematical point of view however, the question has already been investigated by Kingman ([Kin62]), who published a simple necessary and sufficient condition (due to D. G. Kendall) for the two state case: det P > 0. He gave also necessary and sufficient conditions for embeddability of any N × N stochastic matrix, but they are not applicable in practice. At the time, it was already known that for N ≥ 3 more than one intensity matrix could generate the same P ([Spe67]).

However, for the three state case, some simplifications do also occur. Papers [Joh74, Cut73] have shown that necessary and sufficient conditions were in a critical sense dependent upon the nature of the eigenvalues of P. In both of them such conditions were given, except for the case when P has a negative eigenvalue of multiplicity two.

Since then, the problem has mainly been dealt with in the more general context of time-inhomogeneous Markov chains. (See [FS79, JR79, Fry80, Fry83].)

The purpose of this paper is to fill the gap for 3 × 3 stochastic matrices with a negative eigenvalue of multiplicity 2, by providing a method to check whether or not they are embeddable and giving an explicit form of the possible intensity matrices that generate them (Theorem 3.3). Another interesting characterization involving a lower bound for the negative eigenvalues is contained in Theorem 3.7.

2. Notations and preliminaries

The set of positive integers will be denoted by ℕ, the set of real numbers by ℝ and the set of complex numbers by ℂ. The sets of real (resp. complex) m × n matrices is denoted by ℝ^{m×n} (ℂ^{m×n}) and matrices by bold face capital letters. We shall

use the notation diag(p, q, r) for the diagonal matrix (\begin{matrix}p & 0 & 0 \\ 0 & q & 0 \\ 0 & 0 & r \end{matrix}) , p, q, r ∈ ℂ.

Finally, the (i, j)-th element of the n-th power M^{n} of a square matrix M shall be written as M_{ij}^{(n)}, n = 2, 3, …

It was already remarked in [Joh74] that a negative eigenvalue of an embeddable stochastic matrix has to be of even algebraic multiplicity. Indeed, in our case, if P is a 3 × 3 stochastic matrix with an eigenvalue λ < 0, then R has an eigenvalue θ ∈ ℂ with e^{θ} = λ. But then \overline{θ} must also be an eigenvalue of R (R is real), hence the three distinct eigenvalues of R are 0, θ and \overline{θ} forcing R to be diagonalizable:

R = Udiag(0, θ, \overline{θ})U^{−1}, (7)

where U is a nonsingular 3 × 3-matrix. We have then

P = exp R = Udiag(e^{0}, e^{θ}, e\overline{θ})U^{−1}
= Udiag(1, λ, λ)U^{−1}.

Consequently, λ has algebraic multiplicity 2 and −\frac{1}{2} < λ < 0, because 1 + 2λ = trace P ≥ 0. Now P^{∞} = lim_{n→+∞} P^{n} certainly exists (as is the case for every embeddable stochastic matrix P, see [Kin62]) and

(8) P^{∞} = Udiag(1, 0, 0)U^{−1}, since |λ| < 1.

This leads to the following useful relation between P and P^{∞} (see also [Joh74])

P = P^{∞} + λ(I − P^{∞}).

More can be said about the nature of P^{∞}. Indeed, since P^{∞} is a diagonalizable stochastic matrix with eigenvalues 1 and 0 (the latter of algebraic multiplicity 2), P^{∞} must consist of identical rows, equal to say (p_{1} p_{2} p_{3}). Furthermore, none of the p_{i} can be equal to 0: Otherwise, if p_{i} = 0 for some i, then P_{ii} = λ < 0.

Thus, in seeking characterizations for embeddability of P, we only need to focus our attention on these matrices that have the form P = P(λ, P^{∞}), where

P(λ, P^{∞}) = P^{∞} + λ(I − P^{∞}), (9) with

P^{∞} = (\begin{matrix}1 \\ 1 \\ 1 \end{matrix}) (p_{1} p_{2} p_{3}) ,

p_{1} + p_{2} + p_{3} = 1, p_{i} > 0, i = 1, 2, 3,

and λ < 0, λ ≥ κ := − min_{i} \frac{p_{i}}{1 − p_{i}} , (10)

the last condition being a regularity one ensuring that P(λ, P^{∞}) has nonnegative entries.

3. Characterizations of embeddability

A characterization involving the generating intensity matrix. The idea behind this approach is to seek an explicit form of a general matrix R with exp R = P and then to submit the elements R_{ij} to the constraints (4). We begin with a few properties of matrix square roots of P^{∞} − I, which we shall need in the proof of our main result (Theorem 3.3).

Lemma 3.1. Let X be a real square matrix with X^{2} = P^{∞} − I. Then

(a) XP^{∞} = P^{∞}X = 0

(b) exp((2k + 1)πX) = 2P^{∞} − I (k ∈ ℕ)

(c) ∑_{j} X_{ij} = 0 for all i.

Proof. (a) Using (8), we have

P^{∞} − I = Udiag(0, −1, −1)U^{−1},

and so (see [Gan60])

X = UVdiag(0, i, −i)V^{−1}U^{−1},

where V is a real nonsingular matrix that commutes with diag(0, −1, −1). But then V commutes also with diag(1, 0, 0), and

P^{∞}X = UVdiag(1, 0, 0)V^{−1}U^{−1}UVdiag(0, i, −i)V^{−1}U^{−1}
= UV 0 V^{−1}U^{−1}
= 0.

Analogously XP^{∞} = 0.

(b) By X = UVdiag(0, i, −i)V^{−1}U^{−1}, we get

exp((2k + 1)πX) = UVdiag(1, e^{(2k+1)πi}, e^{−(2k+1)πi})V^{−1}U^{−1}
= UVdiag(1, −1, −1)V^{−1}U^{−1}
= 2 UVdiag(1, 0, 0)V^{−1}U^{−1} − UVdiag(1, 1, 1)V^{−1}U^{−1}
= 2P^{∞} − I.

(c) By (a), we have XP^{∞} (\begin{matrix}1 \\ 1 \\ 1 \end{matrix}) = (\begin{matrix}0 \\ 0 \\ 0 \end{matrix}). Hence

X (\begin{matrix}1 \\ 1 \\ 1 \end{matrix}) = (\begin{matrix}0 \\ 0 \\ 0 \end{matrix}) .

This concludes the proof.

Corollary 3.2. If R = ln |λ|(I − P^{∞}) + (2k + 1)πX with X^{2} = P^{∞} − I and k ∈ ℕ, then

exp R = P^{∞} + λ(I − P^{∞}).

Proof. Statement (a) of Lemma 3.1 implies that X commutes with I − P^{∞}, in which case we have

exp R = exp(ln |λ|(I − P^{∞})) · exp((2k + 1)πX).

We now calculate exp(ln |λ|(I − P^{∞})) and use the same notations as in the proof of (a), Lemma 3.1:

exp(ln |λ|(I − P^{∞})) = exp(Udiag(0, ln |λ|, ln |λ|)U^{−1})
= Udiag(1, |λ|, |λ|)U^{−1}
= (1 + λ)Udiag(1, 0, 0)U^{−1} − λUdiag(1, 1, 1)U^{−1}
= (1 + λ)P^{∞} − λI.

Hence

exp R = ((1 + λ)P^{∞} − λI)(2 P^{∞} − I)
= (1 − λ)P^{∞} + λI,

as (P^{∞})^{2} = P^{∞}.

Theorem 3.3. Let P(λ, P^{∞}) be as in (9) and (10). Then the following conditions are equivalent:

(a) P(λ, P^{∞}) is embeddable.

(b) There exist k ∈ ℕ and X ∈ ℝ^{3×3} such that X^{2} = P^{∞} − I and

R = ln |λ|(I − P^{∞}) + (2k + 1)πX is an intensity matrix.

(c) There exists X ∈ ℝ^{3×3} such that X^{2} = P^{∞} − I and

X_{ij} ≥ \frac{1}{π} ln |λ| p_{j}, i ≠ j.

Proof. (a ⇒ b) Let P = exp R, where R is an intensity matrix. As remarked in the preliminaries section, the eigenvalues of R are in this case 0, θ_{k}, and \overline{θ_{k}}, where θ_{k} is a complex number satisfying e^{θ_{k}} = λ, i.e.,

θ_{k} = ln |λ| + (2k + 1)πi (k ∈ ℕ)

and

R = Udiag(0, θ_{k}, \overline{θ_{k}})U^{−1} with U nonsingular

(11) = ln |λ|Udiag(0, 1, 1)U^{−1} + (2k + 1)πi Udiag(0, 1, −1)U^{−1}.

Using (8) and Equation (11), and putting X = i Udiag(0, 1, −1)U^{−1}, we obtain the desired result. (b ⇒ c) Because R is an intensity matrix, we must have R_{ij} ≥ 0, i ≠ j. This means that, by (b), for i ≠ j

ln |λ|(−p_{j}) + (2k + 1)πX_{ij} ≥ 0,

which yields

X_{ij} ≥ \frac{1}{(2k + 1)π} ln |λ|p_{j} ≥ \frac{1}{π} ln |λ|p_{j}.

(c ⇒ a) In this case, by Lemma 3.1 (b) and (c), ln |λ|(I − P^{∞}) + πX defines an intensity matrix, say R, which has the property exp R = P^{∞} + λ(I − P^{∞}) = P(λ, P^{∞}). The proof is now complete.

A connection with embeddable stochastic matrices with complex conjugated eigenvalues. Another approach can be given starting from the following observation, which is valid for all stochastic matrices.

Lemma 3.4. A stochastic matrix P is embeddable if and only if it is the square of a stochastic matrix Q that is also embeddable.

Proof. If P is embeddable, then an intensity matrix R exists with exp R = P. Now, \frac{1}{2}R is also an intensity matrix, hence the matrix Q = exp \frac{1}{2}R is stochastic (see [Fre71], p. 125), embeddable, and has the property Q^{2} = P.

Conversely, suppose that an embeddable stochastic matrix Q exists with Q^{2} = P. Then Q = exp R for some intensity matrix R and

P = Q^{2} = (exp R)^{2} = exp(2 R),

so P is embeddable.

This is interesting, because any stochastic ‘square root’ Q of P(λ, P^{∞}) must have eigenvalues 1, i\sqrt{|λ|} and −i\sqrt{|λ|}. The following characterization of embeddable stochastic matrices of order three with complex conjugate eigenvalues, has been proven in [Joh74].

Theorem 3.5. [Joh74]. A stochastic matrix P of order three with eigenvalues e^{α+iβ} and e^{α−iβ}, 0 < β < π, can be embedded if and only if one of the following conditions holds.

(12) P_{ij}^{(2)}(β(e^{α} cos β − 1) − αe^{α} sin β)

≥ P_{ij}(β(e^{2α} cos 2β − 1) − αe^{2α} sin 2β) for all i ≠ j

(13) P_{ij}^{(2)}((β − 2π)(e^{α} cos β − 1) − αe^{α} sin β)

≥ P_{ij}((β − 2π)(e^{2α} cos 2β − 1) − αe^{2α} sin 2β) for all i ≠ j

A direct application of this result now yields the following characterization.

Theorem 3.6. P(λ, P^{∞}) is embeddable if and only if there exists a stochastic matrix Q of order three such that Q^{2} = P(λ, P^{∞}) and one of the following conditions holds.

(14)  Q_{ij} ≥ (1 + \frac{\sqrt{|λ|}}{π} ln |λ|)p_{i}  for all i ≠ j

(15)  Q_{ij} ≤ (1 − \frac{\sqrt{|λ|}}{3π} ln |λ|)p_{i}  for all i ≠ j

Proof. In view of Theorem 3.5, Lemma 3.4 and the observations made about the eigenvalues of any stochastic square root Q of P(λ, P^{∞}), we have, with the notations used in Theorem 3.5, α = ln \sqrt{|λ|}, β = \frac{π}{2} , and Q_{ij}^{(2)} = (1−λ)p_{i}. Inequality (12) then becomes

Q_{ij} ≥ \frac{1 + ^{\frac{\sqrt{|λ|}}{π}} ln |λ|}{1 + |λ|}(1 − λ)p_{i} = (1 + \frac{\sqrt{|λ|}}{π} ln |λ|)p_{i}, i ≠ j,

and inequality (13)

Q_{ij} ≤ \frac{1 − ^{\frac{\sqrt{|λ|}}{3π}} ln |λ|}{1 + |λ|}(1 − λ)p_{i} = (1 − \frac{\sqrt{|λ|}}{3π} ln |λ|)p_{i}, i ≠ j.

It may be interesting for the sake of consistency to remark the equivalence of the characterizations in Theorems 3.3 and 3.6. Suppose (14) holds. Then for i ≠ j,

Q_{ij} ≥ (1 + \frac{\sqrt{|λ|}}{π} ln |λ|)p_{i}
⇕
\frac{1}{\sqrt{|λ|}}(Q_{ij} − p_{i}) ≥ \frac{1}{π} ln |λ|p_{i}.

Consequently, the matrix X defined as

X = \frac{1}{\sqrt{|λ|}}(Q − P^{∞}) (16)

is the one that satisfies (c) of Theorem 3.3, for it has also the property X^{2} = P^{∞}−I, which remains to be shown:

X^{2} = \frac{1}{|λ|}(Q − P^{∞})^{2}

= \frac{1}{|λ|}(Q^{2} − 2QP^{∞} + (P^{∞})^{2}) (Q and P^{∞} commute)

= \frac{1}{|λ|}(P(λ, P^{∞}) − 2P^{∞} + P^{∞})

= \frac{1}{|λ|}(P(λ, P^{∞}) − P^{∞})

= P^{∞} − I by (9) and λ < 0.

If (15) holds, however, we’ll have to put

X = −\frac{1}{\sqrt{|λ|}}(Q − P^{∞})

to arrive at the same conclusion.

On the other hand, one can easily show using the power series definition

exp(A) = ∑_{p=0}^{∞} \frac{1}{p!}A^{p}

that, starting from (c) Theorem 3.3, the relation (16) defines a matrix Q that is stochastic since it can be written as the exponential of an intensity matrix ^{1} :

Q = \sqrt{|λ|}X + P^{∞} = exp(\frac{1}{2}(ln |λ|(I − P^{∞}) + πX)).

Furthermore, Q^{2} = P(λ, P^{∞}) by Corollary 3.2. Finally, Q satisfies (14) by its very definition.

A characterization involving a lower bound for the negative eigenvalue. Instead of seeking R, we put the following question. For a fixed P^{∞} that has the property (10), what are the possible values of λ < 0 that make the stochastic matrix P(λ, P^{∞}) embeddable? As a consequence of Theorem 3.3 among other things, the following result emerges. Recall the definition of κ from (10).

Theorem 3.7. Let P^{∞} be defined by (10), then there exists Λ ∈]κ, 0[, depending on P^{∞}, such that

P(λ, P^{∞}) embeddable ⇔ Λ ≤ λ < 0.

Proof. Define the map F : [κ, 0[ → M : λ ↦ P(λ, P^{∞}), where M is the set of 3 × 3 stochastic matrices with positive determinant. Let P be the set of all embeddable stochastic 3 × 3 matrices. We have then by the nature of the condition (c) of Theorem 3.3

λ ∈ F^{−1}(P) ⇒ [λ, 0[ ⊂ F^{−1}(P). (17)

Also, if we take an arbitrary real square root X of P^{∞} and a λ_{0} such that ln |λ_{0}| ≤ min_{i≠j} \frac{πX_{ij}}{p_{j}}, then P(λ_{0}, P^{∞}) is embeddable, so

F^{−1}(P) ≠ ∅. (18)

In addition, it was proven in [Kin62] that P is closed in M, so by continuity of F, we have that F^{−1}(P) is closed in [κ, 0[. Hence, in conjunction with (17) and (18), F^{−1}(P) = [Λ, 0[ , for some Λ ∈ [κ, 0[. We still have to show that Λ ≠ κ. Suppose Λ = κ. Then κ ∈ F^{−1}(P) and P(κ, P^{∞}) is embeddable. But then there exists k such that P_{kk}(κ, P^{∞}) = 0, so we can apply Ornstein’s Theorem ([Chu67], II.5, Theorem 2) which states that whenever a stochastic matrix Q, with Q_{ij} = 0 for some i and j, is embeddable, then Q_{ij}^{(n)} = 0, n ≥ 1 and in particular Q_{ij}^{∞} = 0. Consequently lim_{n→+∞} P_{kk}^{(n)}(κ, P^{∞}) = p_{k} = 0, giving a contradiction.

1The exponential of an intensity matrix is always stochastic. For a proof, see [Fre71], p. 151.

4. Illustration of the results

Let p_{1} = p_{2} = p_{3} = 1/3, then κ = −1/2. For P(λ, P^{∞}) with −1/2 < λ < 0 to be embeddable, there must exist by Theorem 3.3 a matrix X ∈ ℝ^{3×3} satisfying X^{2} = P^{∞} − I such that

X_{ij} ≥ \frac{1}{3}c_{λ}, i ≠ j, with c_{λ} = \frac{ln |λ|}{π}. (19)

By [Gan60], such a matrix X assumes the form

X = X(u, v) = V (\begin{matrix}0 & 0 & 0 \\ 0 & u & −\frac{1+u^{2}}{v} \\ 0 & v & −u \end{matrix} ) V^{−1}, u, v ∈ ℝ, v ≠ 0, (20)

where V is a nonsingular matrix satisfying

P^{∞} − I = Vdiag(0, −1, −1)V^{−1}.

It is easy to check that we can take

V = (\begin{matrix}1 & 1/3 & 1/3 \\ 1 & −1/3 & 0 \\ 1 & 0 & −1/3 \end{matrix}) ,

whence

X(u, v) = \frac{1}{3} (\begin{matrix}−\frac{u^{2}−v^{2}+1}{v} & −\frac{u^{2}+2v^{2}+3uv+1}{v} & \frac{2u^{2}+v^{2}+3uv+2}{v} \\ \frac{u^{2}−uv+1}{v} & \frac{u^{2}+2uv+1}{v} & −\frac{2u^{2}+uv+2}{v} \\ u − v & u + 2v & −2u − v \end{matrix}) . (21)

We shall now show that P(λ, P^{∞}) cannot be embeddable for values of λ with −2/\sqrt{5} < c_{λ} < 0.

Suppose P(λ, P^{∞}) is embeddable. Then according to (19), there exist real numbers u and v, v ≠ 0 such that

X_{ij}(u, v) ≥ \frac{1}{3}c_{λ}, i ≠ j.

Using (21), these conditions become algebraic inequalities in u and v. Among others:

(a) −u + v ≤ −c_{λ} for (i, j) = (3, 1)

(b) u + 2v ≥ c_{λ} for (i, j) = (3, 2)

(c) X_{12}(u, v) + X_{21}(u, v) ≥ 2c_{λ}/3, yielding −2u − v ≥ c_{λ}

(d) |v| ≥ 2c_{λ} + 2\sqrt{1 + c_{λ}^{2}} for (i, j) = (1, 2) and (2, 1) It is a straightforward matter to see that (a), (b), (c) and (d) together imply

2c_{λ} + 2\sqrt{1 + c_{λ}^{2}} ≤ −c_{λ}.

The contradiction lies in the fact that this inequality cannot be satisfied if −2/\sqrt{5} < c_{λ} < 0. Hence for P(λ, P^{∞}) to be embeddable, one must necessarily have c_{λ} ≤ −2/\sqrt{5} or λ ≥ −e^{−2π/\sqrt{5}}. In this case, the lower bound Λ from Theorem 3.7 must satisfy Λ ≥ −e^{−2π/\sqrt{5}}. In fact, Λ = −e^{−π\sqrt{3}}, which is a consequence of the following more general result that provides a formula to express Λ in terms of p_{1}, p_{2} and p_{3}:

(\frac{ln |Λ|}{π})^{−2} = \{ \begin{matrix}4p/s^{2} − 1 & if p_{m}(1 − p_{m}) > s/2 \\ p_{m}/(1 − p_{m}) & otherwise \end{matrix}

Embedding Problem 129

where m is an index such that p_{m} = min_{i} p_{i}, p = p_{1}p_{2}p_{3} and s = p_{1}p_{2} +p_{1}p_{3} +p_{2}p_{3}. The proof uses the characterization in Theorem 3.3 and is very technical, so it will be omitted here. Acknowledgement. The author is indebted to the referee for his suggestions resulting in an enhancement of the manuscript.

References

[Bar82] D. J. Bartholomew, Stochastic Models for Social Processes, third ed., John Wiley & Sons, Chichester, 1982.

[Chu67] K. L. Chung, Markov Chains with Stationary Transition Probabilities, second ed., N. Y., Springer, 1967.

[Cut73] James R. Cuthbert, The logarithm function for finite-state Markov semi-groups, J. London Math. Soc. (2) 6 (1973), 524–532.

[Fre71] David Freedman, Markov Chains, Holden-Day, 1971.

[Fry80] Halina Frydman, The embedding problem for Markov chains with three states, Math. Proc. Camb. Phil. Soc. 87 (1980), 285–294.

[Fry83] Halina Frydman, On a number of Poisson matrices in bang-bang representations for 3×3 embeddable matrices, J. Multivariate Analysis 13(3) (1983), 464–472.

[FS79] Halina Frydman and Burton Singer, Total positivity and the embedding problem for Markov chains, Math. Proc. Camb. Phil. Soc. 86 (1979), 339–344.

[Gan60] F. R. Gantmacher, The Theory of Matrices, vol. one, Chelsea Publishing Company, N. Y., 1960.

[Kin62] J. F. C. Kingman, The imbedding problem for finite Markov chains, Z. Wahrscheinlichkeitstheorie 1 (1962), 14–24.

[JR79] Søren Johansen and Fred L. Ramsey, A bang-bang representation for 3 × 3 embeddable stochastic matrices, Z. Wahrscheinlichkeitstheorie verw. Gebiete 47 (1979), 107–118.

[Joh74] Søren Johansen, Some results on the imbedding problem for finite Markov chains, J. London Math. Soc. (2) 8 (1974), 345–351.

[Spe67] J. M. O. Speakman, Two Markov chains with a common skeleton, Z. Warscheinlichkeitstheorie 7 (1967), 224.

[SS77] Burton Singer and Seymour Spilerman, Fitting stochastic models to longitudinal survey data – some examples in the social sciences, Bull. Int. Statist. Inst. (3) 47 (1977a), 283–300.

Centrum voor Statistiek en Operationeel Onderzoek, Vrije Universiteit Brussel, Belgium

phcarett@vub.ac.be