Question

Markov Chains

Answer 1

Calculate \[P_{11}(n)=P(X_n=1|X_0=1)\] where the transition matrix is of the form: \[\left[\begin{matrix}0 & 1 &0 \\ 0 & \dfrac{1}{2} & \dfrac{1}{2} \\ \dfrac{1}{2} & 0 & \dfrac{1}{2}\end{matrix}\right]\] I have gotten to the statement: \[P_{11}(n)=C_1 + \bar{C_2}\bigg(\dfrac{1}{2}\bigg)^ncos\bigg(\dfrac{n \pi}{2}\bigg) + \bar{C_3}\bigg(\dfrac{1}{2}\bigg)^nsin\bigg(\dfrac{n \pi}{2}\bigg)\]
How do I then find the values of my constants?

Thanks for any help

Answer 2

Oh dear it's been a while since I have done Markov chains... Can you explain how you got that equation?

Answer 3

okay so I worked out my eigenvalues of the matrix, and got:\[\lambda = 1, \pm \dfrac{i}{2}\] then \[P_n\] should have the form: \[P_n=C_1(\lambda _ 1)^n + C_2\bigg(\dfrac{i}{2}\bigg)^n + C_3\bigg(\dfrac{-i}{2}\bigg)^n \] (apparently)
Then as we want P_{11}(n) to be real and we know that: \[\bigg(\pm\dfrac{i}{2}\bigg)^n=\bigg(\dfrac{1}{2}\bigg)^n \bigg(\cos\bigg(\dfrac{n \pi}{2} \bigg) \pm i \sin\bigg(\dfrac{n \pi}{2} \bigg) \bigg)\] so I get the equation as above.

Answer 4

Hm can you use the eigendecomposition
\(\large P^n=U\Sigma^n U^{-1}\)

where \(\large \Sigma^n\) is the n-th power of the diagonal matrix of eigenvalues, and
\(\large U\) is the matrix of eigenvectors where the \(i\)-th each column is an eigenvector of \(P\)

Answer 5

I have used that before, but in this particular example it isn't used.
http://keats.kcl.ac.uk/pluginfile.php/794150/mod_resource/content/1/solution%2033.pdf

can you see that (I'm not sure if you have to log in) If you can it's the first question. What I'm struggling with is finding the constants.
He uses the fact that P(0)=1, P(1)=0 and P(2)=0 but I just don't see where he got those values from?

Answer 6

Hmm I'm not sure.. wouldn't \(P_{11}(1)=1/2\) from the transition matrix above o.o

Answer 7

I would guess though that you can find the 2-step transition matrix by just multiplying P with itself. and \[ P_{ij}(0)=\begin{cases} 1&&j=k\\
0&&\text{otherwise}\end{cases}\]

Answer 8

oops if \(i = j\)

Answer 9

I think that it comes from the top left corner of the matrix. Since it's a stochastic matrix we just keep multiplying this matrix by itself multiple times to get different values. So if we raise P to the 0 power we get the identity matrix, hence P(0)=1. Then P(1)=0 and P(2)=0 as well after you square the matrix. Those are all the constants you need so you're good. =)