Question

markov chain
Suppose that a certain queue may contain 0, 1 or 2 items. It is not possible for it to contain 3 or more items. At each time step, one of two things can happen:
(i): with a probability 1/3, one item is removed from the queue, if there is an item to remove, otherwise nothing happens
(ii) with probability 2/3, one item is added to the queue, if there is enough space (i.e if there are not already 2 items in the queue) (otherwise nothing happens).
I need to helps with 3 parts of the question
(a) Formulate a finite markov chain that describes the system.

Answer 1

would the matrix be
[1/3 2/3 0
1/3 0 2/3
0 1/3 1/3]

Answer 2

@phi please help, would really appreciate it

Answer 3

You need to define the states. 0 , 1 or 2 items on the queue. so draw 3 "bubbles"

Answer 4

your matrix does make sense. each row adds up to 1 (so all possibile transitions out of a state are covered)