- Home
- Questions
- IT461 Assignment 3 Markov Chains

# IT461 Assignment 3 Markov Chains

- Simulate the Land of Oz example given in class for 300 days.
- Simulate the stepping stone model of Example 11
*.*2 of Laurie Snell book with *n *= 10 for two colors. Observe what happen’s at after a long time. Can you explain why this is happening?
- Simulate the drunkard’s walk with lattice points {0
*,*1*,*2*,*3*,*4} with 0 and 4 being the absorbing states. Start the random walk in state 1. Estimate i)The expected number of times that the drunkard is in state 3 ii)The expected time to absorbtion iii)Probability that the chain will be absorbed in state 4 . Compare with the theoretical formula’s in section 11*.*2
- Simulate the rat’s maze example (Example 11.22) and keep track of the number of times that the rat is in state
*j *for *j *= 1*,*2*,..,* Compare the empirical stationary distribution of this Markov chain to the theoretical one.
- A discrete time queueing system of capacity
*n *= 50 consists of the person being served and those waiting to be served. The queue length *x *is observed each second. If 0 *< x < n*, then with probability *p *= 0*.*4, the queue size is increased by one by an arrival and, inependently, with probability *r *= 0*.*5, it is decreased by one because the person being served finishes service. If *x *= 0, only an arrival (with probability p) is possible. If *x *= 50, an arrival will depart without waiting for service, and so only the departure (with probability r)of the person being served is possible. This is a Markov chain with states given by the number of customers in the queue. Write a computer program to simulate the queue. Have your program keep track of the proportion of the time that the queue length is *j *for *j *= 0*,*1*,...,n *and the average queue length. Can you estimate the stationary vector ? Test what happens when *p *= 0*.*5 and *r *= 0*.*4?