Algorithms Flow Chart in Robotics - Final Exam | N 1, Assignments of Health sciences

Download Algorithms Flow Chart in Robotics - Final Exam | N 1 and more Assignments Health sciences in PDF only on Docsity! HW: 4 Course: M362M - Introduction to Stochastic Processes I Page: 1 of 4 University of Texas at Austin HW Assignment 4 Note: In all the problems below, let {Xn}n∈N0 be a simple random walk with increments {ξn}n∈N, i.e., Xn = n∑ k=1 ξk, n ∈ N, X0 = 0, where, p = P[ξk = 1] = 1− P[ξk = −1]. Problem 4.1. Which of the following random times are stopping times? (Find the decision functions, or explain clearly why they are not stopping times.) (1) T is first time the average Yn = 1n ∑n k=1 Xk, n ∈ N, Y0 = 0 hits the level l = 1/2. (2) T is third time the random walk {Xn}n∈N0 visits the level l ∈ N0. (3) T = min {n ≥ 2 : Xn −Xn−2 ≥ 2Xn−1} . (4) T = X3 + 3. (5) T = X3 + 6. Solution: (1) Yes, you can write the decision functions in the following way Gn(x0, x1, . . . , xn) = { 1, 1n ∑n k=1 xk = 1 2 , and 1 m ∑m k=1 xk 6= 1 2 , for m < n, 0, otherwise. (2) Yes, Gn(x0, x1, . . . , xn) = { 1, xn = l and exactly 2 of x0, x1, . . . , xn−1 are equal to l, 0, otherwise. (3) Yes, Gn(x0, x1, . . . , xn) = { 1, n ≥ 2, xn − xn−2 ≥ xn−1 and xm − xm−2 < xm−1, for all 2 ≤ m < n, 0, otherwise. (4) This is not a stopping time. If it were, there would exist a (decision) function G0(x0) such that G0(X0) = 1{T=0}. However, T = 0 if and only if X3 = −3, i.e., 1{T=0} = 1{X3=−3}, which is a function of X3 - a quantity unknown at time t = 0. (5) This is a stopping time. The only values T can take are 3, 5, 7 and 9, so Gn(x0, . . . , xn) = 0 if n 6∈ {3, 5, 7, 9}. If n ∈ {3, 5, 7, 9} then Gn(x0, x1, x2, x3, . . . , xn) = { 1, x3 = n− 6, 0, otherwise. Problem 4.2. Let N be a random time, independent of the sequence {ξn}n∈N0 where {ξn}n∈N0 is a sequence of mutually independent Bernoulli ({0, 1}-valued) random variables with parameter pB ∈ (0, 1). Suppose that N has a geometric distribution g(pg) with parameter pg ∈ (0, 1). Compute the distribution of the random sum Y = N∑ k=1 ξk, i.e., find P[Y = i], for i ∈ N0. (Note: You can think of Y as a binomial random variable with “random n”.) Solution: Independence between N and {ξn}n∈N allows us a to use a result from class which states that the generating function PY (s) of Y is given by PY (s) = PN (PB(s)), Instructor: Gordan Žitković Semester: Spring 2008 HW: 4 Course: M362M - Introduction to Stochastic Processes I Page: 2 of 4 where PN (s) = pg 1−qgs is the generating function of N (geometric distribution) and PB(s) = qB + pBs is the generating function of each ξk (Bernoulli distribution). Therefore, PY (s) = pg 1− qg(qb + pBs) = pg 1−qgqB 1− qgpB1−qgqB s = pY 1− qY s , where pY = pg 1− qgqB and qY = 1− pY . PY can be recognized as the generating function of a geometric random variable with parameter pY , so P[Y = i] = qy(py)i−1. Problem 4.3. Let Wn = x + Xn be the “wealth” at time n of a gambler who starts with x ∈ N dollars and repeatedly plays a simple game in which he/she wins a dollar with probability p and loses a dollar with probability q = 1− p (Gambler’s ruin problem). The game is over when Wn hits either a or 0 - whatever comes first. Let p0(x) be the probability that 0 is reached before a, and pa(x) = 1− p0(x) the probability that a gets hit first. The dependence on x will be important so we emphasise it notationally (pa(x) instead of just pa). We have used Wald’s identity in class to show that if p = q = 12 , then pa(x) = x/a. The purpose of this problem is to determine pa for general p ∈ (0, 1). You can freely suppose that p > q (we give the game a bit of a positive spin) and that 0 ≤ x ≤ a. (1) For x = a or x = 0, pa(x) is known. What is it? (2) Suppose now that x 6= a and x 6= 0. Explain why the following relationship pa(x) = q pa(x− 1) + p pa(x + 1) (4.1) should hold between pa(x) and its “neighbors” pa(x− 1) and pa(x + 1), for x = 1, . . . , a− 1. (3) Determine two different constants A1 and A2 such that both pa(x) = (A1)x and pa(x) = (A2)x solve (4.1). (4) Find constants C1 and C2 such that pa(x) = C1(A1)x + C2(A2)x solves (4.1) and the “boundary” conditions from part (1) above. That is your pa(x)! (5) Use Mathematica to sketch the graph of pa(x), x = 0, . . . , a for some (realistic) values of a ∈ N and p > 12 of your choice. Solution: (1) pa(0) = 0 and pa(a) = 1. If you start bankrupt, there is no chance of reaching a. If you start at a you are already there ! (2) Let us apply the law of total proabability to the first step: the value of W1 can be x − 1 with probability q and x + 1 with probability q. So, if we define the event A by A = {a is reached before 0}, we get pa(x) = P[A] = P[A|W1 = x + 1]P[W1 = x + 1] + P[A|W1 = x− 1]P[W1 = x− 1] (4.2) Once W1 = x + 1 or W1 = x − 1 is reached, the conditional probabilities P[A|W1 = x + 1] and P[A|W1 = x− 1] are equal to pa(x + 1) and pa(x− 1) respectively. Indeed, we can imagine that the gambler with the initial wealth x + 1 (x − 1) has just started playing the game. Therefore, since P[W1 = x + 1] = p and P[W1 = x− 1] = q, we have pa(x) = qpa(x− 1) + ppa(x + 1), x = 1, 2, . . . , a− 1. (3) If we plug pa(x) = Ax, A > 0 into (4.1), we get Ax = qAx−1 + pAx+1. Dividing through by Ax−1 > 0 yields A = q + pA2, which is a quadratic equation for A. Two solutions are A1 = 1 and A2 = qp < 1, as you can easily check. Instructor: Gordan Žitković Semester: Spring 2008