Homework 8: Reinforcement Learning - Machine Learning | CS 410, Assignments of Computer Science

Download Homework 8: Reinforcement Learning - Machine Learning | CS 410 and more Assignments Computer Science in PDF only on Docsity! CS 410/510 Machine Learning Winter, 2006 Homework 8: Reinforcement Learning Due Thursday, March 16. 1. (Adapted From T. M. Mitchell, Machine Learning, problem 13.2.) Consider the determin- istic grid world shown below with the absorbing goal state G. Here the immediate rewards are 10 for the labeled transitions and 0 for all unlabeled transitions. Let the discounting factor γ = 0.8. (a) Give the V ∗ value for every state in this grid world. (b) Give the Q(s, a) value for every transition in this grid world. (c) Show an optimal policy π∗—i.e., the optimal action from each state. (d) Now consider applying the Q-learning algorithm to this grid world, assuming the table of Q̂ values is initialized to zeros. Assume the agent begins in the bottom left grid square and then travels clockwise around the perimeter of the grid until it reaches the absorbing goal state, completing the first training episode. Describe which Q̂ values are modified as a result of this episode, and give their revised values. Answer the question again assuming the agent now performs a second identical episode. Answer it for a third episode. G 10 1010 2. Textbook, problem 13.3. 1