Game Theory: Self-Interested Agents and Matrix Games, Slides of Game Theory

Download Game Theory: Self-Interested Agents and Matrix Games and more Slides Game Theory in PDF only on Docsity! Self-interested agents What is Game Theory? Example Matrix Games Game Theory Intro Lecture 3 Game Theory Intro Lecture 3, Slide 1 Self-interested agents What is Game Theory? Example Matrix Games Lecture Overview 1 Self-interested agents 2 What is Game Theory? 3 Example Matrix Games Game Theory Intro Lecture 3, Slide 2 Self-interested agents What is Game Theory? Example Matrix Games Why Utility? Why would anyone argue with the idea that an agent’s preferences could be described using a utility function? why should a single-dimensional function be enough to explain preferences over an arbitrarily complicated set of alternatives? Why should an agent’s response to uncertainty be captured purely by the expected value of his utility function? It turns out that the claim that an agent has a utility function is substantive. There’s a famous theorem (von Neumann & Morgenstern, 1944) that derives the existence of a utility function from a more basic preference ordering and axioms on such orderings. see Theorem 3.1.18 in the book, which includes a proof. Game Theory Intro Lecture 3, Slide 4 Self-interested agents What is Game Theory? Example Matrix Games Why Utility? Why would anyone argue with the idea that an agent’s preferences could be described using a utility function? why should a single-dimensional function be enough to explain preferences over an arbitrarily complicated set of alternatives? Why should an agent’s response to uncertainty be captured purely by the expected value of his utility function? It turns out that the claim that an agent has a utility function is substantive. There’s a famous theorem (von Neumann & Morgenstern, 1944) that derives the existence of a utility function from a more basic preference ordering and axioms on such orderings. see Theorem 3.1.18 in the book, which includes a proof. Game Theory Intro Lecture 3, Slide 4 Self-interested agents What is Game Theory? Example Matrix Games Lecture Overview 1 Self-interested agents 2 What is Game Theory? 3 Example Matrix Games Game Theory Intro Lecture 3, Slide 5 Self-interested agents What is Game Theory? Example Matrix Games Non-Cooperative Game Theory What is it? mathematical study of interaction between rational, self-interested agents Why is it called non-cooperative? while it’s most interested in situations where agents’ interests conflict, it’s not restricted to these settings the key is that the individual is the basic modeling unit, and that individuals pursue their own interests cooperative/coalitional game theory has teams as the central unit, rather than agents Game Theory Intro Lecture 3, Slide 6 Self-interested agents What is Game Theory? Example Matrix Games Non-Cooperative Game Theory What is it? mathematical study of interaction between rational, self-interested agents Why is it called non-cooperative? while it’s most interested in situations where agents’ interests conflict, it’s not restricted to these settings the key is that the individual is the basic modeling unit, and that individuals pursue their own interests cooperative/coalitional game theory has teams as the central unit, rather than agents Game Theory Intro Lecture 3, Slide 6 Self-interested agents What is Game Theory? Example Matrix Games TCP Backoff Game Game Theory Consider this situation as a two-player game: both use a correct implementation: both get 1 ms delay one correct, one defective: 4 ms delay for correct, 0 ms for defective both defective: both get a 3 ms delay. Should you send your packets using correctly-implemented TCP (which has a “backoff” mechanism) or using a defective implementation (which doesn’t)? Game Theory Intro Lecture 3, Slide 7 Self-interested agents What is Game Theory? Example Matrix Games TCP Backoff Game Consider this situation as a two-player game: both use a correct implementation: both get 1 ms delay one correct, one defective: 4 ms delay for correct, 0 ms for defective both defective: both get a 3 ms delay. Questions: What action should a player of the game take? Would all users behave the same in this scenario? What global patterns of behaviour should the system designer expect? Under what changes to the delay numbers would behavior be the same? What effect would communication have? Repetitions? (finite? infinite?) Does it matter if I believe that my opponent is rational? Game Theory Intro Lecture 3, Slide 7 Self-interested agents What is Game Theory? Example Matrix Games Defining Games Finite, n-person game: 〈N,A, u〉: N is a finite set of n players, indexed by i A = A1 × . . .×An, where Ai is the action set for player i a ∈ A is an action profile, and so A is the space of action profiles u = 〈u1, . . . , un〉, a utility function for each player, where ui : A 7→ R Writing a 2-player game as a matrix: row player is player 1, column player is player 2 rows are actions a ∈ A1, columns are a′ ∈ A2 cells are outcomes, written as a tuple of utility values for each player Game Theory Intro Lecture 3, Slide 8 Self-interested agents What is Game Theory? Example Matrix Games Games in Matrix Form Here’s the TCP Backoff Game written as a matrix (“normal form”). 56 3 Competition and Coordination: Normal form games when congestion occurs. You have wo possible strategies: C (for using a Correct implementation) and D (for using a Defective one). If both you and your colleague adopt C the your average packet delay is 1ms (millisecond). If you both adopt D the delay is 3ms, because of additional overhead at the network router. Finally, if one of you adopts D and the other adopts C then the D adopter will experience no delay at all, but the C adopter will experience a delay of 4ms. These consequences are shown in Figure 3.1. Your options are the two rows, and your colleague’s options are the columns. In each cell, the first number represents your payoff (or, minus your delay), and the second number represents your colleague’s payoff.1TCP user’s game Prisoner’s dilemma game C D C −1,−1 −4, 0 D 0,−4 −3,−3 Figure 3.1 The TCP user’s (aka the Prisoner’s) Dilemma. Given these options what should you adopt, C or D? Does it depend on what you think your colleague will do? Furthermore, from the perspective of the network opera- tor, what kind of behavior can he expect from the two users? Will any two users behave the same when presented with this scenario? Will the behavior change if the network operator allows the users to communicate with each other before making a decision? Under what changes to the delays would the users’ decisions still be the same? How would the users behave if they have the opportunity to face this same decision with the same counterpart multiple times? Do answers to the above questions depend on how rational the agents are and how they view each other’s rationality? Game theory gives answers to many of these questions. It tells us that any rational user, when presented with this scenario once, will adopt D—regardless of what the other user does. It tells us that allowing the users to communicate beforehand will not change the outcome. It tells us that for perfectly rational agents, the decision will remain the same even if they play multiple times; however, if the number of times that the agents will play this is infinite, or even uncertain, we may see them adopt C. 3.2 Games in normal form The normal form, also known as the strategic or matrix form, is the most familiargame in strategic form game in matrix form representation of strategic interactions in game theory. 1. The term ‘Prisoners’ Dilemma’ for this famous game theoretic situation derives from the original story accompanying the numbers. Imagine the players of the game are two prisoners suspected of a crime rather than network users, that you each can either Confess to the crime or Deny it, and that the absolute values of the numbers represent the length of jail term each of you will get in each scenario. c©Shoham and Leyton-Brown, 2006 Game Theory Intro Lecture 3, Slide 9 Self-interested agents What is Game Theory? Example Matrix Games Games of Pure Competition Players have exactly opposed interests There must be precisely two players (otherwise they can’t have exactly opposed interests) For all action profiles a ∈ A, u1(a) + u2(a) = c for some constant c Special case: zero sum Thus, we only need to store a utility function for one player in a sense, it’s a one-player game Game Theory Intro Lecture 3, Slide 12 Self-interested agents What is Game Theory? Example Matrix Games Matching Pennies One player wants to match; the other wants to mismatch. 3.2 Games in normal form 59 constant-sum games are meaningful primarily in the context of two-player (though not necessarily two-strategy) games. Definition 3.2.3 A normal form game is constant sum if there exists a constant c such that for each strategy profile a ∈ A1 ×A2 it is the case that u1(a) + u2(a) = c. For convenience, when we talk of constant-sum games going forward we will always assume that c = 0, that is, that we have a zero-sum game. If common-payoff games represent situations of pure coordination, zero-sum games represent situations of pure competition; one player’s gain must come at the expense of the other player. As in the case of common-payoff games, we can use an abbreviated matrix form to represent zero-sum games, in which we write only one payoff value in each cell. This value represents the payoff of player 1, and thus the negative of the payoff of player 2. Note, though, that whereas the full matrix representation is unambiguous, when we use the abbreviation we must explicit state whether this matrix represents a common-payoff game or a zero-sum one. A classical example of a zero-sum game is the game of matching pennies. In this matching pennies gamegame, each of the two players has a penny, and independently chooses to display either heads or tails. The two players then compare their pennies. If they are the same then player 1 pockets both, and otherwise player 2 pockets them. The payoff matrix is shown in Figure 3.5. Heads Tails Heads 1 −1 Tails −1 1 Figure 3.5 Matching Pennies game. The popular children’s game of Rock, Paper, Scissors, also known as Rochambeau, Rock, Paper, Scissors, or Rochambeau game provides a three-strategy generalization of the matching-pennies game. The payoff matrix of this zero-sum game is shown in Figure 3.6. In this game, each of the two players can choose either Rock, Paper, or Scissors. If both players choose the same action, there is no winner, and the utilities are zero. Otherwise, each of the actions wins over one of the other actions, and loses to the other remaining action. In general, games tend to include elements of both coordination and competition. Prisoner’s Dilemma does, although in a rather paradoxical way. Here is another well- known game that includes both elements. In this game, called Battle of the Sexes, a husband and wife wish to go to the movies, and they can select among two movies: “Violence Galore (VG)” and “Gentle Love (GL)”. They much prefer to go together rather than to separate movies, but while the wife prefers VG the husband prefers GL. The payoff matrix is shown in Figure 3.7. We will return to this game shortly. Multi Agent Systems, draft of February 11, 2006 Play this game with someone near you, repeating five times. Game Theory Intro Lecture 3, Slide 13 Self-interested agents What is Game Theory? Example Matrix Games Matching Pennies One player wants to match; the other wants to mismatch. 3.2 Games in normal form 59 constant-sum games are meaningful primarily in the context of two-player (though not necessarily two-strategy) games. Definition 3.2.3 A normal form game is constant sum if there exists a constant c such that for each strategy profile a ∈ A1 ×A2 it is the case that u1(a) + u2(a) = c. For convenience, when we talk of constant-sum games going forward we will always assume that c = 0, that is, that we have a zero-sum game. If common-payoff games represent situations of pure coordination, zero-sum games represent situations of pure competition; one player’s gain must come at the expense of the other player. As in the case of common-payoff games, we can use an abbreviated matrix form to represent zero-sum games, in which we write only one payoff value in each cell. This value represents the payoff of player 1, and thus the negative of the payoff of player 2. Note, though, that whereas the full matrix representation is unambiguous, when we use the abbreviation we must explicit state whether this matrix represents a common-payoff game or a zero-sum one. A classical example of a zero-sum game is the game of matching pennies. In this matching pennies gamegame, each of the two players has a penny, and independently chooses to display either heads or tails. The two players then compare their pennies. If they are the same then player 1 pockets both, and otherwise player 2 pockets them. The payoff matrix is shown in Figure 3.5. Heads Tails Heads 1 −1 Tails −1 1 Figure 3.5 Matching Pennies game. The popular children’s game of Rock, Paper, Scissors, also known as Rochambeau, Rock, Paper, Scissors, or Rochambeau game provides a three-strategy generalization of the matching-pennies game. The payoff matrix of this zero-sum game is shown in Figure 3.6. In this game, each of the two players can choose either Rock, Paper, or Scissors. If both players choose the same action, there is no winner, and the utilities are zero. Otherwise, each of the actions wins over one of the other actions, and loses to the other remaining action. In general, games tend to include elements of both coordination and competition. Prisoner’s Dilemma does, although in a rather paradoxical way. Here is another well- known game that includes both elements. In this game, called Battle of the Sexes, a husband and wife wish to go to the movies, and they can select among two movies: “Violence Galore (VG)” and “Gentle Love (GL)”. They much prefer to go together rather than to separate movies, but while the wife prefers VG the husband prefers GL. The payoff matrix is shown in Figure 3.7. We will return to this game shortly. Multi Agent Systems, draft of February 11, 2006 Play this game with someone near you, repeating five times. Game Theory Intro Lecture 3, Slide 13 Self-interested agents What is Game Theory? Example Matrix Games Coordination Game Which side of the road should you drive on? 58 3 Competition and Coordination: Normal form games C D C a, a b, c D c, b d, d Figure 3.3 Any c > a > d > b define an instance of Prisoner’s Dilemma. To fully understand the role of the payoff numbers we would need to enter into a discussion of utility theory. Here, let us just mention that for most purposes, theutility theory analysis of any game is unchanged if the payoff numbers undergo any positive affinepositive affine transformation transformation; this simply means that each payoff x is replaced by a payoff ax + b, where a is a fixed positive real number and b is a fixed real number. There are some restricted classes of normal-form games that deserve special men- tion. The first is the class of common-payoff games. These are games in which, for every action profile, all players have the same payoff. Definition 3.2.2 A common payoff game, or team game, is a game in which for allcommon-payoff game team game action profiles a ∈ A1 × · · · × An and any pair of agents i, j, it is the case that ui(a) = uj(a). Common-payoff games are also called pure coordination games, since in such gamespure- coordination game the agents have no conflicting interests; their sole challenge is to coordinate on an action that is maximally beneficial to all. Because of their special nature, we often represent common value games with an abbreviated form of the atrix in which we list only one payoff in each of the cells. As an example, imagine two drivers driving towards each other in a country without traffic rules, and who must independently decide whether to drive on the left or on the right. If the players choose the same side (left or right) they have some high utility, and otherwise they have a low utility. The game matrix is shown in Figure 3.4. Left Right Left 1 0 Right 0 1 Figure 3.4 Coordination game. At the other end of the spectrum from pure coordination games lie zero-sum games,zero-sum game which (bearing in mind the comment we made earlier about positive affine transforma- tions) are more properly called constant-sum games. Unlike common-payoff games,constant-sum games c©Shoham and Leyton-Brown, 2006 Play this game with someone near you. Then find a new partner and play again. Play five times in total. Game Theory Intro Lecture 3, Slide 16 Self-interested agents What is Game Theory? Example Matrix Games Coordination Game Which side of the road should you drive on? 58 3 Competition and Coordination: Normal form games C D C a, a b, c D c, b d, d Figure 3.3 Any c > a > d > b define an instance of Prisoner’s Dilemma. To fully understand the role of the payoff numbers we would need to enter into a discussion of utility theory. Here, let us just mention that for most purposes, theutility theory analysis of any game is unchanged if the payoff numbers undergo any positive affinepositive affine transformation transformation; this simply means that each payoff x is replaced by a payoff ax + b, where a is a fixed positive real number and b is a fixed real number. There are some restricted classes of normal-form games that deserve special men- tion. The first is the class of common-payoff games. These are games in which, for every action profile, all players have the same payoff. Definition 3.2.2 A common payoff game, or team game, is a game in which for allcommon-payoff game team game action profiles a ∈ A1 × · · · × An and any pair of agents i, j, it is the case that ui(a) = uj(a). Common-payoff games are also called pure coordination games, since in such gamespure- coordination game the agents have no conflicting interests; their sole challenge is to coordinate on an action that is maximally beneficial to all. Because of their special nature, we often represent common value games with an abbreviated form of the atrix in which we list only one payoff in each of the cells. As an example, imagine two drivers driving towards each other in a country without traffic rules, and who must independently decide whether to drive on the left or on the right. If the players choose the same side (left or right) they have some high utility, and otherwise they have a low utility. The game matrix is shown in Figure 3.4. Left Right Left 1 0 Right 0 1 Figure 3.4 Coordination game. At the other end of the spectrum from pure coordination games lie zero-sum games,zero-sum game which (bearing in mind the comment we made earlier about positive affine transforma- tions) are more properly called constant-sum games. Unlike common-payoff games,constant-sum games c©Shoham and Leyton-Brown, 2006 Play this game with someone near you. Then find a new partner and play again. Play five times in total. Game Theory Intro Lecture 3, Slide 16 Self-interested agents What is Game Theory? Example Matrix Games General Games: Battle of the Sexes The most interesting games combine elements of cooperation and competition. 60 3 Competition and Coordination: Normal form games Rock Paper Scissors Rock 0 −1 1 Paper 1 0 −1 Scissors −1 1 0 Figure 3.6 Rock, Paper, Scissors game. B F B 2, 1 0, 0 F 0, 0 1, 2 Figure 3.7 Battle of the Sexes game. 3.2.2 Strategies in normal-form games We have so far defined the actions available to each player in a game, but not yet his set of strategies, or his available choices. Certainly one kind of strategy is to select a single action and play it; we call such a strategy a pure strategy, and we will usepure strategy the notation we have already developed for actions to represent it. There is, however, another, less obvious type of strategy; a player can choose to randomize over the set of available actions according to some probability distribution; such a strategy is called a mixed strategy. Although it may not be immediately obvious why a player shouldmixed strategy introduce randomness into his choice of action, in fact in a multi-agent setting the role of mixed strategies is critical. We will return to this when we discuss solution concepts for games in the next section. We define a mixed strategy for a normal form game as follows. Definition 3.2.4 Let (N, (A1, . . . , An), O, µ, u) be a normal form game, and for any setX let Π(X) be the set of all probability distributions overX . Then the set of mixed strategies for player i is Si = Π(Ai). The set of mixed strategy profiles is simply themixed strategy profiles Cartesian product of the individual mixed strategy sets, S1 × · · · × Sn. By si(ai) we denote the probability that an action ai will be played under mixed strategy si. The subset of actions that are assigned positive probability by the mixed strategy si is called the support of si. Definition 3.2.5 The support of a mixed strategy si for a player i is the set of pure strategies {ai|si(ai) > 0}. c©Shoham and Leyton-Brown, 2006 Play this game with someone near you. Then find a new partner and play again. Play five times in total. Game Theory Intro Lecture 3, Slide 17