Distributed Software Development: Self-interested Agents and Game Theory, Study notes of Software Engineering

Download Distributed Software Development: Self-interested Agents and Game Theory and more Study notes Software Engineering in PDF only on Docsity! Distributed Software Development Self-interested Agents Chris Brooks Department of Computer Science University of San Francisco Department of Computer Science — University of San Francisco – p. 1/?? 23-2: Engineering systems vs Engineering agents • Recall that at the end of Thursday’s class, we were talking about ant algorithms. ◦ By specifying a simple set of rules, we can achieve interesting large-scale behavior. • Ant-type approaches lead us to think about how we can build systems that produce the effects we want. • “Given that agents will act in a particular way, how can we constrain the environment to achieve a desirable outcome?” • This method of problem solving is best applied to problems involving self-interested agents. Department of Computer Science — University of San Francisco – p. 2/?? 23-5: Example: Clarke tax • Assume that we want to find the shortest path through a graph. • Each edge is associated with an agent. • Each edge has a privately known transmission cost. ◦ Agents might choose to lie about their transmission cost. • How can we find the shortest path? Department of Computer Science — University of San Francisco – p. 5/?? 23-6: Clarke tax • Rule: ◦ Accept each agent’s bid. ◦ If they are not on the shortest path, they get 0. ◦ If they are on the shortest path, they get: • Cost of next shortest path - (cost of shortest path without their contribution). Department of Computer Science — University of San Francisco – p. 6/?? 23-7: Example • Assume each agent bids truthfully. • Agents A, B, and C are each paid 8 - (6 - 2) = 4 ◦ This is their contribution to the ’best solution’ • Other agents are paid nothing. Department of Computer Science — University of San Francisco – p. 7/?? 23-10: Solution concepts • There are a number of potential solution concepts we can use: ◦ Social welfare - sum of all agent utility. ◦ Pareto efficiency • Is there a solution that makes one agent better off without making anyone worse off? ◦ Individual rationality • An agent who participates in the solution should be better off than if it hadn’t participated. ◦ Stability • The mechanism should not be able to be manipulated by one or more agents. • It’s not usually possible to optimize all of these at the same time. Department of Computer Science — University of San Francisco – p. 10/?? 23-11: Stability • Ideally, we can design mechanisms with dominant strategies ◦ A dominant strategy is the best thing to do no matter what any other agent does. ◦ In the previous example, truth-telling was a dominant strategy. ◦ We would say that the mechanism is non-manipulable. (lying can’t break it.) • Unfortunately, many problems don’t have a dominant strategy. • Instead, the best thing for agent 1 to do depends on what agents 2,3,4,... do. Department of Computer Science — University of San Francisco – p. 11/?? 23-12: Nash equilibrium • This leads to the concept of a Nash equilibrium • A set of actions is a Nash equilibrium if, for every agent, given that the other agents are playing those actions, it has no incentive to change. • Example: big monkey and little monkey ◦ Monkeys usually eat ground-level fruit ◦ Occasionally they climb a tree to get a coconut (1 per tree) ◦ A Coconut yields 10 Calories ◦ Big Monkey expends 2 Calories climbing the tree.(net 8 calories) ◦ Little Monkey expends 0 Calories climbing the tree. (net 10 calories) Department of Computer Science — University of San Francisco – p. 12/?? 23-15: Nash equilibrium • What should Big Monkey do? • If BM waits, LM will climb (1 is better than 0): BM gets 9 • If BM climbs, LM will wait :BM gets 4 • BM should wait. • What about LM? ◦ LM should do the opposite of BM. • This is a Nash equilibrium. For each monkey, given the other’s choice, it doesn’t want to change. • Each monkey is playing a best response. Department of Computer Science — University of San Francisco – p. 15/?? 23-16: Nash equilibrium • Nash eqilibria are nice in systems with rational agents. • If I assume other agents are rational, then I can assume they’ll play a best response. • I only need to consider Nash equilibria. • They are efficient (in the Pareto sense). • Problems: ◦ There can be many Nash equilibria. (the cake-cutting problem has an infinite number of Nash equilibria) ◦ Some games have no Nash equilibrium. ◦ There may be ways for groups of agents to cheat. Department of Computer Science — University of San Francisco – p. 16/?? 23-17: Selecting between equilibria • Given that there are lots of possible Nash equilibria in a problem, how does an agent choose a strategy? • In some cases, external forces are used to make one equilibrium more attractive. ◦ Government regulation, taxes or penalties • In other cases a natural focal point exists. ◦ There is a solution that is attractive or sensible outside the scope of the game. Department of Computer Science — University of San Francisco – p. 17/?? 23-20: Auctions • Private-value auctions are easier to think about at first. • In this case, the value agent A places on a job has nothing to do with the value that agent B places on the object. ◦ For example, an hour of computing time. • In common-value auctions, the value an agent places on an item depends on how much others value it. ◦ Example: art, collectibles, precious metals. Department of Computer Science — University of San Francisco – p. 20/?? 23-21: English auctions • An English (or first-price) auction is the kind we’re most familiar with. • Bids start low and rise. All agents see all bids. • May be a reserve price involved. • Dominant strategy: bid ǫ more than the highest price, until your threshold is reached. • Problems: requires multiple rounds, not efficient for the seller, requires agents to reveal their valuations to each other. • There may be technical problems to solve with making sure all agents see all bids within a limited period of time. Department of Computer Science — University of San Francisco – p. 21/?? 23-22: First-price sealed-bid auction • Each agent submits a single sealed bid. Highest wins and pays what they bid. ◦ This is how you buy a house. • Single round of bidding. All preferences remain private. • Problems: No Nash equilibrium - agents need to counterspeculate. Item may not go to the agent who valued it most. (inefficient). Department of Computer Science — University of San Francisco – p. 22/?? 23-25: Example • Angel, Buffy and Cordelia are bidding on a sandwich. ◦ Angel is willing to pay $5, Buffy $3, and Cordelia $2. • Each participant bids the amount they’re willing to pay. • Angel gets the sandwich and pays $3. Department of Computer Science — University of San Francisco – p. 25/?? 23-26: Proof • Let’s prove that truth-telling is a dominant strategy. • Angel: ◦ If he overbids, he still pays $3. No advantage. ◦ If he bids between $3 and $5, he still pays $3. No advantage. ◦ If he bids less than $3, then he doesn’t get the sandwich - but he was willing to pay $5, so this is a loss. Department of Computer Science — University of San Francisco – p. 26/?? 23-27: Proof • Buffy (the same reasoning will hold for Cordelia) ◦ If she bids less than $3, she still doesn’t get the sandwich. (notice that we assume she doesn’t care how much Angel pays.) ◦ If she bids between $3 and $5, she still doesn’t get the sandwich. No benefit. ◦ If she bids more than $5, she gets the sandwich and pays $5. But she was only willing to pay $3, so this is a loss. Department of Computer Science — University of San Francisco – p. 27/?? 23-30: Common and correlated-value auctions • Everything we’ve said so far applies only to private value auctions. • Common or correlated-value auctions are much less predictable. • In particular, common-value auctions are subject to the winner’s curse ◦ As soon as you win a common-value auction, you know you’ve paid too much. Department of Computer Science — University of San Francisco – p. 30/?? 23-31: Winner’s curse • Example: Oil drilling ◦ Suppose that four firms are bidding on drilling rights. Each has an estimate of how much oil is available in that plot. • A thinks $5M, B thinks $10M, C thinks $12M, and D thinks $20M. • Let’s say it’s really $10 M, but the firms don’t know this. ◦ In an English auction, D will win for $12M (plus 1 dollar) ◦ They lose $2M on this deal. ◦ Problem: The winner is the firm who tended to overestimate by the most. ◦ (Assumption: all firms have access to the same information.) Department of Computer Science — University of San Francisco – p. 31/?? 23-32: Winner’s curse • This also explains why sports free agents seem to underperform their contracts. ◦ They’re not underperforming, they’re overpaid. • How to avoid the winner’s curse: ◦ Better information gathering ◦ Caution in bidding Department of Computer Science — University of San Francisco – p. 32/?? 23-35: Winner-determination problem • We can then write the winner-determination problem as an optimization problem: ◦ Find the set of allocations that maximizes:∑ i∈N (v, s)xi(s) ◦ This problem can be solved in a number of ways; integer linear programming or backtracking search are the most common. Department of Computer Science — University of San Francisco – p. 35/?? 23-36: Winner-determination problem • Problem: The size of the WDP is exponential in the number of items that can be sold. ◦ Every possible bundle must be considered. • Formulating the problem as ILP helps some ◦ This problem has been studied since the 50s, so good heuristic techniques exist. Department of Computer Science — University of San Francisco – p. 36/?? 23-37: Winner-determination problem • Other solutions: ◦ Limiting the sorts of bundles allowed. ◦ OR bids and XOR bids. • This transforms the problem into the knapsack problem. • Still NP-hard, but good heuristics exist • Limiting size of bundles. • Approximation algorithms Department of Computer Science — University of San Francisco – p. 37/??