Download Slides on Self Interested MAS - Distributed Software Develop | CS 682 and more Study notes Software Engineering in PDF only on Docsity! Distributed Software Development Self-interested MAS Chris Brooks Department of Computer Science University of San Francisco Department of Computer Science — University of San Francisco – p. 1/?? Engineering systems vs Engineering agents • Recall that at the end of Thursday’s class, we were talking about ant algorithms. • By specifying a simple set of rules, we can achieve interesting large-scale behavior. • Ant-type approaches lead us to think about how we can build systems that produce the effects we want. • “Given that agents will act in a particular way, how can we constrain the environment to achieve a desirable outcome?” • This method of problem solving is best applied to problems involving self-interested agents. Department of Computer Science — University of San Francisco – p. 2/?? Example: Clarke tax • Assume that we want to find the shortest path through a graph. • Each edge is associated with an agent. • Each edge has a privately known transmission cost. • Agents might choose to lie about their transmission cost. • How can we find the shortest path? Department of Computer Science — University of San Francisco – p. 5/?? Clarke tax • Rule: • Accept each agent’s bid. • If they are not on the shortest path, they get 0. • If they are on the shortest path, they get: • Cost of next shortest path - (cost of shortest path without their contribution). Department of Computer Science — University of San Francisco – p. 6/?? Example start finish 5 (f) 3 (g) 4 (d) 4 (e) 7 (h) 2 (a) 2 (b) 2(c) • Assume each agent bids truthfully. • Agents A, B, and C are each paid 8 - (6 - 2) = 4 • This is their contribution to the ’best solution’ • Other agents are paid nothing. Department of Computer Science — University of San Francisco – p. 7/?? Solution concepts • There are a number of potential solution concepts we can use: • Social welfare - sum of all agent utility. • Pareto efficiency • Is there a solution that makes one agent better off without making anyone worse off? • Individual rationality • An agent who participates in the solution should be better off than if it hadn’t participated. • Stability • The mechanism should not be able to be manipulated by one or more agents. • It’s not usually possible to optimize all of these at the same time. Department of Computer Science — University of San Francisco – p. 10/?? Stability • Ideally, we can design mechanisms with dominant strategies • A dominant strategy is the best thing to do no matter what any other agent does. • In the previous example, truth-telling was a dominant strategy. • We would say that the mechanism is non-manipulable. (lying can’t break it.) • Unfortunately, many problems don’t have a dominant strategy. • Instead, the best thing for agent 1 to do depends on what agents 2,3,4,... do. Department of Computer Science — University of San Francisco – p. 11/?? Nash equilibrium • This leads to the concept of a Nash equilibrium • A set of actions is a Nash equilibrium if, for every agent, given that the other agents are playing those actions, it has no incentive to change. • Example: big monkey and little monkey • Monkeys usually eat ground-level fruit • Occasionally they climb a tree to get a coconut (1 per tree) • A Coconut yields 10 Calories • Big Monkey expends 2 Calories climbing the tree.(net 8 calories) • Little Monkey expends 0 Calories climbing the tree. (net 10 calories) Department of Computer Science — University of San Francisco – p. 12/?? Nash equilibrium • What should Big Monkey do? • If BM waits, LM will climb (1 is better than 0): BM gets 9 • If BM climbs, LM will wait :BM gets 4 • BM should wait. • What about LM? • LM should do the opposite of BM. • This is a Nash equilibrium. For each monkey, given the other’s choice, it doesn’t want to change. • Each monkey is playing a best response. Department of Computer Science — University of San Francisco – p. 15/?? Nash equilibrium • Nash eqilibria are nice in systems with rational agents. • If I assume other agents are rational, then I can assume they’ll play a best response. • I only need to consider Nash equilibria. • They are efficient (in the Pareto sense). • Problems: • There can be many Nash equilibria. (the cake-cutting problem has an infinite number of Nash equilibria) • Some games have no Nash equilibrium. • There may be ways for groups of agents to cheat. Department of Computer Science — University of San Francisco – p. 16/?? Selecting between equilibria • Given that there are lots of possible Nash equilibria in a problem, how does an agent choose a strategy? • In some cases, external forces are used to make one equilibrium more attractive. • Government regulation, taxes or penalties • In other cases a natural focal point exists. • There is a solution that is attractive or sensible outside the scope of the game. Department of Computer Science — University of San Francisco – p. 17/?? Auctions • Private-value auctions are easier to think about at first. • In this case, the value agent A places on a job has nothing to do with the value that agent B places on the object. • For example, an hour of computing time. • In common-value auctions, the value an agent places on an item depends on how much others value it. • Example: art, collectibles, precious metals. Department of Computer Science — University of San Francisco – p. 20/?? English auctions • An English (or first-price) auction is the kind we’re most familiar with. • Bids start low and rise. All agents see all bids. • May be a reserve price involved. • Dominant strategy: bid ǫ more than the highest price, until your threshold is reached. • Problems: requires multiple rounds, not efficient for the seller, requires agents to reveal their valuations to each other. • There may be technical problems to solve with making sure all agents see all bids within a limited period of time. Department of Computer Science — University of San Francisco – p. 21/?? First-price sealed-bid auction • Each agent submits a single sealed bid. Highest wins and pays what they bid. • This is how you buy a house. • Single round of bidding. All preferences remain private. • Problems: No Nash equilibrium - agents need to counterspeculate. Item may not go to the agent who valued it most. (inefficient). Department of Computer Science — University of San Francisco – p. 22/?? Example • Angel, Buffy and Cordelia are bidding on a sandwich. • Angel is willing to pay $5, Buffy $3, and Cordelia $2. • Each participant bids the amount they’re willing to pay. • Angel gets the sandwich and pays $3. Department of Computer Science — University of San Francisco – p. 25/?? Proof • Let’s prove that truth-telling is a dominant strategy. • Angel: • If he overbids, he still pays $3. No advantage. • If he bids between $3 and $5, he still pays $3. No advantage. • If he bids less than $3, then he doesn’t get the sandwich - but he was willing to pay $5, so this is a loss. Department of Computer Science — University of San Francisco – p. 26/?? Proof • Buffy (the same reasoning will hold for Cordelia) • If she bids less than $3, she still doesn’t get the sandwich. (notice that we assume she doesn’t care how much Angel pays.) • If she bids between $3 and $5, she still doesn’t get the sandwich. No benefit. • If she bids more than $5, she gets the sandwich and pays $5. But she was only willing to pay $3, so this is a loss. Department of Computer Science — University of San Francisco – p. 27/?? Common and correlated-value auctions • Everything we’ve said so far applies only to private value auctions. • Common or correlated-value auctions are much less predictable. • In particular, common-value auctions are subject to the winner’s curse • As soon as you win a common-value auction, you know you’ve paid too much. Department of Computer Science — University of San Francisco – p. 30/?? Winner’s curse • Example: Oil drilling • Suppose that four firms are bidding on drilling rights. Each has an estimate of how much oil is available in that plot. • A thinks $5M, B thinks $10M, C thinks $12M, and D thinks $20M. • Let’s say it’s really $10 M, but the firms don’t know this. • In an English auction, D will win for $12M+1 • They lose $2M on this deal. • Problem: The winner is the firm who tended to overestimate by the most. • (Assumption: all firms have access to the same information.) Department of Computer Science — University of San Francisco – p. 31/?? Winner’s curse • This also explains why sports free agents seem to underperform their contracts. • They’re not underperforming, they’re overpaid. • How to avoid the winner’s curse: • Better information gathering • Caution in bidding Department of Computer Science — University of San Francisco – p. 32/?? Winner-determination problem • Finding the winner for a single-item Vickrey auction is easy. • Finding the winner for a combinatorial auction is (computationally) hard. • Formulation: • Given: n bidders, m items • Let a bundle S be a subset of the m items. • A bid b is a pair (v,s), where v is the amount an agent will pay for s. • An allocation xi(S)is described by a mapping from (i,s) into {0,1}. ((i,s) = 0 if i does not get s, and 1 if he does.) Department of Computer Science — University of San Francisco – p. 35/?? Winner-determination problem • We can then write the winner-determination problem as an optimization problem: • Find the set of allocations that maximizes:∑ i∈N (v, s)xi(s) • This problem can be solved in a number of ways; integer linear programming or backtracking search are the most common. Department of Computer Science — University of San Francisco – p. 36/?? Winner-determination problem • Problem: The size of the WDP is exponential in the number of items that can be sold. • Every possible bundle must be considered. • Formulating the problem as ILP helps some • This problem has been studied since the 50s, so good heuristic techniques exist. Department of Computer Science — University of San Francisco – p. 37/??