Multiagent Systems II - Lecture Slides | CS 682, Study notes of Software Engineering

Download Multiagent Systems II - Lecture Slides | CS 682 and more Study notes Software Engineering in PDF only on Docsity! Distributed Software Development Multiagent Systems II Chris Brooks Department of Computer Science University of San Francisco Department of Computer Science — University of San Francisco – p.1/?? 20-0: Reminder We were talking about how to achieve coordination among groups of intelligent agents. We’ll start by talking about groups of cooperative agents. We started with contract net, which is a nice algorithm for task allocation. Department of Computer Science — University of San Francisco – p.2/?? 20-1: Distributed Planning Contract Net works nicely for environments that have discrete one-shot tasks. In many cases, domains are more complex. Agents are specialized A ’task’ may require a sequence of steps These steps may be accomplished in several ways. This leads into the realm of distributed planning Department of Computer Science — University of San Francisco – p.3/?? 20-2: Planning in Five Minutes A plan is a sequence of operations meant to accomplish a goal. The goal is specified declaratively: at(luggage, airport), at(brooks, airport), at(students, airport) Actions are ways of accomplishing parts of a plan They have preconditions and effects. Preconditions must be true to perform the action Effects must hold after the action is taken PutLuggageInTrunk. pre: holding(luggage) effect: in(luggage, trunk) Planning is the process of finding a sequence of actions that accomplishes a goal. Department of Computer Science — University of San Francisco – p.4/?? 20-3: Planning example Start Remove(Spare,Trunk)At(Spare,Trunk) Remove(Flat,Axle)At(Flat,Axle) PutOn(Spare,Axle) At(Spare,Ground) At(Flat,Axle) FinishAt(Spare,Axle) At(Flat,Axle) At(Spare,Trunk) ¬ (From Russell and Norvig) Notice that planning operators can be fully described by their preconditions and effects. Planning involves putting these operators into a partial order that accomplishes the conditions in the goal state. Challenges: conflicts, including disjunctive effects, allowing flexibility at run time. Department of Computer Science — University of San Francisco – p.5/?? 20-4: Planning in Five Minutes Many algorithms exist for building and repairing plans. Issues: Ordering constraints Dealing with failure Adapting to changes in the world Scaling A common way to deal with large planning problems is to construct a hierarchical plan. Department of Computer Science — University of San Francisco – p.6/?? 20-5: Hierarchical Planning Many times, aspects of a problem can be solved independently. Example: taking a trip to Peru can be decomposed into: Buying tickets Getting everyone to the airport Getting on the plane and flying there. I can figure out how to solve each of these problems more or less independently. Each subproblem can be represented as an AND/OR graph Some decisions made at runtime. Caveat: decisions made in one subproblem may affect possible choices in other subproblems.Department of Computer Science — University of San Francisco – p.7/?? 20-6: Example This is an example from a military logistics scenario. Resources must be moved C1 to C2. There are many ways to accomplish some of the subtasks. Planners would like to leave as much flexibility as possible. Department of Computer Science — University of San Francisco – p.8/?? 20-7: Distributed Planning Distributed planning comes about when multiple agents (usually with hierarchical plans) must share resources. A communication channel, a bridge, a power supply Alternatively, there may be opportunities for synergy. Both agents plan to deliver packages to the same location. How can agents synchronize their plans? Department of Computer Science — University of San Francisco – p.9/?? 20-8: Distributed Planning Solution 1: Submit plans to a centralized coordinator. Doesn’t scale Agents may not be willing to share more information than is needed. Solution 2: broadcast top-level constraints to each other. This allows agents to detect whether there is a top-level conflict. Plans will either be totally serialized or totally parallel. Department of Computer Science — University of San Francisco – p.10/?? 20-9: Distributed Planning A better solution: Detect whether there is: No problem: all possible interactions may be interleaved. No solution: plans must be serialized. Some solution: We then ’step down a level’ in the plans and force agents to commit to particular alternatives. Tradeoff: Deeper level requires more communication and interleaving (an exponential problem), but produces finer-grained coordination. Department of Computer Science — University of San Francisco – p.11/?? 20-10: Example Agents begin by exchanging top-level plans. As conflicts are detected, they: Eliminate possible alternatives If necessary, drill down to more detailed plans. If no solutions exist, agents must move back up and coordinate at a more abstract level. Department of Computer Science — University of San Francisco – p.12/?? 20-23: Solution concepts There are a number of potential solution concepts we can use: Social welfare - sum of all agent utility. Pareto efficiency Is there a solution that makes one agent better off without making anyone worse off? Individual rationality An agent who participates in the solution should be better off than if it hadn’t participated. Stability The mechanism should not be able to be manipulated by one or more agents. It’s not usually possible to optimize all of these at the same time. Department of Computer Science — University of San Francisco – p.25/?? 20-24: Stability Ideally, we can design mechanisms with dominant strategies A dominant strategy is the best thing to do no matter what any other agent does. In the previous example, truth-telling was a dominant strategy. We would say that the mechanism is non-manipulable. (lying can’t break it.) Unfortunately, many problems don’t have a dominant strategy. Instead, the best thing for agent 1 to do depends on what agents 2,3,4,... do. Department of Computer Science — University of San Francisco – p.26/?? 20-25: Nash equilibrium This leads to the concept of a Nash equilibrium A set of actions is a Nash equilibrium if, for every agent, given that the other agents are playing those actions, it has no incentive to change. Example: big monkey and little monkey Monkeys usually eat ground-level fruit Occasionally they climb a tree to get a coconut (1 per tree) A Coconut yields 10 Calories Big Monkey expends 2 Calories climbing the tree.(net 8 calories) Little Monkey expends 0 Calories climbing the tree. (net 10 calories) Department of Computer Science — University of San Francisco – p.27/?? 20-26: Nash equilibrium If BM climbs the tree BM gets 6 C, LM gets 4 C LM eats some before BM gets down If LM climbs the tree BM gets 9 C, LM gets 1 C BM eats almost all before LM gets down If both climb the tree BM gets 7 C, LM gets 3 C BM hogs coconut How should the monkeys each act so as to maximize their own calorie gain? Department of Computer Science — University of San Francisco – p.28/?? 20-27: Nash equilibrium Assume BM decides first Two choices: wait or climb LM has four choices: Always wait, always climb, same as BM, opposite of BM. Department of Computer Science — University of San Francisco – p.29/?? 20-28: Nash equilibrium What should Big Monkey do? If BM waits, LM will climb (1 is better than 0): BM gets 9 If BM climbs, LM will wait :BM gets 4 BM should wait. What about LM? LM should do the opposite of BM. This is a Nash equilibrium. For each monkey, given the other’s choice, it doesn’t want to change. Each monkey is playing a best response. Department of Computer Science — University of San Francisco – p.30/?? 20-29: Nash equilibrium Nash eqilibria are nice in systems with rational agents. If I assume other agetns are rational, then I can assume they’ll play a best response. I only need to consider Nash equilibria. They are efficient (in the Pareto sense). Problems: There can be many Nash equilibria. Some games have no Nash equilibrium. There may be ways for groups of agents to cheat. Department of Computer Science — University of San Francisco – p.31/?? 20-30: Auctions An auction is a negotiation mechanism where: The mechanism is well-specified (it runs according to explicit rules) The negotiation is mediated by an intermediary Exchanges are market/currency-based Agents place bids on items or collections of items. An auctioneer determines how goods are allocated. Desiderata: the auction should be fair, efficient, easy to use, and computationally efficient. We’ll need to trade these against each other. Department of Computer Science — University of San Francisco – p.32/?? 20-31: Auctions Private-value auctions are easier to think about at first. In this case, the value agent A places on a job has nothing to do with the value that agent B places on the object. For example, an hour of computing time. In common-value auctions, the value an agent places on an item depends on how much others value it. Example: art, collectibles, precious metals. Department of Computer Science — University of San Francisco – p.33/?? 20-32: English auctions An English (or first-price) auction is the kind we’re most familiar with. Bids start low and rise. All agents see all bids. May be a reserve price involved. Dominant strategy: bid  more than the highest price, until your threshold is reached. Problems: requires multiple rounds, not efficient for the seller, requires agents to reveal their valuations to each other. There may be technical problems to solve with making sure all agents see all bids within a limited period of time. Department of Computer Science — University of San Francisco – p.34/?? 20-33: First-price sealed-bid auction Each agent submits a single sealed bid. Highest wins and pays what they bid. This is how you buy a house. Single round of bidding. All preferences remain private. Problems: No Nash equilibrium - agents need to counterspeculate. Item may not go to the agent who valued it most. (inefficient). Department of Computer Science — University of San Francisco – p.35/?? 20-34: Dutch auction Prices start high and decline. First agent to bid wins. Strategically equivalent to first-price sealed-bid. In practice, closes quickly. Department of Computer Science — University of San Francisco – p.36/?? 20-35: Vickrey auction The Vickrey, or second-price, auction, has a number of appealing aspects from a computational point of view. Single round of bidding. Efficient allocation of goods. Truth-telling is the dominant strategy. Rule: each agent bids. Highest bid wins, but pays the second price. (the example we used earlier is isomorphic to the Vickrey auction). Department of Computer Science — University of San Francisco – p.37/?? 20-36: Example Angel, Buffy and Cordelia are bidding on a sandwich. Angel is willing to pay $5, Buffy $3, and Cordelia $2. Each participant bids the amount they’re willing to pay. Angel gets the sandwich and pays $3. Department of Computer Science — University of San Francisco – p.38/?? 20-37: Proof Let’s prove that truth-telling is a dominant strategy. Angel: If he overbids, he still pays $3. No advantage. If he bids between $3 and $5, he still pays $3. No advantage. If he bids less than $3, then he doesn’t get the sandwich - but he was willing to pay $5, so this is a loss. Department of Computer Science — University of San Francisco – p.39/?? 20-38: Proof Buffy (the same reasoning will hold for Cordelia) If she bids less than $3, she still doesn’t get the sandwich. (notice that we assume she doesn’t care how much Angel pays.) If she bids between $3 and $5, she still doesn’t get the sandwich. No benefit. If she bids more than $5, she gets the sandwich and pays $5. But she was only willing to pay $3, so this is a loss. Department of Computer Science — University of San Francisco – p.40/?? 20-39: Vickrey Auctions in real life Because of these properties, Vickrey auctions have been adopted for: Allocation of computer resources Distribution of electrical power Bandwidth allocation Scheduling problems. Interestingly, they are not widely used in human auctions. Perhaps people are not rational ... Department of Computer Science — University of San Francisco – p.41/?? 20-40: Next time Dealing with common-value auctions. Combinations of goods. Computational markets and double auctions. Department of Computer Science — University of San Francisco – p.42/??