Download Search Algorithms: Understanding Heuristics and Optimal Strategies and more Study Guides, Projects, Research Design in PDF only on Docsity! Set 3: Informed Heuristic Search ICS 271 Fall 2016 Kalev Kask Basic search scheme • We have 3 kinds of states – explored (past) – only graph search – frontier (current) – unexplored (future) – implicitly given • Initially frontier=start state • Loop until found solution or exhausted state space – pick/remove first node from frontier using search strategy • priority queue – FIFO (BFS), LIFO (DFS), g (UCS), f (A*), etc. – check if goal – add this node to explored, – expand this node, add children to frontier (graph search : only those children whose state is not in explored list) – Q: what if better path is found to a node already on explored list? 271-fall 2016 Heuristic Search • State-Space Search: every problem is like search of a map • A problem solving agent finds a path in a state-space graph from start state to goal state, using heuristics h= 253 h=329 h=374 Heuristic = straight-line distance 271-Fall 2016 State Space for Path Finding in a Map 271-Fall 2016 State Space for Path Finding on a Map 271-Fall 2016 State Space of the 8 Puzzle Problem 1 2 3 4 5 6 7 8 h1=4 h1=5 h1 = number of misplaced tiles h2=9h2=9 h2 = Manhattan distance 271-Fall 2016 What are Heuristics • Rule of thumb, intuition • A quick way to estimate how close we are to the goal. How close is a state to the goal.. • Pearl: “the ever-amazing observation of how much people can accomplish with that simplistic, unreliable information source known as intuition.” 8-puzzle – h1(n): number of misplaced tiles – h2(n): Manhattan distance – h3(n): Gaschnig’s • Path-finding on a map – Euclidean distance h1(S) = ? 8 h2(S) = ? 3+1+2+2+2+3+3+2 = 18 h3(S) = ? 8 Problem: Finding a Minimum Cost Path • Previously we wanted an path with minimum number of steps. Now, we want the minimum cost path to a goal G – Cost of a path = sum of individual steps along the path • Examples of path-cost: – Navigation • path-cost = distance to node in miles – minimum => minimum time, least fuel – VLSI Design • path-cost = length of wires between chips – minimum => least clock/signal delay – 8-Puzzle • path-cost = number of pieces moved – minimum => least time to solve the puzzle • Algorithm: Uniform-cost search … still somewhat blind 271-Fall 2016 Greedy Best-First Search • Evaluation function f(n) = h(n) (heuristic) = estimate of cost from n to goal • e.g., hSLD(n) = straight-line distance from n to Bucharest • Greedy best-first search expands the node that appears to be closest to goal 271-Fall 2016 Greedy Best-First Search Example
366
StraightHine distance
tb Bucharest
Arad 356
Bucharest 0
Cranva 160
Dobreta 249
Eforie 16
Fagaras 176
Giurgiu 7
Hirsova 151
las 26
Lugoj Et
Mehadia a41
Neamt Eel
Oradea 290
Pitesti 10
Rimnku Vileea = [93
Sibi 253
Timisoara 329
Urziceni 80
Vaslui 199
Zerind 374
271-Fall 2016
Greedy Best-First Search Example
253
Arad >
329
imisoara,
StraightHine distance
Bucharest
Arad
Bucharest
(Craiova
Dobreta
Eforie
Fagaras
Hirsova
Iasi
Lugoj
Mehadia
Neamt
Oradea
Pitesti
Rimnicu V ikea
Sibiu
Timisoara
Urziceni
Vashui
Zerind
365
o
160
242
Lol
176
TW
ISL
1
2H
241
24
Lo
193
33
319
Le
374
Problems with Greedy Search • Not complete – Gets stuck on local minimas and plateaus • Infinite loops • Irrevocable • Not optimal • Can we incorporate heuristics in systematic search? 271-Fall 2016 Informed Search - Heuristic Search • How to use heuristic knowledge in systematic search? • Where? (in node expansion? hill-climbing ?) • Best-first: – select the best from all the nodes encountered so far in OPEN. – “good” use heuristics • Heuristic estimates value of a node – promise of a node – difficulty of solving the subproblem – quality of solution represented by node – the amount of information gained. • f(n) - heuristic evaluation function. – depends on n, goal, search so far, domain 271-Fall 2016 Best-First Algorithm BF (*) 1. Put the start node s on a list called OPEN of unexpanded nodes. 2. If OPEN is empty exit with failure; no solutions exists. 3. Remove the first OPEN node n at which f is minimum (break ties arbitrarily), and place it on a list called CLOSED to be used for expanded nodes. 4. If n is a goal node, exit successfully with the solution obtained by tracing the path along the pointers from the goal back to s. 5. Otherwise expand node n, generating all it’s successors with pointers back to n. 6. For every successor n’ on n: a. Calculate f (n’). b. if n’ was neither on OPEN nor on CLOSED, add it to OPEN. Attach a pointer from n’ back to n. Assign the newly computed f(n’) to node n’. c. if n’ already resided on OPEN or CLOSED, compare the newly computed f(n’) with the value previously assigned to n’. If the old value is lower, discard the newly generated node. If the new value is lower, substitute it for the old (n’ now points back to n instead of to its previous predecessor). If the matching node n’ resides on CLOSED, move it back to OPEN. 7. Go to step 2. * With tests for duplicate nodes. 271-Fall 2016 A’ Search Example
imisoare
993=1404253 44721184329 AMD=T 54374
StraightHine distance
> Bucharest
Arad 66
Bucharest Qo
(Craiova 160
Dobreta 242
Eforie Lol
Fagaras 176
Giurgiu TW
Hirsova LL
las 220
Lugoj 244
Mehadia 241
Neamt ayy
Oradea 380
Pitesti lo
Rimnicu Vilees = [93
Sibiu 353
Timisoara 9
Urziceni 30
Vaslui 199
Zerind 314
271-Fall 2016
A’ Search Example
imisoara,
44721184329 AMD=7 54374
a> Caan
646=2004966 415=230+176 671=291+980 413=220+199
Straight-line distance
> Bucharest
Arad 66
Bucharest Qo
(Craiova 160
Dobreta 242
Eforie Lol
Fagaras 176
Giurgiu TW
Hirsova LL
las 220
Lugoj 244
Mehadia 241
Neamt ayy
Oradea 380
Pitesti lo
Rimnicu Vilees = [93
Sibiu 353
Timisoara 9
Urziceni 30
Vaslui 199
Zerind 314
271-Fall 2016
271-Fall 2016
A’ Search Example
imisoara,
44721184329
646=2804366 415=2304176 67 1=291+380 f
Ga> iar - ci
526=366+160 44729974100 559=3004253
StraightHine distance
Bucharest
Arad
Bucharest
(Craiova
Dobreta
Eforie
Fagaras
Giurgiu
Hirsova
last
Lugoj
Mehadia
Neamt
Oradea
Pitesti
365
Qo
160
242
Le
176
WT
15h
226
24
241
a4
Lo
193
353
339
199
374
A* on 8-Puzzle with h(n) = # misplaced tiles 271-Fall 2016 A*- a Special Best-First Search • Goal: find a minimum sum-cost path • Notation: – c(n,n’) - cost of arc (n,n’) – g(n) = cost of current path from start to node n in the search tree. – h(n) = estimate of the cheapest cost of a path from n to a goal. – evaluation function: f = g+h • f(n) estimates the cheapest cost solution path that goes through n. – h*(n) is the true cheapest cost from n to a goal. – g*(n) is the true shortest path from the start s, to n. – C* is the cost of optimal solution. • If the heuristic function, h always underestimates the true cost (h(n) is smaller than h*(n)), then A* is guaranteed to find an optimal solution. 271-Fall 2016 S G A B D E C F 4.06.710.4 11.0 8.9 6.9 3.0 1 4 S G A B D E C F 2 2 5 2 4 3 5 271-Fall 2016 Behavior of A* - Termination/Completeness • Theorem (completeness) (Hart, Nilsson and Raphael, 1968) – A* always terminates with a solution path (h is not necessarily admissible) if • costs on arcs are positive, above epsilon • branching degree is finite. • Proof: The evaluation function f of nodes expanded must increase eventually (since paths are longer and more costly) until all the nodes on a solution path are expanded. 271-Fall 2016 Admissible A* • The heuristic function h(n) is called admissible if h(n) is never larger than h*(n), namely h(n) is always less or equal to true cheapest cost from n to the goal. • A* is admissible if it uses an admissible heuristic, and h(goal) = 0. • If the heuristic function, h always underestimates the true cost (h(n) is smaller than h*(n)), then A* is guaranteed to find an optimal solution. 271-Fall 2016 A’ with inadmissible h
—
44721184329 4H=7SHSTS
G46=2804386 415-2394176 671=2914380 4413=2204193
orsesa0288 StraightJine distance
1 Bucharest
Arad 6
Bucharest Qo
(Craiova 160
Dobreta 44
Eforie lel
Fagaras 176
Giurgiu 7
Hirsowa ISL
Tosi 236
Lugoj a4
Mehadia 241
Neamt 234
Oradea 290
Pitesti io
Rimnicu Vikea [93 —293
Sibiw 453
Timisoara 329
Urziceni 30
Vaslui 199
271-Fall 2016 Zerind 474
Behavior of A* - Optimality • Theorem (completeness for optimal solution) (HNL, 1968): – If the heuristic function is • admissible (tree search or graph search with explored node re-opening) • consistent (graph search w/o explored node re-opening) – then A* finds an optimal solution. • Proof: – 1. A*(admissible/consistent) will expand only nodes whose f-values are less (or equal) to the optimal cost path C* (f(n) is less-or-equal C*). – 2. The evaluation function of a goal node along an optimal path equals C*. • Lemma: – Anytime before A*(admissible/consistent) terminates there exists and OPEN node n’ on an optimal path with f(n’) <= C*. 271-Fall 2016 Requirements for Optimality • Tree search – Need admissibility • Graph search, without re-opening closed nodes – Need consistency • Graph search, with re-opening closed nodes – Admissibility is enough 271-Fall 2016 Inconsistent but admissible 271-Fall 2016 Consistency : h(ni) <= c(ni,nj) + h(nj) or c(ni,nj) >= h(ni) - h(nj) or c(ni,nj) >= ∆h Summary of Consistent Heuristics • h is consistent if the heuristic function satisfies triangle inequality for every n and its child node n’: h(ni) <= h(nj) + c(ni,nj) • When h is consistent, the f values of nodes expanded by A* are never decreasing. • When A* selected n for expansion it already found the shortest path to it. • When h is consistent every node is expanded once. • Normally the heuristics we encounter are consistent – the number of misplaced tiles – Manhattan distance – straight-line distance 271-Fall 2016 A* properties • A* expands every path along which f(n) < C* • A* will never expand any node such that f(n) > C* • If h is consistent A* will expand any node such that f(n) < C* • Therefore, A* expands all the nodes for which f(n) < C* and a subset of the nodes for which f(n) = C*. • Therefore, if h1(n) < h2(n) clearly the subset of nodes expanded by h2 is smaller. 271-Fall 2016 Complexity of A* • A* is optimally efficient (Dechter and Pearl 1985): – It can be shown that all algorithms that do not expand a node which A* did expand (inside the contours) may miss an optimal solution • A* worst-case time complexity: – is exponential unless the heuristic function is very accurate • If h is exact (h = h*) – search focus only on optimal paths • Main problem: – space complexity is exponential – Not anytime; all or nothing … but largest f expanded is lower bound on C* • Effective branching factor: – Number of nodes generated by a “typical” search node – Approximately : b* = N^(1/d) • Q: what is you are given a solution (not necessarily optimal); can you improve A* performance? 271-Fall 2016 S G A B D E C F 4.06.710.4 11.0 8.9 6.9 3.0 S G A B D E C F 2 1 2 5 4 2 4 3 5 271-Fall 2016 Example of Branch and Bound in Action S A D 2+10.4=12.4 5+8.9=13.9 271-Fall 2016 DB 4+8.9=12.9 3+6.7=9.7 1 2 3 EC 8+6.9=14.9 7+4=11 4 5 D F 10+8.9=18.912+3=15 6 G L=15+0=15 7 E 6+6.9=12.9 8 9 F10+3=13 10 G L=13+0=13 11 B 11+6.7=17.7 1 4A B C S G D E F 2 2 5 2 4 3 5 S G A B D E C F 4.06.710.4 11.0 8.9 6.9 3.0 Example of A* Algorithm in Action 7 + 4 = 11 S A D B D C E E B F G 2 +10.4 = 12.4 5 + 8.9 = 13.9 3 + 6.7 = 9.7 8 + 6.9 = 14.9 4 + 8.9 = 12.9 6 + 6.9 = 12.9 11 + 6.7 = 17.7 10 + 3.0 = 13 13 + 0 = 13 Dead End 271-Fall 2016 1 4 S G A B D E C F 2 2 5 2 4 3 5 S G A B D E C F 4.06.710.4 11.0 8.9 6.9 3.0 1 2 3 4 5 6 7 8 Iterative Deepening A* (IDA*) (combining Branch-and-Bound and A*) • Initialize: f <-- the evaluation function of the start node • until goal node is found – Loop: • Do Branch-and-bound with upper-bound L equal to current evaluation function f. • Increment evaluation function to next contour level – end • Properties: – Guarantee to find an optimal solution – time: exponential, like A* – space: linear, like B&B. • Problems: The number of iterations may be large - ∆f may be ɛ. 271-Fall 2016 Relationships among Search Algorithms
Depth first
(LIFO ordering)
f = depth
(Breadth first)
h=0
(Uniform cost)
(Best-first search)
(Generic graph-search
algorithms)
Effectiveness of heuristic search • How quality of the heuristic impacts search? • What is the time and space complexity? • Is any algorithm better? Worse? • Case study: the 8-puzzle 271-Fall 2016 Dominance • Definition: If h2(n) ≥ h1(n) for all n (both admissible) then h2 dominates h1 • Is h2 better for search? • Typical search costs (average number of nodes expanded): • d=12 IDS = 3,644,035 nodes A*(h1) = 227 nodes A*(h2) = 73 nodes • d=24 IDS = out of memory A*(h1) = 39,135 nodes A*(h2) = 1,641 nodes 271-Fall 2016 Heuristic’s Dominance and Pruning Power • Definition: – A heuristic function h2 (strictly) dominates h1 if both are admissible and for every node n, h2(n) is (strictly) greater than h1(n). • Theorem (Hart, Nilsson and Raphale, 1968): – An A* search with a dominating heuristic function h2 has the property that any node it expands is also expanded by A* with h1. • Question: Does Manhattan distance dominate the number of misplaced tiles? • Extreme cases – h = 0 – h = h* 271-Fall 2016 Inventing Heuristics automatically • Examples of Heuristic Functions for A* – The 8-puzzle problem • The number of tiles in the wrong position – is this admissible? • Manhattan distance – is this admissible? – How can we invent admissible heuristics in general? • look at “relaxed” problem where constraints are removed – e.g.., we can move in straight lines between cities – e.g.., we can move tiles independently of each other 271-Fall 2016 Generating heuristics (cont.) • Example: TSP • Find a tour. A tour is: – 1. A graph with subset of edges – 2. Connected – 3. Total length of edges minimized – 4. Each node has degree 2 • Eliminating 4 yields MST. 271-Fall 2016
Relaxed problems contd.
Well-known example: travelling salesperson problem (TSP)
Find the shortest tour visiting all cities exactly once
Minimum spanning tree can be computed in O(n?)
and is a lower bound on the shortest (open) tour
Chapter 4, Sections 12,485
271-Fall 2016
Automating Heuristic generation • Use STRIPs language representation: • Operators: – pre-conditions, add-list, delete list • 8-puzzle example: – on(x,y), clear(y) adj(y,z) ,tiles x1,…,x8 • States: conjunction of predicates: – on(x1,c1),on(x2,c2)….on(x8,c8),clear(c9) • move(x,c1,c2) (move tile x from location c1 to location c2) – pre-cond: on(x1,c1), clear(c2), adj(c1,c2) – add-list: on(x1,c2), clear(c1) – delete-list: on(x1,c1), clear(c2) • Relaxation: – Remove from precondition: clear(c2), adj(c2,c3) #misplaced tiles – Remove clear(c2) Manhattan distance – Remove adj(c2,c3) h3, a new procedure that transfers to the empty location a tile appearing there in the goal • The space of relaxations can be enriched by predicate refinements – adj(y,z) = iff neigbour(y,z) and same-line(y,z) 271-Fall 2016 Improving Heuristics • Reinforcement learning. • Pattern Databases: you can solve optimally a sub-problem 271-Fall 2016 Pattern Databases • For sliding tiles and Rubic’s cube • For a subset of the tiles compute shortest path to the goal using breadth-first search • For 15 puzzles, if we have 7 fringe tiles and one blank, the number of patterns to store are 16!/(16-8)! = 518,918,400. • For each table entry we store the shortest number of moves to the goal from the current location. • Use different subsets of tiles and take the max heuristic during IDA* search. The number of nodes to solve 15 puzzles was reduced by a factor of 346 (Culberson and Schaeffer) • How can this be generalized? (a possible project) 271-Fall 2016 Beyond Classical Search • AND/OR search spaces – Decomposable independent problems – Searching with non-deterministic actions (erratic vacuum) – Using AND/OR search spaces; solution is a contingent plan • Local search for optimization – Greedy hill-climbing search, simulated annealing, local beam search, genetic algorithms. – Local search in continuous spaces – SLS : "Like climbing Everest in thick fog with amnesia" • Searching with partial observations – Using belief states • Online search agents and unknown environments – Actions, costs, goal-tests are revealed in state only – Exploration problems. Safely explorable 271-Fall 2016 271-Fall 2016 AND/OR search spaces non-deterministic actions : the erratic vacuum world AND/OR Graphs • Nodes represent subproblems – AND links represent subproblem decompositions – OR links represent alternative solutions – Start node is initial problem – Terminal nodes are solved subproblems • Solution graph – It is an AND/OR subgraph such that: • It contains the start node • All its terminal nodes (nodes with no successors) are solved primitive problems • If it contains an AND node A, it must contain the entire group of AND links that leads to children of A. 271-Fall 2016 Algorithms searching AND/OR graphs • All algorithms generalize using hyper-arc successors rather than simple arcs. • AO*: is A* that searches AND/OR graphs for a solution subgraph. • The cost of a solution graph is the sum cost of it arcs. It can be defined recursively as: k(n,N) = c_n+k(n1,N)+…k(n_k,N) • h*(n) is the cost of an optimal solution graph from n to a set of goal nodes • h(n) is an admissible heuristic for h*(n) • Monotonicity: • h(n)<= c+h(n1)+…h(nk) where n1,…nk are successors of n • AO* is guaranteed to find an optimal solution when it terminates if the heuristic function is admissible 271-Fall 2016