Algorithm Strategies: Recursive, Backtracking, Divide & Conquer, Dynamic Programming - Pro, Study notes of Computer Science

Download Algorithm Strategies: Recursive, Backtracking, Divide & Conquer, Dynamic Programming - Pro and more Study notes Computer Science in PDF only on Docsity! 1 Algorithm Strategies Fawzi Emad Chau-Wen Tseng Department of Computer Science University of Maryland, College Park General Concepts Algorithm strategy Approach to solving a problem May combine several approaches Algorithm structure Iterative ⇒ execute action in loop Recursive ⇒ reapply action to subproblem(s) Problem type Satisfying ⇒ find any satisfactory solution Optimization ⇒ find best solutions (vs. cost metric) 2 Some Algorithm Strategies Recursive algorithms Backtracking algorithms Divide and conquer algorithms Dynamic programming algorithms Greedy algorithms Brute force algorithms Branch and bound algorithms Heuristic algorithms Recursive Algorithm Based on reapplying algorithm to subproblem Approach 1. Solves base case(s) directly 2. Recurs with a simpler subproblem 3. May need to convert solution(s) to subproblems 5 Divide and Conquer Based on dividing problem into subproblems Approach 1. Divide problem into smaller subproblems Subproblems must be of same type Subproblems do not need to overlap 2. Solve each subproblem recursively 3. Combine solutions to solve original problem Usually contains two or more recursive calls Divide and Conquer – Examples Quicksort Partition array into two parts around pivot Recursively quicksort each part of array Concatenate solutions Mergesort Partition array into two parts Recursively mergesort each half Merge two sorted arrays into single sorted array 6 Dynamic Programming Algorithm Based on remembering past results Approach 1. Divide problem into smaller subproblems Subproblems must be of same type Subproblems must overlap 2. Solve each subproblem recursively May simply look up solution 3. Combine solutions into to solve original problem 4. Store solution to problem Generally applied to optimization problems Fibonacci Algorithm Fibonacci numbers fibonacci(0) = 1 fibonacci(1) = 1 fibonacci(n) = fibonacci(n-1) + fibonacci(n-2) Recursive algorithm to calculate fibonacci(n) If n is 0 or 1, return 1 Else compute fibonacci(n-1) and fibonacci(n-2) Return their sum Simple algorithm ⇒ exponential time O(2n) 7 Dynamic Programming – Example Dynamic programming version of fibonacci(n) If n is 0 or 1, return 1 Else solve fibonacci(n-1) and fibonacci(n-2) Look up value if previously computed Else recursively compute Find their sum and store Return result Dynamic programming algorithm ⇒ O(n) time Since solving fibonacci(n-2) is just looking up value Dynamic Programming – Example Djikstra’s Shortest Path Algorithm { S } = ∅ C[X] = 0 C[Y] = ∞ for all other nodes while ( not all nodes in { S } ) find node K not in { S } with smallest C[K] add K to { S } for each node M not in { S } adjacent to K C[M] = min ( C[M] , C[K] + cost of (K,M) ) Stores results of smaller subproblems 10 Branch and Bound Algorithm Based on limiting search using current solution Approach Track best current solution found Eliminate partial solutions that can not improve upon best current solution Reduces amount of backtracking Not guaranteed to avoid exponential time O(2n) Branch and Bound – Example Branch and bound algorithm for TSP Find possible paths using recursive backtracking Track cost of best current solution found Stop searching path if cost > best current solution Return lowest cost path If good solution found early, can reduce search May still require exponential time O(2n) 11 Heuristic Algorithm Based on trying to guide search for solution Heuristic ⇒ “rule of thumb” Approach Generate and evaluate possible solutions Using “rule of thumb” Stop if satisfactory solution is found Can reduce complexity Not guaranteed to yield best solution Heuristic Algorithm – Example Heuristic algorithm for TSP Find possible paths using recursive backtracking Search 2 lowest cost edges at each node first Calculate cost of each path Return lowest cost path from first 100 solutions Not guaranteed to find best solution Heuristics used frequently in real applications 12 Summary Wide range of strategies Choice depends on Properties of problem Expected problem size Available resources