Algorithm Design Techniques: Brute Force, Divide-and-Conquer, Dynamic Programming, Greedy , Study notes of Computer Science

Download Algorithm Design Techniques: Brute Force, Divide-and-Conquer, Dynamic Programming, Greedy and more Study notes Computer Science in PDF only on Docsity! Algorithm Design Techniques CSC 3102 0.1 B.B. Karki, LSU B.B. Karki, LSU0.2CSC 3102 Algorithm Design Techniques  Various design techniques exist:  Classifying algorithms based on design ideas or commonality.  General-problem solving strategies.  Brute force  Divide-and-conquer  Decrease-and-conquer  Transform-and-conquer  Space-and-time tradeoffs  Dynamic programming  Greedy techniques B.B. Karki, LSU0.5CSC 3102 The Simplest Approach  Brute force - the simplest of the design strategies  Is a straightforward approach to solving a problem, usually directly based on the problem’s statement and definitions of the concepts involved.  Just do it - the brute-force strategy is easiest to apply.  Results in an algorithm that can be improved with a modest amount of time.  Brute force is important due to its wide applicability and simplicity.  Weakness is subpar efficiency of most brute-force algorithms.  Important examples:  Selection sort, String matching, Convex-hull problem, and Exhaustive search. B.B. Karki, LSU0.6CSC 3102 Selection Sort Algorithm SelectionSort (A[0..n-1]) //Sorts a given array //Input: An array A[0..n-1] of orderable elements //Output: Array A[0..n-1] sorted in ascending order for i ← 0 to n - 2 do min ← i for j ← i + 1 to n - 1 do if A[j] < A[min] min ← j swap A[i] and A[min] € C(n) = 1 j= i+1 n−1 ∑ = [(n −1) − (i +1) +1] = (n −1− i) i= 0 n−2 ∑ i= 0 n−2 ∑ i= 0 n−2 ∑ = (n −1)n2 ∈ Θ(n 2)  Scan the list repeatedly to find the elements, one at a time, in an non- decreasing order.  On the ith pass through the list, search for the smallest item among the last n - i elements and swap it with A[i]. After n - 1 passes, the list is sorted. B.B. Karki, LSU0.7CSC 3102 Brute-Force String Matching Algorithm BruteForceStringMatching (T[0..n - 1], P[0..m - 1]) //Implements string matching //Input: An array T[0..n - 1] of n characters representing a text // an array P[0..m - 1] of m characters representing a pattern //Output: The position of the first character in the text that starts the first // matching substring if the search is successful and -1 otherwise. for i ← 0 to n - m do j ← 0 while j < m and P[j] = T[i + j] do j ← j + 1 if j = m return i return -1 In the worst case, the algorithm is Θ(mn). But the average-case efficiency is shown to be linear, i.e., Θ(m + n) = Θ(n) for searching in random texts.  Align the pattern against the first m characters of the text and start matching the corresponding pairs of characters from left to right until all m pairs match. But, if a mismatching pair is found, then the pattern is shift one position to the right and character comparisons are resumed.  The goal is to find i - the index of the leftmost character of the first matching substring in the text such that ti = p0,….ti+j = pj, ….. Ti+m-1 = pm-1 B.B. Karki, LSU0.10CSC 3102 Exhaustive Search  Exhaustive search is a brute-force approach to combinatorial problems.  It suggests generating each and every combinatorial object of the problem, selecting those of them of that satisfy the problem’s constraints and then finding a desired object.  Impratical for all but applicable to very small instances of problems.  Examples:  Traveling salesman problem  Finding the shortest tour through a given set of n cities that visits each city exactly once before returning to the city where it started.  Knapsack problem  Finding the most valuable list of out-of n items that fit into the knapscak.  Assignment problem  Finding an assignment of n people to execute n jobs with the smallest total cost.  These problems are the examples of so-called NP-hard problems. B.B. Karki, LSU0.11CSC 3102 Traveling Salesman Problem  Traveling salesman problem  Asks to find the shortest tour through a given set of n cities that visits each city exactly once before returning to the city where it started.  Finding the shortest Hamiltonian circuit of the graph  A cycle that passes through all the vertices of the graph exactly once.  A sequence of n +1 adjacent vertices v0, v1, v2, … vn-1, v0.  Get all tours by generating all the permutations of n - 1 intermediate cities, compute the tour lengths, and find the shortest among them.  Efficiency:  Total number of permutations = (n - 1)!/2  Impractical but ok very small values of n. 5 1 7 8 2 3 Tour Length a b c d a 2+8+1+7 = 18 a b d c a 2+3+1+5 = 11 a c b d a 23 a c d b a 11 a d b c a 23 a d c b a 18 More than one optimal solutions. a b dc B.B. Karki, LSU0.12CSC 3102 Knapsack Problem  Given n items of known weights w1, …., wn and values v1, …, vn and a knapsack of capacity W, find the most valuable subset of the items that fit into the knapsack.  Consider all the subsets of the set of n items given,  Computing the total weight of each subset in order to identify feasible subsets (the ones with the total not exceeding the knapsack’s capacity).  Finding a subset of the largest value among them.  The number of subsets of an n- element set is 2n so the exhaustive search leads to Ω(2n) algorithm. Knapsack item1 item2 item3 item4 W = 10 w = 7 v = $42 w = 3 v = $12 w = 4 v = $40 w = 5 v = $25 Subset Total weight Total value Null 0 $0 {1} 7 $42 {2} 3 $12 {3} 4 $40 {4} 5 $25 {1,2} 10 $36 {1,3} 11 not feasible {1,4} 12 not feasible {2,3} 7 $52 {2,4} 8 $37 {3,4} 9 $65 {1,2,3} 14 not feasible {1,2,4} 15 not feasible … {1,2,3,4} 19 not feasible