Analyzing Algorithms using Recurrence Equations, Study notes of Data Structures and Algorithms

Download Analyzing Algorithms using Recurrence Equations and more Study notes Data Structures and Algorithms in PDF only on Docsity! CS230 Data Structures Handout #33 Prof. Lyn Turbak Thursday, April 19, 2007 Wellesley College RECURRENCES --------------------------------------------------------------------------------------------------------------------- Overview of Today’s Lecture • Recurrence equations: a tool for analyzing algorithms • Applications of recurrence equations --------------------------------------------------------------------------------------------------------------------- Two-Step Strategy for Analyzing Algorithms 1. Characterize running-time (space, etc.) of algorithm by a recurrence equation. 2. Solve the recurrence equation, expressing answer in asymptotic notation. --------------------------------------------------------------------------------------------------------------------- Recurrence Equations A recurrence equation is just a recursive function definition. It defines a function at one input in terms of its value on smaller inputs. We use recurrence equations to characterize the running time of algorithms. T(n) typically stands for the running-time of a given algorithm on an input of size n for a particular case (worst-case by default, but could be average-case or best case). Recurrence equations for divide-and-conquer algorithms typically have the form: General Case (n ≥ 1) T(n) = [# of subproblems] T ([size of each subproblem]) + [cost of divide&glue] Base Case (n < 1) T(n) = 0 Because the base case is always the same, it is usually omitted when defining T(n). 2 --------------------------------------------------------------------------------------------------------------------- Deriving Recurrence Equations: Examples 1. Insert an element into a sorted list of length n. T(n) = ______T(__________) + __________ 2. Insertion-sort a list of length n. T(n) = ______T(__________) + __________ 3. Binary search on an array of n elements. T(n) = ______T(__________) + __________ 4. Find the maximum value in a balanced binary tree (not a BST) with n numeric leaves. T(n) = ______T(__________) + __________ 5. Merge-sort a list of n elements. T(n) = ______T(__________) + __________ 6. Create a BST for a list of n elements, assuming that the BST remains balanced after each insertion. T(n) = ______T(__________) + __________ 7. Given a balanced binary tree and an element x, multiply each node n on the path to x by the sum of the elements in the subtree rooted at n. T(n) = ______T(__________) + __________ 8. Towers of Hanoi T(n) = ______T(__________) + __________ 5 Case Recurrence Solution Worst Best Average --------------------------------------------------------------------------------------------------------------------- Matching Theoretical Results with Performance Numbers for Insertion Sort Timings for insertion sort on Java vectors (in milliseconds) n Sorted Reverse Sorted Random 1000 1 105 43 2000 0 350 173 4000 1 1413 706 8000 4 5707 2850 16000 7 27769 12416 32000 14 145265 59143 6 Analyzing Selection Sort NED U N S O R T E D UN S O R T ED UNS O R TED USO R T UN SO R TED UN SO R TED UN SO R TED UN SO R TED UN SO R TED Case Recurrence Solution Worst Best Average --------------------------------------------------------------------------------------------------------------------- Performance Numbers for Selection Sort Timings for selection sort on Java vectors (in milliseconds) n Sorted Reverse Sorted Random 1000 84 81 81 2000 314 320 331 4000 1254 1279 1785 8000 6777 7373 7263 16000 35752 36209 37659 32000 174791 166806 173995 Idea: On the ith step of the algorithm, store into A[i] the smallest element of A[i..n]. Invariant: After step i, the elements A[1..i] are in their final sorted positions. 7 --------------------------------------------------------------------------------------------------------------------- Recurrence 3: T(n) = T(n/2) + 1 Also derivable: T(n/2) = T(n/4) + 1 T(n /4) = T(n/8) + 1 etc. 1 1 1 . . . lg(n) + 1 levels T(n) T(n/2) T(n/4) 1 1 1 ... 1 Total cost of nodes = number of nodes = lg(n) + 1 = Θ(lg(n)) (In this handout, we use lg(n) to stand for log2(n). ) Recurrence 4: T(n) = 2T(n/2) + n Also derivable: T(n/2) = 2T(n/4) + n/2 T(n/4) = 2T(n/8) + n/4 T(n) ... n . . . T(n/2) T(n/2) n T(n/4) T(n/4) T(n/4) T(n/4) 1 1 11 1 . .. level 1 level 2 level 3 level lg(n) + 1 n/2 n/2 n n/2 n/2 n/4 1 1n/4 1n/4 1n/4 1 1 n n n n Total cost = (n)(number of levels) = n(lg(n) + 1) = Θ(n⋅ lg(n)) Information Services Comment: (We start with one T(n) node and generate a number of subnodes equal to the number of times n can be successively divided by 2. The latter quantity is lg(n); including the 1 generated by the original T(n) node gives lg(n) + 1 nodes.) 10 Analyzing Merge Sort U N S O R T E D U N S O R T E D U N S O R T E D U N S O R T E D U N S O R T E D UN SO R T ED UN SO R TED UN SO R TED Case Recurrence Solution Worst Best Average --------------------------------------------------------------------------------------------------------------------- Performance Numbers for Merge Sort Timings for merge sort on Java vectors (in milliseconds) n Sorted Reverse Sorted Random 1000 23 4 6 2000 11 10 12 4000 23 24 27 8000 51 49 60 16000 110 113 130 32000 242 250 287 Idea: Recursively sort subarrays and then merge them into a single sorted array. 11 Analyzing Quick Sort U S O R TQ I C K U SO R TQI C K U SO R TQI CK O I CK U S R TQ OIC K IC K IK I K O C I K US RT Q US RT Q SR TQ SR TQ S RQ S RQ SR SR Case Recurrence Solution Worst Best Average --------------------------------------------------------------------------------------------------------------------- Performance Numbers for Quick Sort Timings for quick sort on Java vectors (in milliseconds) n Sorted Reverse Sorted Random 1000 48 45 2 2000 170 171 5 4000 680 685 12 8000 2801 2739 26 16000 15246 12062 58 32000 79110 52943 130 Idea: Choose A[0] as the pivot. Step 1: Partition A into two subarrays by moving elements in A: 1. lesses: all elements < pivot 2. greaters: all elements >= pivot The lesses subarray should occupy the left portion of A and greaters should occupy the right portion of A. One way to partition is to use two fingers moving inward starting at opposite ends of arrays and swapping when left element is >= pivot and right element is < pivot. Step 2: Recursively sort lesses and greaters. 12 ------------------------------------------------------------------------------------------------------------ Recurrence 6: T(n) = T(n-1) + lg(n) Also derivable: T(n-1) = T(n-2) + lg(n-1) T(n-2) = T(n-3) + lg(n-2) Picture goes here: T(n) = lg(n) + lg(n-1) + lg(n-2) … + lg(2) + lg(1) = lg(n ⋅ (n-1) ⋅ (n-2) ⋅ … ⋅ 3 ⋅ 2 ⋅ 1) {Because lg(a) + lg(b) = log(a ⋅ b)} = lg(n!) Θ(lg(n!)) ≈ Θ(n lg(n)) by Stirling’s approximation