Importance of Growth Order - Lecture Slides | CPSC 5155U, Study notes of Computer Architecture and Organization

Download Importance of Growth Order - Lecture Slides | CPSC 5155U and more Study notes Computer Architecture and Organization in PDF only on Docsity! Slide 9.1 Importance of Growth Order Suppose that a given algorithm takes 1 minute to solve a problem of size 10. How long to solve a problem of size 20? How long for size 30? Complexity Ratio (20) Time (20) Ratio (30) Time (30) O(log N) log(20)/log(10) log(30)/log(10) 1.301 1.3 minutes 1.477 90 seconds O(N) 20/10 = 2 2 minutes 30/10 = 3 3 minutes O(N log N) 2 * 1.301 = 2.6 2.6 minutes 3 * 1.477 = 4.4 4.4 minutes O(N2) 202 / 102 = 4 4 minutes 302 / 102 = 9 9 minutes O(N3) 203 / 103 = 303 / 103 = 8000/ 1000 = 8 8 minutes 27000 / 1000 27 minutes O(2N) 220 / 210 = 210 230 / 210 = 220 = 1024 17 hours = 1,048,576 728 days Slide 10.1 Asymptotic Growth Rates There are three sets of interest to our investigation of the “big picture” when studying algorithmic time complexity. O(G(N)) the “Big O” notation, and (G(N)) the “Big Omega” notation, and (G(N)) the “Big Theta” notation. The set O(G(N)) is the set of functions that grow no faster than G(N). The set (G(N)) is the set of functions that grow no slower than G(N). The set (G(N)) is the set of functions that grow exactly as fast as G(N). This terminology is a bit odd, arising as it does from set theory. The function 2N2 + 5N – 17 is a member of the set (N2), which is the set of quadratic functions. We normally just say that it is a quadratic. Slide 10.4 Some Sets of Functions ( log(N) ) the set of logarithmic functions: log2(N), ln(N), and log10(N); (N) the set of linear functions; (N2) the set of quadratic functions; (N3) the set of cubic functions; NOTE: T(N)  ( G(N) ) if and only if both T(N)  O( G(N) ) and T(N)  ( G(N) ). Slide 14.1 Use of Calculus in Establishing Complexity Consider a function T(N). We say the following 1) If )N(G )N(T lim N  > 0, then T(N) is (G(N)). Note that the limit could be . 2) If )N(G )N(T lim N  = C for some positive constant C, then T(N) is (G(N)). 3) If )N(G )N(T lim N  < , then T(N) is O(G(N)). Note that the limit could be 0. When we call for C > 0 as a constant, we mean a finite number. Slide 14.2 Calculus for Complexity Measures (Part 2) Consider T(N) =13N2 + 52N + 17. We use the calculus methods to prove some properties. Claim: T(N) is (N) N )N(T lim N  = N 17N52N13 lim 2 N   =         N 17 52N13lim N = . Claim: T(N) is (N2) N )N(T lim 2N  = N 17N52N13 lim 2 2 N   =         N 17 N 52 13lim 2 N = 13. Claim: T(N) is (N2) N )N(T lim 2N  = N 17N52N13 lim 2 2 N   =         N 17 N 52 13lim 2 N = 13, as just above. Slide 15.2 L’Hôpital’s Rule (Part 2) T(N) = Nlog(N) + N + 3 Show T(N) is O(N2). We evaluate N 3N)Nlog(N lim 2 N   =         N 3 N 1 N )Nlog( lim 2 N =      N )Nlog( lim N + 0 + 0 But  )Nlog(limN  =  and  NlimN  = , so the rule applies. Also dN )N(logd = 1 / N and dN )N(d = 1, so      N )Nlog( lim N =      1 N/1 lim N = 0 and N 3N)Nlog(N lim 2 N   =      N )Nlog( lim N + 0 + 0 = 0. T(N) is O(N2). Slide 15.3 More Calculus T(N) = Nlog(N) + N + 3 Show T(N) is O( Nlog(N) ). We evaluate )Nlog(N 3N)Nlog(N lim N    =          )Nlog(N 3 )Nlog( 1 1lim N  001lim N   = 1. As        )Nlog( 1 lim N = 0 and        )Nlog(N 3 lim N = 0. So T(N) is ( Nlog(N) ). Slide 19.1 Introduction to Loop Counting We need one new sum formula to consider loop counting. Consider the following loop structure For J = 1 to N do // the outer loop For K = 1 to N do // the inner loop {Something} For each execution of the outer loop, the inner loop is executed N times. The inner loop is executed N  N = N2 times. This is easy. Slide 27.2 Time Complexity: Fibonacci Numbers There are many algorithms to calculate Fibonacci numbers. The recursive algorithm is as follows: Fibonacci numbers, introduced by Leonardo Fibonacci in 1202, are defined by the following recurrence: F(0) = 0 F(1) = 1 F(N) = F(N – 1) + F(N – 2) for N > 1. The sequence begins: 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, etc. The time complexity for the recursive computation is given by T(N) = T(N – 1) + T(N – 2) for N > 1. Slide 27.3 Time Complexity: The Master Theorem Theorem: Let T(N) satisfy the following recurrence equation T(N) = AT(N / B) + F(N) T(1) = C where A ≥ 1, B ≥ 1, and C > 0. If F(N)  (Nd), where d ≥ 0, then T(N)  (Nd) if A < Bd T(N)  (Ndlog(N)) if A = Bd T(N)  (NP) , P = LogB(A) if A > Bd Please note that the recurrence T(N) = AT(N – B) + F(N) cannot be solved using the Master Theorem. Slide 27.4 Time Complexity: The Master Theorem T(N) = 4T(N / 2) + N Here A = 4, B = 2, and F(N) = N. F(N)  (Nd) for d = 1. Bd = 21 = 2, so A > Bd and we calculate logB(A) = log2(4) = 2. We conclude that T(N)  (N2). T(N) = 4T(N / 2) + N2 Here A = 4, B = 2, and F(N) = N2. F(N)  (Nd) for d = 2. Bd = 22 = 4, so A = Bd and T(N)  (N2log(N)). T(N) = 4T(N / 2) + N3 Here A = 4, B = 2, and F(N) = N3. F(N)  (Nd) for d = 3. Bd = 23 = 8, so A < Bd and T(N)  (N3).