Asymptotic Notation: O-, Ω-, Θ-notation and Recurrences, Assignments of Computer science

Download Asymptotic Notation: O-, Ω-, Θ-notation and Recurrences and more Assignments Computer science in PDF only on Docsity! September 12, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.1 Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method • Iterating the recurrence • Recursion tree • Master method Prof. Erik Demaine September 12, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.2 Asymptotic notation O-notation (upper bounds): We write f(n) = O(g(n)) if there exist constants c > 0, n0 > 0 such that 0 ≤ f(n) ≤ cg(n) for all n ≥ n0. e rite f(n) (g(n)) if there exist constants c 0, n0 0 such that 0 f(n) cg(n) for all n n0. September 12, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.5 Asymptotic notation O-notation (upper bounds): We write f(n) = O(g(n)) if there exist constants c > 0, n0 > 0 such that 0 ≤ f(n) ≤ cg(n) for all n ≥ n0. e rite f(n) (g(n)) if there exist constants c 0, n0 0 such that 0 f(n) cg(n) for all n n0. EXAMPLE: 2n2 = O(n3) (c = 1, n0 = 2) functions, not values funny, “one-way” equality September 12, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.6 Set definition of O-notation O(g(n)) = { f(n) : there exist constants c > 0, n0 > 0 such that 0 ≤ f(n) ≤ cg(n) for all n ≥ n0 } (g(n)) { f(n) : there exist constants c 0, n0 0 such that 0 f(n) cg(n) for all n n0 } September 12, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.7 Set definition of O-notation O(g(n)) = { f(n) : there exist constants c > 0, n0 > 0 such that 0 ≤ f(n) ≤ cg(n) for all n ≥ n0 } (g(n)) { f(n) : there exist constants c 0, n0 0 such that 0 f(n) cg(n) for all n n0 } EXAMPLE: 2n2 ∈ O(n3) September 12, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.10 Macro substitution Convention: A set in a formula represents an anonymous function in the set. f(n) = n3 + O(n2) means f(n) = n3 + h(n) for some h(n) ∈ O(n2) . EXAMPLE: September 12, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.11 Macro substitution Convention: A set in a formula represents an anonymous function in the set. n2 + O(n) = O(n2) means for any f(n) ∈ O(n): n2 + f(n) = h(n) for some h(n) ∈ O(n2) . EXAMPLE: September 12, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.12 Ω-notation (lower bounds) O-notation is an upper-bound notation. It makes no sense to say f(n) is at least O(n2). September 12, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.15 Θ-notation (tight bounds) Θ(g(n)) = O(g(n)) ∩ Ω(g(n))(g(n)) (g(n)) (g(n)) September 12, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.16 Θ-notation (tight bounds) Θ(g(n)) = O(g(n)) ∩ Ω(g(n))(g(n)) (g(n)) (g(n)) )(2 22 2 1 nnn Θ=−EXAMPLE: September 12, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.17 ο-notation and ω-notation O-notation and Ω-notation are like ≤ and ≥. o-notation and ω-notation are like < and >. ο(g(n)) = { f(n) : for any constant c > 0, there is a constant n0 > 0 such that 0 ≤ f(n) < cg(n) for all n ≥ n0 } (g(n)) { f(n) : for any constant c 0, there is a constant n0 0 such that 0 f(n) cg(n) for all n n0 } EXAMPLE: (n0 = 2/c)2n2 = o(n3) September 12, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.20 Substitution method 1. Guess the form of the solution. 2. Verify by induction. 3. Solve for constants. The most general method: September 12, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.21 Substitution method 1. Guess the form of the solution. 2. Verify by induction. 3. Solve for constants. The most general method: EXAMPLE: T(n) = 4T(n/2) + n • [Assume that T(1) = Θ(1).] • Guess O(n3) . (Prove O and Ω separately.) • Assume that T(k) ≤ ck3 for k < n . • Prove T(n) ≤ cn3 by induction. September 12, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.22 Example of substitution 3 33 3 3 ))2/(( )2/( )2/(4 )2/(4)( cn nnccn nnc nnc nnTnT ≤ −−= += +≤ += desired – residual whenever (c/2)n3 – n ≥ 0, for example, if c ≥ 2 and n ≥ 1. desired residual September 12, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.25 A tighter upper bound? We shall prove that T(n) = O(n2). September 12, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.26 A tighter upper bound? We shall prove that T(n) = O(n2). Assume that T(k) ≤ ck2 for k < n: )( )2/(4 )2/(4)( 2 2 2 nO ncn nnc nnTnT = += +≤ += September 12, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.27 A tighter upper bound? We shall prove that T(n) = O(n2). Assume that T(k) ≤ ck2 for k < n: )( )2/(4 )2/(4)( 2 2 2 nO ncn nnc nnTnT = += +≤ += Wrong! We must prove the I.H. September 12, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.30 A tighter upper bound! IDEA: Strengthen the inductive hypothesis. • Subtract a low-order term. Inductive hypothesis: T(k) ≤ c1k2 – c2k for k < n. T(n) = 4T(n/2) + n = 4(c1(n/2)2 – c2(n/2)) + n = c1n2 – 2c2n + n = c1n2 – c2n – (c2n – n) ≤ c1n2 – c2n if c2 ≥ 1. September 12, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.31 A tighter upper bound! IDEA: Strengthen the inductive hypothesis. • Subtract a low-order term. Inductive hypothesis: T(k) ≤ c1k2 – c2k for k < n. T(n) = 4T(n/2) + n = 4(c1(n/2)2 – c2(n/2)) + n = c1n2 – 2c2n + n = c1n2 – c2n – (c2n – n) ≤ c1n2 – c2n if c2 ≥ 1. Pick c1 big enough to handle the initial conditions. September 12, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.32 Recursion-tree method • A recursion tree models the costs (time) of a recursive execution of an algorithm. • The recursion-tree method can be unreliable, just like any method that uses ellipses (…). • The recursion-tree method promotes intuition, however. • The recursion tree method is good for generating guesses for the substitution method. September 12, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.35 Example of recursion tree Solve T(n) = T(n/4) + T(n/2) + n2: T(n/4) T(n/2) n2 September 12, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.36 Example of recursion tree Solve T(n) = T(n/4) + T(n/2) + n2: n2 (n/4)2 (n/2)2 T(n/16) T(n/8) T(n/8) T(n/4) September 12, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.37 Example of recursion tree Solve T(n) = T(n/4) + T(n/2) + n2: n2 (n/16)2 (n/8)2 (n/8)2 (n/4)2 (n/4)2 (n/2)2 Θ(1) … September 12, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.40 Example of recursion tree Solve T(n) = T(n/4) + T(n/2) + n2: (n/16)2 (n/8)2 (n/8)2 (n/4)2 (n/4)2 Θ(1) … 2 16 5 n 2n 2 256 25 n n2 (n/2)2 … September 12, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.41 Example of recursion tree Solve T(n) = T(n/4) + T(n/2) + n2: (n/16)2 (n/8)2 (n/8)2 (n/4)2 (n/4)2 Θ(1) … 2 16 5 n 2n 2 256 25 n ( ) ( )( ) 1 3 16 52 16 5 16 52 L++++n = Θ(n2) … Total = n2 (n/2)2 geometric series September 12, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.42 The master method The master method applies to recurrences of the form T(n) = a T(n/b) + f (n) , where a ≥ 1, b > 1, and f is asymptotically positive. September 12, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.45 Three common cases (cont.) Compare f (n) with nlogba: 3. f (n) = Ω(nlogba + ε) for some constant ε > 0. • f (n) grows polynomially faster than nlogba (by an nε factor), and f (n) satisfies the regularity condition that a f (n/b) ≤ c f (n) for some constant c < 1. Solution: T(n) = Θ( f (n)) . September 12, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.46 Examples EX. T(n) = 4T(n/2) + n a = 4, b = 2 ⇒ nlogba = n2; f (n) = n. CASE 1: f (n) = O(n2 – ε) for ε = 1. ∴ T(n) = Θ(n2). September 12, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.47 Examples EX. T(n) = 4T(n/2) + n a = 4, b = 2 ⇒ nlogba = n2; f (n) = n. CASE 1: f (n) = O(n2 – ε) for ε = 1. ∴ T(n) = Θ(n2). EX. T(n) = 4T(n/2) + n2 a = 4, b = 2 ⇒ nlogba = n2; f (n) = n2. CASE 2: f (n) = Θ(n2lg0n), that is, k = 0. ∴ T(n) = Θ(n2lg n). September 12, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.50 f (n/b) Idea of master theorem Recursion tree: f (n/b) f (n/b) Τ (1) … … f (n) a f (n/b2)f (n/b2) f (n/b2)… a September 12, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.51 f (n/b) Idea of master theorem Recursion tree: f (n/b) f (n/b) Τ (1) … … f (n) a f (n/b2)f (n/b2) f (n/b2)… a f (n) a f (n/b) a2 f (n/b2) … September 12, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.52 f (n/b) Idea of master theorem f (n/b) f (n/b) Τ (1) … Recursion tree: … f (n) a f (n/b2)f (n/b2) f (n/b2)… ah = logbn f (n) a f (n/b) a2 f (n/b2) … September 12, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.55 f (n/b) Idea of master theorem Recursion tree: f (n/b) f (n/b) Τ (1) … … f (n) a f (n/b2)f (n/b2) f (n/b2)… ah = logbn f (n) a f (n/b) a2 f (n/b2) CASE 2: (k = 0) The weight is approximately the same on each of the logbn levels. ASE 2: (k = 0) The eight is approxi ately the sa e on each of the logbn levels. Θ(nlogbalg n) nlogbaΤ (1) … September 12, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.56 f (n/b) Idea of master theorem Recursion tree: f (n/b) f (n/b) Τ (1) … … f (n) a f (n/b2)f (n/b2) f (n/b2)… ah = logbn f (n) a f (n/b) a2 f (n/b2) …CASE 3: The weight decreases geometrically from the root to the leaves. The root holds a constant fraction of the total weight. ASE 3: The eight decreases geo etrically fro the root to the leaves. The root holds a constant fraction of the total eight. nlogbaΤ (1) Θ( f (n)) September 12, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.57 Appendix: geometric series 1 11 1 2 x xxxx n n − −=++++ + L for x ≠ 1 1 11 2 x xx − =+++ L for |x| < 1 Return to last slide viewed.