Approximation Algorithms - Algorithm and Complexity Analysis - Lecture Slides, Slides of Computer Science

Download Approximation Algorithms - Algorithm and Complexity Analysis - Lecture Slides and more Slides Computer Science in PDF only on Docsity! Approximation Algorithms 2 Coping With NP-Hardness Suppose you need to solve NP-hard problem X. ■ Theory says you aren’t likely to find a polynomial algorithm. ■ Should you just give up? ! Probably yes, if you’re goal is really to find a polynomial algorithm. ! Probably no, if you’re job depends on it. 3 Coping With NP-Hardness Brute-force algorithms. ■ Develop clever enumeration strategies. ■ Guaranteed to find optimal solution. ■ No guarantees on running time. Heuristics. ■ Develop intuitive algorithms. ■ Guaranteed to run in polynomial time. ■ No guarantees on quality of solution. Approximation algorithms. ■ Guaranteed to run in polynomial time. ■ Guaranteed to find "high quality" solution, say within 1% of optimum. ■ Obstacle: need to prove a solution’s value is close to optimum, without even knowing what optimum value is! 4 Approximation Algorithms and Schemes ρ-approximation algorithm. ■ An algorithm A for problem P that runs in polynomial time. ■ For every problem instance, A outputs a feasible solution within ratio ρ of true optimum for that instance. Polynomial-time approximation scheme (PTAS). ■ A family of approximation algorithms {Aε : ε > 0} for a problem P. ■ Aε is a (1 + ε) - approximation algorithm for P. ■ Aε is runs in time polynomial in input size for a fixed ε. Fully polynomial-time approximation scheme (FPTAS). ■ PTAS where Aε is runs in time polynomial in input size and 1 / ε . Docsity.com 5 Approximation Algorithms and Schemes Types of approximation algorithms. ■ Fully polynomial-time approximation scheme. ■ Constant factor. 6 Knapsack Problem Knapsack problem. ■ Given N objects and a "knapsack." ■ Item i weighs wi > 0 Newtons and has value vi > 0. ■ Knapsack can carry weight up to W Newtons. ■ Goal: fill knapsack so as to maximize total value. Item Value Weight 1 1 1 2 6 2 3 18 5 4 22 6 5 28 7 W = 11 OPT value = 40: { 3, 4 } Greedy = 35: { 5, 2, 1 } vi / wi 7 Knapsack is NP-Hard KNAPSACK: Given a finite set X, nonnegative weights wi , nonnegative values vi , a weight limit W, and a desired value V, is there a subset S ⊆ X such that: SUBSET-SUM: Given a finite set X, nonnegative values ui , and an integer t, is there a subset S ⊆ X whose elements sum to t? Claim. SUBSET-SUM ≤ P KNAPSACK. Proof: Given instance (X, t) of SUBSET-SUM, create KNAPSACK instance: ■ vi = wi = ui ■ V = W = t Vv Ww Si i Si i ≥ ≤ ∑ ∑ ∈ ∈ tu tu Si i Si i ≥∑ ≤∑ ∈ ∈ 8 Knapsack: Dynamic Programming Solution 1 OPT(n, w) = max profit subset of items {1, . . . , n} with weight limit w. ■ Case 1: OPT selects item n. – new weight limit = w – wn – OPT selects best of {1, 2, . . . , n – 1} using this new weight limit ■ Case 2: OPT does not select item n. – OPT selects best of {1, 2, . . . , n – 1} using weight limit w Directly leads to O(N W) time algorithm. ■ W = weight limit. ■ Not polynomial in input size! { }    −−+− >− = = otherwise),1(),,1(max ww if),1( 0n if0 ),( n nn wwnOPTvwnOPT wnOPTwnOPT Docsity.com 17 Traveling Salesperson Problem TSP: Given a graph G = (V, E), nonnegative edge weights c(e), and an integer C, is there a Hamiltonian cycle whose total cost is at most C? Is there a tour of length at most 1570? Yes, red tour = 1565. 18 Hamiltonian Cycle Reduces to TSP HAM-CYCLE: given an undirected graph G = (V, E), does there exists a simple cycle C that contains every vertex in V. TSP: Given a complete (undirected) graph G, integer edge weights c(e) ≥ 0, and an integer C, is there a Hamiltonian cycle whose total cost is at most C? Claim. HAM-CYCLE is NP-complete. Proof. (HAM-CYCLE transforms to TSP) ■ Given G = (V, E), we want to decide if it is Hamiltonian. ■ Create instance of TSP with G’ = complete graph. ■ Set c(e) = 1 if e ∈ E, and c(e) = 2 if e ∉ E, and choose C = |V|. ■ Γ Hamiltonian cycle in G ⇔ Γ has cost exactly |V| in G’. Γ not Hamiltonian in G ⇔ Γ has cost at least |V| + 1 in G’. a dc b G a dc b G’ 1 2 19 TSP TSP-OPT: Given a complete (undirected) graph G = (V, E) with integer edge weights c(e) ≥ 0, find a Hamiltonian cycle of minimum cost? Claim. If P ≠ NP, there is no ρ-approximation for TSP for any ρ ≥ 1 . Proof (by contradiction). ■ Suppose A is ρ-approximation algorithm for TSP. ■ We show how to solve instance G of HAM-CYCLE. ■ Create instance of TSP with G’ = complete graph. ■ Let C = |V|, c(e) = 1 if e ∈ E, and c(e) = ρ |V | + 1 if e ∉ E. ■ Γ Hamiltonian cycle in G ⇔ Γ has cost exactly |V| in G’ Γ not Hamiltonian in G ⇔ Γ has cost more than ρ |V| in G’ ■ Gap ⇒ If G has Hamiltonian cycle, then A must return it. 20 TSP Heuristic APPROX-TSP(G, c) ■ Find a minimum spanning tree T for (G, c). h c b a d e gf MST h c b d e gf Input (assume Euclidean distances) a Docsity.com 21 TSP Heuristic APPROX-TSP(G, c) ■ Find a minimum spanning tree T for (G, c). ■ W ← ordered list of vertices in preorder walk of T. ■ H ← cycle that visits the vertices in the order L. h c b a d e gf Hamiltonian Cycle H a b c h d e f g a h c b a d e g Preorder Traversal Full Walk W a b c b h b a d e f e g e d a f 22 TSP Heuristic APPROX-TSP(G, c) ■ Find a minimum spanning tree T for (G, c). ■ W ← ordered list of vertices in preorder walk of T. ■ H ← cycle that visits the vertices in the order L. h c b a d e gf Hamiltonian Cycle H: 19.074 (assuming Euclidean distances) h c b a d e gf An Optimal Tour: 14.715 23 TSP With Triangle Inequality ∆-TSP: TSP where costs satisfy ∆-inequality: ■ For all u, v, and w: c(u,w) ≤ c(u,v) + c(v,w). Claim. ∆-TSP is NP-complete. Proof. Transformation from HAM-CYCLE satisfies ∆-inequality. Ex. Euclidean points in the plane. ■ Euclidean TSP is NP-hard, but not known to be in NP. ■ PTAS for Euclidean TSP. (Arora 1996, Mitchell 1996) u v w (0,0) (5, 9)(-10, 5) K000.3741595510 222222 =+++++ 24 TSP With Triangle Inequality Theorem. APPROX-TSP is a 2-approximation algorithm for ∆-TSP. Proof. Let H* denote an optimal tour. Need to show c(H) ≤ 2c(H*). ■ c(T) ≤ c(H*) since we obtain spanning tree by deleting any edge from optimal tour. h c b a d e gf An Optimal Tour h c b a d e gf MST T Docsity.com 25 TSP With Triangle Inequality Theorem. APPROX-TSP is a 2-approximation algorithm for ∆-TSP. Proof. Let H* denote an optimal tour. Need to show c(H) ≤ 2c(H*). ■ c(T) ≤ c(H*) since we obtain spanning tree by deleting any edge from optimal tour. ■ c(W) = 2c(T) since every edge visited exactly twice. h c b a d e gf MST T h c b a d e gf Walk W a b c b h b a d e f e g e d a 26 TSP With Triangle Inequality Theorem. APPROX-TSP is a 2-approximation algorithm for ∆-TSP. Proof. Let H* denote an optimal tour. Need to show c(H) ≤ 2c(H*). ■ c(T) ≤ c(H*) since we obtain spanning tree by deleting any edge from optimal tour. ■ c(W) = 2c(T) since every edge visited exactly twice. ■ c(H) ≤ c(W) because of ∆-inequality. h c b a d e gf Walk W a b c b h b a d e f e g e d a h c b a d e gf Hamiltonian Cycle H a b c h d e f g a 27 TSP: Christofides Algorithm Theorem. There exists a 1.5-approximation algorithm for ∆-TSP. CHRISTOFIDES(G, c) ■ Find a minimum spanning tree T for (G, c). ■ M ← min cost perfect matching of odd degree nodes in T. h c b a d e gf MST T h c b e gf Matching M 28 TSP: Christofides Algorithm Theorem. There exists a 1.5-approximation algorithm for ∆-TSP. CHRISTOFIDES(G, c) ■ Find a minimum spanning tree T for (G, c). ■ M ← min cost perfect matching of odd degree nodes in T. ■ G’ ← union of spanning tree and matching edges. h c b a d e gf G’ = MST + Matching h c b e gf Matching M Docsity.com 37 Greedy algorithm. ■ Consider jobs in some fixed order. ■ Assign job j to machine whose load is smallest so far. ■ Note: this is an "on-line" algorithm. FOR i = 1 to m Ti ← 0, Si ← φ FOR j = 1 to n i = argmink Tk Si ← Si ∪ {j} Ti ← Ti + pj LIST-SCHEDULING (m, n, p1, . . . , pn) machine with smallest load assign job j to machine i Load Balancing 38 Load Balancing Theorem (Graham, 1966). Greedy algorithm is a 2-approximation. ■ First worst-case analysis of an approximation algorithm. ■ Need to compare resulting solution with optimal makespan T*. Lemma 1. The optimal makespan is at least ■ The total processing time is Σ j p j . ■ One of m machines must do at least a 1/m fraction of total work. Lemma 2. The optimal makespan is at least ■ Some machine must process the most time-consuming job. .max* jj pT ≥ .* 1 ∑≥ j jm pT 39 Load Balancing Lemma 1. The optimal makespan is at least Lemma 2. The optimal makespan is at least Theorem. Greedy algorithm is a 2-approximation. Proof. Consider bottleneck machine i that works for T units of time. ■ Let j be last job scheduled on machine i. ■ When job j assigned to machine i, i has smallest load. It’s load before assignment is Ti - pj ⇒ Ti - pj ≤ Tk for all 1 ≤ k ≤ m. .* 1 ∑≥ j jm pT Machine 3 Machine 2 Machine 1A D F B C j = G E IH J 0 T = TiTi - pi .max* jj pT ≥ Machine i 40 Load Balancing Lemma 1. The optimal makespan is at least Lemma 2. The optimal makespan is at least Theorem. Greedy algorithm is a 2-approximation. Proof. Consider bottleneck machine i that works for T units of time. ■ Let j be last job scheduled on machine i. ■ When job j assigned to machine i, i has smallest load. It’s load before assignment is Ti - pj ⇒ Ti - pj ≤ Tk for all 1 ≤ k ≤ n. ■ Sum inequalities over all k and divide by m, and then apply L1. ■ Finish off using L2. * 1 1 T p TpT k km k kmji ≤ = ≤− ∑ ∑ *2 ** )( T TT ppTT jjii = +≤ +−= .* 1 ∑≥ j jm pT .max* jj pT ≥ Docsity.com 41 Load Balancing Is our analysis tight? ■ Essentially yes. ■ We give instance where solution is almost factor of 2 from optimal. – m machines, m(m-1) jobs with of length 1, 1 job of length m – 10 machines, 90 jobs of length 1, 1 job of length 10 Machine 55 15 25 35 45 55 65 75 85 Machine 66 16 26 36 46 56 66 76 86 Machine 77 17 27 37 47 57 67 77 87 Machine 88 18 28 38 48 58 68 78 88 Machine 21 11 21 31 41 51 61 71 81 Machine 22 12 22 32 42 52 62 72 82 Machine 33 13 23 33 43 53 63 73 83 Machine 44 14 24 34 44 54 64 74 84 Machine 99 19 29 39 49 59 69 79 89 Machine 1010 20 30 40 50 60 70 80 90 91 List Schedule makespan = 19 42 Load Balancing Is our analysis tight? ■ Essentially yes. ■ We give instance where solution is almost factor of 2 from optimal. – m machines, m(m-1) jobs with of length 1, 1 job of length m – 10 machines, 90 jobs of length 1, 1 job of length 10 Machine 55 15 25 35 45 55 65 75 85 Machine 66 16 26 36 46 56 66 76 86 Machine 77 17 27 37 47 57 67 77 87 Machine 88 18 28 38 48 58 68 78 88 Machine 11 11 21 31 41 51 61 71 81 Machine 22 12 22 32 42 52 62 72 82 Machine 33 13 23 33 43 53 63 73 83 Machine 44 14 24 34 44 54 64 74 84 Machine 99 19 29 39 49 59 69 79 89 Machine 1091 Optimal makespan = 10 50 60 70 80 10 20 30 40 90 43 Load Balancing: State of the Art What’s known. ■ 2-approximation algorithm. ■ 3/2-approximation algorithm: homework. ■ 4/3-approximation algorithm: extra credit. ■ PTAS. Docsity.com