Download Algorithmic Complexity: Understanding Upper and Lower Bounds of Algorithmic Problems and more Study notes Computer Science in PDF only on Docsity! On the Board • Homework 5: due Monday • Next exam: handed out next week Algorithmic Complexity Last lecture, we returned again to the idea that we can place lower and upper bounds on the solution of algorithmic problems. To establish an upper bound, we can exhibit an algorithm. Given some problem, an algorithm to solve the problem constructively establishes an upper bound — the problem can be solved in the given time bound. Lower bounds are established by a proof — such proofs are usually subtle and hard to construct. Recall that an algorithm has polynomial complexity if running time < c · f(size of problem) where f(N) is a polynomial in N and c is a constant. Algorithms that are faster than polynomial take constant time — that is, running time < c for some constant c, independent of the size of the input set. Finding the first element of a list takes constant, or O(1), time. Algorithms whose running time cannot be bounded by a polynomial function are said to take superpolynomial time. Examples of superpolynomial functions include 2N, N!, and functions that grow even faster, such as Ackerman’s function. Problems with Exponential Complexity Some problems have complexities that are not polynomial. Consider the monkey-puzzle problems. You know about these “kids” puzzles. They consist of a picture subdivided into a square number of cards. The picture is such that each edge between two cards has a picture crossing the boundary. The goal is to discover an orientation for the cards that satisfies each boundary constraint — it lines up all the pictures correctly. COMP 200: Elements of Computer Science Fall 2004 Lecture 23: October 24, 2004 Upper & Lower Bounds, NP vs. P The obvious way to solve this problem is to enumerate all of the orientations. If the puzzle has N cards, there are N! orientations. Starting at one position, choose a card — there are 25 choices. The next position has 24 choices, followed by 23, and so on. Thus, total number of choices is 25 · 24 · 23 · 22 · 21 · … · 3 · 2 · 1 or, for N cards, N! possibilities. The obvious algorithm takes O(N!) time. The monkey puzzle problem is genuinely hard. N! grows so rapidly that we cannot solve it effectively for most values of N. N N! 4 24 9 362880 16 20,922,789,888,000 25 1.551121004333098e+25 While computers keep getting faster (twice as fast every 18 to 24 months, over the last twenty-five years), that kind of growth cannot begin to keep up with O(N!). Is there a better algorithm for the monkey puzzle problem? We can improve the algorithm by having it trim the choices that are infeasible — when the first k choices result in some misalignment of the cards. This improved algorithm still takes O(N!) time — recall, O is a limit, not a tight function describing the actual behavior. In fact, any algorithm to solve this problem requires O(N!) time in the number of cards. It is a classic example of a problem that cannot be solved effectively — an intractable problem. Problems with complexity bounded by a polynomial function are considered tractable. The notion that algorithms with polynomial complexity are tractable and those with superpolynomial (greater than polynomial) complexity are intractable might create a divided universe of algorithms. Pictorially, we could represent it as: nonpolynomial
