Guessing Game: NP-Complete?

Transcription

1 Guessing Game: NP-Complete? 1. LONGEST-PATH: Given a graph G = (V, E), does there exists a simple path of length at least k edges? YES 2. SHORTEST-PATH: Given a graph G = (V, E), does there exists a simple path of length at most k edges? In P 3. 2-SAT: Give a formula Φ such that each clause has at most 2 literals, is Φ is satisfiable? In P 4. 3-COLOR: Given a graph G = (V, E), can we color the nodes of G with 3 colors such that no two nodes joined by an edge have the same coloring YES 5. Factoring: Give an integer N. Find the factors of N. INAPPLICABLE 1 Copyright 2000, Kevin Wayne 1

3 Coping With NP-Completeness Q. Suppose I need to solve an NP-complete problem. What should I do? A. Theory says you're unlikely to find poly-time algorithm. Must sacrifice one of three desired features. Solve problem to optimality. Solve problem in polynomial time. Solve arbitrary instances of the problem. This lecture. Solve some special cases of NP-complete problems that arise in practice. 3 Copyright 2000, Kevin Wayne 3

11 Independent Set on Trees Independent set on trees. Given a tree, find a maximum cardinality subset of nodes such that no two share an edge. Fact. A tree on at least two nodes has at least two leaf nodes. degree = 1 Key observation. If v is a leaf, there exists a maximum size independent set containing v. u Pf. (exchange argument) Consider a max cardinality independent set S. If v S, we're done. If u S and v S, then S { v } is independent S not maximum. IF u S and v S, then S { v } { u } is independent. v 11 Copyright 2000, Kevin Wayne 11

12 Independent Set on Trees: Greedy Algorithm Theorem. The following greedy algorithm finds a maximum cardinality independent set in forests (and hence trees). Independent-Set-In-A-Forest(F) { S φ while (F has at least one edge) { Let e = (u, v) be an edge such that v is a leaf Add v to S Delete from F nodes u and v, and all edges incident to them. } return S } Pf. Correctness follows from the previous key observation. Remark. Can implement in O(n) time by considering nodes in postorder. 12 Copyright 2000, Kevin Wayne 12

14 Approximation Algorithms Q. Suppose I need to solve an NP-hard problem. What should I do? A. Theory says you're unlikely to find a poly-time algorithm. Must sacrifice one of three desired features. Solve problem to optimality. Solve problem in poly-time. Solve arbitrary instances of the problem. ρ-approximation algorithm. Guaranteed to run in poly-time. Guaranteed to solve arbitrary instance of the problem Guaranteed to find solution within ratio ρ of true optimum. Challenge. Need to prove a solution's value is close to optimum, without even knowing what optimum value is! 14 Copyright 2000, Kevin Wayne 14

24 The Probabilistic Method Corollary. For any instance of 3-SAT, there exists a truth assignment that satisfies at least a 7/8 fraction of all clauses. Pf. Random variable is at least its expectation some of the time. Probabilistic method. We showed the existence of a non-obvious property of 3-SAT by showing that a random construction produces it with positive probability! 24 Copyright 2000, Kevin Wayne 24

28 What to do if the problem you want to solve is NP-hard More on approximation algorithms Recent research has classified problems based on what kinds of approximations are possible if P NP Best: (1+ε) factor for any ε>0. packing and some scheduling problems, TSP in plane Some fixed constant factor > 1, e.g. 2, 3/2, 100 Vertex Cover, TSP in space, other scheduling problems Θ(log n) factor Set Cover, Graph Partitioning problems Worst: Ω(n 1- ε ) factor for any ε>0 Clique, Independent-Set, Coloring Slides courtesy of Paul Beame 28 Copyright 2000, Kevin Wayne 28

29 What to do if the problem you want to solve is NP-hard Try an algorithm that is provably fast on average. To even try this one needs a model of what a typical instance is. Typically, people consider random graphs e.g. all graphs with a given # of edges are equally likely Problems: real data doesn t look like the random graphs distributions of real data aren t analyzable Slides courtesy of Paul Beame 29 Copyright 2000, Kevin Wayne 29

30 What to do if the problem you want to solve is NP-hard Try to search the space of possible hints/certificates in a more efficient way and hope it is quick enough Backtracking search E.g. For SAT there are 2 n possible truth assignments If we set the truth values one-by-one we might be able to figure out whole parts of the space to avoid, e.g. After setting x 1 1, x 2 0 we don t even need to set x 3 or x 4 to know that it won t satisfy ( x 1 x 2 ) ( x 2 x 3 ) (x 4 x 3 ) (x 1 x 4 ) Related technique: branch-and-bound Backtracking search can be very effective even with exponential worst-case time For example, the best SAT algorithms used in practice are all variants on backtracking search and can solve surprisingly large problems Slides courtesy of Paul Beame 30 Copyright 2000, Kevin Wayne 30

31 What to do if the problem you want to solve is NP-hard Use heuristic algorithms and hope they give good answers No guarantees of quality Many different types of heuristic algorithms Many different options, especially for optimization problems, such as TSP, where we want the best solution. We ll mention several on following slides Slides courtesy of Paul Beame 31 Copyright 2000, Kevin Wayne 31

32 Heuristic algorithms for NP-hard problems local search for optimization problems need a notion of two solutions being neighbors Start at an arbitrary solution S While there is a neighbor T of S that is better than S S T Usually fast but often gets stuck in a local optimum and misses the global optimum With some notions of neighbor can take a long time in the worst case Slides courtesy of Paul Beame 32 Copyright 2000, Kevin Wayne 32

33 e.g., Neighboring solutions for TSP Solution S Solution T Two solutions are neighbors iff there is a pair of edges you can swap to transform one to the other Slides courtesy of Paul Beame 33 Copyright 2000, Kevin Wayne 33

34 Heuristic algorithms for NP-hard problems randomized local search start local search several times from random starting points and take the best answer found from each point more expensive than plain local search but usually much better answers simulated annealing like local search but at each step sometimes move to a worse neighbor with some probability probability of going to a worse neighbor is set to decrease with time as, presumably, solution is closer to optimal helps avoid getting stuck in a local optimum but often slow to converge (much more expensive than randomized local search) analogy with slow cooling to get to lowest energy state in a crystal (or in forging a metal) Slides courtesy of Paul Beame 34 Copyright 2000, Kevin Wayne 34

35 Heuristic algorithms artificial neural networks based on very elementary model of human neurons Set up a circuit of artificial neurons each artificial neuron is an analog circuit gate whose computation depends on a set of connection strengths Train the circuit Adjust the connection strengths of the neurons by giving many positive & negative training examples and seeing if it behaves correctly The network is now ready to use useful for ill-defined classification problems such as optical character recognition but not typical cut & dried problems Slides courtesy of Paul Beame 35 Copyright 2000, Kevin Wayne 35

36 Other directions Quantum computing Use physical processes at the quantum level to implement weird kinds of circuit gates unitary transformations Quantum objects can be in a superposition of many pure states at once can have n objects together in a superposition of 2 n states Each quantum circuit gate operates on the whole superposition of states at once inherent parallelism but classical randomized algorithms have a similar parallelism: not enough on its own Advantage over classical: parallel copies interfere with each other. Need totally new kinds of algorithms to work well. Theoretically able to factor efficiently but huge practical problems: errors, decoherence. Slides courtesy of Paul Beame 36 Copyright 2000, Kevin Wayne 36

37 Loose Ends Space Complexity: Amount of memory used by an algorithm If an algorithm runs in time T, then it uses at most T units of memory Every poly-time algorithm uses poly-space If an algorithm uses S units of memory, it run in time O(2 ) PSPACE: class of algorithms solvable by algorithms that use a polynomial amount of space. P PSPACE Another big question in complexity is whether P = PSPACE. 37 Copyright 2000, Kevin Wayne 37

Approximation Algorithms Chapter Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of

Approximation Algorithms 11 Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of three

Approximation Algorithms Chapter Approximation Algorithms Q. Suppose I need to solve an NP-hard problem. What should I do? A. Theory says you're unlikely to find a poly-time algorithm. Must sacrifice one

2.3 Scheduling jobs on identical parallel machines There are jobs to be processed, and there are identical machines (running in parallel) to which each job may be assigned Each job = 1,,, must be processed

CMPSCI611: Approximating MAX-CUT Lecture 20 For the next two lectures we ll be seeing examples of approximation algorithms for interesting NP-hard problems. Today we consider MAX-CUT, which we proved to

Spectral graph theory Uri Feige January 2010 1 Background With every graph (or digraph) one can associate several different matrices. We have already seen the vertex-edge incidence matrix, the Laplacian

Near Optimal Solutions Many important optimization problems are lacking efficient solutions. NP-Complete problems unlikely to have polynomial time solutions. Good heuristics important for such problems.

Introduction to Logic in Computer Science: Autumn 2006 Ulle Endriss Institute for Logic, Language and Computation University of Amsterdam Ulle Endriss 1 Plan for Today Now that we have a basic understanding

Lecture 7: Approximation via Randomized Rounding Often LPs return a fractional solution where the solution x, which is supposed to be in {0, } n, is in [0, ] n instead. There is a generic way of obtaining

Lecture 19 NP-Completeness I 19.1 Overview In the past few lectures we have looked at increasingly more expressive problems that we were able to solve using efficient algorithms. In this lecture we introduce

CS787: Advanced Algorithms Lecture 5: Applications of Network Flow In the last lecture, we looked at the problem of finding the maximum flow in a graph, and how it can be efficiently solved using the Ford-Fulkerson

NP-complete? NP-hard? Some Foundations of Complexity Prof. Sven Hartmann Clausthal University of Technology Department of Informatics Tractability of Problems Some problems are undecidable: no computer

Quantum and Non-deterministic computers facing NP-completeness Thibaut University of Vienna Dept. of Business Administration Austria Vienna January 29th, 2013 Some pictures come from Wikipedia Introduction

8 8 Introduction to Algorithms May 7, 2004 Massachusetts Institute of Technology 6.046J/18.410J Professors Erik Demaine and Shafi Goldwasser Handout 25 Problem Set 7 Solutions This problem set is due in

276 The P vs. NP problem is a major unsolved problem in computer science It is one of the seven Millennium Prize Problems selected by the Clay Mathematics Institute to carry a $ 1,000,000 prize for the

Lecture 11 The Lovász ϑ Function 11.1 Perfect graphs We begin with some background on perfect graphs. graphs. First, we define some quantities on Definition 11.1. Given a graph G on n vertices, we define

3.1 Algorithm complexity The algorithms A, B are given. The former has complexity O(n 2 ), the latter O(2 n ), where n is the size of the instance. Let n A 0 be the size of the largest instance that can

Introduction to computer science Michael A. Nielsen University of Queensland Goals: 1. Introduce the notion of the computational complexity of a problem, and define the major computational complexity classes.

CS787: Advanced Algorithms Lecture 10: LP Relaxation and Rounding In this lecture we will design approximation algorithms using linear programming. The key insight behind this approach is that the closely

Distributed Computing over Communication Networks: Maximal Independent Set What is a MIS? MIS An independent set (IS) of an undirected graph is a subset U of nodes such that no two nodes in U are adjacent.

Triangle deletion Ernie Croot February 3, 2010 1 Introduction The purpose of this note is to give an intuitive outline of the triangle deletion theorem of Ruzsa and Szemerédi, which says that if G = (V,

CAD Algorithms The Classes P and NP Mohammad Tehranipoor ECE Department 6 September 2010 1 P and NP P and NP are two families of problems. P is a class which contains all of the problems we solve using

Chapter 4 Trees 4.1 Basics A tree is a connected graph with no cycles. A forest is a collection of trees. A vertex of degree one, particularly in a tree, is called a leaf. Trees arise in a variety of applications.

Discrete Mathematics Lent 2009 MA210 Solutions to Exercises 8 (1) Suppose that G is a graph in which every vertex has degree at least k, where k 1, and in which every cycle contains at least 4 vertices.

GRAPH THEORY and APPLICATIONS Trees Properties Tree: a connected graph with no cycle (acyclic) Forest: a graph with no cycle Paths are trees. Star: A tree consisting of one vertex adjacent to all the others.

Solving the ILP using branch-and-cut Solving ILPs is a main topic in combinatorial optimization. We will take a brief look at the branch-and-cut approach. Branch-and-cut makes use of two techniques: Cutting

COLORED GRAPHS AND THEIR PROPERTIES BEN STEVENS 1. Introduction This paper is concerned with the upper bound on the chromatic number for graphs of maximum vertex degree under three different sets of coloring

Complexity Classes P and NP MATH 3220 Supplemental Presentation by John Aleshunas The cure for boredom is curiosity. There is no cure for curiosity Dorothy Parker Computational Complexity Theory In computer

CME 305: Discrete Mathematics and Algorithms 1 Basic Definitions and Concepts in Graph Theory A graph G(V, E) is a set V of vertices and a set E of edges. In an undirected graph, an edge is an unordered

Notes on NP Completeness Rich Schwartz November 10, 2013 1 Overview Here are some notes which I wrote to try to understand what NP completeness means. Most of these notes are taken from Appendix B in Douglas

Basic Notions on Graphs Planar Graphs and Vertex Colourings Presented by Joe Ryan School of Electrical Engineering and Computer Science University of Newcastle, Australia Planar graphs Graphs may be drawn

CoNP and Function Problems conp By definition, conp is the class of problems whose complement is in NP. NP is the class of problems that have succinct certificates. conp is therefore the class of problems

Computational complexity theory Goal: A general theory of the resources needed to solve computational problems What types of resources? Time What types of computational problems? decision problem Decision

Universality in the theory of algorithms and computer science Alexander Shen Computational models The notion of computable function was introduced in 1930ies. Simplifying (a rather interesting and puzzling)

Notes on Complexity Theory Last updated: August, 2011 Jonathan Katz Lecture 1 1 Turing Machines I assume that most students have encountered Turing machines before. (Students who have not may want to look