Lecture 19: Sparsest Cut and L 1 Embeddings

Transcription

1 Advanced Approximation Algorithms (CMU B, pring 2008) Lecture 19: parsest Cut and L 1 Embeddings 25 March, 2008 Lecturer: Anupam Gupta cribe: Amitabh Basu 1 parsest Cut: Problem Definition We will be studying the parsest cut problem in this lecture. In this context we will see how metric methods help in the design of approximation algorithms. We proceed to define the problem and briefly give some motivation for studying the problem. The input to the parsest Cut problem is where and A weighted graph G = (V, E) with positive edge weights (or costs or capacities, as they are called in this context) c e for every edge e E. As is usual, n = V. A set of pairs of vertices {(s 1, t 1 ), (s 2, t 2 )...(s k, t k )}, with associated demands D i between s i and t i. Given such a graph, we define sparsity of a cut V to be c(, ) = D(, ) = Φ() = c(, ) D(, ) e s.t. e crosses the cut, c(e) i s.t. s i,t i are separated by, The objective of the parsest Cut problem is to find a cut which minimizes; let is define Φ = Φ( ) = min V Φ(). As an example, see Figure 1. The dashed edges are the demand edges of demand value 1. The solid edges are edges of the graph with capacity 1. The sparsest cut value for this graph is 1. D i 2 Motivation Consider the special case of the parsest Cut problem when we have unit demand between every pair of vertices: i.e., the demands consist of all pairs ( V 2) and Dxy = 1 for all x y V. Then we 1

2 Figure 1: Example illustrating the parsest Cut problem are minimizing the following quantity: Φ unit = min V c(, ). (1) If n/2, then n/2 n, then the expression (1) is the same (upto a factor of {n/2, n} ) as min c(, )/. (2) n 2 The above quantity the cost of the cut edges divided by the size of the separated set is called the expansion of the set in a weighted graph. (In unit weight graphs, this corresponds exactly to the notion of edge-expansion of graphs). Having a good approximation for this problem enables us to find good balanced separators, which are extremely useful for divide-and-conquer type of algorithms on graphs. ee, e.g. [5] for a survey on how to find balanced separators using the sparsest cut problem as a subroutine, and for applications to approximation algorithms. 2.1 Expander Graphs As mentioned above, for unit demands and unit edge-weights, the sparsity is approximately the same as the edge-expansion of a graph. Graphs with high (edge-)expansion are often called expanders. Examples of expanders are For example, K n has Θ(n) expansion; however, this is not surprising, since the degree of the graph is also large. Infinite 3-regular tree: this has finite degree, but is an infinite graph. ee Figure 2. The challenge is in constructing families of finite expanders with bounded-degree i.e., we want to construct a family of graphs {G n } where G n has n vertices, for infinitely many values of n, such that the expansion of graphs in this family is bounded below by a constant, and the degree is bounded above by a constant. It is not difficult to show that random bounded-degree graphs are 2

3 Figure 2: Infinite 3-regular tree expanders with high probability, but how does one construct explicit expanders? By now there are many constructions known (see, e.g., the survey by Hoory, Linial, and Wigderson [3]). Here is one such construction due to Margluis. Consider the n n grid graph. For every vertex (i, j), add edges to four other vertices : (i, i + 2j), (i, i + 2j + 1), (i + 2j, j), (i + 2j + 1, j). The proof that this is an expander is quite non-trivial, but the intuition comes from considering the uniform unit square in R 2, considering two mappings T(x, y) (x + 2y, y) and T (x, y) (x, x + 2y), and showing that if we take any region R, such that area(r) 1 2, then T(R) T (R) is large. 3 Algorithms for parsest Cut Here is a simple claim for sparsest cuts. Claim 3.1. There exists a sparsest cut (, ), such that the graphs G[] and G[ ] are connected. (We assume that the graph was connected to begin with.) Proof. WLOG, assume G[] is not connected. ay it has components C 1... C t. Let the total capacity of edges from C i to C i be c i and the demand be d i. The sparsity of cut is c c t d d t. Now since all the quantities c i, d i are non-negative, by simple arithmetic, there exists i such that c i /d i c c t d d t. This implies that the cut C i is at least as good as. 3.1 parsest Cut on Trees Using claim 3.1, we know that the sparsest cut on trees will be exactly one edge. Therefore, the sparsest cut problem on trees becomes easy to solve in polynomial time. 3.2 Metric paces - A Digression A metric space on a set V is defined as a distance measure d : V V R, with 3 properties: 1. d(x, y) = 0 iff x = y 3

4 2. d(x, y) = d(y, x) 3. d(x, y) + d(y, z) d(x, z) Typical examples are R d, equipped with the l p norms. Recall that the l p norm is the following - d(x, y) = ( d i=1 x i y i p ) 1 p Cut Metrics Another example is that associated with cuts (i.e., subsets of V ): given a cut V, the cut metric associated with is δ, where { 0 if x, y or x, y δ (x, y) = 1 otherwise Viewing Metrics as Vectors in R (n 2) Any n-point metric can be associated with vectors in R (n 2), with each coordinate corresponding to a pair of vertices from the metric space. Given the metric d, we refer to the corresponding vector as d. In this setting, the following facts are trivial : 1. α d + (1 α) d is a metric for all 0 α 1 2. For all α 0 and metrics d, α d is a metric. Therefore, the set of all metrics forms a convex cone in R (n 2). In this setting the parsest Cut problem can be restated as min all cut metrics c δ (3) where c is the vector in R (n 2) with cij being the capacity of the edge between vertex i and j, and D ij being the demand between vertex i and j; of course, c δ denotes the dot product of the two corresponding vectors. Let us denote the positive cone generated by all cut metrics by CUT n, i.e., CUT n = { d d = V α δ, α 0} ince the optimum of (3) will be achieved at an extreme point (or in this case, an extreme ray), we have Φ = min d CUT n c δ (4) 4

5 3.2.3 What is CUT n? It turns out that the set CUT n is the same as a natural class of metric spaces. Fact 3.2. CUT n = all n-point subsets of R t under the l 1 norm. Proof. Consider any metric in CUT n. For every with α > 0, we have a dimension and in that dimension we put value 0 for x and α for x. This shows CUT n l 1 metrics. For the other direction, consider a set of n-points from R n. Take one dimension d and sort the points in increasing value along that dimension. ay we get v 1,...,v k as the set of distinct values. Define k 1 cut metrics i = {x x d v i+1 }. Also let α i = v i+1 v i. Now along this dimension, x d y d = k i=1 α iδ i. We can construct cut metrics for every dimension. Thus we have a metric in CUT n for every n-point metric in l 1. Note that the above proof can easily be made algorithmic: Lemma 3.3. Given a metric µ l 1 with D dimensions, there is a procedure taking time poly(n, D) that outputs a set of at most nd values α 0 for V such that µ = α δ Rewriting Φ as optimizing over CUT n o we can rewrite (4) as Φ c δ = min d l 1 (5) ince the sparsest cut problem is NP-hard, we can t hope to solve the above optimization problem over metrics in l 1. Hence we consider a relaxation of this problem. We relax the domain of d to the set of all metrics λ := min d metric c δ (6) Clearly, this quantity λ Φ. Moreover, we can compute it via a linear program. min c ij d ij subject to d ij d ik + d kj D ij d ij = 1 d ij 0 (7) 5

6 3.3 Metric Embeddings uppose we solve the LP to find the metric d that achieves the optimal value λ in (6): how can we obtain a cut (, ) such that Φ() is close to Φ? The plan is to embed the metric returned by the linear program above into an l 1 metric so that the distances are not changed by too much. Then if we can somehow recover a cut metric from the l 1 metric with the same objective, then we would be done (modulo what we lose because of the embedding). More formally, we have the following theorem: Theorem 3.4. uppose for each metric (V, d), there exists a metric µ = µ(d) l 1 such that d(x, y) µ(x, y) αd(x, y) for all x, y V. Then the parsest cut LP (7) has an integrality gap of at most α. Proof. Take the metric d returned by the linear program and consider the metric µ l 1, such that d µ αd. Then using the fact that µ α d. Moreover since µ d. Hence c µ α c d (8) D µ D d (9) Φ(µ) := D µ c µ α D d c d = αλ αφ. (10) In other words, we have found a solution to (5) with value at most α times the (optimal) sparsest cut, implying that the integrality gap of the LP is at most α. We have the following embedding theorem due to Bourgain [2] (with the claim about the dimensions due to Linial et al. [4]): Theorem 3.5. For all metrics d, there exists µ l 1 such that α = O(log n). Moreover, the number of dimensions needed is at most O(log 2 n). Corollary 3.6. The LP relaxation of parsest Cut has integrality gap of O(log n). 3.4 How do we find the parsest Cut? We have shown that the integrality gap between (5) and (6) is small by taking the metric d obtained by (6) and embedding it into l 1. However, we want a cut (, ) with small Φ() how do we obtain that? To put in this final piece of the solution, we need to construct a cut (, ) from the l 1 metric µ such that Φ() Φ(µ). 6

7 To do this, recall Fact 3.2 which says that any metric in l 1 can be written as a positive linear combination of cuts. I.e., the metric µ can be written as α δ with α. Given this representation, we get, Φ(µ) = c µ D µ = c ( α δ ) D ( α δ ) α ( c δ ) = α ( D δ ) (11) (12) α ( c δ ) min,α >0 α ( D δ ) = min Φ(). (13),α >0 o, we can simply pick the best cut amongst the ones with non-zero α in the cut-decomposition of µ. Finally, we use Lemma 3.3 and Theorem 3.5 to argue that the representation of µ as a positive linear combination of cut metrics can be found in poly(n) time, with at most O(n log 2 n) cuts. This finally proves the result that not only is the integrality gap small (Corollary 3.6), but Theorem 3.7. Given a metric d that is a solution to the LP (6), we can efficiently find a cut (, ) such that Φ() O(log n) LP-value. References [1] Avis D. and Deza M., The cut cone, l 1 -embeddability, complexity and multicommodity flows. Networks 21 (1991) [2] J. Bourgain. On Lipschitz embeddings of finite metric spaces in {Hilbert} space. Israel J. of Math [3] hmolo Hoory, Nathan Linial, and Avi Wigderson. Expander graphs and their applications Bulletin of the AM, 43(2006) [4] Nathan Linial, Eran London, and Yuri Rabinovich. The geometry of graphs and some of its algorithmic applications. Combinatorica, 15(2): , (Preliminary version in 35th FOC, 1994). [5] David B. hmoys. Cut problems and their application to divide-and-conquer. In Dorit. Hochbaum, editor, Approximation Algorithms for NP-hard Problems, pages PW Publishing,

(67902) Topics in Theory and Complexity Nov 2, 2006 Lecturer: Irit Dinur Lecture 7 Scribe: Rani Lekach 1 Lecture overview This Lecture consists of two parts In the first part we will refresh the definition

INCIDENCE-BETWEENNESS GEOMETRY MATH 410, CSUSM. SPRING 2008. PROFESSOR AITKEN This document covers the geometry that can be developed with just the axioms related to incidence and betweenness. The full

Approximation Algorithms Chapter Approximation Algorithms Q. Suppose I need to solve an NP-hard problem. What should I do? A. Theory says you're unlikely to find a poly-time algorithm. Must sacrifice one

Massachusetts Institute of Technology Handout 6 18.433: Combinatorial Optimization February 20th, 2009 Michel X. Goemans 3. Linear Programming and Polyhedral Combinatorics Summary of what was seen in the

Factoring & Primality Lecturer: Dimitris Papadopoulos In this lecture we will discuss the problem of integer factorization and primality testing, two problems that have been the focus of a great amount

Cycles and clique-minors in expanders Benny Sudakov UCLA and Princeton University Expanders Definition: The vertex boundary of a subset X of a graph G: X = { all vertices in G\X with at least one neighbor

Approximation Algorithms Chapter Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of

Near Optimal Solutions Many important optimization problems are lacking efficient solutions. NP-Complete problems unlikely to have polynomial time solutions. Good heuristics important for such problems.

Chapter 4 Ideal Class Group and Units We are now interested in understanding two aspects of ring of integers of number fields: how principal they are (that is, what is the proportion of principal ideals

A Practical Scheme for Wireless Network Operation Radhika Gowaikar, Amir F. Dana, Babak Hassibi, Michelle Effros June 21, 2004 Abstract In many problems in wireline networks, it is known that achieving

LINEAR ALGEBRA MATH 27.6 SPRING 23 (COHEN) LECTURE NOTES Sets and Set Notation. Definition (Naive Definition of a Set). A set is any collection of objects, called the elements of that set. We will most

Chapter 5 The Ideal Class Group We will use Minkowski theory, which belongs to the general area of geometry of numbers, to gain insight into the ideal class group of a number field. We have already mentioned

CS 05: Algorithms (Grad) Feb 2-24, 2005 Approximating Set Cover. Definition An Instance (X, F ) of the set-covering problem consists of a finite set X and a family F of subset of X, such that every elemennt

CSE 291: Geometric algorithms Spring 2013 Lecture 4 Online and streaming algorithms for clustering 4.1 On-line k-clustering To the extent that clustering takes place in the brain, it happens in an on-line

On Rank vs. Communication Complexity Noam Nisan Avi Wigderson Abstract This paper concerns the open problem of Lovász and Saks regarding the relationship between the communication complexity of a boolean

Mathematical Induction (Handout March 8, 01) The Principle of Mathematical Induction provides a means to prove infinitely many statements all at once The principle is logical rather than strictly mathematical,

Mathematics Course 111: Algebra I Part IV: Vector Spaces D. R. Wilkins Academic Year 1996-7 9 Vector Spaces A vector space over some field K is an algebraic structure consisting of a set V on which are

January 10, 2010 CHAPTER SIX IRREDUCIBILITY AND FACTORIZATION 1. BASIC DIVISIBILITY THEORY The set of polynomials over a field F is a ring, whose structure shares with the ring of integers many characteristics.

Approximation Algorithms 11 Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of three

CSE 291: Unsupervised learning Spring 2008 Lecture 6 Online and streaming algorithms for clustering 6.1 On-line k-clustering To the extent that clustering takes place in the brain, it happens in an on-line

THE TURING DEGREES AND THEIR LACK OF LINEAR ORDER JASPER DEANTONIO Abstract. This paper is a study of the Turing Degrees, which are levels of incomputability naturally arising from sets of natural numbers.

Chapter 7 Classification of Cartan matrices In this chapter we describe a classification of generalised Cartan matrices This classification can be compared as the rough classification of varieties in terms

Large induced subgraphs with all degrees odd A.D. Scott Department of Pure Mathematics and Mathematical Statistics, University of Cambridge, England Abstract: We prove that every connected graph of order

Bargaining Solutions in a Social Network Tanmoy Chakraborty and Michael Kearns Department of Computer and Information Science University of Pennsylvania Abstract. We study the concept of bargaining solutions,

The Max-Distance Network Creation Game on General Host Graphs 13 Luglio 2012 Introduction Network Creation Games are games that model the formation of large-scale networks governed by autonomous agents.

The degree, size and chromatic index of a uniform hypergraph Noga Alon Jeong Han Kim Abstract Let H be a k-uniform hypergraph in which no two edges share more than t common vertices, and let D denote the

ON GALOIS REALIZATIONS OF THE 2-COVERABLE SYMMETRIC AND ALTERNATING GROUPS DANIEL RABAYEV AND JACK SONN Abstract. Let f(x) be a monic polynomial in Z[x] with no rational roots but with roots in Q p for

Three observations regarding Schatten p classes Gideon Schechtman Abstract The paper contains three results, the common feature of which is that they deal with the Schatten p class. The first is a presentation

Numerical Analysis Lecture Notes Peter J. Olver 5. Inner Products and Norms The norm of a vector is a measure of its size. Besides the familiar Euclidean norm based on the dot product, there are a number

Prime Numbers and Irreducible Polynomials M. Ram Murty The similarity between prime numbers and irreducible polynomials has been a dominant theme in the development of number theory and algebraic geometry.

CMSC 858T: Randomized Algorithms Spring 2003 Handout 8: The Local Lemma Please Note: The references at the end are given for extra reading if you are interested in exploring these ideas further. You are

ALMOST COMMON PRIORS ZIV HELLMAN ABSTRACT. What happens when priors are not common? We introduce a measure for how far a type space is from having a common prior, which we term prior distance. If a type

Introduction to computer science Michael A. Nielsen University of Queensland Goals: 1. Introduce the notion of the computational complexity of a problem, and define the major computational complexity classes.

Lecture 4. LaSalle s Invariance Principle We begin with a motivating eample. Eample 4.1 (nonlinear pendulum dynamics with friction) Figure 4.1: Pendulum Dynamics of a pendulum with friction can be written