Computer Science DepartmentCopyright (c) 2015 Carnegie Mellon University All rights reserved.http://repository.cmu.edu/compsci
Recent documents in Computer Science Departmenten-usSun, 02 Aug 2015 01:37:18 PDT3600Discovering Structure in Unstructured I/Ohttp://repository.cmu.edu/compsci/2601
http://repository.cmu.edu/compsci/2601Sat, 01 Aug 2015 12:26:16 PDT
Checkpointing is the predominant storage driver in today's petascale supercomputers and is expected to remain as such in tomorrow's exascale supercomputers. Users typically prefer to checkpoint into a shared file yet parallel file systems often perform poorly for shared file writing. A powerful technique to address this problem is to transparently transform shared file writing into many exclusively written as is done in ADIOS and PLFS. Unfortunately, the metadata to reconstruct the fragments into the original file grows with the number of writers. As such, the current approach cannot scale to exaflop supercomputers due to the large overhead of creating and reassembling the metadata. In this paper, we develop and evaluate algorithms by which patterns in the PLFS metadata can be discovered and then used to replace the current metadata. Our evaluation shows that these patterns reduce the size of the metadata by several orders of magnitude, increase the performance of writes by up to 40 percent, and the performance of reads by up to 480 percent. This contribution therefore can allow current checkpointing models to survive the transition from petato exascale.
]]>
Jun He et al.A case for scaling HPC metadata performance through de-specializationhttp://repository.cmu.edu/compsci/2600
http://repository.cmu.edu/compsci/2600Sat, 01 Aug 2015 12:26:08 PDT
Lack of a highly scalable and parallel metadata service is the Achilles heel for many cluster file system deployments in both the HPC world and the Internet services world. This is because most cluster file systems have focused on scaling the data path, i.e. providing high bandwidth parallel I/O to files that are gigabytes in size. But with proliferation of massively parallel applications that produce metadata-intensive workloads, such as large number of simultaneous file creates and large-scale storage management, cluster file systems also need to scale metadata performance. To realize these goals, this paper makes a case for a scalable metadata service middleware that layers on existing cluster file system deployments and distributes file system metadata, including the namespace tree, small directories and large directories, across many servers. Our key idea is to effectively synthesize a concurrent indexing technique to distribute metadata with a tabular, on-disk representation of all file system metadata.
]]>
Swapnil Patil et al.TABLEFS: Embedding a NoSQL Database Inside the Local File Systemhttp://repository.cmu.edu/compsci/2599
http://repository.cmu.edu/compsci/2599Sat, 01 Aug 2015 12:25:59 PDT
Conventional file systems are optimzed for large file transfers instead of workloads that are dominated by metadata and small file accesses. This paper examines using techniques adopted from NoSQLdatabases to manage file system metadata and small files, which feature high rates of change and efficient out-of-core data representation. A FUSE file system prototype was built by storing file systemmetadata and small files into a modern key-value store: LevelDB. We demonstrate that such techniques can improve the performance of modern local file systems in Linux for workloads dominated by metadata and tiny files.
]]>
Kai Ren et al.Scalable Dynamic Partial Order Reductionhttp://repository.cmu.edu/compsci/2598
http://repository.cmu.edu/compsci/2598Sat, 01 Aug 2015 12:25:50 PDT
Systematic testing, first demonstrated in small, specialized cases 15 years ago, has matured sufficiently for large-scale systems developers to begin to put it into practice. With actual deployment come new, pragmatic challenges to the usefulness of the techniques. In this paper we are concerned with scaling dynamic partial order reduction, a key technique for mitigating the state space explosion problem, to very large clusters. In particular, we present a new approach for distributed dynamic partial order reduction. Unlike previous work, our approach is based on a novel exploration algorithm that 1) enables trading space complexity for parallelism, 2) achieves efficient load-balancing through time-slicing, 3) provides for fault tolerance, which we consider a mandatory aspect of scalability, 4) scales to more than a thousand parallel workers, and 5) is guaranteed to avoid redundant exploration of overlapping portions of the state space.
]]>
Jiri Simsa et al.Scale and Concurrency of GIGA+: File System Directories with Millions of Fileshttp://repository.cmu.edu/compsci/2597
http://repository.cmu.edu/compsci/2597Sat, 01 Aug 2015 12:25:42 PDT
We examine the problem of scalable file system directories, motivated by data-intensive applications requiring millions to billions of small files to be ingested in a single directory at rates of hundreds of thousands of file creates every second. We introduce a POSIX-compliant scalable directory design, GIGA+, that distributes directory entries over a cluster of server nodes. For scalability, each server makes only local, independent decisions about migration for load balancing. GIGA+ uses two internal implementation tenets, asynchrony and eventual consistency, to: (1) partition an index among all servers without synchronization or serialization, and (2) gracefully tolerate stale index state at the clients. Applications, however, are provided traditional strong synchronous consistency semantics. We have built and demonstrated that the GIGA+ approach scales better than existing distributed directory implementations, delivers a sustained throughput of more than 98,000 file creates per second on a 32-server cluster, and balances load more efficiently than consistent hashing.
]]>
Swapnil Patil et al.dBug: Systematic Evaluation of Distributed Systemshttp://repository.cmu.edu/compsci/2596
http://repository.cmu.edu/compsci/2596Sat, 01 Aug 2015 12:25:33 PDT
This paper presents the design, implementation and evaluation of “dBug” – a tool that leverages manual instrumentation for systematic evaluation of distributed and concurrent systems. Specifically, for a given distributed concurrent system, its initial state and a workload, the dBug tool systematically explores possible orders in which concurrent events triggered by the workload can happen. Further, dBug optionally uses the partial order reduction mechanism to avoid exploration of equivalent orders. Provided with a correctness check, the dBug tool is able to verify that all possible serializations of a given concurrent workload execute correctly. Upon encountering an error, the tool produces a trace that can be replayed to investigate the error.

We applied the dBug tool to two distributed systems – the Parallel Virtual File System (PVFS) implemented in C and the FAWN-based key-value storage (FAWN-KV) implemented in C++. In particular, we integrated both systems with dBug to expose the non-determinism due to concurrency. This mechanism was used to verify that the result of concurrent execution of a number of basic operations from a fixed initial state meets the high-level specification of PVFS and FAWN-KV. The experimental evidence shows that the dBug tool is capable of systematically exploring behaviors of a distributed system in a modular, practical, and effective manner.

]]>
Jiri Simsa et al.Online Packing and Covering Framework with Convex Objectiveshttp://repository.cmu.edu/compsci/2595
http://repository.cmu.edu/compsci/2595Sat, 01 Aug 2015 12:25:25 PDT
We consider online fractional covering problems with a convex objective, where the covering constraints arrive over time. Formally, we want to solve min{f(x)∣Ax≥1,x≥0}, where the objective function f:Rn→R is convex, and the constraint matrix Am×n is non-negative. The rows of A arrive online over time, and we wish to maintain a feasible solution x at all times while only increasing coordinates of x. We also consider "dual" packing problems of the form max{c⊺y−g(μ)∣A⊺y≤μ,y≥0}, where g is a convex function. In the online setting, variables y and columns of A⊺ arrive over time, and we wish to maintain a non-decreasing solution (y,μ). We provide an online primal-dual framework for both classes of problems with competitive ratio depending on certain "monotonicity" and "smoothness" parameters of f; our results match or improve on guarantees for some special classes of functions f considered previously. Using this fractional solver with problem-dependent randomized rounding procedures, we obtain competitive algorithms for the following problems: online covering LPs minimizing ℓp-norms of arbitrary packing constraints, set cover with multiple cost functions, capacity constrained facility location, capacitated multicast problem, set cover with set requests, and profit maximization with non-separable production costs. Some of these results are new and others provide a unified view of previous results, with matching or slightly worse competitive ratios.
]]>
Niv Buchbinder et al.Greedy Algorithms for Steiner Foresthttp://repository.cmu.edu/compsci/2594
http://repository.cmu.edu/compsci/2594Sat, 01 Aug 2015 12:25:16 PDT
In the Steiner Forest problem, we are given terminal pairs {si,ti}, and need to find the cheapest subgraph which connects each of the terminal pairs together. In 1991, Agrawal, Klein, and Ravi, and Goemans and Williamson gave primal-dual constant-factor approximation algorithms for this problem; until now, the only constant-factor approximations we know are via linear programming relaxations.

We consider the following greedy algorithm: Given terminal pairs in a metric space, call a terminal "active" if its distance to its partner is non-zero. Pick the two closest active terminals (say si,tj), set the distance between them to zero, and buy a path connecting them. Recompute the metric, and repeat. Our main result is that this algorithm is a constant-factor approximation.

We also use this algorithm to give new, simpler constructions of cost-sharing schemes for Steiner forest. In particular, the first "group-strict" cost-shares for this problem implies a very simple combinatorial sampling-based algorithm for stochastic Steiner forest.

]]>
Anupam Gupta et al.How the Experts Algorithm Can Help Solve LPs Onlinehttp://repository.cmu.edu/compsci/2593
http://repository.cmu.edu/compsci/2593Sat, 01 Aug 2015 12:25:08 PDT
We consider the problem of solving packing/covering LPs online, when the columns of the constraint matrix are presented in random order. This problem has received much attention: the main open question is to figure out how large the right-hand sides of the LPs have to be (compared to the entries on the left-hand side of the constraint) to get (1 + ε)-approximations online? It is known that the RHS has to be Ω(ε − 2 logm) times the left-hand sides, where m is the number of constraints.

In this paper we show how to achieve this bound for all packing LPs, and also for a wide class of mixed packing/covering LPs. Our algorithms construct dual solutions using a regret-minimizing online learning algorithm in a black-box fashion, and use them to construct primal solutions. The adversarial guarantee that holds for the constructed duals help us to take care of most of the correlations that arise in the algorithm; the remaining correlations are handled via martingale concentration and maximal inequalities. These ideas lead to conceptually simple and modular algorithms, which we hope will be useful in other contexts.

]]>
Anupam Gupta et al.Cops, Robbers, and Threatening Skeletons: Padded Decomposition for Minor-Free Graphshttp://repository.cmu.edu/compsci/2592
http://repository.cmu.edu/compsci/2592Sat, 01 Aug 2015 12:24:59 PDT
We prove that any graph excluding Kr as a minor has can be partitioned into clusters of diameter at most Δ while removing at most O(r/Δ) fraction of the edges. This improves over the results of Fakcharoenphol and Talwar, who building on the work of Klein, Plotkin and Rao gave a partitioning that required to remove O(r2/Δ) fraction of the edges. Our result is obtained by a new approach that relates the topological properties (excluding a minor) of a graph to its geometric properties (the induced shortest path metric). Specifically, we show that techniques used by Andreae in his investigation of the cops and robbers game on graphs excluding a fixed minor, can be used to construct padded decompositions of the metrics induced by such graphs. In particular, we get probabilistic partitions with padding parameter O(r) and strong-diameter partitions with padding parameter O(r2) for Kr-free graphs, O(k) for treewidth-k graphs, and O(log g) for graphs with genus g.
]]>
Ittai Abraham et al.Changing Bases: Multistage Optimization for Matroids and Matchingshttp://repository.cmu.edu/compsci/2591
http://repository.cmu.edu/compsci/2591Sat, 01 Aug 2015 12:24:51 PDT
This paper is motivated by the fact that many systems need to be maintained continually while the underlying costs change over time. The challenge is to continually maintain near-optimal solutions to an underlying optimization problem, without creating too much churn in the solution itself. We model this as a multistage combinatorial optimization problem where the input is a sequence of cost functions (one for each time step); while we can change the solution from step to step, we incur an additional cost for every such change.

We first study the multistage matroid maintenance problem, where we need to maintain a base of a matroid in each time step under changing cost functions and acquisition costs for adding new elements. The online version generalizes online paging. E.g., given a graph, we need to maintain a spanning tree Tt at each step: we pay ct (Tt ) for the cost of the tree at time t, and also | Tt ∖ Tt − 1 | for the number of edges changed at this step. Our main result is a polynomial time O(logm logr)-approximation to the online problem, where m is the number of elements/edges and r is the rank of the matroid. This improves on results of Buchbinder et al. [7] who addressed the fractional version of this problem under uniform acquisition costs, and Buchbinder, Chen and Naor [8] who studied the fractional version of a more general problem. We also give an O(logm) approximation for the offline version of the problem. These bounds hold when the acquisition costs are non-uniform, in which case both these results are the best possible unless P=NP.

We also study the perfect matching version of the problem, where we maintain a perfect matching at each step under changing cost functions and costs for adding new elements. Surprisingly, the hardness drastically increases: for any constant ε > 0, there is no O(n 1 − ε )-approximation to the multistage matching maintenance problem, even in the offline case.

]]>
Anupam Gupta et al.Minimum d-dimensional arrangement with fixed pointshttp://repository.cmu.edu/compsci/2590
http://repository.cmu.edu/compsci/2590Sat, 01 Aug 2015 12:24:42 PDT
n the Minimum d-Dimensional Arrangement Problem (d-dimAP) we are given a graph with edge weights, and the goal is to find a 1–1 map of the vertices into ℤd (for some fixed dimension d ≥ 1) minimizing the total weighted stretch of the edges. This problem arises in VLSI placement and chip design.

Motivated by these applications, we consider a generalization of d-dimAP, where the positions of some k of the vertices (pins) is fixed and specified as part of the input. We are asked to extend this partial map to a map of all the vertices, again minimizing the weighted stretch of edges. This generalization, which we refer to as d-dimAP+, arises naturally in these application domains (since it can capture blocked-off parts of the board, or the requirement of power-carrying pins to be in certain locations, etc.). Perhaps surprisingly, very little is known about this problem from an approximation viewpoint.

For dimension d = 2, we obtain an O (k1/2 · logn)-approximation algorithm, based on a strengthening of the spreading-metric LP for 2-dimAP. The integrality gap for this LP is shown to be Ω(k1/4). We also show that it is NP-hard to approximate 2-DIMAP+ within a factor better than Ω(k1/4–∊). We also consider a (conceptually harder, but practically even more interesting) variant of 2-dimAP+, where the target space is the grid , instead of the entire integer lattice ℤ2. For this problem, we obtain a O(klogklogn)-approximation using the same LP relaxation. We complement this upper bound by showing an integrality gap of Ω(k1/2), and an Ω(k1/2–∊)-inapproximability result.

]]>
Anupam Gupta et al.Online Steiner Tree with Deletionshttp://repository.cmu.edu/compsci/2589
http://repository.cmu.edu/compsci/2589Sat, 01 Aug 2015 12:24:33 PDT
In the online Steiner tree problem, the input is a set of vertices that appear one-by-one, and we have to maintain a Steiner tree on the current set of vertices. The cost of the tree is the total length of edges in the tree, and we want this cost to be close to the cost of the optimal Steiner tree at all points in time. If we are allowed to only add edges, a tight bound of Θ(log n) on the competitiveness has been known for two decades. Recently it was shown that if we can add one new edge and make one edge swap upon every vertex arrival, we can still maintain a constant-competitive tree online.

But what if the set of vertices sees both additions and deletions? Again, we would like to obtain a low-cost Steiner tree with as few edge changes as possible. The original paper of Imase and Waxman (SIAM J. Disc. Math, 4(3):369–384, 1991) had also considered this model, and it gave an algorithm that made at most O(n3/2) edge changes for the first n requests, and maintained a constant-competitive tree online. In this paper we improve on these results:

]]>
Anupam Gupta et al.Towards (1 + ∊)-Approximate Flow Sparsifiershttp://repository.cmu.edu/compsci/2588
http://repository.cmu.edu/compsci/2588Sat, 01 Aug 2015 12:24:25 PDT
A useful approach to “compress” a large network G is to represent it with a flow-sparsifier, i.e., a small network H that supports the same flows as G, up to a factor q ≥ 1 called the quality of sparsifier. Specifically, we assume the network G contains a set of k terminals T, shared with the network H, i.e., T ⊆ V(G)∩V(H), and we want H to preserve all multicommodity flows that can be routed between the terminals T. The challenge is to construct H that is small.

These questions have received a lot of attention in recent years, leading to some known tradeoffs between the sparsifier's quality q and its size |V(H)|. Nevertheless, it remains an outstanding question whether every G admits a flow-sparsifier H with quality q = 1 + ∊, or even q = O(1), and size |V(H)| ≤ f(k, ∊) (in particular, independent of |V(G)| and the edge capacities).

Making a first step in this direction, we present new constructions for several scenarios:

Our main result is that for quasi-bipartite networks G, one can construct a (1 + ∊)-flow-sparsifier of size poly(k/∊). In contrast, exact (q = 1) sparsifiers for this family of networks are known to require size 2Ω(k).

For networks G of bounded treewidth w, we construct a flow-sparsifier with quality q = O(logw/loglogw) and size O(w·poly(k)).

For general networks G, we construct a sketch sk(G), that stores all the feasible multicommodity flows up to factor q = 1 + ∊, and its size (storage requirement) is f(k, ∊).

]]>
Alexandr Andoni et al.Random Rates for 0-Extension and Low-Diameter Decompositionshttp://repository.cmu.edu/compsci/2587
http://repository.cmu.edu/compsci/2587Sat, 01 Aug 2015 12:24:16 PDT
Consider the problem of partitioning an arbitrary metric space into pieces of diameter at most ∆, such every pair of points is separated with relatively low probability. We propose a rate-based algorithm inspired by multiplicatively-weighted Voronoi diagrams, and prove it has optimal trade-offs. This also gives us another algorithm for the 0-extension problem
]]>
Anupam Gupta et al.The Power of Deferral: Maintaining a Constant-Competitive Steiner Tree Onlinehttp://repository.cmu.edu/compsci/2586
http://repository.cmu.edu/compsci/2586Sat, 01 Aug 2015 12:24:08 PDT
In the online Steiner tree problem, a sequence of points is revealed one-by-one: when a point arrives, we only have time to add a single edge connecting this point to the previous ones, and we want to minimize the total length of edges added. Here, a tight bound has been known for two decades: the greedy algorithm maintains a tree whose cost is O(log n) times the Steiner tree cost, and this is best possible. But suppose, in addition to the new edge we add, we have time to change a single edge from the previous set of edges: can we do much better? Can we, e.g., maintain a tree that is constant-competitive?

We answer this question in the affirmative. We give a primal-dual algorithm that makes only a single swap per step (in addition to adding the edge connecting the new point to the previous ones), and such that the tree's cost is only a constant times the optimal cost. Our dual-based analysis is quite different from previous primal-only analyses. In particular, we give a correspondence between radii of dual balls and lengths of tree edges; since dual balls are associated with points and hence do not move around (in contrast to edges), we can closely monitor the edge lengths based on the dual radii. Showing that these dual radii cannot change too rapidly is the technical heart of the paper, and allows us to give a hard bound on the number of swaps per arrival, while maintaining a constant-competitive tree at all times. Previous results for this problem gave an algorithm that performed an amortized constant number of swaps: for each n, the number of swaps in the first $n$ steps was O(n). We also give a simpler tight analysis for this amortized case.

]]>
Albert Gu et al.Sparsest Cut on Bounded Treewidth Graphs: Algorithms and Hardness Resultshttp://repository.cmu.edu/compsci/2585
http://repository.cmu.edu/compsci/2585Sat, 01 Aug 2015 12:23:59 PDT
We give a 2-approximation algorithm for the non-uniform Sparsest Cut problem that runs in time nO(k), where k is the treewidth of the graph. This improves on the previous 22k-approximation in time poly(n) 2O(k) due to Chlamtac et al. [18].

To complement this algorithm, we show the following hardness results: If the non-uniform Sparsest Cut has a ρ-approximation for series-parallel graphs (where ρ ≥ 1), then the MaxCut problem has an algorithm with approximation factor arbitrarily close to 1/ρ. Hence, even for such restricted graphs (which have treewidth 2), the Sparsest Cut problem is NP-hard to approximate better than 17/16 - ε for ε > 0; assuming the Unique Games Conjecture the hardness becomes 1/αGW - ε. For graphs with large (but constant) treewidth, we show a hardness result of 2 - ε assuming the Unique Games Conjecture.

Our algorithm rounds a linear program based on (a subset of) the Sherali-Adams lift of the standard Sparsest Cut LP. We show that even for treewidth-2 graphs, the LP has an integrality gap close to 2 even after polynomially many rounds of Sherali-Adams. Hence our approach cannot be improved even on such restricted graphs without using a stronger relaxation.

]]>
Anupam Gupta et al.A Stochastic Probing Problem with Applicationshttp://repository.cmu.edu/compsci/2584
http://repository.cmu.edu/compsci/2584Sat, 01 Aug 2015 12:23:50 PDT
We study a general stochastic probing problem defined on a universe V, where each elemente ∈ V is “active” independently with probability pe . Elements have weights {we :e ∈ V} and the goal is to maximize the weight of a chosen subset S of active elements. However, we are given only the pe values—to determine whether or not an element e is active, our algorithm must probee. If element e is probed and happens to be active, then e must irrevocably be added to the chosen set S; if e is not active then it is not included in S. Moreover, the following conditions must hold in every random instantiation:

the set Q of probed elements satisfy an “outer” packing constraint,

the set S of chosen elements satisfy an “inner” packing constraint.

The kinds of packing constraints we consider are intersections of matroids and knapsacks. Our results provide a simple and unified view of results in stochastic matching [1, 2] and Bayesian mechanism design [3], and can also handle more general constraints. As an application, we obtain the first polynomial-time Ω(1/k)-approximate “Sequential Posted Price Mechanism” under k-matroid intersection feasibility constraints, improving on prior work [3-5].

This problem is closely related to the Asymmetric TSP (ATSP) problem, which seeks to find a tour (instead of an s-t path) visiting all the nodes: for ATSP, a ρ-approximation guarantee implies an O(ρ)-approximation for ATSPP. However, no such connection is known for the integrality gaps of the linear programming relxations for these problems: the current-best approximation algorithm for ATSPP is O(logn/loglogn), whereas the best bound on the integrality gap of the natural LP relaxation (the subtour elmination LP) for ATSPP is O(logn).

In this paper, we close this gap, and improve the current best bound on the integrality gap from O(logn) to O(logn/loglogn). The resulting algorithm uses the structure of narrow s-t cuts in the LP solution to construct a (random) tree witnessing this integrality gap. We also give a simpler family of instances showing the integrality gap of this LP is at least 2.

]]>
Zachary Friggstad et al.Thrifty Algorithms for Multistage Robust Optimizationhttp://repository.cmu.edu/compsci/2582
http://repository.cmu.edu/compsci/2582Sat, 01 Aug 2015 12:23:33 PDT
We consider a class of multi-stage robust covering problems, where additional information is revealed about the problem instance in each stage, but the cost of taking actions increases. The dilemma for the decision-maker is whether to wait for additional information and risk the inflation, or to take early actions to hedge against rising costs. We study the “k-robust” uncertainty model: in each stage i = 0, 1, …, T, the algorithm is shown some subset of size kithat completely contains the eventual demands to be covered; here k 1 > k 2 > ⋯ > kT which ensures increasing information over time. The goal is to minimize the cost incurred in theworst-case possible sequence of revelations.

For the multistage k-robust set cover problem, we give an O(logm + logn)-approximation algorithm, nearly matching the Ω(logn+logmloglogm) hardness of approximation [4] even for T = 2 stages. Moreover, our algorithm has a useful “thrifty” property: it takes actions on just two stages. We show similar thrifty algorithms for multi-stage k-robust Steiner tree, Steiner forest, and minimum-cut. For these problems our approximation guarantees are O( min { T, logn, logλmax }), where λ max is the maximum inflation over all the stages. We conjecture that these problems also admit O(1)-approximate thrifty algorithms.

]]>
Anupam Gupta et al.Online Primal-Dual For Non-linear Optimization with Applications to Speed Scalinghttp://repository.cmu.edu/compsci/2581
http://repository.cmu.edu/compsci/2581Sat, 01 Aug 2015 12:23:23 PDT
We give a principled method to design online algorithms (for potentially non-linear problems) using a mathematical programming formulation of the problem, and also to analyze the competitiveness of the resulting algorithm using the dual program. This method can be viewed as an extension of the online primal-dual method for linear programming problems, to nonlinear programs. We show the application of this method to two online speed-scaling problems: one involving scheduling jobs on a speed scalable processor so as to minimize energy plus an arbitrary sum scheduling objective, and one involving routing virtual circuit connection requests in a network of speed scalable routers so as to minimize the aggregate power or energy used by the routers. This analysis shows that competitive algorithms exist for problems that had resisted analysis using the dominant potential function approach in the speed-scaling literature, and provides alternate cleaner analysis for other known results. This gives us another tool in the design and analysis of primal-dual algorithms for online problems.
]]>
Anupam Gupta et al.Approximating Sparse Covering Integer Programs Onlinehttp://repository.cmu.edu/compsci/2580
http://repository.cmu.edu/compsci/2580Sat, 01 Aug 2015 12:23:14 PDT
A covering integer program (CIP) is a mathematical program of the form: min{c⊤x∣Ax≥1,0≤x≤u,x∈\ensuremathZn},

where A∈Rm×n≥0,c,u∈\ensuremathRn≥0. In the online setting, the constraints (i.e., the rows of the constraint matrix A) arrive over time, and the algorithm can only increase the coordinates of x to maintain feasibility. As an intermediate step, we consider solving thecovering linear program (CLP) online, where the requirement x ∈ ℤ n is replaced by x ∈ ℝ n .

Our main results are (a) an O(logk)-competitive online algorithm for solving the CLP, and (b) an O(logk ·logℓ)-competitive randomized online algorithm for solving the CIP. Here k ≤ n and ℓ ≤ m respectively denote the maximum number of non-zero entries in any row and column of the constraint matrix A. By a result of Feige and Korman, this is the best possible for polynomial-time online algorithms, even in the special case of set cover (where A ∈ {0,1} m ×nand c, u ∈ {0,1} n ).

The novel ingredient of our approach is to allow the dual variables to increase and decrease throughout the course of the algorithm. We show that the previous approaches, which either only raise dual variables, or lower duals only within a guess-and-double framework, cannot give a performance better than O(logn), even when each constraint only has a single variable (i.e., k = 1).

]]>
Anupam Gupta et al.Iterative Constructions and Private Data Releasehttp://repository.cmu.edu/compsci/2579
http://repository.cmu.edu/compsci/2579Sat, 01 Aug 2015 12:23:05 PDT
In this paper we study the problem of approximately releasing the cut function of a graph while preserving differential privacy, and give new algorithms (and new analyses of existing algorithms) in both the interactive and non-interactive settings.

Our algorithms in the interactive setting are achieved by revisiting the problem of releasing differentially private, approximate answers to a large number of queries on a database. We show that several algorithms for this problem fall into the same basic framework, and are based on the existence of objects which we call iterative database construction algorithms. We give a new generic framework in which new (efficient) IDC algorithms give rise to new (efficient) interactive private query release mechanisms. Our modular analysis simplifies and tightens the analysis of previous algorithms, leading to improved bounds. We then give a new IDC algorithm (and therefore a new private, interactive query release mechanism) based on the Frieze/Kannan low-rank matrix decomposition. This new release mechanism gives an improvement on prior work in a range of parameters where the size of the database is comparable to the size of the data universe (such as releasing all cut queries on dense graphs).

We also give a non-interactive algorithm for efficiently releasing private synthetic data for graph cuts with error O(|V|1.5). Our algorithm is based on randomized response and a non-private implementation of the SDP-based, constant-factor approximation algorithm for cut-norm due to Alon and Naor. Finally, we give a reduction based on the IDC framework showing that an efficient, private algorithm for computing sufficiently accurate rank-1 matrix approximations would lead to an improved efficient algorithm for releasing private synthetic data for graph cuts. We leave finding such an algorithm as our main open problem.

]]>
Anupam Gupta et al.Approximation Algorithms for Correlated Knapsacks and Non-Martingale Banditshttp://repository.cmu.edu/compsci/2578
http://repository.cmu.edu/compsci/2578Sat, 01 Aug 2015 12:22:56 PDT
In the stochastic knapsack problem, we are given a knapsack of size B, and a set of items whose sizes and rewards are drawn from a known probability distribution. To know the actual size and reward we have to schedule the item-when it completes, we get to know these values. The goal is to schedule the items (possibly making adaptive decisions based on the sizes seen so far) to maximize the expected total reward of items which successfully pack into the knapsack. We know constant-factor approximations when (i) the rewards and sizes are independent, and (ii) we cannot prematurely cancel items after we schedule them. What if either or both assumptions are relaxed? Related stochastic packing problems are the multi-armed bandit (and budgeted learning) problems, here one is given several arms which evolve in a specified stochastic fashion with each pull, and the goal is to (adaptively) decide which arms to pull, in order to maximize the expected reward obtained after B pulls in total. Much recent work on this problem focuses on the case when the evolution of each arm follows a martingale, i.e., when the expected reward from one pull of an arm is the same as the reward at the current state. What if the rewards do not form a martingale? In this paper, we give O(1)-approximation algorithms for the stochastic knapsack problem with correlations and/or cancellations. Extending the ideas developed here, we give O(1)-approximations for MAB problems without the martingale assumption. Indeed, we can show that previously proposed linear programming relaxations for these problems have large integrality gaps. So we propose new time-indexed LP relaxations, using a decomposition and "gap-filling" approach, we convert these fractional solutions to distributions over strategies, and then use the LP values and the time ordering information from these strategies to devise randomized adaptive scheduling algorithms.
]]>
Anupam Gupta et al.Privately Releasing Conjunctions and the Statistical Query Barrierhttp://repository.cmu.edu/compsci/2577
http://repository.cmu.edu/compsci/2577Sat, 01 Aug 2015 12:22:48 PDT
Suppose we would like to know all answers to a set of statistical queries C on a data set up to small error, but we can only access the data itself using statistical queries. A trivial solution is to exhaustively ask all queries in C. Can we do any better?

1. We show that the number of statistical queries necessary and sufficient for this task is—up to polynomial factors—equal to the agnostic learning complexity of C in Kearns’ statistical query (SQ) model. This gives a complete answer to the question when running time is not a concern.

2. We then show that the problem can be solved efficiently (allowing arbitrary error on a small fraction of queries) whenever the answers to C can be described by a submodular function. This includes many natural concept classes, such as graph cuts and Boolean disjunctions and conjunctions.

While interesting from a learning theoretic point of view, our main applications are in privacypreserving data analysis: Here, our second result leads to an algorithm that efficiently releases differentially private answers to all Boolean conjunctions with 1% average error. This presents significant progress on a key open problem in privacy-preserving data analysis. Our first result on the other hand gives unconditional lower bounds on any differentially private algorithm that admits a (potentially nonprivacy-preserving) implementation using only statistical queries. Not only our algorithms, but also most known private algorithms can be implemented using only statistical queries, and hence are constrained by these lower bounds. Our result therefore isolates the complexity of agnostic learning in the SQ-model as a new barrier in the design of differentially private algorithms.

]]>
Anupam Gupta et al.Approximation Algorithms for Network Design: A Surveyhttp://repository.cmu.edu/compsci/2576
http://repository.cmu.edu/compsci/2576Sat, 01 Aug 2015 12:22:39 PDT
In a typical instance of a network design problem, we are given a directed or undirected graph G = (V,E), non-negative edge-costs ce for all e ∈ E, and our goal is to find a minimum-cost subgraph H of G that satisfies some design criteria. For example, we may wish to find a minimum-cost set of edges that induces a connected graph (this is the minimum-cost spanning tree problem), or we might want to find a minimum-cost set of arcs in a directed graph such that every vertex can reach every other vertex (this is the minimum-cost strongly connected subgraph problem). This abstract model for network design problems has a large number of practical applications; the design process of telecommunication and traffic networks, and VLSI chip design are just two examples.

Many practically relevant instances of network design problems are NP-hard, and thus likely intractable. This survey focuses on approximation algorithms as one possible way of circumventing this impasse. Approximation algorithms are efficient (i.e., they run in polynomial-time), and they compute solutions to a given instance of an optimization problem whose objective values are close to those of the respective optimum solutions. More concretely, most of the problems discussed in this survey are minimization problems. We then say that an algorithm is an α-approximation for a given problem if the ratio of the cost of an approximate solution computed by the algorithm to that of an optimum solution is at most α over all instances. In the following we will also sometimes refer to α as the performance guarantee of the respective approximation algorithm.

The last 30 years have seen a tremendous amount of research on approximation algorithms for network design problems. And over this period, several technical themes have emerged, and have been explored and exploited to give algorithms and analyze their performance. Our aim in this survey is to provide an overview over these techniques. Each of the following sections focuses on one technique and has two main parts: first, we present an introductory application to the well-known classical minimum-spanning tree problem. The second part of each section demonstrates more sophisticated recent example applications of the respective technique. Throughout we assume that the reader is familiar with fundamental concepts of graph theory, combinatorial optimization, and approximation algorithms. While we may recap certain key definitions, we rely on the reader to be familiar with others. We refer to the excellent text books [45, 160, 163] for background reading.

The minimum spanning tree problem has been studied for at least a century, and it is clearly one of the most prominent network design problems. The input to an instance of this problem consists of an undirected graph G = (V,E) each of whose edges e ∈ E is endowed by an arbitrary cost ce, and the goal is to compute a spanning tree of smallest cost. The earliest known algorithm for this problem was developed by Boruvka [ ˙ 21], and since then a vast number of techniques have been developed and subsequently used in order to devise increasingly sophisticated algorithms.

]]>
Anupam Gupta et al.Scalably Scheduling Power-Heterogeneous Processorshttp://repository.cmu.edu/compsci/2575
http://repository.cmu.edu/compsci/2575Sat, 01 Aug 2015 12:22:31 PDT
We show that a natural online algorithm for scheduling jobs on a heterogeneous multiprocessor, with arbitrary power functions, is scalable for the objective function of weighted flow plus energy.
]]>
Anupam Gupta et al.Nonclairvoyantly Scheduling Power-Heterogeneous Processorshttp://repository.cmu.edu/compsci/2574
http://repository.cmu.edu/compsci/2574Sat, 01 Aug 2015 12:22:22 PDT
We show that a natural nonclairvoyant online algorithm for scheduling jobs on a power-heterogeneous multiprocessor is bounded-speed bounded-competitive for the objective of flow plus energy.
]]>
Anupam Gupta et al.Com2: Fast Automatic Discovery of Temporal (’Comet’) Communitieshttp://repository.cmu.edu/compsci/2573
http://repository.cmu.edu/compsci/2573Thu, 30 Jul 2015 14:58:15 PDT
Given a large network, changing over time, how can we find patterns and anomalies? We propose Com2, a novel and fast, incremental tensor analysis approach, which can discover both transient and periodic/ repeating communities. The method is (a) scalable, being linear on the input size (b) general, (c) needs no user-defined parameters and (d) effective, returning results that agree with intuition.

We apply our method on real datasets, including a phone-call network and a computer-traffic network. The phone call network consists of 4 million mobile users, with 51 million edges (phonecalls), over 14 days. Com2 spots intuitive patterns, that is, temporal communities (comet communities).

We report our findings, which include large ‘star’-like patterns, nearbipartite- cores, as well as tiny groups (5 users), calling each other hundreds of times within a few days.

]]>
Miguel Araujo et al.OddBall: Spotting Anomalies in Weighted Graphshttp://repository.cmu.edu/compsci/2572
http://repository.cmu.edu/compsci/2572Thu, 30 Jul 2015 14:58:08 PDT
Given a large, weighted graph, how can we find anomalies? Which rules should be violated, before we label a node as an anomaly? We propose the oddball algorithm, to find such nodes. The contributions are the following: (a) we discover several new rules (power laws) in density, weights, ranks and eigenvalues that seem to govern the so-called “neighborhood sub-graphs” and we show how to use these rules for anomaly detection; (b) we carefully choose features, and design oddball, so that it is scalable and it can work un-supervised (no user-defined constants) and (c) we report experiments on many real graphs with up to 1.6 millionnodes, where oddball indeed spots unusual nodes that agree with intuition.
]]>
Leman Akoglu et al.Extending the Graphics Pipeline with Adaptive, Multi-Rate Shadinghttp://repository.cmu.edu/compsci/2571
http://repository.cmu.edu/compsci/2571Thu, 30 Jul 2015 14:58:00 PDT
Due to complex shaders and high-resolution displays (particularly on mobile graphics platforms), fragment shading often dominates the cost of rendering in games. To improve the efficiency of shading on GPUs, we extend the graphics pipeline to natively support techniques that adaptively sample components of the shading function more sparsely than per-pixel rates. We perform an extensive study of the challenges of integrating adaptive, multi-rate shading into the graphics pipeline, and evaluate two- and three-rate implementations that we believe are practical evolutions of modern GPU designs. We design new shading language abstractions that simplify development of shaders for this system, and design adaptive techniques that use these mechanisms to reduce the number of instructions performed during shading by more than a factor of three while maintaining high image quality.
]]>
Yong He et al.Self-Refining Games using Player Analyticshttp://repository.cmu.edu/compsci/2570
http://repository.cmu.edu/compsci/2570Thu, 30 Jul 2015 14:57:52 PDT
Data-driven simulation demands good training data drawn from a vast space of possible simulations. While fully sampling these large spaces is infeasible, we observe that in practical applications, such as gameplay, users explore only a vanishingly small subset of the dynamical state space. In this paper we present a sampling approach that takes advantage of this observation by concentrating precomputation around the states that users are most likely to encounter. We demonstrate our technique in a prototype self-refining game whose dynamics improve with play, ultimately providing realistically rendered, rich fluid dynamics in real time on a mobile device. Our results show that our analytics-driven training approach yields lower model error and fewer visual artifacts than a heuristic training strategy.
]]>
Matt Stanton et al.Efficient BVH Construction via Approximate Agglomerative Clusteringhttp://repository.cmu.edu/compsci/2569
http://repository.cmu.edu/compsci/2569Thu, 30 Jul 2015 14:57:45 PDT
We introduce Approximate Agglomerative Clustering (AAC), an efficient, easily parallelizable algorithm for generating high-quality bounding volume hierarchies using agglomerative clustering. The main idea of AAC is to compute an approximation to the true greedy agglomerative clustering solution by restricting the set of candidates inspected when identifying neighboring geometry in the scene. The result is a simple algorithm that often produces higher quality hierarchies (in terms of subsequent ray tracing cost) than a full sweep SAH build yet executes in less time than the widely used top-down, approximate SAH build algorithm based on binning.
]]>
Yan Gu et al.Caveat-Scriptor: Write Anywhere Shingled Diskshttp://repository.cmu.edu/compsci/2568
http://repository.cmu.edu/compsci/2568Thu, 30 Jul 2015 14:57:38 PDT
In this paper we will present a simple model for Host- Managed Caveat-Scriptor, describe a simple FUSE-based file system for Host-Managed Caveat-Scriptor, construct and describe a file system aging tool and report initial performance comparisons between Strict-Append and Caveat-Scriptor. We will show the potential for Caveat-Scriptor to help limit heavy tail response times for shingled disks.
]]>
Saurabh Kadekodi et al.BatchFS: Scaling the File System Control Plane with Client-Funded Metadata Servershttp://repository.cmu.edu/compsci/2567
http://repository.cmu.edu/compsci/2567Thu, 30 Jul 2015 14:57:31 PDT
Parallel file systems are often characterized by a layered architecture that decouples metadata management from I/O operations, allowing file systems to facilitate fast concurrent access to file contents. However, metadata intensive workloads are still likely to bottleneck at the file system control plane due to namespace synchronization, which taxes application performance through lock contention on directories, transaction serialization, and RPC overheads. In this paper, we propose a client-driven file system metadata architecture, BatchFS, that is optimized for noninteractive, or batch, workloads. To avoid metadata bottlenecks, BatchFS features a relaxed consistency model marked by lazy namespace synchronization and optimistic metadata verification. Capable of executing namespace operations on client-provisioned resources without contacting any metadata server, BatchFS clients are able to delay namespace synchronization until synchronization is really needed. Our goal in this vision paper is to handle these delayed operations securely and efficiently with metadata verification and bulk insertion. Preliminary experiments demonstrate that our client-funded metadata architecture outperforms a traditional synchronous file system by orders of magnitude.
]]>
Qing Zheng et al.Will They Blend?: Exploring Big Data Computation atop Traditional HPC NAS Storagehttp://repository.cmu.edu/compsci/2566
http://repository.cmu.edu/compsci/2566Thu, 30 Jul 2015 14:57:23 PDT
The Apache Hadoop framework has rung in a new era in how data-rich organizations can process, store, and analyze large amounts of data. This has resulted in increased potential for an infrastructure exodus from the traditional solution of commercial database ad-hoc analytics on network-attached storage (NAS). While many data-rich organizations can afford to either move entirely to Hadoop for their Big Data analytics, or to maintain their existing traditional infrastructures and acquire a new set of infrastructure solely for Hadoop jobs, most supercomputing centers do not enjoy either of those possibilities. Too much of the existing scientific code is tailored to work on massively parallel file systems unlike the Hadoop Distributed File System (HDFS), and their datasets are too large to reasonably maintain and/or ferry between two distinct storage systems. Nevertheless, as scientists search for easier-to-program frameworks with a lower time-to-science to post-process their huge datasets after execution, there is increasing pressure to enable use of MapReduce within these traditional High Performance Computing (HPC) architectures. Therefore, in this work we explore potential means to enable use of the easy-to-program Hadoop MapReduce framework without requiring a complete infrastructure overhaul from existing HPC NAS solutions. We demonstrate that retaining function-dedicated resources like NAS is not only possible, but can even be effected efficiently with MapReduce. In our exploration, we unearth subtle pitfalls resultant from this mash-up of new-era Big Data computation on conventional HPC storage and share the clever architectural configurations that allow us to avoid them. Last, we design and present a novel Hadoop File System, the Reliable Array of Independent NAS File System (RainFS), and experimentally demonstrate its improvements in performance and reliability over the previous architectures we have investigated.
]]>
Ellis H. Wilson III et al.TABLEFS: Enhancing Metadata Efﬁciency in the Local File Systemhttp://repository.cmu.edu/compsci/2565
http://repository.cmu.edu/compsci/2565Thu, 30 Jul 2015 14:57:15 PDT
File systems that manage magnetic disks have long recognized the importance of sequential allocation and large transfer sizes for file data. Fast random access has dominated metadata lookup data structures with increasing use of B-trees on-disk. Yet our experiments with workloads dominated by metadata and small file access indicate that even sophisticated local disk file systems like Ext4, XFS and Btrfs leave a lot of opportunity for performance improvement in workloads dominated by metadata and small files.

In this paper we present a stacked file system, TABLEFS, which uses another local file system as an object store. TABLEFS organizes all metadata into a single sparse table backed on disk using a Log-Structured Merge (LSM) tree, LevelDB in our experiments. By stacking, TABLEFS asks only for efficient large file allocation and access from the underlying local file system. By using an LSM tree, TABLEFS ensures metadata is written to disk in large, non-overwrite, sorted and indexed logs. Even an inefficient FUSE based user level implementation of TABLEFS can perform comparably to Ext4, XFS and Btrfs on data-intensive benchmarks, and can outperform them by 50% to as much as 1000% for metadata-intensive workloads. Such promising performance results from TABLEFS suggest that local disk file systems can be significantly improved by more aggressive aggregation and batching of metadata updates.

]]>
Kai Ren et al.Active Property Testinghttp://repository.cmu.edu/compsci/2564
http://repository.cmu.edu/compsci/2564Tue, 28 Jul 2015 14:58:57 PDT
One motivation for property testing of boolean functions is the idea that testing can provide a fast preprocessing step before learning. However, in most machine learning applications, it is not possible to request for labels of arbitrary examples constructed by an algorithm. Instead, the dominant query paradigm in applied machine learning, called active learning, is one where the algorithm may query for labels, but only on points in a given (polynomial-sized) unlabeled sample, drawn from some underlying distribution D. In this work, we bring this well-studied model to the domain of testing. We develop both general results for this active testing model as well as efficient testing algorithms for several importantproperties for learning, demonstrating that testing can still yield substantial benefits in this restricted setting. For example, we show that testing unions of d intervals can be done with O(1) label requests in our setting, whereas it is known to √ require Ω(d) labeled examples for learning (and Ω(√d) for passivetesting [22] where the algorithm must pay for every example drawn from D). In fact, our results fortesting unions of intervals also yield improvements on prior work in both the classic query model (where any point in the domain can be queried) and the passive testing model as well. For the problem oftesting linear separators in Rn over the Gaussian distribution, we show that both active and passivetesting can be done with O(√n) queries, substantially less than the Ω(n) needed for learning, with near-matching lower bounds. We also present a general combination result in this model for building testableproperties out of others, which we then use to provide testers for a number of assumptions used in semi-supervised learning. In addition to the above results, we also develop a general notion of thetesting dimension of a given property with respect to a given distribution, that we show characteriz- s (up to constant factors) the intrinsic number of label requests needed to test that property. We develop such notions for both the active and passive testing models. We then use these dimensions to prove a number of lower bounds, including for linear separators and the class of dictator functions.
]]>
Maria-Florina Balcan et al.The Johnson-Lindenstrauss Transform Itself Preserves Differential Privacyhttp://repository.cmu.edu/compsci/2563
http://repository.cmu.edu/compsci/2563Tue, 28 Jul 2015 14:58:48 PDT
This paper proves that an "old dog", namely - the classical Johnson-Lindenstrauss transform, "performs new tricks" - it gives a novel way of preserving differential privacy. We show that if we take two databases, D and D', such that (i) D'-D is a rank-1 matrix of bounded norm and (ii) all singular values of D and D' are sufficiently large, then multiplying either D or D' with a vector of iid normal Gaussians yields two statistically close distributions in the sense of differential privacy. Furthermore, a small, deterministic and public alteration of the input is enough to assert that all singular values of D are large.

We apply the Johnson-Lindenstrauss transform to the task of approximating cut-queries: the number of edges crossing a (S, S)-cut in a graph. We show that the JL transform allows us to publish a sanitized graph that preserves edge differential privacy (where two graphs are neighbors if they differ on a single edge) while adding only O(|S|ϵ) random noise to any given query (w.h.p). Comparing the additive noise of our algorithm to existing algorithms for answering cut-queries in a differentially private manner, we outperform all others on small cuts (|S| = o(n)).

We also apply our technique to the task of estimating the variance of a given matrix in any given direction. The JL transform allows us to publish a sanitized covariance matrix that preserves differential privacy w.r.t bounded changes (each row in the matrix can change by at most a norm-1 vector) while adding random noise of magnitude independent of the size of the matrix (w.h.p). In contrast, existing algorithms introduce an error which depends on the matrix dimensions.

]]>
Jeremiah Blocki et al.Additive Approximation for Near-Perfect Phylogeny Constructionhttp://repository.cmu.edu/compsci/2562
http://repository.cmu.edu/compsci/2562Tue, 28 Jul 2015 14:58:39 PDT
We study the problem of constructing phylogenetic trees for a given set of species. The problem is formulated as that of finding a minimum Steiner tree on n points over the Boolean hypercube of dimension d. It is known that an optimal tree can be found in linear time [1] if the given dataset has a perfect phylogeny, i.e. cost of the optimal phylogeny is exactly d. Moreover, if the data has a near-perfect phylogeny, i.e. the cost of the optimal Steiner tree isd + q, it is known [2] that an exact solution can be found in running time which is polynomial in the number of species and d, yet exponential in q. In this work, we give a polynomial-time algorithm (in both d and q) that finds a phylogenetic tree of cost d + O(q 2). This provides the best guarantees known—namely, a (1 + o(1))-approximation—for the case log(d)≪q≪d√, broadening the range of settings for which near-optimal solutions can be efficiently found. We also discuss the motivation and reasoning for studying such additive approximations.
]]>
Pranjal Awasthi et al.Welfare and Profit Maximization with Production Costshttp://repository.cmu.edu/compsci/2561
http://repository.cmu.edu/compsci/2561Tue, 28 Jul 2015 14:58:30 PDT
Combinatorial Auctions are a central problem in Algorithmic Mechanism Design: pricing and allocating goods to buyers with complex preferences in order to maximize some desired objective (e.g., socialwelfare, revenue, or profit). The problem has been well-studied in the case of limited supply (one copy of each item), and in the case of digital goods (the seller can produce additional copies at no cost). Yet in the case of resources -- oil, labor, computing cycles, etc. -- neither of these abstractions is just right: additional supplies of these resources can be found, but at increasing difficulty (marginal cost) as resources are depleted. In this work, we initiate the study of the algorithmic mechanism design problem of combinatorial pricing under increasing marginal cost. The goal is to sell these goods to buyers with unknown and arbitrary combinatorial valuation functions to maximize either the social welfare, or the seller's profit, specifically we focus on the setting of posted item prices with buyers arriving online. We give algorithms that achieve constant factor approximations for a class of natural cost functions - linear, low-degree polynomial, logarithmic - and that give logarithmic approximations for more general increasing marginal cost functions (along with a necessary additive loss). We show that these bounds are essentially best possible for these settings.
]]>
Avrim Blum et al.Center-based clustering under perturbation stabilityhttp://repository.cmu.edu/compsci/2560
http://repository.cmu.edu/compsci/2560Tue, 28 Jul 2015 14:58:20 PDT
Clustering under most popular objective functions is NP-hard, even to approximate well, and so unlikely to be efficiently solvable in the worst case. Recently, Bilu and Linial (2010) [11] suggested an approach aimed at bypassing this computational barrier by using properties of instances one might hope to hold in practice. In particular, they argue that instances in practice should be stable to small perturbations in the metric space and give an efficient algorithm for clustering instances of the Max-Cut problem that are stable to perturbations of size O(n1/2). In addition, they conjecture that instances stable to as little as O(1) perturbations should be solvable in polynomial time. In this paper we prove that this conjecture is true for any center-based clustering objective (such as k-median, k-means, and k -center). Specifically, we show we can efficiently find the optimal clustering assuming only stability to factor-3 perturbations of the underlying metric in spaces without Steiner points, and stability to factor perturbations for general metrics. In particular, we show for such instances that the popular Single-Linkage algorithm combined with dynamic programming will find the optimal clustering. We also present NP-hardness results under a weaker but related condition.
]]>
Pranjal Awasthi et al.A Discriminative Model for Semi-Supervised Learninghttp://repository.cmu.edu/compsci/2559
http://repository.cmu.edu/compsci/2559Tue, 28 Jul 2015 14:58:10 PDT
Supervised learning—that is, learning from labeled examples—is an area of Machine Learning that has reached substantial maturity. It has generated general-purpose and practically successful algorithms and the foundations are quite well understood and captured by theoretical frameworks such as the PAC-learning model and the Statistical Learning theory framework. However, for many contemporary practical problems such as classifying web pages or detecting spam, there is often additional information available in the form ofunlabeled data, which is often much cheaper and more plentiful than labeled data. As a consequence, there has recently been substantial interest in semi-supervised learning—using unlabeled data together with labeled data—since any useful information that reduces the amount of labeled data needed can be a significant benefit. Several techniques have been developed for doing this, along with experimental results on a variety of different learning problems. Unfortunately, the standard learning frameworks for reasoning about supervised learning do not capture the key aspects and the assumptions underlying these semi-supervised learning methods.

In this article, we describe an augmented version of the PAC model designed for semi-supervised learning, that can be used to reason about many of the different approaches taken over the past decade in the Machine Learning community. This model provides a unified framework for analyzing when and why unlabeled data can help, in which one can analyze both sample-complexity and algorithmic issues. The model can be viewed as an extension of the standard PAC model where, in addition to a concept class C, one also proposes a compatibility notion: a type of compatibility that one believes the target concept should have with the underlying distribution of data. Unlabeled data is then potentially helpful in this setting because it allows one to estimate compatibility over the space of hypotheses, and to reduce the size of the search space from the whole set of hypotheses C down to those that, according to one's assumptions, are a-priori reasonable with respect to the distribution. As we show, many of the assumptions underlying existing semi-supervised learning algorithms can be formulated in this framework.

After proposing the model, we then analyze sample-complexity issues in this setting: that is, how much of each type of data one should expect to need in order to learn well, and what the key quantities are that these numbers depend on. We also consider the algorithmic question of how to efficiently optimize for natural classes and compatibility notions, and provide several algorithmic results including an improved bound for Co-Training with linear separators when the distribution satisfies independence given the label.

]]>
Maria-Florina Balcan et al.Trading off Mistakes and Don’t-Know Predictionshttp://repository.cmu.edu/compsci/2558
http://repository.cmu.edu/compsci/2558Tue, 28 Jul 2015 14:58:02 PDT
We discuss an online learning framework in which the agent is allowed to say I don't know'' as well as making incorrect predictions on given examples. We analyze the trade off between saying I don't know'' and making mistakes. If the number of don't know predictions is forced to be zero, the model reduces to the well-known mistake-bound model introduced by Littlestone [Lit88]. On the other hand, if no mistakes are allowed, the model reduces to KWIK framework introduced by Li et. al. [LLW08]. We propose a general, though inefficient, algorithm for general finite concept classes that minimizes the number of don't-know predictions if a certain number of mistakes are allowed. We then present specific polynomial-time algorithms for the concept classes of monotone disjunctions and linear separators
]]>
Amin Sayedi et al.Stability Yields a PTAS for k-Median and k-Means Clusteringhttp://repository.cmu.edu/compsci/2557
http://repository.cmu.edu/compsci/2557Tue, 28 Jul 2015 14:57:53 PDT
We consider fc-median clustering in finite metric spaces and fc-means clustering in Euclidean spaces, in the setting where k is part of the input (not a constant). For the fc-means problem, Ostrovsky et al. show that if the optimal (k - 1)-means clustering of the input is more expensive than the optimal fc-means clustering by a factor of 1/∈2, then one can achieve a (1 + f(∈))-approximation to the fc-means optimal in time polynomial in n and k by using a variant of Lloyd's algorithm. In this work we substantially improve this approximation guarantee. We show that given only the condition that the (k - 1)-means optimal is more expensive than the fc-means optimal by a factor 1 + α for some constant α > 0, we can obtain a PTAS. In particular, under this assumption, for any ∈ > 0 we achieve a (1 + ∈)-approximation to the fc-means optimal in time polynomial in n and k, and exponential in 1/e and 1/α. We thus decouple the strength of the assumption from the quality of the approximation ratio. We also give a PTAS for the fc-median problem in finite metrics under the analogous assumption as well. For fc-means, we in addition give a randomized algorithm with improved running time of no(1) (k log n)poly(1/∈,1/α) Our technique also obtains a PTAS under the assumption of Balcan et al. that all (1 + α) approximations are δ-close to a desired target clustering, in the case that all target clusters have size greater than δn and α > 0 is constant. Note that the motivation of Balcan et al. is that for many clustering problems, the objective function is only a proxy for the true goal of getting close to the target. From this perspective, our improvement is that for fc-means in Euclidean spaces we reduce the distance of the clustering found to the target from O(δ) to δ when all target clusters are large, and for fc-median we improve- - the "largeness" condition needed in to get exactly δ-close from O(δn) to δn. Our results are based on a new notion of clustering stability.
]]>
Pranjal Awasthi et al.On Nash-Equilibria of Approximation-Stable Gameshttp://repository.cmu.edu/compsci/2556
http://repository.cmu.edu/compsci/2556Tue, 28 Jul 2015 14:57:43 PDT
One reason for wanting to compute an (approximate) Nash equilibrium of a game is to predict how players will play. However, if the game has multiple equilibria that are far apart, or ε-equilibria that are far in variation distance from the true Nash equilibrium strategies, then this prediction may not be possible even in principle. Motivated by this consideration, in this paper we define the notion of games that are approximation stable, meaning that all ε-approximate equilibria are contained inside a small ball of radius Δ around a true equilibrium, and investigate a number of their properties. Many natural small games such as matching pennies and rock-paper-scissors are indeed approximation stable. We show furthermore there exist 2-player n-by-n approximation-stable games in which the Nash equilibrium and all approximate equilibria have support Ω(logn). On the other hand, we show all (ε,Δ) approximation-stable games must have an ε-equilibrium of support O(Δ2−o(1)ϵ2logn), yielding an immediate nO(Δ2−o(1)ϵ2logn)-time algorithm, improving over the bound of [11] for games satisfying this condition. We in addition give a polynomial-time algorithm for the case that Δ and ε are sufficiently close together. We also consider an inverse property, namely that all non-approximate equilibria are far from some true equilibrium, and give an efficient algorithm for games satisfying that condition.
]]>
Pranjal Awasthi et al.Learning Conditional Abstractionshttp://repository.cmu.edu/compsci/2555
http://repository.cmu.edu/compsci/2555Tue, 28 Jul 2015 14:57:34 PDT
Abstraction is central to formal verification. In term-level abstraction, the design is abstracted using a fragment of first-order logic with background theories, such as the theory of uninterpreted functions with equality. The main challenge in using term-level abstraction is determining what components to abstract and under what conditions. In this paper, we present an automatic technique to conditionally abstract register transfer level (RTL) hardware designs to the term level. Our approach is a layered approach that combines random simulation and machine learning inside a counter-example guided abstraction refinement (CEGAR) loop. First, random simulation is used to determine modules that are candidates for abstraction. Next, machine learning is used on the resulting simulation traces to generate candidate conditions under which those modules can be abstracted. Finally, a verifier is invoked. If spurious counterexamples arise, we refine the abstraction by performing a further iteration of random simulation and machine learning. We present an experimental evaluation on processor designs.
]]>
Bryan Brady et al.Introductory Computer Science Education at Carnegie Mellon University: A Deans’ Perspectivehttp://repository.cmu.edu/compsci/2554
http://repository.cmu.edu/compsci/2554Tue, 28 Jul 2015 14:57:25 PDT
The School of Computer Science at Carnegie Mellon University is planning major revisions to its introductory course sequence in ways that will affect not just our own students, but also the many students from across campus who take computer science courses. Major changes include: 1) revising our introductory courses to promote the principles of computational thinking, for both majors and nonmajors, 2) increasing our emphasis on the need to make software systems highly reliable and the means to achieve this, and 3) preparing students for a future in which programs will achieve high performance by exploiting parallel execution.
]]>
Randal E. Bryant et al.