Similar presentations

2
Linear Programming Linear Programming is the most powerful optimization problem that can be solved in polynomial time – Some generalizations [semidefinite programming] can be much more appropriate to use Fast LP solvers exist – Write an LP, solve it – Fast in theory/fast in practice... Important theoretical tool Duality

3
Linear Programming Definition – A Linear Program (LP) is an optimization problem over real variables x 1,…,x n – Maximizes/minimizes a linear function of x: max/min C(x)=  i c i x i ( c i are real coefficients) – Constraints: feasible solutions are those that satisfy a system of linear inqualities: A is a real m £ n matrix, b a vector All x with Ax · b are feasible – We are looking for a feasible x with maximum C(x)

7
Duality Given an LP A,b,c in standard form We call this the primal LP The dual LP is defined as follow: – There are m variables y i – Minimize  i b i y i – Constraints:  i=1 m A[i,j] y i ¸ c j y ¸ 0

8
Weak Duality Claim: – Let x be a feasible solution for the primal LP and y a feasible solution for the dual LP, then

9
Proof

10
Strong Duality Strong duality means that feasible solutions with the same value exist for the primal and the dual Compare Max Flow/Min Cut theorem

11
Nonstandard form The dual of a standard primal has a variable for each constraint of the primal and a constraint for each variable – And all variables are nonnegative If the primal has an equation  a i x i = b then the dual has a variable which is not constrained to be nonnegative

13
Constraints One constraint for each edge (u,v) – f(u,v) appears in 3 constraints in the primal Constraints in the dual y(s)=1 Note: y(s) and y(t) extra `variables’ y(t)=0 For each edge (u,v): x(u,v) - y(u) + y(v) ¸ 0 For each edge: x(u,v) ¸ 0 Interpretation: – For all s-t- paths p: sum of weights x(u,v) on p is at least 1

14
Equivalence An s,t-cut (L,R) of capacity c yields a solution of cost c – Each edge from L to R receives weight 1 – Cost is now equal to cut capacity – Constraints are true for every s,t path At least we know that the program is a relaxation of s,t-Min Cut How can we extract a cut from a solution of the dual LP?

16
Finding the cut  (u,v) C(u,v) Prob  [u 2 L and v 2 R] But Prob  [u 2 L and v 2 R] · d(v)-d(u) · x(u,v) Hence the expectation is at most  (u,v) C(u,v) x(u,v) Which is the value of the program I.e., given a solution of the LP of cost at most c there is a cut L,R of cost at most c I.e., the optimum of the LP has the same cost as the Min st cut Finding the cut: exercise.

17
Linear Programming Some facts: – In practice LP‘s are usually solved with the Simplex Algorithm – Many math programs/libraries have LP solvers – Simplex is exponential in the worst case – There is a theoretical explanation why Simplex is fast in practice [smoothed analysis] – Worst case polynomial time algorithms: Ellipsoid Interior Point Methods

19
Geometry Consider R n Every inequality corresponds to a halfspace Feasible solutions lie in an intersection of m halfspaces This set is a convex polytope The objective function points in some direction

20
Types of LPs For an LP the objective function can be unbounded – Solution set must be unbounded There can be no solution – Solution set empty Or there is a nonempty set of optimal solutions – A face of the polytope – There is always an optimal vertex

21
Slack Form We can express constraints as equations as follows:  Take  A[i,j] x j · b i  Introduce a new variable x n+i and set  x n+i = b i -  A[i,j]x j and x n+i ¸ 0 x n+i are called slack variables Doing this with all inequalities yields a system with n+m variables and m equations and the global nonnegativity constraint x ¸ 0 The variables on the left side of the equations are called basic variables

22
Simplex Algorithm Use slack form Start at a vertex of the convex polytope (how?)

23
Simplex Algorithm Iteration: – follow an edge to a better vertex We reach a local maximum Due to convexity and linearity of the cost function this is also a global maximum

24
Simplex Algorithm Input: LP in slack form m basic variables and n nonbasic variables – basic variables do not appear in the objective function Basic solution: – Set all nonbasic variables to 0 and all basic variables via their equations – Assume (for now) this is a feasible solution – Then this is a vertex of the polytope Increase a nonbasic variable to improve the solution (until not possible anymore) Exchange the nonbasic variable with a basic variable – Rewrite the LP Continue until (local) optimum is reached

28
Change of the slack form Choose a nonbasic variable x e with c(e)>0. Increasing x e increases the objective function Increasing x e changes the value of basic variables Increase x e until a basic variable is 0 – If this is never the case the objective function is unbounded Change the LP to make x e a basic variable – This is called pivoting Iterate Geometry: move along edges of the polytope

29
Operation pivot Input: – slack form LP: N: Indices of nonbasic variables H: Indices of basic variables A,b: Matrix and vector of the LP v: constant in the objective function c: coefficients of the objective function e: Index of some nonbasic variable l: Index of a basic variable Output: all these – new slack form LP with x e as basic variable and x l as nonbasic variable

35
Termination The objective function never decreases It might not increase This is called a degenerate iteration. It can happen that Simplex get‘s into a loop

36
Termination Claim: – If Simplex uses more than Iterations, then it it in a loop Proof: – For a given LP and set of basic variables the slack form is unique – There are n+m variables und m basic variables – There are choices of basic variables – More iterations means loop So we can at least stop Loops are rare in practice

37
Preventing loops Loops happen, if the objective function does not increase – Solution: perturb the input slightly Bland‘s rule: – Always choose the variables with the smallest index (among candidates for x e,x l ) Under Bland‘s Rule there are no loops

38
Running Time The running time is bounded by the number of vertices of the polytope Polytope has at most n+m facets and lies in an n-dimensional subspace Vertex: 0-dimensional, intersection of n facets Number of vertices is at most

39
Correctness (II) We have to show that the solution obtained in the end is optimal This is proved using the dual

40
Strong Duality and Optimality Claim: – If  c j x j =  b i y i, and x is feasible for the primal and y is feasible for the dual, then both are optimal Proof: – By weak duality no solution x‘ of the primal can have larger value than the optimum of the dual.

41
Strong Duality Primal and Dual LP have feasible solutions x,y that have the same objective value Theorem: – Assume Simplex produces solution x 1,…,x n – H and N are the basic and nonbasic variables of the last slack form – c‘ coefficients of the objective function of the last slack form – define y by:y i =-c‘ n+i if n+i 2 N =0 otherwise – Then x is optimal for the primal and y for the dual – x and y have the same objective value