A greedy algorithm always makes the choice that looks best at the moment

Key point Greed makes a locally optimal choice in the hope that this choice will lead to a globally optimal solution

Note Greedy algorithms do not always yield optimal solutions, but for SOME problems they do

2Greed

When do we use greedy algorithms?

When we need a heuristic (e.g., hard problems like the Traveling Salesman Problem)

When the problem itself is greedy

Greedy Choice Property (CLRS 16.2)

Optimal Substructure Property (shared with DP) (CLRS 16.2)

Examples

Minimum Spanning Tree (Kruskals algorithm)

Optimal Prefix Codes (Huffmans algorithm)

3Elements of the Greedy Algorithm

Greedy-choice property A globally optimal solution can be arrived at by making a locally optimal (greedy) choice.

Must prove that a greedy choice at each step yields a globally optimal solution

Optimal substructure property A problem exhibits optimal substructure if an optimal solution to the problem contains within it optimal solutions to subproblems. This property is a key ingredient of assessing the applicability of greedy algorithm and dynamic programming.

4Greedy Algorithm Minimum Spanning Tree

Kruskals minimum spanning tree algorithm

INPUT edge-weighted graph G (V,E), with V n

OUTPUT T a spanning tree of G (touches all vertices, and therefore has n-1 edges) of minimum cost ( total edge weight)

Will grow a set of edges T until it contains n-1 edges

Algorithm Start with T empty, then iteratively add to T the smallest (minimum-cost) remaining edge in the graph if the edge does not form a cycle in T

Claim Greedy-MST is correct.

Proof (by induction)

Define a promising edge set E ? E to be any edge set that is a subgraph of some MST. We will show by induction that as T is constructed, T is always promising.

5Proof of Kruskals Algorithm

Basis T 0, trivial.

Induction Step T is promising by I.H., so it is a subgraph of some MST, call it S. Let ei be the smallest edge in E, s.t. T?ei has no cycle, ei?T.

If ei?S, were done.

Suppose ei?S, then S S ? ei has a unique cycle containing ei, and all other arcs in cycle ? ei (because S is an MST!)

Call the cycle C. Observe that C with ei cannot be in T, because T ? ei is acyclic (because Kruskal adds ei)

6Proof of Kruskals Algorithm

Then C must contains some edge ej s.t. ej?S, and we also know c(ej)?c(ei).

Let S S ? ei \ ej

S is an MST, so T?ei is promising

7Greedy Algorithm Huffman Codes

Prefix codes

one code per input symbol

no code is a prefix of another

Why prefix codes?

Easy decoding

Since no codeword is a prefix of any other, the codeword that begins an encoded file is unambiguous

Identify the initial codeword, translate it back to the original character, and repeat the decoding process on the remainder of the encoded file

8Greedy Algorithm Huffman Codes

Huffman coding

Given frequencies with which with which source symbols (e.g., A, B, C, , Z) appear in a message

Goal is to minimize the expected encoded message length

Create tree (leaf) node for each symbol that occurs with nonzero frequency

Node weights frequencies

Find two nodes with smallest frequency

Create a new node with these two nodes as children, and with weight equal to the sum of the weights of the two children

Continue until have a single tree

9Greedy Algorithm Huffman Codes

Example

10Greedy Algorithm Huffman Codes

Step 1 Step 2

Step 3 Step 4

11Greedy Algorithm Huffman Codes

Step 5

Step 6

12Greedy Algorithm Huffman Codes

Step 7

Step 8

13Greedy Algorithm Huffman Codes

Step 9

14Greedy Algorithm Huffman Codes

Finally

Note that the 0s (left branches) and 1s (right branches) give the code words for each symbol

15Proof That Huffmans Merge is Optimal

Let T be an optimal prefix-code tree in which a, b are siblings at deepest level, L(a) L(b)

Suppose that x, y are two other nodes that are merged by the Huffman algorithm

x, y have lowest weights because Huffman chose them

WLOG w(x) w(a), w(y) w(b) L(a) L(b) ? L(x), L(y)

Swap a and x cost difference between T and new T is

w(x)L(x) w(a)L(a) w(x)L(a) w(a)L(x)

(w(a) w(x))(L(a) L(x)) // both factors non-neg

? 0

Similar argument for b, y

? Huffman choice also optimal

16Dynamic Programming

Dynamic programming Divide problem into overlapping subproblems recursively solve each in the same way.

Similar to DQ, so whats the difference

DQ partition the problem into independent subproblems.

DP breaking it into overlapping subproblems, that is, when subproblems share subproblems.

So DP saves work compared with DQ by solving every subproblems just once ( when subproblems are overlapping).

17Elements of Dynamic Programming

Optimal substructure A problem exhibits optimal substructure if an optimal solution to the problem contains within it optimal solutions to subproblems.

Whenever a problem exhibits optimal substructure, it is a good clue that DP might apply.(a greedy method might apply also.)

Overlapping subproblems A recursive algorithm for the problem solves the same subproblems over and over, rather than always generating new subproblems.

18Dynamic Programming Matrix Chain Product

Matrix-chain multiplication problem Give a chain of n matrices ?A1, A2, , An? to be multiplied, how to get the product A1 A2 An. with minimum number of scalar multiplications.

Because of the associative law of matrix multiplication, there are many possible orderings to calculate the product for the same matrix chain

Longest Common Subsequence Give two strings a1 a2 am and b1 b2 bn, what is the largest value P such that

For indices 1? i1 ? i2 ? ? ip ? m, and

1? j1 ? j2 ? ? jp ? n,

We have aix bjx, for 1? x? P

Example

So P 4, i 1, 2, 3, 5, j 3, 4, 5, 6

b a a b a c b a c b a a a 25Dynamic Programming Longest Common Subsequence

Let L(k, l) denote length of LCS for a1 a2 ak and b1 b2 bl.

Then we have facts

L(p, q) ? L(p-1, q-1).

L(p, q) L(p-1, q-1) 1 if ap bq when ap and bq are both in LCS.

L(p, q) ? L(p-1, q) when ap is not in LCS.

L(p, q) ? L(p, q-1)

when bq is not in LCS.

26Dynamic Programming Longest Common Subsequence

ALGORITHM

for i 1 to m

for j 1 to n

if ai bj then L(i, j) L(i-1, j-1) 1

else L(i, j) maxL(i, j-1),

L(i-1, j)

Time complexity ?(n2).

27Dynamic Programming Knapsack

The problem The knapsack problem is a particular type of integer program with just one constraint Each item that can go into the knapsack has a size and a benefit. The knapsack has a certain capacity. What should go into the knapsack so as to maximize the total benefit?

Hint Recall shortest path method.

Define Fk(y) max (0?k?n)

with (0?y?b)

Then, what is Fk(y)?

Max value possible using only first k items when weight limit is y.

28Dynamic Programming Knapsack

B.C.s 1. F0(y) 0 ?y ?no items chosen

2. Fk(0) 0 ?k ?weight limit 0

3. F1(y) ?y/w1?v1

Generally speaking

Fk(y) maxFk-1(y), Fk(y-wk)vk

Then we could build matrix use entries above, here is an example

29Dynamic Programming Knapsack

Example k4, b10, ypounds, kitem types allowed

v11 w12 v23 w23

v35 w34 v49 w47

Fk(y) maxFk-1(y), Fk(y-wk)vk

Fk(y)

n?b table

30Dynamic Programming Knapsack

Note 12 max(11, 9 F4(3))max(11,93)12

What is missing here? (Like in SP, we know the SPs cost, but we dont know SP itself)

PowerShow.com is a leading presentation/slideshow sharing website. Whether your application is business, how-to, education, medicine, school, church, sales, marketing, online training or just for fun, PowerShow.com is a great resource. And, best of all, most of its cool features are free and easy to use.

You can use PowerShow.com to find and download example online PowerPoint ppt presentations on just about any topic you can imagine so you can learn how to improve your own slides and
presentations for free. Or use it to find and download high-quality how-to PowerPoint ppt presentations with illustrated or animated slides that will teach you how to do something new, also for free. Or use it to upload your own PowerPoint slides so you can share them with your teachers, class, students, bosses, employees, customers, potential investors or the world. Or use it to create really cool photo slideshows - with 2D and 3D transitions, animation, and your choice of music - that you can share with your Facebook friends or Google+ circles. That's all free as well!

For a small fee you can get the industry's best online privacy or publicly promote your presentations and slide shows with top rankings. But aside from that it's free. We'll even convert your presentations and slide shows into the universal Flash format with all their original multimedia glory, including animation, 2D and 3D transition effects, embedded music or other audio, or even video embedded in slides. All for free. Most of the presentations and slideshows on PowerShow.com are free to view, many are even free to download. (You can choose whether to allow people to download your original PowerPoint presentations and photo slideshows for a fee or free or not at all.) Check out PowerShow.com today - for FREE. There is truly something for everyone!

presentations for free. Or use it to find and download high-quality how-to PowerPoint ppt presentations with illustrated or animated slides that will teach you how to do something new, also for free. Or use it to upload your own PowerPoint slides so you can share them with your teachers, class, students, bosses, employees, customers, potential investors or the world. Or use it to create really cool photo slideshows - with 2D and 3D transitions, animation, and your choice of music - that you can share with your Facebook friends or Google+ circles. That's all free as well!

For a small fee you can get the industry's best online privacy or publicly promote your presentations and slide shows with top rankings. But aside from that it's free. We'll even convert your presentations and slide shows into the universal Flash format with all their original multimedia glory, including animation, 2D and 3D transition effects, embedded music or other audio, or even video embedded in slides. All for free. Most of the presentations and slideshows on PowerShow.com are free to view, many are even free to download. (You can choose whether to allow people to download your original PowerPoint presentations and photo slideshows for a fee or free or not at all.) Check out PowerShow.com today - for FREE. There is truly something for everyone!