1
00:00:00 --> 00:00:00
2
00:00:00 --> 00:00:02
The following content is
provided under a creative
3
00:00:02 --> 00:00:03
commons license.
4
00:00:03 --> 00:00:06
Your support will help MIT
OpenCourseWare continue to
5
00:00:06 --> 00:00:10
offer high-quality educational
resources for free.
6
00:00:10 --> 00:00:13
To make a donation or view
additional materials from
7
00:00:13 --> 00:00:16
hundreds of MIT courses,
visit MIT OpenCourseWare
8
00:00:16 --> 00:00:21
at ocw.mit.edu.
9
00:00:21 --> 00:00:25
PROFESSOR: Last time we were
talking about binary search and
10
00:00:25 --> 00:00:29
I sort of left a promise to
you which I need to pick up.
11
00:00:29 --> 00:00:34
I want to remind you, we were
talking about search, which is
12
00:00:34 --> 00:00:36
a very fundamental thing that
we do in a whole lot
13
00:00:36 --> 00:00:37
of applications.
14
00:00:37 --> 00:00:39
We want to go find things
in some data set.
15
00:00:39 --> 00:00:41
And I'll remind you that
we sort of a separated
16
00:00:41 --> 00:00:42
out two cases.
17
00:00:42 --> 00:00:48
We said if we had an
ordered list, we could
18
00:00:48 --> 00:00:50
use binary search.
19
00:00:50 --> 00:00:54
And we said that was log
rhythmic, took log n time where
20
00:00:54 --> 00:00:56
n is the size of the list.
21
00:00:56 --> 00:01:01
If it was an unordered list,
we were basically stuck
22
00:01:01 --> 00:01:03
with linear search.
23
00:01:03 --> 00:01:05
Got to walk through the
whole list to see if
24
00:01:05 --> 00:01:05
the thing is there.
25
00:01:05 --> 00:01:08
So that was of order in.
26
00:01:08 --> 00:01:11
And then one of the things that
I suggested was that if we
27
00:01:11 --> 00:01:15
could figure out some way to
order it, and in particular, if
28
00:01:15 --> 00:01:23
we could order it in n log n
time, and we still haven't done
29
00:01:23 --> 00:01:26
that, but if we could do that,
then we said the complexity
30
00:01:26 --> 00:01:27
changed a little bit.
31
00:01:27 --> 00:01:30
But it changed in a way
that I want to remind you.
32
00:01:30 --> 00:01:33
And the change was, that in
this case, if I'm doing a
33
00:01:33 --> 00:01:36
single search, I've
got a choice.
34
00:01:36 --> 00:01:41
I could still do the linear
case, which is order n or I
35
00:01:41 --> 00:01:44
could say, look, take the
list, let's sort it
36
00:01:44 --> 00:01:45
and then search it.
37
00:01:45 --> 00:01:50
But in that case, we said well
to sort it was going to take n
38
00:01:50 --> 00:01:52
log n time, assuming
I can do that.
39
00:01:52 --> 00:01:59
Once I have it sorted I can
search it in log n time, but
40
00:01:59 --> 00:02:02
that's still isn't as
good as just doing n.
41
00:02:02 --> 00:02:09
And this led to this idea of
amortization, which is I need
42
00:02:09 --> 00:02:12
to not only factor in the cost,
but how am I going to use it?
43
00:02:12 --> 00:02:14
And typically, I'm not going to
just search once in a list, I'm
44
00:02:14 --> 00:02:16
going to search multiple times.
45
00:02:16 --> 00:02:21
So if I have to do k searches,
then in the linear case, I got
46
00:02:21 --> 00:02:23
to do order n things k times.
47
00:02:23 --> 00:02:25
It's order k n.
48
00:02:25 --> 00:02:30
Whereas in the ordered case, I
need to get them sorted, which
49
00:02:30 --> 00:02:34
is still n log n, but then
the search is only log n.
50
00:02:34 --> 00:02:38
I need to do k of those.
51
00:02:38 --> 00:02:44
And we suggested well this
is better than that.
52
00:02:44 --> 00:02:49
This is certainly better than
that. m plus k all times log
53
00:02:49 --> 00:02:51
n is in general going to be
much better than k times n.
54
00:02:51 --> 00:02:54
It depends on n and k but
obviously as n gets big, that
55
00:02:54 --> 00:02:56
one is going to be better.
56
00:02:56 --> 00:03:00
And that's just a way of
reminding you that we want to
57
00:03:00 --> 00:03:02
think carefully, but what are
the things we're trying
58
00:03:02 --> 00:03:04
to measure when we talk
about complexity here?
59
00:03:04 --> 00:03:06
It's both the size of the
thing and how often are
60
00:03:06 --> 00:03:07
we going to use it?
61
00:03:07 --> 00:03:10
And there are some trade offs,
but I still haven't said how
62
00:03:10 --> 00:03:13
I'm going to get an n log n
sorting algorithm, and that's
63
00:03:13 --> 00:03:14
what I want to do today.
64
00:03:14 --> 00:03:17
One of the two things
I want to do today.
65
00:03:17 --> 00:03:19
To set the stage for this,
let's go back just for a
66
00:03:19 --> 00:03:22
second to binary search.
67
00:03:22 --> 00:03:26
At the end of the lecture I
said binary search was an
68
00:03:26 --> 00:03:39
example of a divide and
conquer algorithm.
69
00:03:39 --> 00:03:42
Sort of an Attila the Hun
kind of approach to doing
70
00:03:42 --> 00:03:43
things if you like.
71
00:03:43 --> 00:03:46
So let me say -- boy, I could
have made a really bad
72
00:03:46 --> 00:03:48
political joke there, which
I will forego, right.
73
00:03:48 --> 00:03:50
Let's say what this actually
means, divide and conquer.
74
00:03:50 --> 00:03:53
Divide and conquer says
basically do the following:
75
00:03:53 --> 00:04:14
split the problem into several
sub-problems of the same type.
76
00:04:14 --> 00:04:15
I'll come back in a second
to help binary searches
77
00:04:15 --> 00:04:17
matches in that, but that's
what we're going to do.
78
00:04:17 --> 00:04:22
For each of those sub-problems
we're going to solve them
79
00:04:22 --> 00:04:27
independently, and then we're
going to combine
80
00:04:27 --> 00:04:34
those solutions.
81
00:04:34 --> 00:04:36
And it's called divide and
conquer for the obvious reason.
82
00:04:36 --> 00:04:39
I'm going to divide it up into
sub-problems with the hope that
83
00:04:39 --> 00:04:41
those sub-problems get easier.
84
00:04:41 --> 00:04:43
It's going to be easier to
conquer if you like, and then
85
00:04:43 --> 00:04:45
I'm going to merge them back.
86
00:04:45 --> 00:04:47
Now, in the binary search
case, in some sense, this
87
00:04:47 --> 00:04:51
is a little bit trivial.
88
00:04:51 --> 00:04:52
What was the divide?
89
00:04:52 --> 00:04:56
The divide was breaking a big
search up into half a search.
90
00:04:56 --> 00:04:59
We actually threw half of the
list away and we kept dividing
91
00:04:59 --> 00:05:01
it down, until ultimately
we got something of
92
00:05:01 --> 00:05:03
size one to search.
93
00:05:03 --> 00:05:04
That's really easy.
94
00:05:04 --> 00:05:07
The combination was also sort
of trivial in this case because
95
00:05:07 --> 00:05:11
the solution to the sub-problem
was, in fact, the solution
96
00:05:11 --> 00:05:13
to the larger problem.
97
00:05:13 --> 00:05:15
But there's the idea of
divide and conquer.
98
00:05:15 --> 00:05:18
I'm going to use exactly that
same ideas to tackle sort.
99
00:05:18 --> 00:05:20
Again, I've got an unordered
list of n elements.
100
00:05:20 --> 00:05:24
I want to sort it into a
obviously a sorted list.
101
00:05:24 --> 00:05:27
And that particular algorithm
is actually a really nice
102
00:05:27 --> 00:05:29
algorithm called merge sort.
103
00:05:29 --> 00:05:37
And it's actually a
fairly old algorithm.
104
00:05:37 --> 00:05:41
It was invented in 1945 by
John von Neumann one of the
105
00:05:41 --> 00:05:46
pioneers of computer science.
106
00:05:46 --> 00:05:50
And here's the idea behind
merge sort, actually I'm going
107
00:05:50 --> 00:05:53
to back into it in a funny way.
108
00:05:53 --> 00:05:56
Let's assume that I could
somehow get to the stage where
109
00:05:56 --> 00:05:59
I've got two sorted lists.
110
00:05:59 --> 00:06:02
How much work do I have
to do to actually
111
00:06:02 --> 00:06:04
merge them together?
112
00:06:04 --> 00:06:05
So let me give you an example.
113
00:06:05 --> 00:06:13
Suppose I want to merge two
lists, and they're sorted.
114
00:06:13 --> 00:06:22
Just to give you an example,
here's one list, 3121724
115
00:06:22 --> 00:06:25
Here's another list, 12430.
116
00:06:25 --> 00:06:29
I haven't said how I'm going to
get those sorted lists, but
117
00:06:29 --> 00:06:31
imagine I had two sorted
lists like that.
118
00:06:31 --> 00:06:34
How hard is it to merge them?
119
00:06:34 --> 00:06:35
Well it's pretty easy, right?
120
00:06:35 --> 00:06:38
I start at the beginning of
each list, and I say is
121
00:06:38 --> 00:06:39
one less than three?
122
00:06:39 --> 00:06:41
Sure.
123
00:06:41 --> 00:06:45
So that says one should be the
first element in my merge list.
124
00:06:45 --> 00:06:49
Now, compare the first element
in each of these lists.
125
00:06:49 --> 00:06:53
Two is less than three, so
two ought to be the next
126
00:06:53 --> 00:06:54
element of the list.
127
00:06:54 --> 00:06:55
And you get the idea.
128
00:06:55 --> 00:06:56
What am I going to do next?
129
00:06:56 --> 00:07:00
I'm going to compare
three against four.
130
00:07:00 --> 00:07:04
Three is the smallest one, and
I'm going to compare four games
131
00:07:04 --> 00:07:07
twelve, which is going
to give me four.
132
00:07:07 --> 00:07:07
And then what do?
133
00:07:07 --> 00:07:10
I have to do twelve
against thirty, twelve is
134
00:07:10 --> 00:07:15
smaller, take that out.
135
00:07:15 --> 00:07:25
Seventeen against thirty,
twenty-four against thirty And
136
00:07:25 --> 00:07:27
by this stage I've got nothing
left in this element, so I just
137
00:07:27 --> 00:07:31
add the rest of that list in.
138
00:07:31 --> 00:07:33
Wow I can sort two lists,
so I can merge two lists.
139
00:07:33 --> 00:07:35
I said it poorly.
140
00:07:35 --> 00:07:37
What's the point?
141
00:07:37 --> 00:07:40
How many operations did
it take me to do this?
142
00:07:40 --> 00:07:41
Seven comparisons, right?
143
00:07:41 --> 00:07:41
I've got eight elements.
144
00:07:41 --> 00:07:48
It took me seven comparisons,
because I can take advantage of
145
00:07:48 --> 00:07:50
the fact I know I only ever
have to look at the first
146
00:07:50 --> 00:07:53
element of each sub-list.
147
00:07:53 --> 00:07:54
Those are the only things I
need to compare, and when I run
148
00:07:54 --> 00:07:58
out of one list, I just add
the rest of the list in.
149
00:07:58 --> 00:08:02
What's the order of
complexity of merging?
150
00:08:02 --> 00:08:03
I heard it somewhere
very quietly.
151
00:08:03 --> 00:08:05
STUDENT: n.
152
00:08:05 --> 00:08:06
PROFESSOR: Sorry,
and thank you.
153
00:08:06 --> 00:08:08
Linear, absolutely right?
154
00:08:08 --> 00:08:10
And what's n by the way here?
155
00:08:10 --> 00:08:11
What's it measuring?
156
00:08:11 --> 00:08:14
STUDENT: [UNINTELLIGIBLE]
157
00:08:14 --> 00:08:15
PROFESSOR: In both
lists, right.
158
00:08:15 --> 00:08:21
So this is linear, order n and
n is this sum of the element,
159
00:08:21 --> 00:08:30
or sorry, the number of
elements in each list.
160
00:08:30 --> 00:08:32
I said I was going to
back my way into this.
161
00:08:32 --> 00:08:37
That gives me a way
to merge things.
162
00:08:37 --> 00:08:40
So here's what merge
sort would do.
163
00:08:40 --> 00:08:47
Merge sort takes this idea of
divide and conquer, and it does
164
00:08:47 --> 00:08:58
the following: it says let's
divide the list in half.
165
00:08:58 --> 00:09:00
There's the divide and conquer.
166
00:09:00 --> 00:09:05
And let's keep dividing each of
those lists in half until we
167
00:09:05 --> 00:09:07
get down to something that's
really easy to sort.
168
00:09:07 --> 00:09:08
What's the simplest
thing to sort?
169
00:09:08 --> 00:09:12
A list of size one, right?
170
00:09:12 --> 00:09:26
So continue until we
have singleton lists.
171
00:09:26 --> 00:09:29
Once I got a list of size
one they're sorted,
172
00:09:29 --> 00:09:31
and then combine them.
173
00:09:31 --> 00:09:36
Combine them by doing
emerge the sub-lists.
174
00:09:36 --> 00:09:42
And again, you see that flavor.
175
00:09:42 --> 00:09:45
I'm going to just keep dividing
it up until I get something
176
00:09:45 --> 00:09:47
really easy, and then
I'm going to combine.
177
00:09:47 --> 00:09:49
And this is different than
binary search now, the
178
00:09:49 --> 00:09:50
combine is going to
have to do some work.
179
00:09:50 --> 00:09:55
So, I'm giving you a piece of
code that does this, and I'm
180
00:09:55 --> 00:09:57
going to come back to it in the
second, but it's up there.
181
00:09:57 --> 00:10:00
But what I'd like to do is to
try you sort sort of a little
182
00:10:00 --> 00:10:02
simulation of how
this would work.
183
00:10:02 --> 00:10:03
And I was going to originally
make the TAs come up here and
184
00:10:03 --> 00:10:06
do it, but I don't have enough
t a's to do a full merge sort.
185
00:10:06 --> 00:10:10
So I'm hoping, so I also have
these really high-tech props.
186
00:10:10 --> 00:10:12
I spent tons and tons
of department money on
187
00:10:12 --> 00:10:13
them as you can see.
188
00:10:13 --> 00:10:15
I hope you can see this because
I'm going to try and simulate
189
00:10:15 --> 00:10:16
what a merge sort does.
190
00:10:16 --> 00:10:19
I've got eight things I want to
sort here, and those initially
191
00:10:19 --> 00:10:21
start out here at top level.
192
00:10:21 --> 00:10:23
The first step is
divide them in half.
193
00:10:23 --> 00:10:27
All right?
194
00:10:27 --> 00:10:30
I'm not sure how to mark
it here, remember I need
195
00:10:30 --> 00:10:31
to come back there.
196
00:10:31 --> 00:10:33
I'm not yet done.
197
00:10:33 --> 00:10:33
What do I do?
198
00:10:33 --> 00:10:41
Divide them in half again.
199
00:10:41 --> 00:10:42
You know, if I had like
shells and peas here I
200
00:10:42 --> 00:10:44
could make some more money.
201
00:10:44 --> 00:10:44
What do I do?
202
00:10:44 --> 00:10:50
I divide them in
half one more time.
203
00:10:50 --> 00:10:53
Let me cluster them because
really what I have,
204
00:10:53 --> 00:10:56
sorry, separate them out.
205
00:10:56 --> 00:10:59
I've gone from one problem
size eight down to eight
206
00:10:59 --> 00:11:01
problems of size one.
207
00:11:01 --> 00:11:03
At this stage I'm at
my singleton case.
208
00:11:03 --> 00:11:04
So this is easy.
209
00:11:04 --> 00:11:04
What do I do?
210
00:11:04 --> 00:11:05
I merge.
211
00:11:05 --> 00:11:17
And the merge is,
put them in order.
212
00:11:17 --> 00:11:18
What do I do next?
213
00:11:18 --> 00:11:19
Obvious thing, I merge these.
214
00:11:19 --> 00:11:22
And that as we saw was a
nice linear operation.
215
00:11:22 --> 00:11:31
It's fun to do it upside down,
and then one more merge which
216
00:11:31 --> 00:11:34
is I take the smallest
elements of each one until
217
00:11:34 --> 00:11:40
I get to where I want.
218
00:11:40 --> 00:11:42
Wow aren't you impressed.
219
00:11:42 --> 00:11:45
No, don't please don't
clap, not for that one.
220
00:11:45 --> 00:11:48
Now let me do it a second
time to show you that --
221
00:11:48 --> 00:11:49
I'm saying this poorly.
222
00:11:49 --> 00:11:50
Let me say it again.
223
00:11:50 --> 00:11:52
That's the general idea.
224
00:11:52 --> 00:11:53
What should you
see out of that?
225
00:11:53 --> 00:11:56
I just kept sub-dividing down
until I got really easy
226
00:11:56 --> 00:11:59
problems, and then I
combine them back.
227
00:11:59 --> 00:12:02
I actually misled you slightly
there or maybe a lot, because
228
00:12:02 --> 00:12:03
I did it in parallel.
229
00:12:03 --> 00:12:06
In fact, let me just shuffle
these up a little bit.
230
00:12:06 --> 00:12:08
Really what's going to happen
here, because this is a
231
00:12:08 --> 00:12:11
sequential computer, is that
we're going to start off up
232
00:12:11 --> 00:12:19
here, at top level, we're going
to divide into half, then we're
233
00:12:19 --> 00:12:21
going to do the complete
subdivision and merge
234
00:12:21 --> 00:12:24
here before we ever come
back and do this one.
235
00:12:24 --> 00:12:30
We're going to do a division
here and then a division there.
236
00:12:30 --> 00:12:32
At that stage we can merge
these, and then take this
237
00:12:32 --> 00:12:35
down, do the division merge
and bring them back up.
238
00:12:35 --> 00:12:41
Let me show you an
example by running that.
239
00:12:41 --> 00:12:44
I've got a little list I've
made here called test.
240
00:12:44 --> 00:12:53
Let's run merge sort on it, and
then we'll look at the code.
241
00:12:53 --> 00:12:57
OK, what I would like you to
see is I've been printing out,
242
00:12:57 --> 00:12:59
as I went along, actually let's
back up slightly and
243
00:12:59 --> 00:13:00
look at the code.
244
00:13:00 --> 00:13:03
There's merge sort.
245
00:13:03 --> 00:13:04
Takes in a list.
246
00:13:04 --> 00:13:04
What does it say to do?
247
00:13:04 --> 00:13:07
It says check to see if
I'm in that base case.
248
00:13:07 --> 00:13:10
It's the list of
length less than two.
249
00:13:10 --> 00:13:11
Is it one basically?
250
00:13:11 --> 00:13:16
In which case, just
return a copy the list.
251
00:13:16 --> 00:13:17
That's the simple case.
252
00:13:17 --> 00:13:18
Otherwise, notice
what it says to do.
253
00:13:18 --> 00:13:24
It's says find the mid-point
and split the list in half.
254
00:13:24 --> 00:13:27
Copy of the back end, sorry,
copy of the left side,
255
00:13:27 --> 00:13:28
copy of the right side.
256
00:13:28 --> 00:13:30
Run merge sort on those.
257
00:13:30 --> 00:13:32
By induction, if it does the
right thing, I'm going to get
258
00:13:32 --> 00:13:37
back two lists, and I'm going
to then merge them together.
259
00:13:37 --> 00:13:37
Notice what I'm going to do.
260
00:13:37 --> 00:13:40
I'm going to print here the
list if we go into it, and
261
00:13:40 --> 00:13:44
print of the when we're done
and then just return that.
262
00:13:44 --> 00:13:45
Merge up here.
263
00:13:45 --> 00:13:46
There's a little
more code there.
264
00:13:46 --> 00:13:48
I'll let you just grok it but
you can see it's basically
265
00:13:48 --> 00:13:51
doing what I did over there.
266
00:13:51 --> 00:13:54
Setting up two indices for the
two sub-list, it's just walking
267
00:13:54 --> 00:13:56
down, finding the smallest
element, putting it
268
00:13:56 --> 00:13:57
into a new list.
269
00:13:57 --> 00:14:01
When it gets to the end of one
of the lists, it skips to the
270
00:14:01 --> 00:14:03
next part, and only one of
these two pieces will get
271
00:14:03 --> 00:14:05
called because only one of
them is going to have
272
00:14:05 --> 00:14:06
things leftovers.
273
00:14:06 --> 00:14:08
It's going to add the
other pieces in.
274
00:14:08 --> 00:14:10
OK, if you look at that
then, let's look at what
275
00:14:10 --> 00:14:11
happened when we ran this.
276
00:14:11 --> 00:14:16
We started off with a
call with that list.
277
00:14:16 --> 00:14:19
Ah ha, split it in half.
278
00:14:19 --> 00:14:21
It's going down the
left side of this.
279
00:14:21 --> 00:14:23
That got split in half, and
that got split in half until
280
00:14:23 --> 00:14:27
I got to a list of one.
281
00:14:27 --> 00:14:28
Here's the first
list of size one.
282
00:14:28 --> 00:14:30
There's the second
list of size one.
283
00:14:30 --> 00:14:31
So I merged them.
284
00:14:31 --> 00:14:33
It's now in the right
order, and that's coming
285
00:14:33 --> 00:14:35
from right there.
286
00:14:35 --> 00:14:37
Having done that, it goes
back up and picks the
287
00:14:37 --> 00:14:42
second sub-list, which
came from there.
288
00:14:42 --> 00:14:44
It's a down to base
case, merges it.
289
00:14:44 --> 00:14:48
When these two merges are done,
we're basically at a stage in
290
00:14:48 --> 00:14:51
that branch where we can now
merge those two together, which
291
00:14:51 --> 00:14:56
gives us that, and it goes
through the rest of it.
292
00:14:56 --> 00:15:00
A really nice algorithm.
293
00:15:00 --> 00:15:03
As I said, an example
of divide and conquer.
294
00:15:03 --> 00:15:06
Notice here that it's different
than the binary search case.
295
00:15:06 --> 00:15:10
We're certainly dividing down,
but the combination now
296
00:15:10 --> 00:15:12
actually takes some work.
297
00:15:12 --> 00:15:13
I'll have to actually
figure out how to put
298
00:15:13 --> 00:15:15
them back together.
299
00:15:15 --> 00:15:17
And that's a general thing you
want to keep in mind when
300
00:15:17 --> 00:15:19
you're thinking about designing
a divide and conquer
301
00:15:19 --> 00:15:21
kind of algorithm.
302
00:15:21 --> 00:15:23
You really want to get the
power of dividing things up,
303
00:15:23 --> 00:15:26
but if you end up doing a ton
of work at the combination
304
00:15:26 --> 00:15:28
stage, you may not
have gained anything.
305
00:15:28 --> 00:15:31
So you really want to think
about that trade off.
306
00:15:31 --> 00:15:38
All right, having said that,
what's the complexity here?
307
00:15:38 --> 00:15:40
Boy, there's a dumb question,
because I've been telling you
308
00:15:40 --> 00:15:42
for the last two lectures the
complexity is n log n, but
309
00:15:42 --> 00:15:43
let's see if it really is.
310
00:15:43 --> 00:15:46
What's the complexity here?
311
00:15:46 --> 00:16:01
If we think about it, we start
off with the problem of size n.
312
00:16:01 --> 00:16:02
What do we do?
313
00:16:02 --> 00:16:05
We split it into two
problems of size n over 2.
314
00:16:05 --> 00:16:08
Those get split each into two
problems of size n over 4, and
315
00:16:08 --> 00:16:15
we keep doing that until we get
down to a level in this
316
00:16:15 --> 00:16:20
tree where we have only
singletons left over.
317
00:16:20 --> 00:16:23
Once we're there, we
have to do the merge.
318
00:16:23 --> 00:16:24
Notice what happens here.
319
00:16:24 --> 00:16:30
We said each of the merge
operations was of order n.
320
00:16:30 --> 00:16:31
But n is different.
321
00:16:31 --> 00:16:31
Right?
322
00:16:31 --> 00:16:33
Down here, I've just got two
things to merge, and then
323
00:16:33 --> 00:16:35
I've got things of size two
to merge and then things
324
00:16:35 --> 00:16:37
of size four to merge.
325
00:16:37 --> 00:16:38
But notice a trade off.
326
00:16:38 --> 00:16:43
I have n operations if you
like down there of size one.
327
00:16:43 --> 00:16:47
Up here I have n over two
operations of size two.
328
00:16:47 --> 00:16:50
Up here I've got n over four
operations of size four.
329
00:16:50 --> 00:16:54
So I always have to do
a merge of n elements.
330
00:16:54 --> 00:16:57
How much time does that take?
331
00:16:57 --> 00:17:01
Well, we said it, right?
332
00:17:01 --> 00:17:02
Where did I put it?
333
00:17:02 --> 00:17:04
Right there, order n.
334
00:17:04 --> 00:17:16
So I have order n operations
at each level in the tree.
335
00:17:16 --> 00:17:17
And then how many
levels deep am I?
336
00:17:17 --> 00:17:20
Well, that's the divide, right?
337
00:17:20 --> 00:17:26
So how many levels do I have?
338
00:17:26 --> 00:17:29
Log n, because at each
stage I'm cutting
339
00:17:29 --> 00:17:30
the problem in half.
340
00:17:30 --> 00:17:31
So I start off with n then
it's n over two n over
341
00:17:31 --> 00:17:33
four n over eight.
342
00:17:33 --> 00:17:40
So I have n operations log n
times, there we go, n log n.
343
00:17:40 --> 00:17:42
Took us a long time to get
there, but it's a nice
344
00:17:42 --> 00:17:45
algorithm to have.
345
00:17:45 --> 00:17:51
Let me generalize
this slightly.
346
00:17:51 --> 00:17:56
When we get a problem, a
standard tool to try and attack
347
00:17:56 --> 00:17:59
it with is to say, is there
some way to break this problem
348
00:17:59 --> 00:18:02
down into simpler, I shouldn't
say simpler, smaller versions
349
00:18:02 --> 00:18:05
of the same problem.
350
00:18:05 --> 00:18:07
If I can do that, it's
a good candidate for
351
00:18:07 --> 00:18:08
divide and conquer.
352
00:18:08 --> 00:18:10
And then the things I have
to ask is how much of a
353
00:18:10 --> 00:18:12
division do I want to do?
354
00:18:12 --> 00:18:15
The obvious one is to divide it
in half, but there may be cases
355
00:18:15 --> 00:18:16
where there are different
divisions you want
356
00:18:16 --> 00:18:18
to have take place.
357
00:18:18 --> 00:18:21
The second question I want to
ask is what's the base case?
358
00:18:21 --> 00:18:24
When do I get down to a problem
that's small enough that it's
359
00:18:24 --> 00:18:26
basically trivial to solve?
360
00:18:26 --> 00:18:28
Here it was lists of size one.
361
00:18:28 --> 00:18:30
I could have stopped at
lists of size two right.
362
00:18:30 --> 00:18:31
That's an easy comparison.
363
00:18:31 --> 00:18:34
Do one comparison and return
one of two possible orders on
364
00:18:34 --> 00:18:36
it, but I need to decide that.
365
00:18:36 --> 00:18:39
And the third thing I need to
decide is how do I combine?
366
00:18:39 --> 00:18:42
You know, point out to you
in the binary search case,
367
00:18:42 --> 00:18:44
combination was trivial.
368
00:18:44 --> 00:18:45
The answer to the final
search was just the
369
00:18:45 --> 00:18:47
answer all the way up.
370
00:18:47 --> 00:18:49
Here, a little more work,
and that's why I'll
371
00:18:49 --> 00:18:50
come back to that idea.
372
00:18:50 --> 00:18:54
If I'm basically just squeezing
jello, that is, I'm trying to
373
00:18:54 --> 00:18:56
make the problem simpler, but
the combination turns out to be
374
00:18:56 --> 00:18:59
really complex, I've
not gained anything.
375
00:18:59 --> 00:19:02
So things that are good
candidates for divide and
376
00:19:02 --> 00:19:05
conquer are problems where it's
easy to figure out how to
377
00:19:05 --> 00:19:09
divide down, and the
combination is of
378
00:19:09 --> 00:19:09
little complexity.
379
00:19:09 --> 00:19:12
It would be nice if it was less
than linear, but linear is nice
380
00:19:12 --> 00:19:15
because then I'm going to get
that n log in kind of behavior.
381
00:19:15 --> 00:19:18
And if you ask the TAs in
recitation tomorrow, they'll
382
00:19:18 --> 00:19:20
tell you that you see a lot
of n log n algorithms
383
00:19:20 --> 00:19:21
in computer science.
384
00:19:21 --> 00:19:23
It's a very common class
of algorithms, and it's
385
00:19:23 --> 00:19:28
very useful one to have.
386
00:19:28 --> 00:19:32
Now, one of the questions we
could still ask is, right,
387
00:19:32 --> 00:19:36
we've got binary search, which
has got this nice log behavior.
388
00:19:36 --> 00:19:40
If we can sort things, you
know, we get this n log n
389
00:19:40 --> 00:19:43
behavior, and we got a n
log n behavior overall.
390
00:19:43 --> 00:19:47
But can we actually do better
in terms of searching.
391
00:19:47 --> 00:19:49
I'm going to show you
one last technique.
392
00:19:49 --> 00:19:53
And in fact, we're going to put
quotes around the word better,
393
00:19:53 --> 00:19:58
but it does better than even
this kind of binary search, and
394
00:19:58 --> 00:20:04
that's a method called hashing.
395
00:20:04 --> 00:20:08
You've actually seen hashing,
you just don't know it.
396
00:20:08 --> 00:20:10
Hashing is the the technique
that's used in Python to
397
00:20:10 --> 00:20:12
represent dictionaries.
398
00:20:12 --> 00:20:15
Hashing is used when you
actually come in to Logan
399
00:20:15 --> 00:20:18
Airport and Immigration or
Homeland Security checks your
400
00:20:18 --> 00:20:20
picture against a database.
401
00:20:20 --> 00:20:24
Hashing is used every time you
enter a password into a system.
402
00:20:24 --> 00:20:26
So what in the
world is hashing?
403
00:20:26 --> 00:20:29
Well, let me start with a
simple little example.
404
00:20:29 --> 00:20:35
Suppose I want to represent
a collection of integers.
405
00:20:35 --> 00:20:38
This is an easy little example.
406
00:20:38 --> 00:20:41
And I promise you that the
integers are never going to be
407
00:20:41 --> 00:20:45
anything other than between
the range of zero to nine.
408
00:20:45 --> 00:20:47
OK, so it might be the
collection of one and five.
409
00:20:47 --> 00:20:48
It might be two,
three, four, eight.
410
00:20:48 --> 00:20:50
I mean some collection of
integers, but I guarantee you
411
00:20:50 --> 00:20:52
it's between zero and nine.
412
00:20:52 --> 00:20:55
Here's the trick I can play.
413
00:20:55 --> 00:21:11
I can build -- I can't count --
I could build a list with spots
414
00:21:11 --> 00:21:14
for all of those elements,
zero, one, two, three, four,
415
00:21:14 --> 00:21:16
five, six, seven, eight, nine.
416
00:21:16 --> 00:21:19
And then when I want to create
my set, I could simply put a
417
00:21:19 --> 00:21:26
one everywhere that
that integer falls.
418
00:21:26 --> 00:21:28
So if I wanted to represent,
for example, this is the set
419
00:21:28 --> 00:21:35
two, six and eight, I put
a one in those slots.
420
00:21:35 --> 00:21:37
This seems a little weird, but
bear with me for second, in
421
00:21:37 --> 00:21:40
fact, I've given you a little
piece a code to do it,
422
00:21:40 --> 00:21:45
which is the next piece of
code on the hand out.
423
00:21:45 --> 00:21:48
So let's take a look
at it for second.
424
00:21:48 --> 00:21:53
This little set of code here
from create insert and number.
425
00:21:53 --> 00:21:54
What's create do?
426
00:21:54 --> 00:21:55
It says, given a low and a
high range, in this case
427
00:21:55 --> 00:21:57
it would be zero to nine.
428
00:21:57 --> 00:22:00
I'm going to build a list.
429
00:22:00 --> 00:22:02
Right, you can see that little
loop going through there.
430
00:22:02 --> 00:22:03
What am I doing?
431
00:22:03 --> 00:22:07
I'm creating a list with just
that special symbol none in it.
432
00:22:07 --> 00:22:08
So I'm building the list.
433
00:22:08 --> 00:22:10
I'm returning that as my set.
434
00:22:10 --> 00:22:11
And then to create the
object, I'll simply
435
00:22:11 --> 00:22:13
do a set of inserts.
436
00:22:13 --> 00:22:15
If I want the values two, six
and eight in there, I would do
437
00:22:15 --> 00:22:18
an insert of two into that set,
an insert of six into that
438
00:22:18 --> 00:22:20
set, and an insert of
eight into the set.
439
00:22:20 --> 00:22:21
And what does it do?
440
00:22:21 --> 00:22:25
It marks a one in
each of those spots.
441
00:22:25 --> 00:22:26
Now, what did I want to do?
442
00:22:26 --> 00:22:27
I wanted to check membership.
443
00:22:27 --> 00:22:28
I want to do search.
444
00:22:28 --> 00:22:30
Well that's simple.
445
00:22:30 --> 00:22:32
Given that representation
and some value, I just
446
00:22:32 --> 00:22:36
say gee is it there?
447
00:22:36 --> 00:22:43
What's the order
complexity here?
448
00:22:43 --> 00:22:45
I know I drive you nuts
asking questions?
449
00:22:45 --> 00:22:48
What's the order
complexity here?
450
00:22:48 --> 00:22:57
Quadratic, linear,
log, constant?
451
00:22:57 --> 00:22:58
Any takers?
452
00:22:58 --> 00:23:00
I know I have the wrong glasses
on the see hands up too, but...
453
00:23:00 --> 00:23:04
STUDENT: [UNINTELLIGIBLE]
454
00:23:04 --> 00:23:05
PROFESSOR: Who said it?
455
00:23:05 --> 00:23:06
STUDENT: Constant.
456
00:23:06 --> 00:23:09
PROFESSOR: Constant, why?
457
00:23:09 --> 00:23:09
STUDENT: [UNINTELLIGIBLE]
458
00:23:09 --> 00:23:10
PROFESSOR: Yes, thank you.
459
00:23:10 --> 00:23:11
All right, it is constant.
460
00:23:11 --> 00:23:14
You keep sitting back there
where I can't get to you.
461
00:23:14 --> 00:23:15
Thank you very much.
462
00:23:15 --> 00:23:17
It has a constant.
463
00:23:17 --> 00:23:21
Remember we said we design
lists so that the access, no
464
00:23:21 --> 00:23:24
matter where it was on the
list was of constant time.
465
00:23:24 --> 00:23:27
That is another way of
saying that looking up this
466
00:23:27 --> 00:23:29
thing here is constant.
467
00:23:29 --> 00:23:35
So this is constant
time, order one.
468
00:23:35 --> 00:23:37
Come on, you know, representing
sets of integers,
469
00:23:37 --> 00:23:38
this is pretty dumb.
470
00:23:38 --> 00:23:41
Suppose I want to have
a set of characters.
471
00:23:41 --> 00:23:42
How could I do that?
472
00:23:42 --> 00:23:44
Well the idea of a hash, in
fact, what's called a hash
473
00:23:44 --> 00:23:48
function is to have some way
of mapping any kind of
474
00:23:48 --> 00:23:50
data into integers.
475
00:23:50 --> 00:23:55
So let's look at the second
example, all right, -- I keep
476
00:23:55 --> 00:24:01
doing that -- this piece of
code from here to here gives
477
00:24:01 --> 00:24:06
me a way of now creating
a hash table of size 256.
478
00:24:06 --> 00:24:09
Ord as a built in
python representation.
479
00:24:09 --> 00:24:12
There is lots of them around
that takes any character and
480
00:24:12 --> 00:24:13
gives you back an integer.
481
00:24:13 --> 00:24:17
In fact, just to show that to
you, if I go down here and I
482
00:24:17 --> 00:24:32
type ord, sorry, I
did that wrong.
483
00:24:32 --> 00:24:33
Let me try again.
484
00:24:33 --> 00:24:38
We'll get to exceptions
in a second.
485
00:24:38 --> 00:24:39
I give it some character.
486
00:24:39 --> 00:24:42
It gives me back an
integer representing.
487
00:24:42 --> 00:24:43
It looks weird.
488
00:24:43 --> 00:24:44
Why is three come back
to some other thing?
489
00:24:44 --> 00:24:46
That's the internal
representation that
490
00:24:46 --> 00:24:47
python uses for this.
491
00:24:47 --> 00:24:51
If I give it some other
character, yeah, it would help
492
00:24:51 --> 00:24:54
if I could type, give it
some other character.
493
00:24:54 --> 00:24:57
It gives me back a
representation.
494
00:24:57 --> 00:24:59
So now here's the idea.
495
00:24:59 --> 00:25:04
I build a list 256 elements
long, and I fill it up with
496
00:25:04 --> 00:25:06
those special characters none.
497
00:25:06 --> 00:25:09
That's what create is
going to do right here.
498
00:25:09 --> 00:25:14
And then hash character takes
in any string or character,
499
00:25:14 --> 00:25:16
single character, gives
me back a number.
500
00:25:16 --> 00:25:17
Notice what I do.
501
00:25:17 --> 00:25:21
If I want to create a set or a
sequence representing these
502
00:25:21 --> 00:25:24
things, I simply insert
into that list.
503
00:25:24 --> 00:25:26
It goes through and puts
ones in the right place.
504
00:25:26 --> 00:25:29
And then, if I want to find
out if something's there,
505
00:25:29 --> 00:25:30
I do the same thing.
506
00:25:30 --> 00:25:33
But notice now, hash is
converting the input
507
00:25:33 --> 00:25:38
into an integer.
508
00:25:38 --> 00:25:40
So, what's the idea?
509
00:25:40 --> 00:25:44
If I know what my hash function
does, it maps, in this case
510
00:25:44 --> 00:25:49
characters into a range zero to
256, which is zero to 255,
511
00:25:49 --> 00:25:52
I create a list that long,
and I simply mark things.
512
00:25:52 --> 00:25:55
And my look up is
still constant.
513
00:25:55 --> 00:25:56
Characters are simple.
514
00:25:56 --> 00:25:59
Suppose you want to represent
sets of strings, well you
515
00:25:59 --> 00:26:01
basically just generalize
the hash function.
516
00:26:01 --> 00:26:03
I think one of the classic ones
for strings is called the
517
00:26:03 --> 00:26:06
Rabin-Karp algorithm.
518
00:26:06 --> 00:26:09
And it's simply the same idea
that you have a mapping from
519
00:26:09 --> 00:26:13
your import into a
set of integers.
520
00:26:13 --> 00:26:17
Wow, OK, maybe not so wow,
but this is now constant.
521
00:26:17 --> 00:26:19
This is constant time access.
522
00:26:19 --> 00:26:24
So I can do searching in
constant time which is great.
523
00:26:24 --> 00:26:26
Where's the penalty?
524
00:26:26 --> 00:26:28
What did I trade off here?
525
00:26:28 --> 00:26:31
Well I'm going to suggest
that what I did was I really
526
00:26:31 --> 00:26:40
traded space for time.
527
00:26:40 --> 00:26:44
It makes me sound like an astro
physicist somehow right?
528
00:26:44 --> 00:26:45
What do I mean by that?
529
00:26:45 --> 00:26:49
I have constant time access
which is great, but I paid a
530
00:26:49 --> 00:26:52
price, which is I had
to use up some space.
531
00:26:52 --> 00:26:55
In the case of
integers it was easy.
532
00:26:55 --> 00:26:57
In the case of characters,
so I have to give up a
533
00:26:57 --> 00:26:59
list of 256, no big deal.
534
00:26:59 --> 00:27:01
Imagine now you
want to do faces.
535
00:27:01 --> 00:27:03
You've got a picture of
somebody's face, it's
536
00:27:03 --> 00:27:04
a million pixels.
537
00:27:04 --> 00:27:07
Each pixel has a range of
values from zero to 256.
538
00:27:07 --> 00:27:11
I want to hash a face with some
function into an integer.
539
00:27:11 --> 00:27:13
I may not want to do the full
range of this, but I may
540
00:27:13 --> 00:27:16
decide I have to use a lot
of gigabytes of space in
541
00:27:16 --> 00:27:18
order to do a trade off.
542
00:27:18 --> 00:27:20
The reason I'm showing you this
is it that this is a gain, a
543
00:27:20 --> 00:27:22
common trade off in
computer science.
544
00:27:22 --> 00:27:25
That in many cases, I can gain
efficiency if I'm willing
545
00:27:25 --> 00:27:28
to give up space.
546
00:27:28 --> 00:27:30
Having said that though, there
may still be a problem, or
547
00:27:30 --> 00:27:32
there ought to be a problem
that may be bugging you
548
00:27:32 --> 00:27:37
slightly, which is how do I
guarantee that my hash function
549
00:27:37 --> 00:27:43
takes any input into exactly
one spot in the storage space?
550
00:27:43 --> 00:27:45
The answer is I can't.
551
00:27:45 --> 00:27:48
OK, in the simple case of
integers I can, but in the case
552
00:27:48 --> 00:27:51
of something more complex like
faces or fingerprints or
553
00:27:51 --> 00:27:54
passwords for that matter, it's
hard to design a hash function
554
00:27:54 --> 00:27:57
that has completely even
distribution, meaning that
555
00:27:57 --> 00:28:00
it takes any input into
exactly one output spot.
556
00:28:00 --> 00:28:04
So what you typically do and a
hash case is you design your
557
00:28:04 --> 00:28:05
code to deal with that.
558
00:28:05 --> 00:28:07
You try to design -- actually
I'm going to come back
559
00:28:07 --> 00:28:07
to that in a second.
560
00:28:07 --> 00:28:09
It's like you're trying to use
a hash function that spread
561
00:28:09 --> 00:28:11
things out pretty evenly.
562
00:28:11 --> 00:28:13
But the places you store into
in those lists may have to
563
00:28:13 --> 00:28:16
themselves have a small list in
there, and when you go to check
564
00:28:16 --> 00:28:19
something, you may have to do a
linear search through the
565
00:28:19 --> 00:28:20
elements in that list.
566
00:28:20 --> 00:28:23
The good news is the elements
in any one spot in a hash table
567
00:28:23 --> 00:28:26
are likely to be a small
number, three, four, five.
568
00:28:26 --> 00:28:27
So the search is really easy.
569
00:28:27 --> 00:28:29
You're not searching
a million things.
570
00:28:29 --> 00:28:31
You're searching three or four
things, but nonetheless, you
571
00:28:31 --> 00:28:34
have to do that trade off.
572
00:28:34 --> 00:28:43
The last thing I want to say
about hashes are that they're
573
00:28:43 --> 00:28:46
actually really hard to create.
574
00:28:46 --> 00:28:48
There's been a lot of work done
on these over the years, but
575
00:28:48 --> 00:28:51
in fact, it's pretty hard to
invent a good hash function.
576
00:28:51 --> 00:28:54
So my advice to you is, if you
want to use something was
577
00:28:54 --> 00:28:56
a hash, go to a library.
578
00:28:56 --> 00:28:57
Look up a good hash function.
579
00:28:57 --> 00:29:00
For strings, there's a
classic set of them
580
00:29:00 --> 00:29:01
that work pretty well.
581
00:29:01 --> 00:29:02
For integers, there are
some real simple ones.
582
00:29:02 --> 00:29:05
If there's something more
complex, find a good hash
583
00:29:05 --> 00:29:07
function, but designing a
really good hash function takes
584
00:29:07 --> 00:29:10
a lot of effort because you
want it to have that
585
00:29:10 --> 00:29:11
even distribution.
586
00:29:11 --> 00:29:16
You'd like it to have as few
duplicates if you like in each
587
00:29:16 --> 00:29:22
spot in the hash table for each
one of the things that you use.
588
00:29:22 --> 00:29:25
Let me pull back
for a second then.
589
00:29:25 --> 00:29:28
What have we done over the
last three or four lectures?
590
00:29:28 --> 00:29:31
We've started introducing you
to classes of algorithms.
591
00:29:31 --> 00:29:35
Things that I'd like you to be
able to see are how to do some
592
00:29:35 --> 00:29:37
simple complexity analysis.
593
00:29:37 --> 00:29:40
Perhaps more importantly,
how to recognize a kind of
594
00:29:40 --> 00:29:42
algorithm based on its
properties and know what
595
00:29:42 --> 00:29:43
class it belongs to.
596
00:29:43 --> 00:29:44
This is a hint.
597
00:29:44 --> 00:29:47
If you like, leaning towards
the next quiz, that you oughta
598
00:29:47 --> 00:29:50
be able to say that looks like
a logarithmic algorithm because
599
00:29:50 --> 00:29:51
it's got a particular property.
600
00:29:51 --> 00:29:53
That looks like an n log n
algorithm because it has
601
00:29:53 --> 00:29:54
a particular property.
602
00:29:54 --> 00:29:57
And the third thing we've done
is we've given you now a
603
00:29:57 --> 00:30:01
set of sort of standard
algorithms if you like.
604
00:30:01 --> 00:30:05
Root force, just walk through
every possible case.
605
00:30:05 --> 00:30:08
It works well if the
problem sizes are small.
606
00:30:08 --> 00:30:11
We've had, there are a number
of variants of guess and check
607
00:30:11 --> 00:30:14
or hypothesize and test, where
you try to guess the solution
608
00:30:14 --> 00:30:17
and then check it and use
that to refine your search.
609
00:30:17 --> 00:30:20
Successive approximation,
Newton-Raphson was one nice
610
00:30:20 --> 00:30:23
example, but there's a whole
class of things that get closer
611
00:30:23 --> 00:30:26
and closer, reducing your
errors as you go along.
612
00:30:26 --> 00:30:30
Divide and conquer and actually
I guess in between there
613
00:30:30 --> 00:30:33
bi-section, which is really
just a very difficult of
614
00:30:33 --> 00:30:36
successive approximation, but
divide and conquer is
615
00:30:36 --> 00:30:37
a class of algorithm.
616
00:30:37 --> 00:30:39
These are tools that you
want in your tool box.
617
00:30:39 --> 00:30:41
These are the kinds of
algorithms that you should
618
00:30:41 --> 00:30:42
be able to recognize.
619
00:30:42 --> 00:30:45
And what I'd like you to begin
to do is to look at a problem
620
00:30:45 --> 00:30:49
and say, gee, which kind of
algorithm is most likely to be
621
00:30:49 --> 00:30:54
successful on this problem,
and map it into that case.
622
00:30:54 --> 00:30:56
OK, starting next -- don't
worry I'm not going to quit
623
00:30:56 --> 00:30:58
36 minutes after -- I got
one more topic for today.
624
00:30:58 --> 00:31:02
But jumping ahead, I'm going to
skip in a second now to talk
625
00:31:02 --> 00:31:05
about one last linguistic thing
from Python, but I want to
626
00:31:05 --> 00:31:07
preface Professor Guttag is
going to pick up next week, and
627
00:31:07 --> 00:31:10
what we're going to start doing
then is taking these classes of
628
00:31:10 --> 00:31:13
algorithms and start looking at
much more complex algorithms.
629
00:31:13 --> 00:31:16
Things you're more likely
to use in problems.
630
00:31:16 --> 00:31:19
Things like knapsack
problems as we move ahead.
631
00:31:19 --> 00:31:22
But the tools you've seen so
far are really the things
632
00:31:22 --> 00:31:24
that were going to see as
we build those algorithms.
633
00:31:24 --> 00:31:27
OK, I want to spend the last
portion of this lecture
634
00:31:27 --> 00:31:29
doing one last piece
of linguistics stuff.
635
00:31:29 --> 00:31:32
One last little thing from
Python, and that's to
636
00:31:32 --> 00:31:44
talk about exceptions.
637
00:31:44 --> 00:31:49
OK, you've actually seen
exceptions a lot, you just
638
00:31:49 --> 00:31:51
didn't know that's what they
were, because exceptions show
639
00:31:51 --> 00:31:52
up everywhere in Python.
640
00:31:52 --> 00:31:54
Let me give you a
couple of examples.
641
00:31:54 --> 00:31:58
I'm going to clear
some space here.
642
00:31:58 --> 00:32:01
Before I type in that
expression, I get
643
00:32:01 --> 00:32:02
an error, right?
644
00:32:02 --> 00:32:03
So it's not defined.
645
00:32:03 --> 00:32:06
But in fact, what this did
was it threw an exception.
646
00:32:06 --> 00:32:10
An exception is called a
name error exception.
647
00:32:10 --> 00:32:13
It says you gave me something
I didn't know how to deal.
648
00:32:13 --> 00:32:16
I'm going to throw it, or raise
it, to use the right term to
649
00:32:16 --> 00:32:18
somebody in case they can
handle it, but it's a
650
00:32:18 --> 00:32:21
particular kind of exception.
651
00:32:21 --> 00:32:24
I might do something like,
remind you I have test.
652
00:32:24 --> 00:32:31
If I do this, try and get
the 10th element of a list
653
00:32:31 --> 00:32:32
that's only eight long.
654
00:32:32 --> 00:32:34
I get what looks like an
error, but it's actually
655
00:32:34 --> 00:32:35
throwing an exception.
656
00:32:35 --> 00:32:37
The exception is right there.
657
00:32:37 --> 00:32:41
It's an index error, that is
it's trying to do something
658
00:32:41 --> 00:32:45
going beyond the range of what
this thing could deal with.
659
00:32:45 --> 00:32:47
OK, you say, come on, I've
seen these all the time.
660
00:32:47 --> 00:32:49
Every time I type something
into my program, it does one
661
00:32:49 --> 00:32:50
of these things, right?
662
00:32:50 --> 00:32:54
When we're just interacting
with idol, with the interactive
663
00:32:54 --> 00:32:57
editor or sorry, interactive
environment if you like,
664
00:32:57 --> 00:32:58
that's what you expect.
665
00:32:58 --> 00:33:00
What's happening is that we're
typing in something, an
666
00:33:00 --> 00:33:02
expression it doesn't
know how to deal.
667
00:33:02 --> 00:33:04
It's raising the exception, but
is this simply bubbling up
668
00:33:04 --> 00:33:07
at the top level saying
you've got a problem.
669
00:33:07 --> 00:33:12
Suppose instead you're in the
middle of some deep piece of
670
00:33:12 --> 00:33:15
code and you get one
of these cases.
671
00:33:15 --> 00:33:19
It's kind of annoying if it
throws it all the way back up
672
00:33:19 --> 00:33:21
to top level for you to fix.
673
00:33:21 --> 00:33:23
If it's truly a bug, that's
the right thing to do.
674
00:33:23 --> 00:33:25
You want to catch it.
675
00:33:25 --> 00:33:27
But in many cases exceptions
or things that, in fact,
676
00:33:27 --> 00:33:30
you as a program designer
could have handled.
677
00:33:30 --> 00:33:33
So I'm going to distinguish
in fact between un-handled
678
00:33:33 --> 00:33:39
exceptions, which are the
things that we saw there,
679
00:33:39 --> 00:33:44
and handled exceptions.
680
00:33:44 --> 00:33:45
I'm going to show you in a
second how to handle them, but
681
00:33:45 --> 00:33:46
let's look at an example.
682
00:33:46 --> 00:33:50
What do I mean by a
handled exception?
683
00:33:50 --> 00:33:54
Well let's look at the
next piece of code.
684
00:33:54 --> 00:33:55
OK, it's right here.
685
00:33:55 --> 00:33:56
It's called read float.
686
00:33:56 --> 00:33:58
We'll look at it in a second.
687
00:33:58 --> 00:34:00
Let me sort of set the stage up
for this -- suppose I want to
688
00:34:00 --> 00:34:03
input -- I'm sorry I want
you as a user to input a
689
00:34:03 --> 00:34:05
floating point number.
690
00:34:05 --> 00:34:08
We talked about things
you could do to try
691
00:34:08 --> 00:34:09
make sure that happens.
692
00:34:09 --> 00:34:11
You could run through a little
loop to say keep trying
693
00:34:11 --> 00:34:12
until you get one.
694
00:34:12 --> 00:34:14
But one of the ways I
could deal with it is
695
00:34:14 --> 00:34:15
what's shown here.
696
00:34:15 --> 00:34:17
And what's this little
loop say to do?
697
00:34:17 --> 00:34:21
This little loop says I'm
going to write a function
698
00:34:21 --> 00:34:23
or procedures that
takes in two messages.
699
00:34:23 --> 00:34:25
I'm going to run through a
loop, and I'm going to request
700
00:34:25 --> 00:34:28
some input, which I'm going
to read in with raw input.
701
00:34:28 --> 00:34:30
I'm going to store
that into val.
702
00:34:30 --> 00:34:33
And as you might expect, I'm
going to then try and see if I
703
00:34:33 --> 00:34:36
can convert that into a float.
704
00:34:36 --> 00:34:37
Oh wait a minute, that's a
little different than what
705
00:34:37 --> 00:34:38
we did last time, right?
706
00:34:38 --> 00:34:41
Last time we checked the
type and said if it is
707
00:34:41 --> 00:34:41
a float you're okay.
708
00:34:41 --> 00:34:42
If not, carry on.
709
00:34:42 --> 00:34:43
In this case what would happen?
710
00:34:43 --> 00:34:46
Well float is going to try
and do the cohersion.
711
00:34:46 --> 00:34:50
It's going to try and turn it
into a floating point number.
712
00:34:50 --> 00:34:52
If it does, I'm great, right.
713
00:34:52 --> 00:34:55
And I like just to return val.
714
00:34:55 --> 00:34:59
If it doesn't, floats going to
throw or raise, to use the
715
00:34:59 --> 00:35:00
right term, an exception.
716
00:35:00 --> 00:35:03
It's going to say something
like a type error.
717
00:35:03 --> 00:35:04
In fact, let's try
it over here.
718
00:35:04 --> 00:35:09
I if I go over here, and I say
float of three, it's going
719
00:35:09 --> 00:35:10
to do the conversion.
720
00:35:10 --> 00:35:15
But if I say turn this into
a float, ah it throws a
721
00:35:15 --> 00:35:16
value error exception.
722
00:35:16 --> 00:35:19
It says it's a wrong kind
of value that I've got.
723
00:35:19 --> 00:35:23
So I'm going to write a little
piece of code that says if it
724
00:35:23 --> 00:35:26
gives me a float, I'm set, But
if not, I'd like to have the
725
00:35:26 --> 00:35:29
code handle the exception.
726
00:35:29 --> 00:35:33
And that's what this funky
tri-accept thing does.
727
00:35:33 --> 00:35:43
This is a tri-accept block and
here's the flow of control
728
00:35:43 --> 00:35:45
that takes place in there.
729
00:35:45 --> 00:35:47
When I hit a tri-block.
730
00:35:47 --> 00:35:48
It's going to
literally do that.
731
00:35:48 --> 00:35:51
It's going to try and
execute the instructions.
732
00:35:51 --> 00:35:54
If it can successfully execute
the instructions, it's going to
733
00:35:54 --> 00:35:58
skip past the except block and
just carry on with the
734
00:35:58 --> 00:35:59
rest of the code.
735
00:35:59 --> 00:36:03
If, however, it raises an
exception, that exception, at
736
00:36:03 --> 00:36:07
least in this case where it's a
pure accept with no tags on it,
737
00:36:07 --> 00:36:11
is going to get, be like thrown
directly to the except block,
738
00:36:11 --> 00:36:13
and it's going to try
and execute that.
739
00:36:13 --> 00:36:14
So notice what's going
to happen here, then.
740
00:36:14 --> 00:36:17
If I give it something that can
be turned into a float, I come
741
00:36:17 --> 00:36:20
in here, I read the input, if
it can be turned into a float,
742
00:36:20 --> 00:36:22
I'm going to just return
the value and I'm set.
743
00:36:22 --> 00:36:25
If not, it's basically going to
throw it to this point, in
744
00:36:25 --> 00:36:28
which case I'm going to print
out an error message and oh
745
00:36:28 --> 00:36:30
yeah, I'm still in that while
loop, so it's going
746
00:36:30 --> 00:36:31
to go around.
747
00:36:31 --> 00:36:35
So in fact, if I go here
and, let me un-comment
748
00:36:35 --> 00:36:39
this and run the code.
749
00:36:39 --> 00:36:41
It says enter a float.
750
00:36:41 --> 00:36:48
And if I give it something that
can be -- sorry, I've got, yes,
751
00:36:48 --> 00:36:51
never mind the grades crap.
752
00:36:51 --> 00:36:54
Where did I have that?
753
00:36:54 --> 00:36:58
Let me comment that out.
754
00:36:58 --> 00:37:00
Somehow it's appropriate in the
middle of my lecture for it to
755
00:37:00 --> 00:37:04
say whoops at me but that
wasn't what I intended.
756
00:37:04 --> 00:37:09
And we will try this again.
757
00:37:09 --> 00:37:10
OK, says it says enter a float.
758
00:37:10 --> 00:37:11
I give it something that
can be converted into
759
00:37:11 --> 00:37:13
a float, it says fine.
760
00:37:13 --> 00:37:15
I'm going to go back and
run it again though.
761
00:37:15 --> 00:37:21
If I run it again, it
says enter a float.
762
00:37:21 --> 00:37:24
Ah ha, it goes into that accept
portion, prints out a message,
763
00:37:24 --> 00:37:27
and goes back around the
while loop to say try again.
764
00:37:27 --> 00:37:30
And it's going to keep doing
this until I give it something
765
00:37:30 --> 00:37:33
that does serve as a float.
766
00:37:33 --> 00:37:37
Right, so an exception then
has this format that I can
767
00:37:37 --> 00:37:39
control as a programmer.
768
00:37:39 --> 00:37:40
Why would I want to use this?
769
00:37:40 --> 00:37:44
Well some things I can actually
expect may happen and
770
00:37:44 --> 00:37:45
I want to handle them.
771
00:37:45 --> 00:37:46
The float example
is a simple one.
772
00:37:46 --> 00:37:47
I'm going to generalize
in a second.
773
00:37:47 --> 00:37:48
Here's a better example.
774
00:37:48 --> 00:37:52
I'm writing a piece of code
that wants to input a file.
775
00:37:52 --> 00:37:54
I can certainly imagine
something that says give me
776
00:37:54 --> 00:37:56
the file name, I'm going
to do something with it.
777
00:37:56 --> 00:37:59
I can't guarantee that the file
may exist under that name, but
778
00:37:59 --> 00:38:01
I know that's something
that might occur.
779
00:38:01 --> 00:38:04
So a nice way to handle it is
to write it as an exception
780
00:38:04 --> 00:38:06
that says, here's what I want
to do if I get the file.
781
00:38:06 --> 00:38:10
But just in case the file name
is not there, here's what I
782
00:38:10 --> 00:38:11
want to do in that
case to handle it.
783
00:38:11 --> 00:38:15
Let me specify what the
exception should do.
784
00:38:15 --> 00:38:17
In the example I just wrote
here, this is pretty
785
00:38:17 --> 00:38:17
trivial, right.
786
00:38:17 --> 00:38:19
OK, I'm trying to input floats.
787
00:38:19 --> 00:38:21
I could generalize
this pretty nicely.
788
00:38:21 --> 00:38:24
Imagine the same kind of idea
where I want to simply say I
789
00:38:24 --> 00:38:27
want to take input of anything
and try and see how to make
790
00:38:27 --> 00:38:28
sure I get the right
kind of thing.
791
00:38:28 --> 00:38:35
I want to make it polymorphic.
792
00:38:35 --> 00:38:38
Well that's pretty easy to do.
793
00:38:38 --> 00:38:43
That is basically the next
example, right here.
794
00:38:43 --> 00:38:49
In fact, let me
comment this one out.
795
00:38:49 --> 00:38:50
I can do exactly the
same kind of thing.
796
00:38:50 --> 00:38:55
Now what I'm going to try and
do is read in a set of values,
797
00:38:55 --> 00:38:59
but I'm going to give a type of
value as well as the messages.
798
00:38:59 --> 00:39:00
The format is the same.
799
00:39:00 --> 00:39:02
I'm going to ask for some
input, and then I am going to
800
00:39:02 --> 00:39:07
use that procedure to check, is
this the right type of value.
801
00:39:07 --> 00:39:10
And I'm trying to use that to
do the coercion if you like.
802
00:39:10 --> 00:39:12
Same thing if it works, I'm
going to skip that, if
803
00:39:12 --> 00:39:13
it not, it's going to
throw the exception.
804
00:39:13 --> 00:39:16
Why is this much nice?
805
00:39:16 --> 00:39:18
Well, that's a handy
piece of code.
806
00:39:18 --> 00:39:20
Because imagine I've got that
now, and I can now store that
807
00:39:20 --> 00:39:25
away in some file name, input
dot p y, and import into every
808
00:39:25 --> 00:39:28
one of my procedure functions,
pardon me, my files of
809
00:39:28 --> 00:39:30
procedures, because it's
a standard way of now
810
00:39:30 --> 00:39:33
giving me the input.
811
00:39:33 --> 00:39:35
OK, so far though, I've just
shown you what happens
812
00:39:35 --> 00:39:36
inside a peace a code.
813
00:39:36 --> 00:39:37
It raises an exception.
814
00:39:37 --> 00:39:39
It goes to that accept clause.
815
00:39:39 --> 00:39:42
We don't have to use it
just inside of one place.
816
00:39:42 --> 00:39:44
We can actually use
it more generally.
817
00:39:44 --> 00:39:47
And that gets me to the last
example I wanted to show you.
818
00:39:47 --> 00:39:55
Let me uncomment this.
819
00:39:55 --> 00:39:56
Let's take a look at this code.
820
00:39:56 --> 00:40:00
This looks like a handy piece
of code to have given what we
821
00:40:00 --> 00:40:02
just recently did to you.
822
00:40:02 --> 00:40:03
All right, get grades.
823
00:40:03 --> 00:40:06
It's a little function that's
going to say give me a file
824
00:40:06 --> 00:40:10
name, and I'm going to go off
and open that up and bind
825
00:40:10 --> 00:40:11
it to a local variable.
826
00:40:11 --> 00:40:14
And if it's successful, then
I'd just like to go off and do
827
00:40:14 --> 00:40:17
some things like turn it into a
list so I can compute average
828
00:40:17 --> 00:40:19
score or distributions
or something else.
829
00:40:19 --> 00:40:21
I don't really care
what's going on here.
830
00:40:21 --> 00:40:24
Notice though what I've done.
831
00:40:24 --> 00:40:28
Open, it doesn't succeed is
going to raise a particular
832
00:40:28 --> 00:40:32
kind of exception
called I O error.
833
00:40:32 --> 00:40:33
And so I've done a little bit
different things here which
834
00:40:33 --> 00:40:39
is I put the accept part of
the block with I O error.
835
00:40:39 --> 00:40:40
What does that say?
836
00:40:40 --> 00:40:43
It says if in the code up here
I get an exception of that
837
00:40:43 --> 00:40:46
sort, I'm going to go to
this place to handle it.
838
00:40:46 --> 00:40:49
On the other hand, if I'm
inside this procedure and some
839
00:40:49 --> 00:40:53
other exception is raised, it's
not tagged by that one, it's
840
00:40:53 --> 00:40:55
going to raise it up the chain.
841
00:40:55 --> 00:40:57
If that procedure was called by
some other procedure it's going
842
00:40:57 --> 00:41:00
to say is there an exception
block in there that
843
00:41:00 --> 00:41:01
can handle that.
844
00:41:01 --> 00:41:02
If not, I am going to
keep going up the chain
845
00:41:02 --> 00:41:05
until eventually I
get to the top level.
846
00:41:05 --> 00:41:06
And you can see that down here.
847
00:41:06 --> 00:41:07
I'm going to run
this in a second.
848
00:41:07 --> 00:41:09
This is just a piece of code
where I'm going to say, gee,
849
00:41:09 --> 00:41:15
if I can get the grades, do
something, if not carry on.
850
00:41:15 --> 00:41:19
And if I go ahead and run
this -- now it's going
851
00:41:19 --> 00:41:19
to say woops, at me.
852
00:41:19 --> 00:41:25
What happened?
853
00:41:25 --> 00:41:28
I'm down here and try, I'm
trying do get grades, which is
854
00:41:28 --> 00:41:34
a call to that function, which
is not bound in my computer.
855
00:41:34 --> 00:41:34
That says it's in here.
856
00:41:34 --> 00:41:36
It's in this tri-block.
857
00:41:36 --> 00:41:41
It raised an exception, but
it wasn't and I O error.
858
00:41:41 --> 00:41:45
So it passes it back, past this
exception, up to this level,
859
00:41:45 --> 00:41:48
which gets to that exception.
860
00:41:48 --> 00:41:51
Let me say this a little
bit better then.
861
00:41:51 --> 00:41:54
I can write exceptions
inside a piece of code.
862
00:41:54 --> 00:41:57
Try this, if it doesn't work I
can have an exception that
863
00:41:57 --> 00:41:59
catches any error
at that level.
864
00:41:59 --> 00:42:02
Or I can say catch only these
kinds of errors at that
865
00:42:02 --> 00:42:05
level, otherwise pass
them up the chain.
866
00:42:05 --> 00:42:07
And that exception will keep
getting passed up the chain of
867
00:42:07 --> 00:42:10
calls until it either gets to
the top level, in which case it
868
00:42:10 --> 00:42:11
looks like what you
see all the time.
869
00:42:11 --> 00:42:13
It looks like an error, but it
tells you what the error came
870
00:42:13 --> 00:42:18
from, or it gets an exception
, it can deal with it.
871
00:42:18 --> 00:42:21
OK, so the last thing to say
about this is what's the
872
00:42:21 --> 00:42:37
difference between an
exception and an assert?
873
00:42:37 --> 00:42:39
We introduced
asserts earlier on.
874
00:42:39 --> 00:42:44
You've actually seen them in
some pieces of code, so what's
875
00:42:44 --> 00:42:47
the difference between
the two of them?
876
00:42:47 --> 00:42:49
Well here's my way
of describing it.
877
00:42:49 --> 00:42:53
The goal of an assert, or an
assert statement, is basically
878
00:42:53 --> 00:42:57
to say, look, you can make sure
that my function is going to
879
00:42:57 --> 00:43:01
give this kind of result if you
give me inputs of a
880
00:43:01 --> 00:43:02
particular type.
881
00:43:02 --> 00:43:03
Sorry, wrong way of saying it.
882
00:43:03 --> 00:43:05
If you give me inputs that
satisfy some particular
883
00:43:05 --> 00:43:07
constraints.
884
00:43:07 --> 00:43:08
That was the kind
of thing you saw.
885
00:43:08 --> 00:43:11
Asserts said here are
some conditions to test.
886
00:43:11 --> 00:43:13
If they're true, I'm going to
let the rest of the code run.
887
00:43:13 --> 00:43:16
If not, I'm going
to throw an error.
888
00:43:16 --> 00:43:19
So the assertion is basically
saying we got some
889
00:43:19 --> 00:43:26
pre-conditions, those are the
clauses inside the assert that
890
00:43:26 --> 00:43:34
have to be true, and there's a
post condition. and in essence,
891
00:43:34 --> 00:43:37
what the assert is saying is,
or rather the programmer is
892
00:43:37 --> 00:43:39
saying using the assert is, if
you give me input that
893
00:43:39 --> 00:43:42
satisfies the preconditions,
I'm guaranteeing to you that my
894
00:43:42 --> 00:43:44
code is going to give you
something that meets
895
00:43:44 --> 00:43:45
the post condition.
896
00:43:45 --> 00:43:46
It's going to do
the right thing.
897
00:43:46 --> 00:43:49
And as a consequence, as you
saw with the asserts, if the
898
00:43:49 --> 00:43:52
preconditions aren't true,
it throws an error.
899
00:43:52 --> 00:43:55
It goes back up the top
level saying stop operation
900
00:43:55 --> 00:43:59
immediately and goes
back up the top level.
901
00:43:59 --> 00:44:01
Asserts in fact are nice in the
sense that they let you check
902
00:44:01 --> 00:44:03
conditions at debugging
time or testing time.
903
00:44:03 --> 00:44:07
So you can actually use them to
see where your code is going.
904
00:44:07 --> 00:44:10
An exception, when you use an
exception, basically what
905
00:44:10 --> 00:44:12
you're saying is, look, you can
do anything you want with my
906
00:44:12 --> 00:44:15
function, and you can be sure
that I'm going to tell you if
907
00:44:15 --> 00:44:16
something is going wrong.
908
00:44:16 --> 00:44:19
And in many cases I'm going
to handle it myself.
909
00:44:19 --> 00:44:23
So as much as possible, the
exceptions are going to try to
910
00:44:23 --> 00:44:27
handle unexpected things,
actually wrong term, you
911
00:44:27 --> 00:44:29
expected them, but not
what the user did.
912
00:44:29 --> 00:44:31
It's going to try to handle
conditions other than
913
00:44:31 --> 00:44:33
the normal ones itself.
914
00:44:33 --> 00:44:37
So you can use the
thing in anyway.
915
00:44:37 --> 00:44:39
If it can't, it's going to try
and throw it to somebody else
916
00:44:39 --> 00:44:42
to handle, and only if there
is no handler for that
917
00:44:42 --> 00:44:45
unexpected condition, will
it come up to top level.
918
00:44:45 --> 00:44:48
So, summarizing better, assert
is something you put in to say
919
00:44:48 --> 00:44:51
to the user, make sure you're
giving me input of this type,
920
00:44:51 --> 00:44:53
but I'm going to guarantee you
the rest of the code
921
00:44:53 --> 00:44:54
works correctly.
922
00:44:54 --> 00:44:57
Exceptions and exception
handlers are saying, here are
923
00:44:57 --> 00:45:00
the odd cases that I might see
and here's what I'd like to do
924
00:45:00 --> 00:45:04
in those cases in order to try
and be able to deal with them.
925
00:45:04 --> 00:45:10
Last thing to say is why would
you want to have exceptions?
926
00:45:10 --> 00:45:14
Well, let's go back to that
case of inputting a simple
927
00:45:14 --> 00:45:16
little floating point.
928
00:45:16 --> 00:45:18
If I'm expecting mostly numbers
in, I can certainly try
929
00:45:18 --> 00:45:19
and do the coercion.
930
00:45:19 --> 00:45:22
I could have done that
just doing the coercion.
931
00:45:22 --> 00:45:26
The problem is, I want to know
if, in fact, I've got something
932
00:45:26 --> 00:45:28
that's not of the
form I expect.
933
00:45:28 --> 00:45:30
I'm much better having an
exception get handled at the
934
00:45:30 --> 00:45:34
time of input than to let that
prop -- that value rather
935
00:45:34 --> 00:45:36
propagate through a whole bunch
of code until eventually it
936
00:45:36 --> 00:45:39
hits an error 17 calls later,
and you have no clue
937
00:45:39 --> 00:45:41
where it came from.
938
00:45:41 --> 00:45:44
So the exceptions are useful
when you want to have the
939
00:45:44 --> 00:45:48
ability to say, I expect in
general this kind of behavior,
940
00:45:48 --> 00:45:50
but I do know there are some
other things that might happen
941
00:45:50 --> 00:45:53
and here's what I'd like to do
in each one of those cases.
942
00:45:53 --> 00:45:56
But I do want to make sure that
I don't let a value that I'm
943
00:45:56 --> 00:45:58
not expecting pass through.
944
00:45:58 --> 00:46:01
That goes back to that idea of
sort of discipline coding.
945
00:46:01 --> 00:46:04
It's easy to have assumptions
about what you think are going
946
00:46:04 --> 00:46:06
to come into the program
when you writ it.
947
00:46:06 --> 00:46:09
If you really know what they
are use them as search, but if
948
00:46:09 --> 00:46:11
you think there's going to be
some flexibility, you want to
949
00:46:11 --> 00:46:14
prevent the user getting
trapped in a bad spot, and
950
00:46:14 --> 00:46:17
exceptions as a consequence
are a good thing to use.
951
00:46:17 --> 00:46:18