2
Exam 1 9 May 2000, 1 week from today Covers material through HW#2 + through Amdahls law/Efficiency in this lecture Closed book, Closed Notes. You may use a calculator. At the end of this lecture –will ask you for a list of things you want reviewed during next lecture.

4
One measurement of granularity Computation to Communication Ratio –(Computation time)/(Communication time) –Increasing this ratio is often a key to good efficiency –How does this measure granularity? CCR = ? Grain

5
Communication Overhead Another important metric is communication overhead – time (measured in instructions) a zero- byte message consumes in a process –Measure time spent on communication that cannot be spent on computation Overlapped Messages – portion of message lifetime that can occur concurrently with computation – time bits are on wire – time bits are in the switch or NIC

6
Many little things add up … Lots of little things add up that add overhead to a parallel program –Efficient implementations demand Overlapping (aka hiding) the overheads as much as possible Keeping non-overlapping overheads as small as possible

7
Speed-Up S(n) = –(Execution time on Single CPU)/(Execution on N parallel processors) –t s /t p –Serial time is for best serial algorithm This may be a different algorithm than a parallel version –Divide-and-conquer Quicksort O(NlogN) vs. Mergesort

11
Example of Amdahls Law Suppose that a calculation has a 4% serial portion, what is the limit of speedup on 16 processors? –16/(1 + (16 – 1)*.04) = 10 –What is the maximum speedup? 1/0.04 = 25

12
More to think about … Amdahls law works on a fixed problem size –This is reasonable if your only goal is to solve a problem faster. –What if you also want to solve a larger problem? Gustafsons Law (Scaled Speedup)

14
More on Gustafsons Law Derived by fixing the parallel execution time (Amdahl fixed the problem size -> fixed serial execution time) –For many practical situations, Gustafsons law makes more sense Have a bigger computer, solve a bigger problem. Amdahls law turns out to be too conservative for high-performance computing.

16
Example questions Given a (scaled) speed up of 20 on 32 processors, what is the serial fraction from Amdahls law?, From Gustafsons Law? A program attains 89% efficiency with a serial fraction of 2%. Approximately how many processors are being used according to Amdahls law?

19
Ping-Pong All the overhead (including startup) are included in every message When message is very long, you get an accurate indication of bandwidth 1 2 Time

20
Send + Ack Overheads of messages are masked Has the effect of decoupling startup latency from bandwidth (concurrency) More accurate with a large # of trials 1 2 Time

21
In the limit … As messages get larger, both methods converge to the same number How does one measure latency? –Ping-pong over multiple trials –Latency = 2* /t What things arent being measured (or are being smeared by these two methods)? –Will talk about cost models and the start of LogP analysis next time.

22
How you measure performance It is important to understand exactly what you are measuring and how you are measuring it.

23
Exam … What topics/questions do you want reviewed? Are there specific questions on the homework/programming assignment that need talking about?