3 Importance of Algorithm Analysis 1/2“Performance” of an algorithm often refers to how quickly it executes and/or how much memory it needsperformance matters when the data grows in size!can observe and/or analyze, then tune or revise algorithmAlgorithm analysis is so important that every Brown CS student is required to take at least one course covering it

4 Importance of Algorithm Analysis 2/2Many important factors that make performance importantcomputing resourcesimplementation (like machine, language)size of data, denoted Nnumber of elements to be sortednumber of elements in ArrayList to iterate throughex: much faster to search through CS15 course-list than the Social Security DatabaseThis lecture: a brief introduction to Algorithm Analysis! Goal: learning how to maximize efficiency of an algorithm and conserve resources

5 RuntimeThe runtime of an algorithm varies with the input and typically grows with the input sizeIn most of computer science we focus on the worst case running time.easier to analyze and important for unforeseen inputsAverage case is what will happen most often. Best case requires the least amount of work and is the best situation you could have.average case is also important, best case is interesting but not insightfulHow to determine runtime?by inspecting pseudocode, can determine the number of statements executed by an algorithm as a function of input sizeallows us to evaluate approximate speed of an algorithm independent of the hardware or software environmentmemory use may be even more important for small and embedded devicesSupplied by CS16, taught at Brown University with Prof. David Laidlaw

6 Elementary OperationsAlgorithmic “time” is measured in elementary operationsmath (+, -, *, /, max, min, log, sin, cos, abs, ...)comparisons ( ==, >, <=, ...)function calls and value returns (not counting the body of the method)variable assignmentvariable increment or decrementarray allocationcreating a new object (careful, object's constructor may have elementary ops too!)For the purpose of algorithm analysis, we assume each of these operations takes the same time: “1 operation” – we are only interested in “asymptotic performance” for large data sets where small differences don’t matter – see S8Supplied by CS16, taught at Brown University with Prof. David Laidlaw

7 Example: Constant runtimepublic int addition(int x, int y) {return x+y; //call, add and return, 3 ops}always 3 operations performedHow many operations performed if this function were to add ten integers? Would it still be constant runtime?Supplied by CS16, taught at Brown University with Prof. David Laidlaw

8 Example: Linear runtimeHow many operations if the list has 1,000 elements?Worst case is an increasing listWorst case varies proportional to the size of the input list: 4n + 2We’ll be in the for loop longer as the input list growsThe runtime increases linearly//find max of a set of integerspublic int maxElement(int[] a) {int max = 0; //assignment, 1 opfor (i=0; i<a.length; i++){ //1 op per loopif (a[i]>max){ //2 ops per loopmax = a[i]; //1 op per loop, sometimes}return max; //1 opSupplied by CS16, taught at Brown University with Prof. David Laidlaw

9 Example: Quadratic Runtimepublic void printPossibleSums(int[ ] a, int arrayLength) {for (i=0; i<arrayLength; i++) { //1 op per loopfor (j=0; j<arrayLength; j++) { //1 op per loopSystem.out.println(a[i] + a[j]); // 4 ops per loop}Requires about 5n2 + n operations (okay to approximate!)Number of operations executed grows quadratically!If add one element to list: element must be multiplied with every other element in listNotice that the linear algorithm on the previous slide had only one for loop, while this quadratic one has two for loops, nestedSupplied by CS16, taught at Brown University with Prof. David Laidlaw

11 1+2+3+…+N =  i = N(N+1)/2, which is O(N2)Big-O ConstantsRemember, only largest N expression without constants mattersWe aren’t concerned about runtime with small numbers of data – we care about running operations on large amounts of inputs3N2 and 500N2 are both O(N2) – unrealistic if N is small, of courseN/2 is O(N)4N2 + 2N is O(N2)Useful sum that recurs frequently in analysis:1+2+3+…+N =  i = N(N+1)/2, which is O(N2)i =1N

12 Social Security Database Example 1/3Hundreds of millions of people in US, each have a number associated to themIf 100,000 people named John Smith, each has an individual SSNIf gov’t wants to look up information they have on John Smith, they use his SSN

13 Social Security Database Example 2/3Say it takes 10-4 seconds to perform a constant set of operations on one SSNrunning an algorithm on 5 social security numbers may take 5x10-4 seconds, and running an algorithm on 50 will only take 5x10-3 secondsboth are incredibly fast, a difference in runtime might not be noticeable by an interactive userthis changes with large amounts of data, i.e., the actual SS Data Base

14 Social Security Database Example 3/3Say it takes 10-4 seconds to perform a constant set of operations on one SSNto perform an algorithm with O(n) on 300 million people, it will take 8.3 hoursO(n2) takes 285,000 yearsWith large amounts of data, differences between O(n) and O(n2) are huge

20 SortingWe can use our analysis of runtimes to help choose the best algorithm to solve a problemA popular application is sorting, along with searching (future lecture) a huge consumer of computer time. They often go together…

21 Sorting – Social Security NumbersConsider example where run-time influences approachHow would you sort every SSN in the Social Security Database in increasing order?A number of ways to sort a listthese ways vary in the amount of time they take

22 Sorting – Bubble SortIterate through sequence, compare each element to right neighborExchange adjacent elements if necessaryEnd up with a sorted sub-array on the right. Each time we go through the list, need to switch one fewer itemImages courtesy of Brown cs16 with David Laidlaw

23 Bubble Sort - PseudocodeIterate through sequence, compare each element to right neighborExchange adjacent elements if necessaryEnd up with a sorted sub-array on the right. Each time we go through the list, need to switch one fewer itemN is number of objects in sequencePseudocode:

25 Sorting – Insertion SortLike inserting a new card into a partially sorted hand by bubbling to the left into sorted subarray on left; little less brute-force than bubble sortAdd one element a[i] at a timeFind proper position, j+1, to the left by shifting to the right a[i-1], a[i-2], ..., a[j+1] left neighbors, until a[j] < a[i]Move a[i] into vacated a[j+1]After iteration i<n, the original a[0] ... a[i] are in sorted order, but not necessarily in final position

31 Comparison of Elementary Sorting AlgorithmsThe differences in Best and Worst case performance result from the state (ordering) of the input before sortingSelection wins on data movementBut for small data, even the worst sort, Bubble is fine!

38 Consider it graphicallyEach part of the tree performs n operations to merge the two subproblems below itBecause we divide the sequence by two, the algorithm makes log2n merges.O(nlog2n) which is way better than O(N2)You will learn much more about how to find the runtime of these types of algorithms in cs16!Photo from

40 What’s next?Now that you've been introduced to how different approaches to solving problems can speed/slow up and dramatically affect speed of algorithm…Subsequent lectures will cover other data structures for collections beyond arrays and arraylists that you can use to significantly improve speed of your algorithm

42 Data StructuresYou have been introduced to techniques for storing a collection of elements using Arrays and ArrayLists, and for analyzing the run-time performance of an algorithmThere are other ways to store a collection of elements.Choose which data structure to use based on the way data is accessed and stored in your algorithmThe access and store operations of different data structures can have very different impacts on an algorithm’s overall efficiency

43 What is a Linked List?A Linked List is a collection of nodes that form a sequence of elementsThe Linked List holds a reference to the first nodeEach node holds an element and a reference to the next node in the list

45 Arrays ArrayLists Linked Listsindexed (allows explicit access to nth item)indices of other items don’t change if an item is replaced or deleted from a specific index, but will change if new item is inserted at an index before endcannot change size dynamicallyArrayListsindexed (allows explicit access to nth item)indices of other items are updated following an inserted or deleted itemcan grow/shrink dynamicallyJava uses an Array as the underlying data structureLinked Listsnot indexed. In order to access the nth element, must start at the beginning and go to the next node n times → no random accesscan grow/shrink dynamicallyuse nodes instead of arrayscan insert/remove in middle of list without data movement throughout rest of list

51 The Node Classpublic class Node<Type> {private Type _element;private Node<Type> _next;public Node(Type element) {_element=element;_next=null;}public void setNext(Node<Type> next){_next = next;public Node<Type> getNext() {return _next;/*similar mutator and accessor method for _element are elided.*/Node class holds its element and a reference to the next NodeIts methods are made up of mutators and accessors for these variablesThat’s it.

52 Announcements DoodleJump DQs due Sunday at 2PMHelp Session Sunday at 6PM in B&H 166DoodleJump early handin is Friday 10/31 at 10PM.On time handin is Sunday 11/2 at 11:59PMLate handin is Tuesday 11/4 at 11:59PM