3 Sorting & related conceptsSorting: A process of arranging the data items in a data structure to support ascending or descending order of the items.Typically the data structure is an array.Why sorting?Key: When each data item is composed of multiple attributes, one of the attributes must be selected as the “key”.Sorting is based on comparing the key values of the data items.Comparability : The key values must be comparable.CSCI 3333 Data Structures

5 Bubble sortSource:A simple sorting algorithm of O(N2).Also called ‘sink sort’. Why?Exercise: Sort an array of the five items with bubble sort and count the number of comparisons.Question: How does the ‘sorted section’ grow with each pass?CSCI 3333 Data Structures

6 Selection sortSource:O(N2)Exercise: Sort an array of the five items with selection sort and count the number of comparisons.Question: How does the ‘sorted section’ grow with each pass? Where is the ‘sorted section’ located?CSCI 3333 Data Structures

7 Insertion sort O(N2) Only appropriate for small N.A good algorithm when most items are already sorted.More efficient in practice than most other simple quadratic algorithms such as selection sort or bubble sort; the best case (nearly sorted input) is O(n).CSCI 3333 Data Structures

10 Theorems wrt insertion sortTheorem 8.1: The average number of inversions in an array of N distinct numbers is N(N-1)/4.An inversion is a pair of elements that are out of order in an array.Formally, let A[1..n] be an array of n distinct numbers. If i < j and A[i] > A[j], then the pair (i,j) is called an inversion of A. (http://en.wikipedia.org/wiki/Inversion_%28discrete_mathematics%29)The number of inversions in an array measures its unsortedness.For data sets that are already substantially sorted, the time complexity of insertion sort is O(n + d), where d is the number of inversions. That is, the average cost is still O(N2).CSCI 3333 Data Structures

11 Theorems wrt insertion sortTheorem 8.2: Any algorithm that sorts by exchanging adjacent elements requires time on average.True for insertion, bubble, and selection sorts, all of which perform adjacent exchanges.CSCI 3333 Data Structures

12 Insertion vs Selection sortSource:While insertion sort typically makes fewer comparisons than selection sort, it requires more writes because the inner loop can require shifting large sections of the sorted portion of the array.In general, insertion sort will write to the array O(n2) times, whereas selection sort will write only O(n) times.Question: Do you agree with the above statement? Is there a way of verifying it?For this reason selection sort may be preferable in cases where writing to memory is significantly more expensive than reading, such as with EEPROM or flash memory.CSCI 3333 Data Structures

14 Shellsort Shellsort uses a sequence called the increment sequence.After a phase, using some increment hk, we have for every i where i + hk is a valid index; all elements spaced hk apart are sorted.The array is then said to be hk-sorted.Exercise 1: Shellsort the array below using the shellsort() method shown above.Exercise 2: Repeat the shellsort but use the sequence {1,3,5}. Compare their performance.Exercise 3: Would the sequence {1,3,7} be better?CSCI 3333 Data Structures

15 Shellsort Also called diminishing gap sortFor each gap, it performs a gap insertion sort.When gap becomes 1, it performs exactly the insertion sort.Question: The shell sort contains three loops. How can it be possible that it’s more efficient than the insertion sort, which contains only two loops?CSCI 3333 Data Structures

16 ShellsortThe running time of Shellsort depends heavily on the choice of increment sequences.Better sequences (than what Shell proposed) are known.Odd gaps only: When the gap is even, add 1 to it.Divide the gap by 2.2, instead of 2 as in the Shell’s incrementsCSCI 3333 Data Structures

17 Quicksort A divide-and-conquer algorithmAverage running time is O(N logN)Worst case: O(N2), but can be avoided by choosing the pivot rightThe basic idea:Given a set of items, choose one of them as the pivot, p.Partition the items into three groups: those that are larger than p (L), those that are smaller than p (R), and those that are the same as p (S).Continue the same process with L and R to sort them.When done, combine L, S, and R.CSCI 3333 Data Structures

19 Quicksort The basic algorithm Quicksort(S):If the number of elements in S is 0 or 1, then return. //base conditionPick an element v in S. It is called the pivot.Partition S – { v} ( the remaining elements in S) into two disjoint groups: L = {x in S-{v} <= v} and R = {x in S-{v} >= v}.Return the result of Quicksort(L) + v + Quicksort(R).CSCI 3333 Data Structures

20 Example implementationQuicksort (a[ ], low, high)If size(a) < CUTOFF then insertionSort (a,low,high);ElseSort the low, middle, high elementsChoose the middle as the pivotPlace the pivot at the high-1 positionPartitioning the range from low to pivot-1:Search from low toward the pivot until an item >= the pivot is found (let i = the index of that item)Search from the pivot down toward low until an item <= the pivot is found (let j = the index of that item)Place the pivot to the right position, i.Quicksort (a, low, i-1)Quicksort (a, i+1, high)CSCI 3333 Data Structures

24 Exercises Ex 8.1: Sort the sequence 8,1,4,1,5,9,2,6,5 usingInsertion sortShellsort for the increments {1,3,5}Quicksort, with the middle element as the pivot and no cutoff (show all steps)Quicksort, with median-of-three pivot selection and a cutoff of 3CSCI 3333 Data Structures