6. Running time measurement - running time cannot be measured by seconds, millseconds & so on - reason is o dependence of speed of a computer o dependence of quality of algorithm o complier used in generating the machine code o difficulty in clocking the actual running time.

7. Basic Operation - The operation contributing the most of the total time & count the number of times the basic operation is executed.

- Basic operation is usually the most time-consuming operation in the algorithms inner loop. - T(n) = Cop * Cn

8. Worst-Case efficiency of an algorithm - is the maximum number of steps that an algorithm can take for any collection of data values. - Is its efficiency for the worst case input of size n, which is an input of size n for which the algorithm runs the longest among all possible inputs of that size. - Example: Sequential Search. Searched value may at the at the end or may not be.

9. Best-Case efficiency of an algorithm - is the minimum number of steps that an algorithm can take any collection of data values - is its efficiency for the best-case input of size n, which is an input of size n, for which the algorithm runs the fastest among all possible inputs of that size. - Example: Sequential Search. Searched value may at the first position. o The algorithm makes the smallest number of comparisons. Cbest (n) = 1 O(1)

10. Average-Case efficiency of an algorithm - can not be obtained by taking the average of the worst-case and best-case efficiencies. o Sometimes this is may be correct. - Efficiency averaged on possible inputs. - Some assumptions about the possible inputs of size is made. - Example: Sequential Search. Searched value may at the middle position. o The algorithm makes the average number of comparisons.

11. Asymptotic Notations - The efficiency analysis framework concentrates on the order of growth of an algorithms basic operation count as the principal indicator of the algorithms efficiency. - To compare and rank such orders of growth , 3 notations are used. o O (big oh), (big omega), (big theta)

2. Identify the basic operation (located in innermost loop) 3. Check whether the number of times the basic operations is executed depended only on the input size of an input. If it also depends on some additional property, worst, average, best case efficiencies are investigated. 4. Set up a sum expressing the number of times the algorithms basic operations is executed. 5. Using standard formulas and rules of sum manipulation, either find closed-form formula for the count or at the very best establish its order of growth.

16. Steps for analyzing efficiency of recursive algorithms 1. decide on a parameter indicating an input size 2. identify the algorithms basic operation 3. check whether the number of times the basic operation executed can vary on different inputs of the same size. If it can , investigate worst, best, average case. 4. set up a recursive relation, with an appropriate initial condition for the number of times the basic operation is executed. 5. solve the recurrence or at least ascertain the order of growth of its solution.

17. Time Efficiency Time efficiency is measured by counting the number of time the algorithm basic operation is executed.

18. Space Efficiency Space efficiency is measured by counting the number of extra memory units consumed by the algorithm.

19. Need for case efficiencies The efficiencies of some algorithms may differ significantly for inputs of the same size. For such algorithms, case (best, worst, average) efficiencies is needed.

20. Need for algorithm analysis framework The frameworks primary interest lies in the order of growth of the algorithms running time as its input size goes to infinity.

21. Efficiency classes The efficiencies of a large number of algorithms fall into the following few classes; Constant, Logarithmic, Linear, n-log-n, Quadratic, Cubic, and Exponential.

22. Main tool for anlaysing the time efficiency of a non-recursive algorithm Setting up a sum expressing the number of executions of its basic operation and ascertain the sums order of growth.

23. Main tool for anlaysing the time efficiency of a non-recursive algorithm Setting up a recurrence expressing the number of executions of its basic operation and ascertain the solutions order of growth.

24. Brute Force Method Brute force is a straightforward approach to solving a problem, usually directly based on the problems statement and definitions of the concepts involved. Examples: Matrix multiplication, selection sort, sequential search.

26. Exhaustive Search Exhaustive search is a brute force approach to combinatorial problems. It suggests generating each and every combinatorial object of the problem, selecting those of them that satisfy the problems constraints and then finding a desired object. This search is impractical for large problems.

UNIT II DIVIDE AND CONQUER METHODOLOGY

27. What is Divide and conquer methodology? 1. A problem instance is divided into several smaller instances of the same problem, ideally of about the same size. 2. The smaller instances are solved (typically recursively though sometimes a different algorithm is employed when instances become small enough) 3. If necessary, the solutions obtained for the smaller instances are combined to get a solution to the original problem. Algorithm DANDC(P) { if SMALL(P) then return SOLUTION(P) else { divide P into smaller instances, P1,P2,..Pk k1 Apply DANDC to each of these subproblems Return COMBINE(DANDC(P1),DANDC(P2),.,DANDC(Pk)) } }

28. Computing time for finding the sum of n numbers

Time efficiency T(n) of many divide and conquer algorithms satisfies the equation T(n)=aT(n/b) + f(n).

29. Merge Sort A merge sort works as follows: 1. If the list is of length 0 or 1, then it is already sorted. Otherwise: 2. Divide the unsorted list into two sublists of about half the size. 3. Sort each sublist recursively by re-applying merge sort. 4. Merge the two sublists back into one sorted list. Data structure used Array Worst case performance (nlogn) Best case performance (nlogn) Average case performance (nlogn) Worst case space complexity (n)

31. Recurrence relation for the number of key comparisons in Merge sort

0 n=1 C(n) = 2C(n/2) +Cmerge(n) n>1

Cmerge(n) is the number of key comparisons performed during the merging stage. In worst case, neither of the two arrays becomes empty before other one contains just one element. Cmerge(n) = n-1

Worst case 0 n=1 Cworst(n) = 2Cworst(n/2) + n-1 n>1

if n=2k then Cworst(n) = nlog2n n+1 & T(n) = O(n log n)

Best case 0 n1 Cbest(n) = 2Cbest(n/2) + n/2 n>1

32. Disadvantages of Merge sort 1. Extra space (auxiliary array) n is needed during merge process 2. Stack space is increased by the use of recursion. Maximum depth of the stack is log n 3. Time is spent on recursion instead of sorting

33. Quick Sort

Quicksort sorts by employing a divide and conquer strategy to divide a list into two sub-lists. The steps are: 1. Pick an element, called a pivot, from the list. 2. Reorder the list so that all elements which are less than the pivot come before the pivot and so that all elements greater than the pivot come after it (equal values can go either way). After this partitioning, the pivot is in its final position. This is called the partition operation. 3. Recursively sort the sub-list of lesser elements and the sub-list of greater elements. Works by partitioning its inputs elements according to their value relative to some preselected element, called pivot element. The elements to the left of pivot should be less than the pivot element and the elements to the right of pivot should be greater than the pivot element. Worst case performance (n2) Best case performance (nlogn) Average case performance (nlogn) comparisons Worst case space complexity Varies by implementationOptimal Result Sometimes

34. Binary Search Binary search is a O(log n) algorithm for searching in sorted arrays. Search a sorted array by repeatedly dividing the search interval in half. Begin with an interval covering the whole array. If the value of the search key is less than the item in the middle of the interval, narrow the interval to the lower half. Otherwise narrow it to the upper half. Repeatedly check until the value is found or the interval is empty. Data structure Array

35. Worst-case analysis of binary search - key comparisons in the worst case is Cw(n) worst case no match, first or last element matches the key - after one comparison, half the size of array is considered, so recurrence relation for Cw(n)

{ 1 n=1 Cw(n)= { { Cw|n/2| + 1 n>1 - standard way of solving recurrences is to assume n=2k (n is a power of 2) and solve by backward substitution

36. Multiplication Of Large Integers - decrease the total number of multiplications performed at the expense of a slight increase in the number of addition. - Modern cryptology, require manipulation of integers that are over 100 decimal digits long Such integers are too long to fit in a single word of a modern computer, so they require special treatment DANC technique - to multiply 2 n-digit numbers - reduces the number of multiplication - Representation C = a * b a first number bsecond number n digits A = a1ao b= b1b0

= a1*10n/2 + ao = b1*10n/2 + b0

C = (a1*10n/2 + ao)* (b1*10n/2 + b0) [10n/2*10n/2=10n/2+n/2

= (a1* b1)10n + (a1* b0 +a0* b1)10n/2+(a0* b0) =102n/2 ]

- Formula C= C210n + C110n/2 + C0

C2 = a1* b1

C0 = a0* b0

C1 = (a1 + a0)* (b1+ b0) (C2+C0)

37. How many digit multiplications does algorithm for multiplying 2 large integers make? - Multiplication of n-digit numbers requires 3 multiplications of n/2 digit numbers. - Recurrence relation for number of multiplications is

DANDC algorithm for multiplying two n-digit integers requires about n1.585

One-digit multiplications.

38. Strassens Algorithm Strassens algorithm needs only seven multiplications to multiply two 2-by-2 matrices but requires more addition than the definition based algorithm. By exploiting the DANDC technique, this algorithm can multiply two n-by-n matrices with about n1.585 multiplications.

1 1 1 1 1 1 Y = 1 1 1 1 1 1 Add zeroes to the last row and last column of the X & Y matrices, respectively OR zeroes to the first row and first column of the X & Y matrices, respectively

1 1 1 1 X = 1 1 1 1 1 1 1 1 0 0 0 0

1 1 1 0 1 1 1 0 Y= 1 1 1 0 1 1 1 0

Z = X x Y Apply Strassens method

GREEDY METHOD

42. What is Greedy method? - The greedy approach suggests constructing a solution through a sequence of steps, each expanding a partially constructed solution obtained so far, until a complete solution to the problem is reached. - On each step the choice made must be Feasible (ie) it has to satisfy the problems constraints Locally optimal (ie) it has to be the best local choice among all feasible choices available on that step Irrevocable (ie) once made, it cannot be changed on subsequent steps of the algorithm Algorithm GREEDY(a,n) {

43. Spanning Tree - a spanning tree of a connected graph is its connected acyclic subgraph ((ie) a tree) that contains all the vertices of a graph.

44. Minimum Spanning tree - a minimum spanning tree of a weighted connected graph is its spanning tree of the smallest weight, where the weight of a tree is defined as the sum of the weights on all its edges.

45. Prims algorithm Prims algorithm is a greedy algorithm for constructing a minimum spanning tree of a weighted connected graph. It works by attaching to a previously constructed subtree a vertex closest to the vertices already in the tree.

Prim's algorithm is an algorithm that finds a minimum spanning tree for a connected weighted graph. This means it finds a subset of the edges that forms a tree that includes every vertex, where the total weight of all the edges in the tree is minimized.

46. Steps-Prims algorithm 1. consider a vertex as the initial vertex for the minimum spanning tree 2. shortest edge from the initial vertex is selected a. to find the shortest edge the information about the nearest vertices are provided in the following way i. name of the nearest vertices and length(weight) of the corresponding edge. This is Fringe. Fringe contains only the vertices that are not in the tree but are adjacent to at least one tree vertex. These are candidates from which the next tree vertex is selected. ii. Vertices that are not adjacent to any of the tree vertex can be given a label. This is Unseen. All the other vertices of the graph, is called Unseen, because they are yet to be affected by the algorithm. 3. After identifying a vertex u* to be added to the tree, perform 2 operations a. move u* from the set V-VT to the set of tree vertices VT

b. for each remaining vertex u in V-VT that is connected to u* by a shorter edge than the us current distance label, update its labels by u* and weight of the edge between u* & u, respectively.

47. Implementation of Prims algorithm (Data structure used)

- efficiency of Prims algorithm depends on the data structure chosen for the graph Priority queue is the best - If the graph is represented by its weight matrix and priority queue is implemented by as a unordered array, the algorithms running time will be (n2), where n is the number of vertices in the graph.

- On each of the n-1 iterations, the array implementing the priority queue, is traversed to find and delete the minimum and then to update, if necessary, the priorities of the remaining vertices. - The priority queue can be implemented with min-heap. - If the graph is represented by its adjacency linked list and the priority queue is implemented as a min-heap, the running time of the algorithm is O(Elogn) This is because the algorithm performs n-1 deletions of the smallest element and makes E verifications and possibly, changes of an elements priority in a min-heap of size not greater than n. - Each of this operations, is a O(log n) operations - Running time of this implementation of Prims algorithm is ((n-1) + E)O(log n) = O(Elogn) in a connected graph, n-1 E

48. Kruskals Algorithm - This algorithm looks at a MST for a weighted connected graph G=<V,E> as an acyclic subgraph with V-1 edges for which the sum of the edge weights is the smallest. - Algorithm constructs a MST as an expanding sequence of subgraphs, which are always acyclic but not necessarily connected on the intermediate stages of the algorithm - Algorithm begins by sorting the graphs edges in non-decreasing order of their weights. - Then starting with the empty subgraph, it scans this sorted list adding the next edge on the list to the current sub-graph, if such inclusion does not create a cycle and simply skipping the edge otherwise. - It will generally only be a forest since the set of edges t can be completed into a tree iff there are no cycles in t. OR Kruskals algorithm is a greedy algorithm for the minimum spanning tree problem. It constructs a minimum spanning tree by selecting edges in increasing order of their weights provided that the inclusion does not create a cycle. It requires Union-Find algorithms.

49. Dijkstras algorithm Dijkstras algorithm solves the single-source shortest path problem of finding shortest paths from a given vertex(the source) to all the other vertices of a weighted graph or digraph. It works as Prims algorithm but compares path lengths rather than edge lengths. Dijkstras algorithm always yields a correct solution for a graph with non-negative lengths.

51. Difference between Prims & Kruskals algorithm 1. Prims algorithm expands the tree by one vertex at a time, by selecting the minimum vertex available at that stage 2. At any point of time the resultant tree is a tree with no cycles

1. Kruskals algorithm selects the minimum vertex among all vertices

2. At any point time the resultant tree may be a forest 3. All edges are sorted in increasing order

52. Difference between Greedy and Dynamic Programming Method - In the greedy method only one decision sequence is ever generated

- In the dynamic programming, many decision sequences may be generated. - Dynamic programming algorithms often have a polynomial complexity.

53. Single Source Shortest Path (SSSP) Problem (Dijkstras algorithm) - Given a directed graph G = (V,E), with non-negative costs on each edge, and a selected source node v in V, for all w in V, find the cost of the least cost path from v to w. - The cost of a path is simply the sum of the costs on the edges traversed by the path.

54. Data structures used by Dijkstra's algorithm include: - a cost matrix C, where C[i,j] is the weight on the edge connecting node i to node j. If there is no such edge, C[i,j] = infinity. - a set of nodes S, containing all the nodes whose shortest path from the source node is known. Initially, S contains only the source node. - a distance vector D, where D[i] contains the cost of the shortest path (so far) from the source node to node i, using only those nodes in S as intermediaries.

UNIT III DYNAMIC PROGRAMMING

55. What is Dynamic programming? - is a technique for solving problems with overlapping sub-problems o These sub-problems arise from a recurrence relating a solution to a given problem with solutions to its smaller sub-problems of the same type. - Is an algorithm design method that can be used when the solution to a problem can be viewed as the result of a sequence of decisions o An optimal sequence of decisions can be found by making the decisions one at a time and never making an erroneous decision - It suggests solving each smaller sub problem once and recording the results in a table from which a solution to the original problem can be than obtained.

56. Principle of Optimality The principle of optimality state that an optimal sequence of decisions has the property that whatever the initial state and decision are, the remaining decisions must constitute an optimal decision sequence with regard to the state resulting from the first decision. OR An optimal solution to any of its instance must be made up of optimal solutions to its sub-instances.

\|kn, is the number of combinations (subsets) of k elements from an n-element set (0kn)

- The binomial coefficient defined by factorials

||.|

\|kn=! )! (!k k n n

(0kn)

||.|

\|kn= 0 k<0 or k>n - numerous properties of binomial coefficients, concentrates on the following two:

C(n,k)=C(n-1,k-1) + C(n-1,k) for n>k>0 C(n,0) = C(n,n) = 1

C(n,k) = { 1 k=0,k=n { C(n-1,k-1)+C(n-1,k) k>0

- to solve, record the values of the binomial coefficients in a table of n+1 rows and k+1 columns, numbered from 0 to n and from 0 to k, respectively. - to compute C(n,k) fill the table row by row, starting with row 0 and ending with row n. - each row i (0in) is filled left to right, starting with 1 because C(n,0) = 1 - Rows 0 through k also end with 1 on the tables main diagonal C(i,i)=1 (0ik) - Compute other entries, adding the contents of the cells in the preceding row and previous column and in the preceding row and the same column.

58. Time efficiency for computing binomial coefficient - basic operation is Addition - Let A(n,k) be the total number of additions for computing C(n,k) - Computing each entry in the table requires just one addition - The first k+1 rows of the table form a triangle - Remaining n-k rows of the table form a rectangle o So split the sum expressing A(n,k) as

59. Warshalls algorithm - constructs the transitive closure of a given digraph with n vertices through a series of n x n Boolean matrices R0,R1, ..Rk-1,Rk,,Rn

- each of these matrices provides certain information about directed path in the digraph. - The element Rijk in the ith row & jth column of the matrix Rk (k=0,1,n) is equal to 1 iff there exists a directed path from the ith vertex to the jth vertex with each intermediate value, if any, numbered not higher than k. - formula for generating the elements of matrix Rk from the elements of matrix Rk-1 is

rijk= rijk-1 OR (rikk-1 and

rkjk-1)

o if an element rij is 1 in Rk-1, it remains 1 in Rk. o If an element rij is 0 in Rk-1, it remains 1 in Rk iff the element in its row i & column k and the element in its column j & row k are both 1 in Rk-1. - time efficiency is cubic ,O(n3)

60. Floyds Algorithm - used to solve All-pairs shortest-paths problem - uses the idea of Warshalls algorithm - Given a weighted connected graph (undirected or directed), the all-pairs shortest paths problem finds the distances (the lengths of the shortest paths) from each vertex to all other vertices. - The lengths of the shortest path is recorded in an n x n matrix D called Distance matrix. o The element dij in the ith row and the jth column of this matrix indicates the length of the shortest path from the ith vertex to the jth vertex 1i, jn - Computes the distance matrix of a weighted graph with n vertices through a series of n x n matrices.

D0,D1, ..,Dk-1,Dk,,Dn

- each of these matrices contains the lengths of the shortest paths with certain constraints on the paths considered.

- The element Dijk in the ith row & jth column of the matrix Dk (k=0,1,n) is equal to length of the shortest path among all paths from the ith vertex to the jth vertex with each intermediate value, if any, numbered not higher than k. - Shortest path among the paths that use the kth vertex is equal to dikk-1 + dkjk-1

{ Min { dijk-1 , dikk-1 + dkjk-1} k1 dijk = { wij k=0

- the element in the ith row & jth column of the current distance matrix Dk-1 is replaced by the sum of the elements in the same row i & the kth column and in the same column j & kth column iff the latter sum is smaller than its current value. - time efficiency is O(n3)

61. Difference between Warshalls & Floyds algorithm Warshalls - Input is adjacency matrix - Output is transitive closure - If there is no direct edge between vertices, the value in adjacency matrix is zero - Algorithm looks for 1s in the adjacency matrix to find transitive closure matrix - Transitive closure matrix is a Boolean matrix Floyds - Input is weight matrix - Output is distance matrix - If there is no direct edge between vertices, the value in weight matrix is infinity and diagonal elements are zero - Algorithm looks for minimum value in the weight matrix to find distance matrix - Distance matrix is not a Boolean matrix and will not have infinity

62. 0/1 Knapsack Problem - Given n items of known weights w1,.,wn and values v1,..vn and a knapsack of capacity W. Find the most valuable subset of the items that fill into the knapsack. All weights and knapsack capacity are positive integers and the item values can be real values. - Aim is to fill the knapsack in a way that maximizes the value of the included objects, while respecting the capacity constraint. - Let xi be zero is we dont select the object i or 1 if we include object i. - The problem may be stated as Maximize E vixi i=1 to n Subject to Ewixi <W where vi>0, wi>0 and xie{0,1}

- To solve the problem by Dynamic Programming, we set up a table V(1:n,0:W), with one row for each available object and one column for each weight from 0 to W. - The solution of the instance can be found in V(n,W). - Fill the table either row by row or column by column. - the recurrence for knapsack problem is V(i,j)=Max{V(i-1,j), V(i-1,j-wi)+vi} } j-wi0 V(i-1,j) j-wi<0

If V(i-1,j) is larger, the object i is discarded.

If V(i-1,j-Wi)+vi is larger, the object i is included

- initial conditions are V(0,j)=0 j>0 V(i,j)=- for all i when j<0 V(i,0) = 0 i0 - fill the table V, using the above formula - solution is found at V(n,W) - to find solution vector start from V(n,W) and track back the computations in the table - to add or discard an item i, the following criteria should be satisfied for each item. 1. If V(i,j) = V(i-1,j) and V(i,j) = V(i-1,j-wi)+vi then discard item i. 2. If V(i,j) = V(i-1,j) and V(i,j) = V(i-1,j-wi)+vi then include item i. 3. If V(i,j) = V(i-1,j) and V(i,j) = V(i-1,j-wi)+vi then include item i. 4. If V(i,j) = V(i-1,j) and V(i,j) = V(i-1,j-wi)+vi then discard item i. - if item i is added, update the weight of the items included in the sack. - Time & space efficiency is O(nW), time necessary to construct the table - The composition of the optimal load can be determined in a time of O(n+W)

63. Memory Functions - The direct top-down approach to finding a solution to such a recurrence leads to an algorithm that solves common subproblems more than once and hence inefficient (exponential or worse) - The classic 0/1 knapsack of dynamic programming works bottom-up. o The solutions of some of these smaller subproblems are often not necessary for getting a solution to the problem given. - The goal is to get a method that solves only subproblems that are necessary and does it only once. Such a method exists; it is bases on using Memory Functions. o This method solves a given problem in the top-down manner but, in addition, maintains a table of the kind that is used in bottom-up approach. o Initially, all the tables entries are initialized with a special nul symbol to indicate that they have not yet been calculated. o Whenever a new value needs to be calculated, the method checks that corresponding entry in the table first; if this enry is not null, it is simple retrieved from the table; otherwise it is computed by the recursive call whose result is than recorded in the table. o After initializing the table, the recursive function needs to be called with i=n(no. of items) and j=W (knapsack capacity)

63.a. Optimal Binary Search Tree An optimal binary search tree, is a binary search tree where the average cost of looking up an item (the expected search cost) is minimized.

UNIT IV BACKTRACKING

64. Backtracking Backtracking constructs its state-space tree in the depth-first search fashion. If the sequence of choices represented by a current node of the state-space tree can be developed

further without violating the problems constraints, it is done by considering the first remaining legitimate option for the next component. Otherwise, the method backtracks by undoing the last component of the partially built solution and replaces it by the next alternative.

- Unlike the exhaustive search approach, which is doomed to be extremely slow for all instance of a problem, backtracking at least holds a hope for solving some instances of non-trivial sizes in an acceptable amount of time. This is especially true for optimization problems, for which the idea of backtracking can be further enhanced by evaluating the quality of partially constructed solutions.

- Even is backtracking does not eliminate any elements of a problems state space and ends up generating all its elements, it provides a specific technique for doing so, which can be of value in its own right.

65. State Space Tree State-space tree is a rooted tree whose nodes represent partially constructed solutions to the problem in question. Is constructed in the manner of depth first search. Its root represents an initial state before the search for a solution begins. The nodes of the first level in the tree represent the choices made for the first component of a solution; the nodes of the second level represent the choices for the second component and so on.

66. Promising Node A node in a state-space is said to be promising if it corresponds to a partially constructed that may still lead to a complete solution.

67. Explicit Constraints Explicit constraints are rules that restrict each xi to take a value only from a given set. xi0 Si={all positive real numbers} xi=0 or 1 Si={0,1} lixiui Si={a : liaui} The explicit constraints depend on the particular instance I of the problem being solved. All tuples that satisfy the explicit constraints define a possible solution space for I.

68. Implicit Constraints Implicit constraints are rules that determine which of the tuples in the solution space of I satisfy the criterion function. Thus implicit constraints describe the way in which the xi must relate to each other.

69. N Queens Problem Is a combinatorial problem to place n queens on an n-by-n chessboard, so that no two queens attack each other by being in the same row or in the same column or on the same diagonal. The Explicit constraint: Value of xi must be from S={1,2,3,,n}. The solution space consists of nn n-tuples. Xi represents the column numbers

The Implicit constraint: No two xis can be the same (ie) all queens must be on different columns and no two queens can be on the same diagonal.

Two queens lie on the same diagonal iff |j-l| = |i-k|.

70. Hamiltonian Circuit (Cycle) Let G=<V,E> be a connected graph with n vertices. A Hamiltonian cycle is a round-trip path along n edges of G that visits every vertex once and returns to its starting position. If a Hamiltonian cycle begins at some vertex G v e1 and the vertices of G are visited in the order v1,v2,.,vn+1 then edges (vi,vi+1) are in E, 1in and the vi are distinct except for v1 and vn+1

which are equal. Implicit constraints - all vertices should be included in the cycle - all vertices should be distinct except x1,xn+1 - only distinct cycles are output. Explicit constraint - xi= vertex number

71. Sum of Subsets Given positive numbers wi, 1sisn and m. The problem is to find all subsets of the wi

whose sum equals m. The problem can be formulated using either Fixed or Variable sized tuples.

73. Branch and Bound Branch and bound is an algorithm design technique that enhances the idea of generating a state-space tree with the idea of estimating the best value obtainable from a current node of the decision tree: if such an estimate is not superior to the best solution seen up to that point in the processing, the node is eliminated from further consideration.

74. Principal idea behind Branch and Bound Technique - problem is represented in state space tree - a nodes bound value is compared with the value of the best solution seen so far: o if the bound value is not better than the best solution seen so far, (ie) not smaller for a minimization problem and not larger for a maximization problem, the node is non-promising and can be terminated.

o No solution obtained from it can yield a better solution than the one already available.

75. A search path at the current node in a state space tree is terminated for any one of the following 3 reasons:

1. value of the nodes bound is not better that the value of the best solution seen so far. 2. node represents no feasible solutions because the constraints of the problem is already violated. 3. subset of feasible solutions represented by the node consists of a single point (no further choices can be made) o the value of the objective function for this feasible solution is compared with that of the best solution seen so far and update the later with the former, if the new solution is better.

76. Feasible Solution Feasible solution is a point in the problems search space that satisfies all the problems constraints. Example: Cycle in Traveling sales person problem, Items whose weight does not exceed the capacity of bag.

77. Optimal Solution Optimal solution is a feasible solution with the best value of the objective function. Example: shortest path in Traveling Salesperson problem, most valuable items that fits the bag.

78. Best-First Branch and Bound Strategy - In branch and bound, instead of generating a single child of the last promising node, all the children of the most promising among non-terminated leaves in the current tree is generated. (Non-terminated still promising, leaves are live). - Compare the lower bounds of the live nodes to find which of the nodes is most promising and consider a node with the best bound as most promising. - This strategy is called Best First branch and bound.

79. Knapsack Problem - order the items of a given instance in descending order by their value-to-weight ratios (vi/wi) v1/w1 v2/w2 . vn/wn

- Compute upper bound ub = vi + (W-wi)(vi+1/wi+1) - Root of the state space, has no items and total weight of the items already selected W and their total value V is equal to 0 o Value of the ub is computed by formula. - Next level nodes w,v,ub is computed o Levels left nodes ub and right nodes ub is compared. o Higher ub value, node is selected for finding the next value, in maximization problem. o Also w at each level is checked to find whether it exceeds the bag capacity. If a nodes w exceeds the bag capacity, the path will not give feasible solution.

80. Assignment Problem - is the problem of assigning n people to n jobs so that the total cost of the assignment is a small as possible.

- Assignment problem is specified by an n-by-n cost matrix, C. - The problem is stated as follows: o Select one element in each row of the matrix so that no 2 selected elements are in the same column and their sum is the smallest possible.

81. Traveling Salesperson Problem The problem is to find a least-cost tour of the N cities in a sales region. The tour is to visit each city exactly once. To help find an optimum tour, the salesperson has a cost matrix C, where element C(i,j) equals the cost (usually in terms of time, money or distance) of direct travel between city i & city j.

82. Common issues in Backtracking and Branch and Bound Backtracking and Branch & Bound - are used to solve some large instance of difficult combinatorial problems - can be considered as an improvement over exhaustive search - Are based on the construction of a state-space-tree, whose nodes reflect specific choices made for a solutions components. - Terminate a node as soon as it can be guaranteed that no solution, to the problem can be obtained by considering choices that correspond to the nodes descendants.

83. Difference between Backtracking and Branch and Bound Backtracking - Applicable to non-optimization problems. - State-space-tree is developed as Depth-first. Branch & Bound - Applicable only to optimization problems because it is based on computing a bound on possible values of the problems objective function. - Can generate nodes according to several rules; the most natural rule is Best-first rule. - Has both the challenge and opportunity of choosing an order of node generation and finding a good bounding function. - Best first rule may or may not lead to a solution faster than other strategies.

84. Difference between Backtracking and Backward approach Backtracking 1. starts from root of the tree 2. When a node is a non-promising node, moves back to the previous node or level 3. From non-promising node, first moves by BFS and then by DFS in backwards

Backward approach 1. starts from leaf and moves towards root 2. moves either by BFS or DFS in backwards, to reach the root

UNIT V NP-HARD AND NP-COMPLETE PROBLEMS

85.Heuristic approach A heuristic is a common-sense rule drawn from experience rather than from a mathematically proven assertion.

86.Approximation algorithms

Approximation algorithms are often used to find approximate solutions to difficult problems of combinatorial optimization. Approximation algorithm run a length in level of sophistication, many of them use greedy algorithms based on some problem specific heuristic.

87. Accuracy of approximate solution The accuracy of an approximate solution Sa to a problem minimizing some function f can be quantified by the size of the relative error of this approximation

88. Performance Ratio The performance ratio is the principal metric for measuring the accuracy of such approximation algorithms.

89. Nearest Neighbor algorithm Nearest Neighbor is a simple greedy algorithm for approximating a slution to the Traveling salesperson problem. The performance ratio of this algorithm is unbounded above, even for the important subset of Euclidean graphs. Step 1: choose an arbitrary city as the start Step 2: repeat the following operation until all the cities have been visited; go to the unvisited city nearest to the one visited last. (Ties can be broken arbitrary) Step 3: return to the starting city.

90. Twice-around-the-tree algorithm Twice-around-the-tree is an approximation algorithm for the Traveling salesperson problem with the performance ratio of 2 for Euclidean graphs. The algorithm is based on modifying a walk around a minimum spanning tree by shortcuts. Step 1: Construct a Minimum Spanning Tree of the graph corresponding to a given instance of the Traveling salesperson problem. Step 2: Starting at an arbitrary vertex, perform a walk around the Minimum spanning tree recording the vertices passed by Step 3: Scan the list of vertices obtained in Step 2 and eliminate from it all repeated occurrences of the same vertex except the starting one at the end of the list. The vertices remaining on the list will form a Hamiltonian Circuit, which is the output of the algorithm.

91. Approximation Schemes Polynomial-time approximation schemes for discrete version of knapsack problem, which are parametric families of algorithms that allows to get approximations Sa(k) with any predefined accuracy level:

k S f S f ka11) () (*) (+ s for any instance of size n, where k is an integer parameter in the range 0kn.

92. Complexity Theory Complexity theory seeks to classify problems according to their computational complexity. The principal split in between tractable & intractable problems problems that can & cannot be solved in polynomial time, respectively.

93. Halting Problem Halting problem is an example of an undecidable decision problem (ie) it cannot be solved by any algorithm. The halting problem is to determine for an arbitrary deterministic algorithm A and an input I whether algorithm A with input I ever terminates (or enters an infinite loop). This problem is undecidable. Hence there exists no algorithm to solve this problem.

94. Polynomial time An algorithm solves the problem in polynomial time if its worst-case time efficiency belong to O(p(n)) where p(n) is a polynomial of the problems input size n. An algorithm is said to be solvable in polynomial time if the number of steps required to complete the algorithm for a given input is for some nonnegative integer , where is the complexity of the input. Polynomial-time algorithms are said to be "fast." Problems that can be solved in polynomial time are called Tractable, problems that cannot be solved in polynomial time are called intractable.

95. Class P P is the class of all decision problems than can be solved in polynomial time. Class P is a class of decision problems (problems which have Yes/No answers) that can be solved in polynomial time by deterministic algorithms. This class of problems is called Polynomial.

96. Non deterministic Algorithm A non-deterministic algorithm is a two- stage procedure that takes as its input an instance I of a decision problem and does the following: Non-deterministic stage (guessing): An arbitrary string S is generated that can be thought of as a candidate solution to the given instance I. Deterministic stage(verification): A deterministic algorithm takes both I and S as its input and outputs Yes if S represents a solution to instance I. Non-deterministic algorithms are said to solve decision problem iff for every Yes instance of the problem returns Yes on some execution. A non-deterministic algorithm is said to be non-deterministic polynomial if the time efficiency of its verification stage is polynomial.

97. Class NP NP is the class of all decision problems whose randomly generated solutions can be verified in polynomial time. Class NP is the class of decision problems that can be solved by non-deterministic polynomial algorithms. This class of problems is called non-deterministic polynomial.

98. NP- Complete

A decision problem D is said to be NP-complete if 1. it belongs to class NP 2. every problem in NP is polynomially reducible to D. Decision version of a difficult combinatorial problem in NP-Complete. Examples. Traveling salesperson problem, Knapsack problem. It is not known whether P =NP or P is just a proper subset of NP. A discovery of a Polynomial-time algorithm for any of the thousands known NP-complete problem would imply that P=NP.

99. Polynomially-Reducible A decision problem D1 is said to be polynomially reducible to a decision problem D2 if there exists a function t that transforms instanced of D1 to instances of D2 such that 1. t maps all yes instances of D1 to yes instances of D2 and all no instances of D1 to no instances of D2. 2. t is computable by a Polynomial-time algorithm

This definition implies that if a problem D1 is polynomially reducible to some problems D2 that can be solved in polynomial time, then problem D1 can also be solved in polynomial time.

100. Exponential time

In complexity theory, exponential time is the computation time of a problem where the time to complete the computation, m(n), is bounded by an exponential function of the problem size, n (i.e, as the size of the problem increases linearly, the time to solve the problem increases exponentially). Written mathematically, there exists k > 1 such that m(n) = (kn) and there exists c such that m(n) = O(cn).

101. Reason for intractability

1. Arbitrary instance of intractable problems cannot be solved in a reasonable amount of time unless such instances are very small. 2. There is a huge difference between the running times in O(p(n)) for polynomials of drastically different degrees, there are very few useful polynomial-time algorithms with the degree of a polynomial higher than 3. 3. Polynomial functions posses many convenient properties both the sum & composition of two polynomials are always polynomials too. 4. Polynomial algorithm lead to the development of an extensive theory called computational complexity, which seeks to classify problems according to their inherent difficulty.

102. Undecidable Problems Not every decision problems can be solved in polynomial time. Some decision problems cannot be solved at all by any algorithm. Such problems are called Undecidable.

103. Decision Problem

Any problem for which the answer is either zero or one is called decision problem.

104. Optimization problem

Any problem that involves the identification of an optimal (either minimum or maximum) value of a given cost function is known as an optimization problem.

105. Cooks Theorem

Cook's theorem, states that the Boolean satisfiability problem is NP-complete. That is, any problem in NP can be reduced in polynomial time by a deterministic Turing machine to a problem of determining whether a Boolean formula is satisfiable.

106. Satisfiability Problem

The satisfiability problem is to determine whether a formula is true for some assignment of truth values to the variables. CNF-satisfiability problem for CNF (Conjunctive Normal Form) formulas.

Diagram of complexity classes provided that P NP. If P = NP, then all three classes are equal.