How do I learn data structures and algorithms from scratch?

If you don’t know, learn some discrete mathematics and probability. Learn basic combinatorics, graph theory, matrices, probability theory, especially the concept of expected values. Do this on the fly.

Learn basic data structures

Implement singly and doubly linked lists on your own.

Learn stack and queue. Implement them using linked lists. For more fun - try implementing stacks and queues using arrays.

Implement vector data structure in C++ on your own. It is simple - when you run out of space, allocate twice the memory you had allocated previously and copy all the data to newly allocated array.

Implement binary heap data structure and use it as priority queue.

Learn sorting algorithms

Learn somewhat inefficient sorting algorithms - Selection sort, Bubble sort and Insertion sort (the last one works fine on small inputs)

Learn merge sort. Recursive functions can be somewhat hard to understand in the beginning. This teaches you that skill. Trace it very well.

Learn quick sort. This is one of the most important algorithms ever discovered. Learn the importance of randomized quick-sort and pivot selection.

Learn counting sort and why it works in linear sort.

If you are interested, you can go further and learn about “stability” of sorting algorithms. For instance, merge sort and insertion sort are stable.

You might come across performance issues for certain inputs. Why? Now learn about “Balanced” binary search trees like AVL Tree and Red-Black Trees and try implementing them - This is hard, tedious and scary to implement and debug. Expect lot of Segmentation faults.

Learn the concept of hashing and hash-table. Implement a hash-table. (there are two schemes).

Figure out what else can be done with binary search trees. Augmenting data structures with more information? How about adding a feature where kth order statistic can be found? How about binary search on keys?

You might have encountered terms like Divide and Conquer while studying merge sort. They are called “Algorithm design techniques”. You will also come across terms like Brute Force, Dynamic programming, Greedy algorithms. Now let’s dive into it.

You already know Divide and Conquer. Learn Dynamic programming technique. You might find it a bit difficult in the beginning and all of us had been there. Study the solution of some standard problems like Rod Cutting, 0/1 Knapsack, Edit Distance, Matrix Chain Multiplication, Longest Increasing Subsequence etc.

Now it is time to look at another technique - Greedy algorithms. Study standard problems which use greedy technique like selecting largest number of intervals among given ones so that there is no conflict. You’ll see more later.

Learn the difference between all these techniques. DP and D&C differ in sub-problem overlap and Greedy, DP differ in the way decisions are made and revisited.

Study the proofs of these algorithms, especially the greedy ones. Don’t worry about abstract concepts like Matroids for now.

Let’s jump to Graph algorithms now.

Let’s build a maze solver - Learn and implement Depth First Search (DFS) and Breadth First Search (BFS). They are basic searching techniques.

Let’s implement our version of Google Maps - Finding the shortest path between two given places given a roadmap. Learn and implement the algorithms of Dijkstra, Bellman-Ford and Floyd-Warshall.

Bingo, now you are pretty good at data structures and algorithms. But remember that this is just the tip of the iceberg. Once you learn it, implement on your own. Only then you get a feel of it. If you want to learn them in a fun way, try competitive programming.

Here are some resources

Introduction to Algorithms - CLRS (Chapters are in the order I have written. I have skipped some chapters in it). I suggest this one along with the DSA course from Stanford on Coursera.

Popular Posts

Computer programming is one of the fastest growing field. If you are a programmer then you can be hired by a top tech company or you can also work as a freelancer and earn decent amount. So the question arises, how to learn to code ? Initially it was very difficult to learn programming. As there was no internet and you have to relay on books or your colleagues’ code. Now you can find tons of material on the internet. You can learn any computer programming language you want. I am going to tell you some sites which you should visit regularly.

Huffman Code compress data very effectively: saving of 20% to 90% are typical, depending on the characteristics of the data being compressed.Huffman's greedy algorithm uses a table giving how often each character occurs (i.e., its frequency) to build up an optimal way of representing each character as a binary string. This algorithm constructs an optimal prefix code called a Huffman Code.

What is namespace?"In general, a namespace is a container for a set of identifiers Namespaces provide a level of direction to specific identifiers, thus making it possible to distinguish between identifiers with the same exact name. " Or According to Wikipedia “In computer programming namespaces are typically employed for the purpose of grouping symbols and identifiers around a particular functionality and to avoid name collisionsbetween multiple identifiers that share the same name.”