Difference between Multi-Threading and Parallel Programming

Since the use of multi-core computers / PCs is common these days, there are more and more parallel programming frameworks coming up. If you use Scala (or Java) you might have used the AKA framework which is based on Actors. In java 7 there are the Fork/ join APIs. However you might ask why do we need these , since multithreaded apps are already quite common. If you use Servlets and so on you are automatically using multiple threads and JVM is a beast when it comes to scaling up using threads and utilising multiple cores.

However as a programmer when it comes to explicit use of Multi threading or parallel programming it is important to understand the fundamental difference.

The traditional multi-threading was actually used to do time-slicing or take advantage of the CPU idle time, which is that while one of the threads in your program was waiting another thread could execute.

This type of multi threading in java was traditionally used with the object of type Thread. Using threads however does not guarantee that you will optimally use all the cores available in your computer. Esp. if you heavily use thread synchronisation.

That is where the “parallel” programming comes into play. Parallel programming explicitly breaks the task down into smallest unit of execution, where each unit can be executed in parallel on a single CPU core. This way you can have multiple parts of the same task being executed in parallel.

The new Java 7’s Fork / Join framework does this quite nicely. I suspect that the number of apps which use Fork/ Join will out number the traditional multi-threaded apps in the coming future.

Watch out for blogs in the future about how to use Fork / Join, AKA and other parallel programming frameworks/ APIs

3 thoughts on “Difference between Multi-Threading and Parallel Programming”

Just a few humble side notes: multithreading and parallel programming are not entirely different concepts. we can assume that every parallel program is also multithreaded, right? However, a multithreaded program can have different parts executing at the same time if hardware allows that, which makes it sort of a parallel program. And other thing is saying that parallel programming divides the main task into smaller parts can be true for task parallelism, however in data parallelism you do not break the task down to small units of execution but small units of data.