Why deep learning is popular now though existed long before?

Deep learning has a long and rich history and started back to 1940s but gained popularity just recently and mostly in this decade

Emroj Hossain

4 min read

Sun Jan 12 2020

Deep learning is known as an amazing new technology that is changing our life. But one would be amazed to know that it is not a new technology but existed for many years and gained popularity just in recent days. The deep learning started back on the 1940s. It appears to be new as it was unpopular for several decades. There are several reasons for the popularity of deep learning in recent days that didn't exist when deep learning was started.

Changing names: from cybernetics to deep learning

One of the reasons for deep learning being unknown for several decades is the different names used and the name "deep learning" has emerged just recently. In the 1940s - 1960s the area of deep learning was known as cybernetics and during the period 1980s to 1990s, the area was known as connectionism and the area was re-branded as deep learning around 2006. Since, some of the algorithms of the deep learning are inspired by the human brain and use the neural network, which is similar to the neuron, the structural and functional unit of the human brain, the field is of often known as the artificial neural network (ANN).

Increasing datasets sizes

The performance of a deep learning model increases with the increasing size of the datasets. Better algorithm and skills are required for better performance of a machine learning or a deep learning model, but with increasing size of the training data, the amount of skills required to design the algorithms reduces significantly to train the same performance. The deep learning algorithms which use to solve the toy models have now reached to the human level accuracy due to the increase in the training data. That's why the deep learning why earlier was considered as an art is now a part of today's technology and a part of our day today's life. Several datasets i.e. MNIST, image-net provides huge datasets in several categories that are useful for training the deep learning models. Apart from the standard datasets internet provides a source of huge datasets which can be used to train big deep learning models. The videos uploaded in the youtube, the speech used to communicate with voice assistant such as Alexa, Siri, Google assistance, the user behaviour in several websites such as Netflix is used to train and improve the performance of those machine learning software and is used as a big data for them.

Increasing computational power

In the earlier days, the machine learning or the deep learning models were trained in very slow systems and with a limited amount of memory. But in recent years the computation power of modern computers has been increased significantly. That enables the engineers to train a deep learning algorithm at a faster speed. Also, the increase in the storage capacity enables us to train the machine learning models by a relatively larger dataset. It is observed that modern graphical processing units (GPU) perform far better than the central processing units (CPU). So, GPU's are preferred over CPU's in training the machine learning algorithms. Apart from this, the distributed computing enables us to improve the performance of a deep learning model efficiently. Google has recently come up with Tensor Processing Unit (TPU), which is an AI accelerator application-specific integrated circuit specially designed for efficient computing of deep learning models. Intel also came up with Nervana neural network processors enabling distributed learning algorithms and systems to scale up deep learning reasoning, using more advanced forms of AI to go beyond the conversion of data into information-turning data into global knowledge. The improve computational power enables machine learning models to solve the real-world complex problems which were earlier unsolved by the same machine learning models due to the limited computation power of the systems.

Increasing model sizes

Increasing computational power and storage enable engineers to use deep machine learning algorithms to solve more real-world complex problems. Increasing the depth of a deep learning model allows us to solve more realistic and complex problems with better accuracy. One of the reasons that is responsible for human intelligence is a huge number of interconnected neurons. Though deep learning models also use artificial neural networks, their numbers are limited and still is very less than the number of neurons in the human brain. Earlier very less number of artificial neurons in neural network limited the use of it for the complex. But, with the improvement of the computation power and the memory the number of artificial neurons in the deep learning models have increased. This allows the machines learning algorithms to solve more complex real-world problems and as a result, the deep learning models are applicable in day to days life and also earned popularity. The size of artificial neural network almost doubles in every 2.4 years and the growth is enhanced with faster memory, more computation power and large datasets.

Software libraries

In the early days of deep learning and machine learning, the programming a deep learning model was only confined to a small group of programmers and they need to write the code from the beginning that adds to the size of the code and was time consuming and difficult to maintain. But, now there are many open-source software libraries such as Tensorflow, PyTorch, Caffe, Theano that make the programming the deep learning models super easy. The earlier big code to classify handwritten digits can be replaced by Just 8 lines of codes. Moreover, the libraries take care of from Building a deep learning model to deploy it for production. That attract researcher developer, software engineer, and several other communities in the machine learning these days and help to spread it boundaries.

Open-source codes and pre-trained models

Now the machine learning and the deep learning community is quite big and believe in open-source software. The codes of various algorithms are available online for testing using and improving. Not only codes but the pre-trained deep learning models along with weights and biases are available online. The models are generally pre-tarained in GPU over a long time with a huge dataset. As a result, the pre-trained models perform better with small available datasets and with training for very small time. As an example, pre-trained models on image-net datasets are available online and perform better with a small set of training images compared to previously untrained models.

Solving real-world problems and increasing accuracy

In early days deep learning was only used for solving toy models and relatively simple problems. But, now deep learning is used in various technologies and productions and used in day today's life. Google voice assistants, Alexa, Siri acts as virtual assistants. Tesla's and google's self-driving car drive the car without any human interventions. Many other areas such as image processing video editing use machine learning. One of the promising areas of machine learning is in medical sciences. It can help to replace a human organ with electro-mechanical organ, help doctors to predict diseases and also helps to predict cure.

Big companies in deep learning

The success of deep learning attracted several big companies such as Google, Facebook, Nvidia, Intel, Netflix, Apple, Amazon, Baidu, IBM, Adobe and they are making huge investments in the area of deep learning. That is attracting lots of people to learn and work in this area. This also adds to the popularity of machine learning and deep learning. The successful real-world applications of deep learning are attracting much more companies, investors, researchers, engineers, peoples of various areas to work in deep learning and is currently a very popular topic to work on.