Neural Networks

What they are and why they matter

Neural networks are computing systems with interconnected nodes that work much like neurons in the human brain. Using algorithms, they can recognize hidden patterns and correlations in raw data, cluster and classify it, and – over time – continuously learn and improve.

History of neural networks

The first neural network was conceived of by Warren McCulloch and Walter Pitts in 1943. They wrote a seminal paper on how neurons may work and modeled their ideas by creating a simple neural network using electrical circuits.

This breakthrough model paved the way for neural network research in two areas:

AI research quickly accelerated, with Kunihiko Fukushima developing the first true, multilayered neural network in 1975.

The original goal of the neural network approach was to create a computational system that could solve problems like a human brain. However, over time, researchers shifted their focus to using neural networks to match specific tasks, leading to deviations from a strictly biological approach. Since then, neural networks have supported diverse tasks, including computer vision, speech recognition, machine translation, social network filtering, playing board and video games, and medical diagnosis.

As structured and unstructured data sizes increased to big data levels, people developed deep learning systems, which are essentially neural networks with many layers. Deep learning enables the capture and mining of more and bigger data, including unstructured data.

Why are neural networks important?

Neural networks are also ideally suited to help people solve complex problems in real-life situations. They can learn and model the relationships between inputs and outputs that are nonlinear and complex; make generalizations and inferences; reveal hidden relationships, patterns and predictions; and model highly volatile data (such as financial time series data) and variances needed to predict rare events (such as fraud detection). As a result, neural networks can improve decision processes in areas such as:

Credit card and Medicare fraud detection.

Optimization of logistics for transportation networks.

Character and voice recognition, also known as natural language processing.

Our first goal for these neural networks, or models, is to achieve human-level accuracy. Until you get to that level, you always know you can do better.Ivan GomezData Scientist and ConsultantZencos

Types of neural networks

There are different kinds of deep neural networks – and each has advantages and disadvantages, depending upon the use. Examples include:

Convolutional neural networks (CNNs)contain five types of layers: input, convolution, pooling, fully connected and output. Each layer has a specific purpose, like summarizing, connecting or activating. Convolutional neural networks have popularized image classification and object detection. However, CNNs have also been applied to other areas, such as natural language processing and forecasting.

Recurrent neural networks (RNNs)use sequential information such as time-stamped data from a sensor device or a spoken sentence, composed of a sequence of terms. Unlike traditional neural networks, all inputs to a recurrent neural network are not independent of each other, and the output for each element depends on the computations of its preceding elements. RNNs are used in fore­casting and time series applications, sentiment analysis and other text applications.

Feedforward neural networks, in which each perceptron in one layer is connected to every perceptron from the next layer. Information is fed forward from one layer to the next in the forward direction only. There are no feedback loops.

Autoencoder neural networksare used to create abstractions called encoders, created from a given set of inputs. Although similar to more traditional neural networks, autoencoders seek to model the inputs themselves, and therefore the method is considered unsupervised. The premise of autoencoders is to desensitize the irrelevant and sensitize the relevant. As layers are added, further abstractions are formulated at higher layers (layers closest to the point at which a decoder layer is introduced). These abstractions can then be used by linear or nonlinear classifiers.

Neural Networks in today’s world

Neural networks are changing how people and organizations interact with systems, solve problems, and make better decisions and predictions. Learn more about the impact of neural networks.

Neural networks for herd health

A passionate data scientist uses neural networks to detect tuberculosis in elephants. Find out how her research can help prevent the spread of tuberculosis.

Learn neural network modeling

Learn how to choose an appropriate neural network architecture, how to determine the relevant training method, how to implement neural network models in a distributed computing environment and how to construct custom neural networks using the NEURAL procedure.

Neural networks have the ability to identify anomalies. In the future, we can use them to give doctors a second opinion – for example, if something is cancer, or what some unknown problem is. And we’ll be able to provide these second opinions faster and with more accuracy.Leigh Ann HerholdData Scientist and ConsultantZencos

Building a Neural Network Model

In this video, you learn how to use SAS® Visual Data Mining and Machine Learning in the context of neural networks. This example examines the drivers of website visitors and what causes them to download a paper from an IT company’s site.

How neural networks work

A simple neural network includes an input layer, an output (or target) layer and, in between, a hidden layer. The layers are connected via nodes, and these connections form a “network” – the neural network – of interconnected nodes.

A node is patterned after a neuron in a human brain. Similar in behavior to neurons, nodes are activated when there is sufficient stimuli or input. This activation spreads throughout the network, creating a response to the stimuli (output). The connections between these artificial neurons act as simple synapses, enabling signals to be transmitted from one to another. Signals across layers as they travel from the first input to the last output layer – and get processed along the way.

When posed with a request or problem to solve, the neurons run mathematical calculations to figure out if there’s enough information to pass on the information to the next neuron. Put more simply, they read all the data and figure out where the strongest relationships exist. In the simplest type of network, data inputs received are added up, and if the sum is more than a certain threshold value, the neuron “fires” and activates the neurons it’s connected to.

As the number of hidden layers within a neural network increases, deep neural networks are formed. Deep learning architectures take simple neural networks to the next level. Using these layers, data scientists can build their own deep learning networks that enable machine learning, which can train a computer to accurately emulate human tasks, such as recognizing speech, identifying images or making predictions. Equally important, the computer can learn on its own by recognizing patterns in many layers of processing.

So let’s put this definition into action. Data is fed into a neural network through the input layer, which communicates to hidden layers. Processing takes place in the hidden layers through a system of weighted connections. Nodes in the hidden layer then combine data from the input layer with a set of coefficients and assigns appropriate weights to inputs. These input-weight products are then summed up. The sum is passed through a node’s activation function, which determines the extent that a signal must progress further through the network to affect the final output. Finally, the hidden layers link to the output layer – where the outputs are retrieved.