Subscribe to stay notified about new videos: http://3b1b.co/subscribe
Support more videos like this on Patreon: https://www.patreon.com/3blue1brown
Or don't. It's your call really, no pressure.
Special thanks to these supporters: http://3b1b.co/nn1-thanks
Additional funding provided by Amplify Partners. For any early-stage ML entrepreneurs, Amplify would love to hear from you: 3blue1brown@amplifypartners.com
Full playlist: http://3b1b.co/neural-networks
Typo correction: At 14:45, the last index on the bias vector is n, when it's supposed to in fact be a k. Thanks for the sharp eyes that caught that!
For those who want to learn more, I highly recommend the book by Michael Nielsen introducing neural networks and deep learning: https://goo.gl/Zmczdy
There are two neat things about this book. First, it's available for free, so consider joining me in making a donation Nielsen's way if you get something out of it. And second, it's centered around walking through some code and data which you can download yourself, and which covers the same example that I introduce in this video. Yay for active learning!
https://github.com/mnielsen/neural-networks-and-deep-learning
I also highly recommend Chris Olah's blog: http://colah.github.io/
For more videos, Welch Labs also has some great series on machine learning:
https://youtu.be/i8D90DkCLhI
https://youtu.be/bxe2T-V8XRs
For those of you looking to go *even* deeper, check out the text "Deep Learning" by Goodfellow, Bengio, and Courville.
Also, the publication Distill is just utterly beautiful: https://distill.pub/
Lion photo by Kevin Pluck
Music by Vincent Rubinetti:
https://vincerubinetti.bandcamp.com/album/the-music-of-3blue1brown
------------------
3blue1brown is a channel about animating math, in all senses of the word animate. And you know the drill with YouTube, if you want to stay posted on new videos, subscribe, and click the bell to receive notifications (if you're into that).
If you are new to this channel and want to see more, a good place to start is this playlist: http://3b1b.co/recommended
Various social media stuffs:
Website: https://www.3blue1brown.com
Twitter: https://twitter.com/3Blue1Brown
Patreon: https://patreon.com/3blue1brown
Facebook: https://www.facebook.com/3blue1brown
Reddit: https://www.reddit.com/r/3Blue1Brown

published:05 Oct 2017

views:3111232

Just a simple implementation of neural net for evolution of a car to finish the track. The neural network itself doesn't evolve in shape, but in the neuron connection weights.
Made inUnityMusic: "Righteous" by Silent Partner

published:20 Jun 2017

views:515094

Read the complete tutorial about how to implement a machine learning algorithm for the Flappy Bird video game here:
http://www.askforgametask.com/tutorial/machine-learning-algorithm-flappy-bird
This video shows a creation of an artificial intelligence controller for the Flappy Bird game using neural networks and a genetic algorithm.
The program is written in HTML5 using Phaser framework (http://phaser.io/). There is also used Synaptic Neural Network library (https://synaptic.juancazala.com/) to implement entire artificial neural network instead of making a new one from the scratch.
Download source code here:
https://github.com/ssusnic/Machine-Learning-Flappy-Bird
-----------------------------------------------------------------
According to Arthur Samuel, machine learning is the science of getting computers to act without being explicitly programmed. It is a fine tuning process of learning that incrementally improves an initial random system.
The form of machine learning implemented in this program uses a genetic algorithm to train artificial neural networks.
The birds are learning how to flap optimally in order to fly safely through barriers as long as possible.
The main concept is based on these 3 steps:
1. creating the initial population of 10 birds randomly
2. learning as the game is being played
3. applying natural evolution to form the next improved population
-----------------------------------------------------------------
To play the game, each bird has its own neural network consisted of the next 3 layers:
1. an input layer with 2 neurons representing what a bird sees:
- horizontal distance to the closest gap
- height difference to the closest gap
2. a hidden layer with 6 neurons
3. an output layer with 1 neuron to perform flap if its value is greater than 0.5
-----------------------------------------------------------------
To measure the quality of birds and select the best ones, for each bird is calculated its fitness function in this way:
- reward a bird by its total traveled distance
- penalize a bird by its current distance to the closest gap
When the entire population is dead, the fittest four birds are selected to breed a new population by using genetic algorithm operators: selection, crossover and mutation.
-----------------------------------------------------------------
Visit us:
http://www.askforgametask.com
Like us:
https://www.facebook.com/askforgametask
Follow us:
https://twitter.com/ssusnic
Music:
"BedtimeTune" by Jay Man
http://www.ourmusicbox.com

published:10 Aug 2017

views:585180

By popular demand, I threw my own voice into a neural network (3 times) and got it to recreate what it had learned along the way!
This is 3 different recurrent neural networks (LSTM type) trying to find patterns in raw audio and reproduce them as well as they can. The networks are quite small considering the complexity of the data. I recorded 3 different vocal sessions as training data for the network, trying to get more impressive results out of the network each time. The audio is 8-bit and a low sample rate because sound files get very big very quickly, making the training of the network take a very long time. Well over 300 hours of training in total went into the experiments with my voice that led to this video.
The graphs are created from log files made during training, and show the progress that it was making leading up to immediately before the audio that you hear at every point in the video. Their scrolling speeds up at points where I only show a short sample of the sound, because I wanted to dedicated more time to the more impressive parts. I included a lot of information in the video itself where it's relevant (and at the end), especially details about each of the 3 neural networks at the beginning of each of the 3 sections, so please be sure to check that if you'd like more details.
I'm less happy with the results this time around than in my last RNN+voice video (https://www.youtube.com/watch?v=FsVSZpoUdSU), because I've experimented much less with my own voice than I have with higher-pitched voices from various games and haven't found the ideal combination of settings yet. That's because I don't really want to hear the sound of my own voice, but so many people commented on my old video that they wanted to hear a neural network trained on a male English voice, so here we are now! Also, learning from a low-pitched voice is not as easy as with a high-pitched voice, for reasons explained in the first part of the video (basically, the most fundamental patterns are longer with a low-pitched voice).
The neural network software is the open-source "torch-rnn" (https://github.com/jcjohnson/torch-rnn/), although that is only designed to learn from plain text. Frankly, I'm still amazed at what a good job it does of learning from raw audio, with many overlapping patterns over longer timeframes than text. I made a program(*) that substitutes raw bytes in any file (e.g. audio) for valid UTF-8 text characters and torch-rnn happily learned from it. My program also substituted torch-rnn's generated text back into raw bytes to get audio again. I do not understand the mathematics and low-level algorithms that go make a neural network work, and I cannot program my own, so please check the code and .md files at torch-rnn's Github page for details. Also, torch-rnn is actually a more-efficient fork of an earlier software called char-rnn (https://github.com/karpathy/char-rnn), whose project page also has a lot of useful information.
I will probably soon release the program that I wrote to create the line graphs from CSV files. It can make images up to 16383 pixels wide/tall with customisable colours, from CSV files with hundreds of thousands of lines, in a few seconds. All free software I could find failed hideously at this (e.g. OpenOffice Calc took over a minute to refresh the screen with only a fraction of that many lines, during which time it stopped responding; the lines overlapped in an ugly way that meant you couldn't even see the average value; and "exporting" graphs is limited to pressing Print Screen, so you're limited to the width of your screen... really?).
(*)Here is the code rewritten from VB6 in a C++-like pseudocode:
http://robbi-985.homeip.net/information/bintoutf8_pseudo.txt
Also, here is an English explanation of the idea behind how it works:
http://robbi-985.homeip.net/information/bintoutf8_info.txt
EDIT: I have released my BinToUTF8 program to the public! Please have a look here:
http://robbi-985.homeip.net/blog/?p=1845

What's actually happening to a neural network as it learns?
Next video: https://youtu.be/tIeHLnjs5U8
Training data generation: http://3b1b.co/crowdflower
Find the full playlist at http://3b1b.co/neural-networks
The following video is sort of an appendix to this one. The main goal with the follow-on video is to show the connection between the visual walkthrough here, and the representation of these "nudges" in terms of partial derivatives that you will find when reading about backpropagation in other resources, like Michael Nielsen's book or Chis Olah's blog.
Thanks to everyone supporting on Patreon.
http://3b1b.co/nn3-thanks
http://3b1b.co/support
For more on backpropagation:
http://neuralnetworksanddeeplearning.com/chap2.html
https://github.com/mnielsen/neural-networks-and-deep-learning
http://colah.github.io/posts/2015-08-Backprop/
Music by Vincent Rubinetti:
https://vincerubinetti.bandcamp.com/album/the-music-of-3blue1brown
------------------
3blue1brown is a channel about animating math, in all senses of the word animate. And you know the drill with YouTube, if you want to stay posted on new videos, subscribe, and click the bell to receive notifications (if you're into that).
If you are new to this channel and want to see more, a good place to start is this playlist: http://3b1b.co/recommended
Various social media stuffs:
Website: https://www.3blue1brown.com
Twitter: https://twitter.com/3Blue1Brown
Patreon: https://patreon.com/3blue1brown
Facebook: https://www.facebook.com/3blue1brown
Reddit: https://www.reddit.com/r/3Blue1Brown

So I made a program that trains snake AIs with a genetic algorithm (neuroevolution).
Code can be found here: https://github.com/emgoz/Neural-network-snake
Coding Rainbow: Genetic algorthms: https://www.youtube.com/watch?v=9zfeTw-uFCw
Computerphile: Neural Networks:
https://www.youtube.com/watch?v=py5byOOHZM8
Sorry for the offset video, somehow my screen capture software messed it up.
I wrote the whole thing in Java. Don't get angry, I know it's not the best in performance.

For example, a neural network for handwriting recognition is defined by a set of input neurons which may be activated by the pixels of an input image. After being weighted and transformed by a function (determined by the network's designer), the activations of these neurons are then passed on to other neurons. This process is repeated until finally, an output neuron is activated. This determines which character was read.

An algorithm is an effective method that can be expressed within a finite amount of space and time and in a well-defined formal language for calculating a function. Starting from an initial state and initial input (perhaps empty), the instructions describe a computation that, when executed, proceeds through a finite number of well-defined successive states, eventually producing "output" and terminating at a final ending state. The transition from one state to the next is not necessarily deterministic; some algorithms, known as randomized algorithms, incorporate random input.

In relation to logic-based and artificial neural network-based clinical decision support system, which are also computer applications to the medical decision making field, algorithms are less complex in architecture, data structure and user interface. Medical algorithms are not necessarily implemented using digital computers. In fact, many of them can be represented on paper, in the form of diagrams, nomographs, etc.

But what *is* a Neural Network? | Deep learning, chapter 1

Subscribe to stay notified about new videos: http://3b1b.co/subscribe
Support more videos like this on Patreon: https://www.patreon.com/3blue1brown
Or don't. It's your call really, no pressure.
Special thanks to these supporters: http://3b1b.co/nn1-thanks
Additional funding provided by Amplify Partners. For any early-stage ML entrepreneurs, Amplify would love to hear from you: 3blue1brown@amplifypartners.com
Full playlist: http://3b1b.co/neural-networks
Typo correction: At 14:45, the last index on the bias vector is n, when it's supposed to in fact be a k. Thanks for the sharp eyes that caught that!
For those who want to learn more, I highly recommend the book by Michael Nielsen introducing neural networks and deep learning: https://goo.gl/Zmczdy
There are two neat things about this book. First, it's available for free, so consider joining me in making a donation Nielsen's way if you get something out of it. And second, it's centered around walking through some code and data which you can download yourself, and which covers the same example that I introduce in this video. Yay for active learning!
https://github.com/mnielsen/neural-networks-and-deep-learning
I also highly recommend Chris Olah's blog: http://colah.github.io/
For more videos, Welch Labs also has some great series on machine learning:
https://youtu.be/i8D90DkCLhI
https://youtu.be/bxe2T-V8XRs
For those of you looking to go *even* deeper, check out the text "Deep Learning" by Goodfellow, Bengio, and Courville.
Also, the publication Distill is just utterly beautiful: https://distill.pub/
Lion photo by Kevin Pluck
Music by Vincent Rubinetti:
https://vincerubinetti.bandcamp.com/album/the-music-of-3blue1brown
------------------
3blue1brown is a channel about animating math, in all senses of the word animate. And you know the drill with YouTube, if you want to stay posted on new videos, subscribe, and click the bell to receive notifications (if you're into that).
If you are new to this channel and want to see more, a good place to start is this playlist: http://3b1b.co/recommended
Various social media stuffs:
Website: https://www.3blue1brown.com
Twitter: https://twitter.com/3Blue1Brown
Patreon: https://patreon.com/3blue1brown
Facebook: https://www.facebook.com/3blue1brown
Reddit: https://www.reddit.com/r/3Blue1Brown

3:29

Evolution of Neural Networks using Genetic Algorithm for a 3D car made in Unity

Evolution of Neural Networks using Genetic Algorithm for a 3D car made in Unity

Evolution of Neural Networks using Genetic Algorithm for a 3D car made in Unity

Just a simple implementation of neural net for evolution of a car to finish the track. The neural network itself doesn't evolve in shape, but in the neuron connection weights.
Made inUnityMusic: "Righteous" by Silent Partner

Read the complete tutorial about how to implement a machine learning algorithm for the Flappy Bird video game here:
http://www.askforgametask.com/tutorial/machine-learning-algorithm-flappy-bird
This video shows a creation of an artificial intelligence controller for the Flappy Bird game using neural networks and a genetic algorithm.
The program is written in HTML5 using Phaser framework (http://phaser.io/). There is also used Synaptic Neural Network library (https://synaptic.juancazala.com/) to implement entire artificial neural network instead of making a new one from the scratch.
Download source code here:
https://github.com/ssusnic/Machine-Learning-Flappy-Bird
-----------------------------------------------------------------
According to Arthur Samuel, machine learning is the science of getting computers to act without being explicitly programmed. It is a fine tuning process of learning that incrementally improves an initial random system.
The form of machine learning implemented in this program uses a genetic algorithm to train artificial neural networks.
The birds are learning how to flap optimally in order to fly safely through barriers as long as possible.
The main concept is based on these 3 steps:
1. creating the initial population of 10 birds randomly
2. learning as the game is being played
3. applying natural evolution to form the next improved population
-----------------------------------------------------------------
To play the game, each bird has its own neural network consisted of the next 3 layers:
1. an input layer with 2 neurons representing what a bird sees:
- horizontal distance to the closest gap
- height difference to the closest gap
2. a hidden layer with 6 neurons
3. an output layer with 1 neuron to perform flap if its value is greater than 0.5
-----------------------------------------------------------------
To measure the quality of birds and select the best ones, for each bird is calculated its fitness function in this way:
- reward a bird by its total traveled distance
- penalize a bird by its current distance to the closest gap
When the entire population is dead, the fittest four birds are selected to breed a new population by using genetic algorithm operators: selection, crossover and mutation.
-----------------------------------------------------------------
Visit us:
http://www.askforgametask.com
Like us:
https://www.facebook.com/askforgametask
Follow us:
https://twitter.com/ssusnic
Music:
"BedtimeTune" by Jay Man
http://www.ourmusicbox.com

13:41

Neural Network Tries to Generate English Speech (RNN/LSTM)

Neural Network Tries to Generate English Speech (RNN/LSTM)

Neural Network Tries to Generate English Speech (RNN/LSTM)

By popular demand, I threw my own voice into a neural network (3 times) and got it to recreate what it had learned along the way!
This is 3 different recurrent neural networks (LSTM type) trying to find patterns in raw audio and reproduce them as well as they can. The networks are quite small considering the complexity of the data. I recorded 3 different vocal sessions as training data for the network, trying to get more impressive results out of the network each time. The audio is 8-bit and a low sample rate because sound files get very big very quickly, making the training of the network take a very long time. Well over 300 hours of training in total went into the experiments with my voice that led to this video.
The graphs are created from log files made during training, and show the progress that it was making leading up to immediately before the audio that you hear at every point in the video. Their scrolling speeds up at points where I only show a short sample of the sound, because I wanted to dedicated more time to the more impressive parts. I included a lot of information in the video itself where it's relevant (and at the end), especially details about each of the 3 neural networks at the beginning of each of the 3 sections, so please be sure to check that if you'd like more details.
I'm less happy with the results this time around than in my last RNN+voice video (https://www.youtube.com/watch?v=FsVSZpoUdSU), because I've experimented much less with my own voice than I have with higher-pitched voices from various games and haven't found the ideal combination of settings yet. That's because I don't really want to hear the sound of my own voice, but so many people commented on my old video that they wanted to hear a neural network trained on a male English voice, so here we are now! Also, learning from a low-pitched voice is not as easy as with a high-pitched voice, for reasons explained in the first part of the video (basically, the most fundamental patterns are longer with a low-pitched voice).
The neural network software is the open-source "torch-rnn" (https://github.com/jcjohnson/torch-rnn/), although that is only designed to learn from plain text. Frankly, I'm still amazed at what a good job it does of learning from raw audio, with many overlapping patterns over longer timeframes than text. I made a program(*) that substitutes raw bytes in any file (e.g. audio) for valid UTF-8 text characters and torch-rnn happily learned from it. My program also substituted torch-rnn's generated text back into raw bytes to get audio again. I do not understand the mathematics and low-level algorithms that go make a neural network work, and I cannot program my own, so please check the code and .md files at torch-rnn's Github page for details. Also, torch-rnn is actually a more-efficient fork of an earlier software called char-rnn (https://github.com/karpathy/char-rnn), whose project page also has a lot of useful information.
I will probably soon release the program that I wrote to create the line graphs from CSV files. It can make images up to 16383 pixels wide/tall with customisable colours, from CSV files with hundreds of thousands of lines, in a few seconds. All free software I could find failed hideously at this (e.g. OpenOffice Calc took over a minute to refresh the screen with only a fraction of that many lines, during which time it stopped responding; the lines overlapped in an ugly way that meant you couldn't even see the average value; and "exporting" graphs is limited to pressing Print Screen, so you're limited to the width of your screen... really?).
(*)Here is the code rewritten from VB6 in a C++-like pseudocode:
http://robbi-985.homeip.net/information/bintoutf8_pseudo.txt
Also, here is an English explanation of the idea behind how it works:
http://robbi-985.homeip.net/information/bintoutf8_info.txt
EDIT: I have released my BinToUTF8 program to the public! Please have a look here:
http://robbi-985.homeip.net/blog/?p=1845

What is backpropagation really doing? | Deep learning, chapter 3

What's actually happening to a neural network as it learns?
Next video: https://youtu.be/tIeHLnjs5U8
Training data generation: http://3b1b.co/crowdflower
Find the full playlist at http://3b1b.co/neural-networks
The following video is sort of an appendix to this one. The main goal with the follow-on video is to show the connection between the visual walkthrough here, and the representation of these "nudges" in terms of partial derivatives that you will find when reading about backpropagation in other resources, like Michael Nielsen's book or Chis Olah's blog.
Thanks to everyone supporting on Patreon.
http://3b1b.co/nn3-thanks
http://3b1b.co/support
For more on backpropagation:
http://neuralnetworksanddeeplearning.com/chap2.html
https://github.com/mnielsen/neural-networks-and-deep-learning
http://colah.github.io/posts/2015-08-Backprop/
Music by Vincent Rubinetti:
https://vincerubinetti.bandcamp.com/album/the-music-of-3blue1brown
------------------
3blue1brown is a channel about animating math, in all senses of the word animate. And you know the drill with YouTube, if you want to stay posted on new videos, subscribe, and click the bell to receive notifications (if you're into that).
If you are new to this channel and want to see more, a good place to start is this playlist: http://3b1b.co/recommended
Various social media stuffs:
Website: https://www.3blue1brown.com
Twitter: https://twitter.com/3Blue1Brown
Patreon: https://patreon.com/3blue1brown
Facebook: https://www.facebook.com/3blue1brown
Reddit: https://www.reddit.com/r/3Blue1Brown

Snakes, Neural Networks and Genetic Algorithms

So I made a program that trains snake AIs with a genetic algorithm (neuroevolution).
Code can be found here: https://github.com/emgoz/Neural-network-snake
Coding Rainbow: Genetic algorthms: https://www.youtube.com/watch?v=9zfeTw-uFCw
Computerphile: Neural Networks:
https://www.youtube.com/watch?v=py5byOOHZM8
Sorry for the offset video, somehow my screen capture software messed it up.
I wrote the whole thing in Java. Don't get angry, I know it's not the best in performance.

Deep Neural Network Learns Van Gogh's Art | Two Minute Papers #6

Artificial neural networks were inspired by the human brain and simulate how neurons behave when they are shown a sensory input (e.g., images, sounds, etc). They are known to be excellent tools for image recognition, any many other problems beyond that - they also excel at weather predictions, breast cancer cell mitosis detection, brain image segmentation and toxicity prediction among many others. Deep learning means that we use an artificial neural network with multiple layers, making it even more powerful for more difficult tasks.
This time they have been shown to be apt at reproducing the artistic style of many famous painters, such as Vincent Van Gogh and Pablo Picasso among many others. All the user needs to do is provide an input photograph and a target image from which the artistic style will be learned.
______________________
I promised some links, so here they come!
The paper "A Neural Algorithm of Artistic Style" is available here:
http://arxiv.org/abs/1508.06576v1
Disclaimer: I was not part of this research project, I am merely providing commentary on this work.
Recommended for you - Two MinutePapers episode on Artificial Neural Networks:
https://www.youtube.com/watch?v=rCWTOOgVXyE&index=3&list=PLujxSBD-JXgnqDD1n-V30pKtp6Q886x7e
Picasso meets Gandalf:
http://mashable.com/2015/08/29/computer-photos/
A nice website with many results:
https://deepart.io/
More examples with Picasso and some sketches:
http://imgur.com/a/jeJB6
Google DeepMind's Deep Q-learning algorithm plays Atari games:
https://www.youtube.com/watch?v=V1eYniJ0Rnk
The first implementations / source code packages are now available:
1. http://gitxiv.com/posts/jG46ukGod8R7Rdtud/a-neural-algorithm-of-artistic-style
2. https://github.com/kaishengtai/neuralart
3. https://github.com/jcjohnson/neural-style
A great read on Deep DreamingNeural Networks:
http://googleresearch.blogspot.co.uk/2015/06/inceptionism-going-deeper-into-neural.html
Many of you have asked for the code. Some people were experimenting with it in the Machine Learning reddit. Check it out:
https://www.reddit.com/r/MachineLearning/comments/3imx1m/a_neural_algorithm_of_artistic_style/
Subscribe if you would like to see more of these! - http://www.youtube.com/subscription_center?add_user=keeroyz
Splash screen/thumbnail design: Felícia Fehér - http://felicia.hu
Music:
Epilog - Ghostpocalypse by Kevin MacLeod is licensed under a Creative Commons Attribution license (https://creativecommons.org/licenses/by/4.0/)
Source: http://incompetech.com/music/royalty-free/index.html?isrc=USUAN1100666
Artist: http://incompetech.com/
______________________
Károly Zsolnai-Fehér's links:
Patreon → https://www.patreon.com/TwoMinutePapers
Facebook → https://www.facebook.com/TwoMinutePapers/
Twitter → https://twitter.com/karoly_zsolnai
Web → https://cg.tuwien.ac.at/~zsolnai/

12:07

Neural Networks Explained - Machine Learning Tutorial for Beginners

Neural Networks Explained - Machine Learning Tutorial for Beginners

Neural Networks Explained - Machine Learning Tutorial for Beginners

If you know nothing about how a neural network works, this is the video for you! I've worked for weeks to find ways to explain this in a way that is easy to understand for beginners.
Past Videos:
Intro to Machine Learning with Javascript:
https://www.youtube.com/watch?v=9Hz3P1VgLz4&list=PLoYCgNOIyGABWLy_XoLSxTVRe2bltV8GM&index=2&t=0s
Machine Learning 2 - Building a Recommendation Engine:
https://www.youtube.com/watch?v=lvzekeBQsSo&list=PLoYCgNOIyGABWLy_XoLSxTVRe2bltV8GM&index=3&t=0s
Machine learning and neural networks are awesome. This video provides beginners with an easy tutorial explaining how a neural network works - what math is involved, and a step by step explanation of how the data moves through the network.
The example used will be a feed forward neural network with back propagation. It explains the difference between linear and non linear data, the importance of the activation function, learning rate, and momentum configurations.
-~-~~-~~~-~~-~-
LearningWeb Development? Watch the FREE COURSE:
"Web Development for Absolute Beginners"!
https://www.youtube.com/watch?v=gQojMIhELvM
-~-~~-~~~-~~-~-

But what *is* a Neural Network? | Deep learning, chapter 1

Subscribe to stay notified about new videos: http://3b1b.co/subscribe
Support more videos like this on Patreon: https://www.patreon.com/3blue1brown
Or don't. It's your call really, no pressure.
Special thanks to these supporters: http://3b1b.co/nn1-thanks
Additional funding provided by Amplify Partners. For any early-stage ML entrepreneurs, Amplify would love to hear from you: 3blue1brown@amplifypartners.com
Full playlist: http://3b1b.co/neural-networks
Typo correction: At 14:45, the last index on the bias vector is n, when it's supposed to in fact be a k. Thanks for the sharp eyes that caught that!
For those who want to learn more, I highly recommend the book by Michael Nielsen introducing neural networks and deep learning: https://goo.gl/Zmczdy
There are two neat things about thi...

published: 05 Oct 2017

Evolution of Neural Networks using Genetic Algorithm for a 3D car made in Unity

Just a simple implementation of neural net for evolution of a car to finish the track. The neural network itself doesn't evolve in shape, but in the neuron connection weights.
Made inUnityMusic: "Righteous" by Silent Partner

Read the complete tutorial about how to implement a machine learning algorithm for the Flappy Bird video game here:
http://www.askforgametask.com/tutorial/machine-learning-algorithm-flappy-bird
This video shows a creation of an artificial intelligence controller for the Flappy Bird game using neural networks and a genetic algorithm.
The program is written in HTML5 using Phaser framework (http://phaser.io/). There is also used Synaptic Neural Network library (https://synaptic.juancazala.com/) to implement entire artificial neural network instead of making a new one from the scratch.
Download source code here:
https://github.com/ssusnic/Machine-Learning-Flappy-Bird
-----------------------------------------------------------------
According to Arthur Samuel, machine learning is the sc...

published: 10 Aug 2017

Neural Network Tries to Generate English Speech (RNN/LSTM)

By popular demand, I threw my own voice into a neural network (3 times) and got it to recreate what it had learned along the way!
This is 3 different recurrent neural networks (LSTM type) trying to find patterns in raw audio and reproduce them as well as they can. The networks are quite small considering the complexity of the data. I recorded 3 different vocal sessions as training data for the network, trying to get more impressive results out of the network each time. The audio is 8-bit and a low sample rate because sound files get very big very quickly, making the training of the network take a very long time. Well over 300 hours of training in total went into the experiments with my voice that led to this video.
The graphs are created from log files made during training, and show the...

What is backpropagation really doing? | Deep learning, chapter 3

What's actually happening to a neural network as it learns?
Next video: https://youtu.be/tIeHLnjs5U8
Training data generation: http://3b1b.co/crowdflower
Find the full playlist at http://3b1b.co/neural-networks
The following video is sort of an appendix to this one. The main goal with the follow-on video is to show the connection between the visual walkthrough here, and the representation of these "nudges" in terms of partial derivatives that you will find when reading about backpropagation in other resources, like Michael Nielsen's book or Chis Olah's blog.
Thanks to everyone supporting on Patreon.
http://3b1b.co/nn3-thanks
http://3b1b.co/support
For more on backpropagation:
http://neuralnetworksanddeeplearning.com/chap2.html
https://github.com/mnielsen/neural-networks-and-deep-learni...

Snakes, Neural Networks and Genetic Algorithms

So I made a program that trains snake AIs with a genetic algorithm (neuroevolution).
Code can be found here: https://github.com/emgoz/Neural-network-snake
Coding Rainbow: Genetic algorthms: https://www.youtube.com/watch?v=9zfeTw-uFCw
Computerphile: Neural Networks:
https://www.youtube.com/watch?v=py5byOOHZM8
Sorry for the offset video, somehow my screen capture software messed it up.
I wrote the whole thing in Java. Don't get angry, I know it's not the best in performance.

Deep Neural Network Learns Van Gogh's Art | Two Minute Papers #6

Artificial neural networks were inspired by the human brain and simulate how neurons behave when they are shown a sensory input (e.g., images, sounds, etc). They are known to be excellent tools for image recognition, any many other problems beyond that - they also excel at weather predictions, breast cancer cell mitosis detection, brain image segmentation and toxicity prediction among many others. Deep learning means that we use an artificial neural network with multiple layers, making it even more powerful for more difficult tasks.
This time they have been shown to be apt at reproducing the artistic style of many famous painters, such as Vincent Van Gogh and Pablo Picasso among many others. All the user needs to do is provide an input photograph and a target image from which the artisti...

published: 29 Aug 2015

Neural Networks Explained - Machine Learning Tutorial for Beginners

If you know nothing about how a neural network works, this is the video for you! I've worked for weeks to find ways to explain this in a way that is easy to understand for beginners.
Past Videos:
Intro to Machine Learning with Javascript:
https://www.youtube.com/watch?v=9Hz3P1VgLz4&list=PLoYCgNOIyGABWLy_XoLSxTVRe2bltV8GM&index=2&t=0s
Machine Learning 2 - Building a Recommendation Engine:
https://www.youtube.com/watch?v=lvzekeBQsSo&list=PLoYCgNOIyGABWLy_XoLSxTVRe2bltV8GM&index=3&t=0s
Machine learning and neural networks are awesome. This video provides beginners with an easy tutorial explaining how a neural network works - what math is involved, and a step by step explanation of how the data moves through the network.
The example used will be a feed forward neural network with back ...

But what *is* a Neural Network? | Deep learning, chapter 1

Subscribe to stay notified about new videos: http://3b1b.co/subscribe
Support more videos like this on Patreon: https://www.patreon.com/3blue1brown
Or don't. I...

Subscribe to stay notified about new videos: http://3b1b.co/subscribe
Support more videos like this on Patreon: https://www.patreon.com/3blue1brown
Or don't. It's your call really, no pressure.
Special thanks to these supporters: http://3b1b.co/nn1-thanks
Additional funding provided by Amplify Partners. For any early-stage ML entrepreneurs, Amplify would love to hear from you: 3blue1brown@amplifypartners.com
Full playlist: http://3b1b.co/neural-networks
Typo correction: At 14:45, the last index on the bias vector is n, when it's supposed to in fact be a k. Thanks for the sharp eyes that caught that!
For those who want to learn more, I highly recommend the book by Michael Nielsen introducing neural networks and deep learning: https://goo.gl/Zmczdy
There are two neat things about this book. First, it's available for free, so consider joining me in making a donation Nielsen's way if you get something out of it. And second, it's centered around walking through some code and data which you can download yourself, and which covers the same example that I introduce in this video. Yay for active learning!
https://github.com/mnielsen/neural-networks-and-deep-learning
I also highly recommend Chris Olah's blog: http://colah.github.io/
For more videos, Welch Labs also has some great series on machine learning:
https://youtu.be/i8D90DkCLhI
https://youtu.be/bxe2T-V8XRs
For those of you looking to go *even* deeper, check out the text "Deep Learning" by Goodfellow, Bengio, and Courville.
Also, the publication Distill is just utterly beautiful: https://distill.pub/
Lion photo by Kevin Pluck
Music by Vincent Rubinetti:
https://vincerubinetti.bandcamp.com/album/the-music-of-3blue1brown
------------------
3blue1brown is a channel about animating math, in all senses of the word animate. And you know the drill with YouTube, if you want to stay posted on new videos, subscribe, and click the bell to receive notifications (if you're into that).
If you are new to this channel and want to see more, a good place to start is this playlist: http://3b1b.co/recommended
Various social media stuffs:
Website: https://www.3blue1brown.com
Twitter: https://twitter.com/3Blue1Brown
Patreon: https://patreon.com/3blue1brown
Facebook: https://www.facebook.com/3blue1brown
Reddit: https://www.reddit.com/r/3Blue1Brown

Subscribe to stay notified about new videos: http://3b1b.co/subscribe
Support more videos like this on Patreon: https://www.patreon.com/3blue1brown
Or don't. It's your call really, no pressure.
Special thanks to these supporters: http://3b1b.co/nn1-thanks
Additional funding provided by Amplify Partners. For any early-stage ML entrepreneurs, Amplify would love to hear from you: 3blue1brown@amplifypartners.com
Full playlist: http://3b1b.co/neural-networks
Typo correction: At 14:45, the last index on the bias vector is n, when it's supposed to in fact be a k. Thanks for the sharp eyes that caught that!
For those who want to learn more, I highly recommend the book by Michael Nielsen introducing neural networks and deep learning: https://goo.gl/Zmczdy
There are two neat things about this book. First, it's available for free, so consider joining me in making a donation Nielsen's way if you get something out of it. And second, it's centered around walking through some code and data which you can download yourself, and which covers the same example that I introduce in this video. Yay for active learning!
https://github.com/mnielsen/neural-networks-and-deep-learning
I also highly recommend Chris Olah's blog: http://colah.github.io/
For more videos, Welch Labs also has some great series on machine learning:
https://youtu.be/i8D90DkCLhI
https://youtu.be/bxe2T-V8XRs
For those of you looking to go *even* deeper, check out the text "Deep Learning" by Goodfellow, Bengio, and Courville.
Also, the publication Distill is just utterly beautiful: https://distill.pub/
Lion photo by Kevin Pluck
Music by Vincent Rubinetti:
https://vincerubinetti.bandcamp.com/album/the-music-of-3blue1brown
------------------
3blue1brown is a channel about animating math, in all senses of the word animate. And you know the drill with YouTube, if you want to stay posted on new videos, subscribe, and click the bell to receive notifications (if you're into that).
If you are new to this channel and want to see more, a good place to start is this playlist: http://3b1b.co/recommended
Various social media stuffs:
Website: https://www.3blue1brown.com
Twitter: https://twitter.com/3Blue1Brown
Patreon: https://patreon.com/3blue1brown
Facebook: https://www.facebook.com/3blue1brown
Reddit: https://www.reddit.com/r/3Blue1Brown

Evolution of Neural Networks using Genetic Algorithm for a 3D car made in Unity

Just a simple implementation of neural net for evolution of a car to finish the track. The neural network itself doesn't evolve in shape, but in the neuron conn...

Just a simple implementation of neural net for evolution of a car to finish the track. The neural network itself doesn't evolve in shape, but in the neuron connection weights.
Made inUnityMusic: "Righteous" by Silent Partner

Just a simple implementation of neural net for evolution of a car to finish the track. The neural network itself doesn't evolve in shape, but in the neuron connection weights.
Made inUnityMusic: "Righteous" by Silent Partner

Read the complete tutorial about how to implement a machine learning algorithm for the Flappy Bird video game here:
http://www.askforgametask.com/tutorial/machi...

Read the complete tutorial about how to implement a machine learning algorithm for the Flappy Bird video game here:
http://www.askforgametask.com/tutorial/machine-learning-algorithm-flappy-bird
This video shows a creation of an artificial intelligence controller for the Flappy Bird game using neural networks and a genetic algorithm.
The program is written in HTML5 using Phaser framework (http://phaser.io/). There is also used Synaptic Neural Network library (https://synaptic.juancazala.com/) to implement entire artificial neural network instead of making a new one from the scratch.
Download source code here:
https://github.com/ssusnic/Machine-Learning-Flappy-Bird
-----------------------------------------------------------------
According to Arthur Samuel, machine learning is the science of getting computers to act without being explicitly programmed. It is a fine tuning process of learning that incrementally improves an initial random system.
The form of machine learning implemented in this program uses a genetic algorithm to train artificial neural networks.
The birds are learning how to flap optimally in order to fly safely through barriers as long as possible.
The main concept is based on these 3 steps:
1. creating the initial population of 10 birds randomly
2. learning as the game is being played
3. applying natural evolution to form the next improved population
-----------------------------------------------------------------
To play the game, each bird has its own neural network consisted of the next 3 layers:
1. an input layer with 2 neurons representing what a bird sees:
- horizontal distance to the closest gap
- height difference to the closest gap
2. a hidden layer with 6 neurons
3. an output layer with 1 neuron to perform flap if its value is greater than 0.5
-----------------------------------------------------------------
To measure the quality of birds and select the best ones, for each bird is calculated its fitness function in this way:
- reward a bird by its total traveled distance
- penalize a bird by its current distance to the closest gap
When the entire population is dead, the fittest four birds are selected to breed a new population by using genetic algorithm operators: selection, crossover and mutation.
-----------------------------------------------------------------
Visit us:
http://www.askforgametask.com
Like us:
https://www.facebook.com/askforgametask
Follow us:
https://twitter.com/ssusnic
Music:
"BedtimeTune" by Jay Man
http://www.ourmusicbox.com

Read the complete tutorial about how to implement a machine learning algorithm for the Flappy Bird video game here:
http://www.askforgametask.com/tutorial/machine-learning-algorithm-flappy-bird
This video shows a creation of an artificial intelligence controller for the Flappy Bird game using neural networks and a genetic algorithm.
The program is written in HTML5 using Phaser framework (http://phaser.io/). There is also used Synaptic Neural Network library (https://synaptic.juancazala.com/) to implement entire artificial neural network instead of making a new one from the scratch.
Download source code here:
https://github.com/ssusnic/Machine-Learning-Flappy-Bird
-----------------------------------------------------------------
According to Arthur Samuel, machine learning is the science of getting computers to act without being explicitly programmed. It is a fine tuning process of learning that incrementally improves an initial random system.
The form of machine learning implemented in this program uses a genetic algorithm to train artificial neural networks.
The birds are learning how to flap optimally in order to fly safely through barriers as long as possible.
The main concept is based on these 3 steps:
1. creating the initial population of 10 birds randomly
2. learning as the game is being played
3. applying natural evolution to form the next improved population
-----------------------------------------------------------------
To play the game, each bird has its own neural network consisted of the next 3 layers:
1. an input layer with 2 neurons representing what a bird sees:
- horizontal distance to the closest gap
- height difference to the closest gap
2. a hidden layer with 6 neurons
3. an output layer with 1 neuron to perform flap if its value is greater than 0.5
-----------------------------------------------------------------
To measure the quality of birds and select the best ones, for each bird is calculated its fitness function in this way:
- reward a bird by its total traveled distance
- penalize a bird by its current distance to the closest gap
When the entire population is dead, the fittest four birds are selected to breed a new population by using genetic algorithm operators: selection, crossover and mutation.
-----------------------------------------------------------------
Visit us:
http://www.askforgametask.com
Like us:
https://www.facebook.com/askforgametask
Follow us:
https://twitter.com/ssusnic
Music:
"BedtimeTune" by Jay Man
http://www.ourmusicbox.com

Neural Network Tries to Generate English Speech (RNN/LSTM)

By popular demand, I threw my own voice into a neural network (3 times) and got it to recreate what it had learned along the way!
This is 3 different recurren...

By popular demand, I threw my own voice into a neural network (3 times) and got it to recreate what it had learned along the way!
This is 3 different recurrent neural networks (LSTM type) trying to find patterns in raw audio and reproduce them as well as they can. The networks are quite small considering the complexity of the data. I recorded 3 different vocal sessions as training data for the network, trying to get more impressive results out of the network each time. The audio is 8-bit and a low sample rate because sound files get very big very quickly, making the training of the network take a very long time. Well over 300 hours of training in total went into the experiments with my voice that led to this video.
The graphs are created from log files made during training, and show the progress that it was making leading up to immediately before the audio that you hear at every point in the video. Their scrolling speeds up at points where I only show a short sample of the sound, because I wanted to dedicated more time to the more impressive parts. I included a lot of information in the video itself where it's relevant (and at the end), especially details about each of the 3 neural networks at the beginning of each of the 3 sections, so please be sure to check that if you'd like more details.
I'm less happy with the results this time around than in my last RNN+voice video (https://www.youtube.com/watch?v=FsVSZpoUdSU), because I've experimented much less with my own voice than I have with higher-pitched voices from various games and haven't found the ideal combination of settings yet. That's because I don't really want to hear the sound of my own voice, but so many people commented on my old video that they wanted to hear a neural network trained on a male English voice, so here we are now! Also, learning from a low-pitched voice is not as easy as with a high-pitched voice, for reasons explained in the first part of the video (basically, the most fundamental patterns are longer with a low-pitched voice).
The neural network software is the open-source "torch-rnn" (https://github.com/jcjohnson/torch-rnn/), although that is only designed to learn from plain text. Frankly, I'm still amazed at what a good job it does of learning from raw audio, with many overlapping patterns over longer timeframes than text. I made a program(*) that substitutes raw bytes in any file (e.g. audio) for valid UTF-8 text characters and torch-rnn happily learned from it. My program also substituted torch-rnn's generated text back into raw bytes to get audio again. I do not understand the mathematics and low-level algorithms that go make a neural network work, and I cannot program my own, so please check the code and .md files at torch-rnn's Github page for details. Also, torch-rnn is actually a more-efficient fork of an earlier software called char-rnn (https://github.com/karpathy/char-rnn), whose project page also has a lot of useful information.
I will probably soon release the program that I wrote to create the line graphs from CSV files. It can make images up to 16383 pixels wide/tall with customisable colours, from CSV files with hundreds of thousands of lines, in a few seconds. All free software I could find failed hideously at this (e.g. OpenOffice Calc took over a minute to refresh the screen with only a fraction of that many lines, during which time it stopped responding; the lines overlapped in an ugly way that meant you couldn't even see the average value; and "exporting" graphs is limited to pressing Print Screen, so you're limited to the width of your screen... really?).
(*)Here is the code rewritten from VB6 in a C++-like pseudocode:
http://robbi-985.homeip.net/information/bintoutf8_pseudo.txt
Also, here is an English explanation of the idea behind how it works:
http://robbi-985.homeip.net/information/bintoutf8_info.txt
EDIT: I have released my BinToUTF8 program to the public! Please have a look here:
http://robbi-985.homeip.net/blog/?p=1845

By popular demand, I threw my own voice into a neural network (3 times) and got it to recreate what it had learned along the way!
This is 3 different recurrent neural networks (LSTM type) trying to find patterns in raw audio and reproduce them as well as they can. The networks are quite small considering the complexity of the data. I recorded 3 different vocal sessions as training data for the network, trying to get more impressive results out of the network each time. The audio is 8-bit and a low sample rate because sound files get very big very quickly, making the training of the network take a very long time. Well over 300 hours of training in total went into the experiments with my voice that led to this video.
The graphs are created from log files made during training, and show the progress that it was making leading up to immediately before the audio that you hear at every point in the video. Their scrolling speeds up at points where I only show a short sample of the sound, because I wanted to dedicated more time to the more impressive parts. I included a lot of information in the video itself where it's relevant (and at the end), especially details about each of the 3 neural networks at the beginning of each of the 3 sections, so please be sure to check that if you'd like more details.
I'm less happy with the results this time around than in my last RNN+voice video (https://www.youtube.com/watch?v=FsVSZpoUdSU), because I've experimented much less with my own voice than I have with higher-pitched voices from various games and haven't found the ideal combination of settings yet. That's because I don't really want to hear the sound of my own voice, but so many people commented on my old video that they wanted to hear a neural network trained on a male English voice, so here we are now! Also, learning from a low-pitched voice is not as easy as with a high-pitched voice, for reasons explained in the first part of the video (basically, the most fundamental patterns are longer with a low-pitched voice).
The neural network software is the open-source "torch-rnn" (https://github.com/jcjohnson/torch-rnn/), although that is only designed to learn from plain text. Frankly, I'm still amazed at what a good job it does of learning from raw audio, with many overlapping patterns over longer timeframes than text. I made a program(*) that substitutes raw bytes in any file (e.g. audio) for valid UTF-8 text characters and torch-rnn happily learned from it. My program also substituted torch-rnn's generated text back into raw bytes to get audio again. I do not understand the mathematics and low-level algorithms that go make a neural network work, and I cannot program my own, so please check the code and .md files at torch-rnn's Github page for details. Also, torch-rnn is actually a more-efficient fork of an earlier software called char-rnn (https://github.com/karpathy/char-rnn), whose project page also has a lot of useful information.
I will probably soon release the program that I wrote to create the line graphs from CSV files. It can make images up to 16383 pixels wide/tall with customisable colours, from CSV files with hundreds of thousands of lines, in a few seconds. All free software I could find failed hideously at this (e.g. OpenOffice Calc took over a minute to refresh the screen with only a fraction of that many lines, during which time it stopped responding; the lines overlapped in an ugly way that meant you couldn't even see the average value; and "exporting" graphs is limited to pressing Print Screen, so you're limited to the width of your screen... really?).
(*)Here is the code rewritten from VB6 in a C++-like pseudocode:
http://robbi-985.homeip.net/information/bintoutf8_pseudo.txt
Also, here is an English explanation of the idea behind how it works:
http://robbi-985.homeip.net/information/bintoutf8_info.txt
EDIT: I have released my BinToUTF8 program to the public! Please have a look here:
http://robbi-985.homeip.net/blog/?p=1845

What's actually happening to a neural network as it learns?
Next video: https://youtu.be/tIeHLnjs5U8
Training data generation: http://3b1b.co/crowdflower
Find the full playlist at http://3b1b.co/neural-networks
The following video is sort of an appendix to this one. The main goal with the follow-on video is to show the connection between the visual walkthrough here, and the representation of these "nudges" in terms of partial derivatives that you will find when reading about backpropagation in other resources, like Michael Nielsen's book or Chis Olah's blog.
Thanks to everyone supporting on Patreon.
http://3b1b.co/nn3-thanks
http://3b1b.co/support
For more on backpropagation:
http://neuralnetworksanddeeplearning.com/chap2.html
https://github.com/mnielsen/neural-networks-and-deep-learning
http://colah.github.io/posts/2015-08-Backprop/
Music by Vincent Rubinetti:
https://vincerubinetti.bandcamp.com/album/the-music-of-3blue1brown
------------------
3blue1brown is a channel about animating math, in all senses of the word animate. And you know the drill with YouTube, if you want to stay posted on new videos, subscribe, and click the bell to receive notifications (if you're into that).
If you are new to this channel and want to see more, a good place to start is this playlist: http://3b1b.co/recommended
Various social media stuffs:
Website: https://www.3blue1brown.com
Twitter: https://twitter.com/3Blue1Brown
Patreon: https://patreon.com/3blue1brown
Facebook: https://www.facebook.com/3blue1brown
Reddit: https://www.reddit.com/r/3Blue1Brown

What's actually happening to a neural network as it learns?
Next video: https://youtu.be/tIeHLnjs5U8
Training data generation: http://3b1b.co/crowdflower
Find the full playlist at http://3b1b.co/neural-networks
The following video is sort of an appendix to this one. The main goal with the follow-on video is to show the connection between the visual walkthrough here, and the representation of these "nudges" in terms of partial derivatives that you will find when reading about backpropagation in other resources, like Michael Nielsen's book or Chis Olah's blog.
Thanks to everyone supporting on Patreon.
http://3b1b.co/nn3-thanks
http://3b1b.co/support
For more on backpropagation:
http://neuralnetworksanddeeplearning.com/chap2.html
https://github.com/mnielsen/neural-networks-and-deep-learning
http://colah.github.io/posts/2015-08-Backprop/
Music by Vincent Rubinetti:
https://vincerubinetti.bandcamp.com/album/the-music-of-3blue1brown
------------------
3blue1brown is a channel about animating math, in all senses of the word animate. And you know the drill with YouTube, if you want to stay posted on new videos, subscribe, and click the bell to receive notifications (if you're into that).
If you are new to this channel and want to see more, a good place to start is this playlist: http://3b1b.co/recommended
Various social media stuffs:
Website: https://www.3blue1brown.com
Twitter: https://twitter.com/3Blue1Brown
Patreon: https://patreon.com/3blue1brown
Facebook: https://www.facebook.com/3blue1brown
Reddit: https://www.reddit.com/r/3Blue1Brown

Snakes, Neural Networks and Genetic Algorithms

So I made a program that trains snake AIs with a genetic algorithm (neuroevolution).
Code can be found here: https://github.com/emgoz/Neural-network-snake
Codi...

So I made a program that trains snake AIs with a genetic algorithm (neuroevolution).
Code can be found here: https://github.com/emgoz/Neural-network-snake
Coding Rainbow: Genetic algorthms: https://www.youtube.com/watch?v=9zfeTw-uFCw
Computerphile: Neural Networks:
https://www.youtube.com/watch?v=py5byOOHZM8
Sorry for the offset video, somehow my screen capture software messed it up.
I wrote the whole thing in Java. Don't get angry, I know it's not the best in performance.

So I made a program that trains snake AIs with a genetic algorithm (neuroevolution).
Code can be found here: https://github.com/emgoz/Neural-network-snake
Coding Rainbow: Genetic algorthms: https://www.youtube.com/watch?v=9zfeTw-uFCw
Computerphile: Neural Networks:
https://www.youtube.com/watch?v=py5byOOHZM8
Sorry for the offset video, somehow my screen capture software messed it up.
I wrote the whole thing in Java. Don't get angry, I know it's not the best in performance.

Deep Neural Network Learns Van Gogh's Art | Two Minute Papers #6

Artificial neural networks were inspired by the human brain and simulate how neurons behave when they are shown a sensory input (e.g., images, sounds, etc). The...

Artificial neural networks were inspired by the human brain and simulate how neurons behave when they are shown a sensory input (e.g., images, sounds, etc). They are known to be excellent tools for image recognition, any many other problems beyond that - they also excel at weather predictions, breast cancer cell mitosis detection, brain image segmentation and toxicity prediction among many others. Deep learning means that we use an artificial neural network with multiple layers, making it even more powerful for more difficult tasks.
This time they have been shown to be apt at reproducing the artistic style of many famous painters, such as Vincent Van Gogh and Pablo Picasso among many others. All the user needs to do is provide an input photograph and a target image from which the artistic style will be learned.
______________________
I promised some links, so here they come!
The paper "A Neural Algorithm of Artistic Style" is available here:
http://arxiv.org/abs/1508.06576v1
Disclaimer: I was not part of this research project, I am merely providing commentary on this work.
Recommended for you - Two MinutePapers episode on Artificial Neural Networks:
https://www.youtube.com/watch?v=rCWTOOgVXyE&index=3&list=PLujxSBD-JXgnqDD1n-V30pKtp6Q886x7e
Picasso meets Gandalf:
http://mashable.com/2015/08/29/computer-photos/
A nice website with many results:
https://deepart.io/
More examples with Picasso and some sketches:
http://imgur.com/a/jeJB6
Google DeepMind's Deep Q-learning algorithm plays Atari games:
https://www.youtube.com/watch?v=V1eYniJ0Rnk
The first implementations / source code packages are now available:
1. http://gitxiv.com/posts/jG46ukGod8R7Rdtud/a-neural-algorithm-of-artistic-style
2. https://github.com/kaishengtai/neuralart
3. https://github.com/jcjohnson/neural-style
A great read on Deep DreamingNeural Networks:
http://googleresearch.blogspot.co.uk/2015/06/inceptionism-going-deeper-into-neural.html
Many of you have asked for the code. Some people were experimenting with it in the Machine Learning reddit. Check it out:
https://www.reddit.com/r/MachineLearning/comments/3imx1m/a_neural_algorithm_of_artistic_style/
Subscribe if you would like to see more of these! - http://www.youtube.com/subscription_center?add_user=keeroyz
Splash screen/thumbnail design: Felícia Fehér - http://felicia.hu
Music:
Epilog - Ghostpocalypse by Kevin MacLeod is licensed under a Creative Commons Attribution license (https://creativecommons.org/licenses/by/4.0/)
Source: http://incompetech.com/music/royalty-free/index.html?isrc=USUAN1100666
Artist: http://incompetech.com/
______________________
Károly Zsolnai-Fehér's links:
Patreon → https://www.patreon.com/TwoMinutePapers
Facebook → https://www.facebook.com/TwoMinutePapers/
Twitter → https://twitter.com/karoly_zsolnai
Web → https://cg.tuwien.ac.at/~zsolnai/

Artificial neural networks were inspired by the human brain and simulate how neurons behave when they are shown a sensory input (e.g., images, sounds, etc). They are known to be excellent tools for image recognition, any many other problems beyond that - they also excel at weather predictions, breast cancer cell mitosis detection, brain image segmentation and toxicity prediction among many others. Deep learning means that we use an artificial neural network with multiple layers, making it even more powerful for more difficult tasks.
This time they have been shown to be apt at reproducing the artistic style of many famous painters, such as Vincent Van Gogh and Pablo Picasso among many others. All the user needs to do is provide an input photograph and a target image from which the artistic style will be learned.
______________________
I promised some links, so here they come!
The paper "A Neural Algorithm of Artistic Style" is available here:
http://arxiv.org/abs/1508.06576v1
Disclaimer: I was not part of this research project, I am merely providing commentary on this work.
Recommended for you - Two MinutePapers episode on Artificial Neural Networks:
https://www.youtube.com/watch?v=rCWTOOgVXyE&index=3&list=PLujxSBD-JXgnqDD1n-V30pKtp6Q886x7e
Picasso meets Gandalf:
http://mashable.com/2015/08/29/computer-photos/
A nice website with many results:
https://deepart.io/
More examples with Picasso and some sketches:
http://imgur.com/a/jeJB6
Google DeepMind's Deep Q-learning algorithm plays Atari games:
https://www.youtube.com/watch?v=V1eYniJ0Rnk
The first implementations / source code packages are now available:
1. http://gitxiv.com/posts/jG46ukGod8R7Rdtud/a-neural-algorithm-of-artistic-style
2. https://github.com/kaishengtai/neuralart
3. https://github.com/jcjohnson/neural-style
A great read on Deep DreamingNeural Networks:
http://googleresearch.blogspot.co.uk/2015/06/inceptionism-going-deeper-into-neural.html
Many of you have asked for the code. Some people were experimenting with it in the Machine Learning reddit. Check it out:
https://www.reddit.com/r/MachineLearning/comments/3imx1m/a_neural_algorithm_of_artistic_style/
Subscribe if you would like to see more of these! - http://www.youtube.com/subscription_center?add_user=keeroyz
Splash screen/thumbnail design: Felícia Fehér - http://felicia.hu
Music:
Epilog - Ghostpocalypse by Kevin MacLeod is licensed under a Creative Commons Attribution license (https://creativecommons.org/licenses/by/4.0/)
Source: http://incompetech.com/music/royalty-free/index.html?isrc=USUAN1100666
Artist: http://incompetech.com/
______________________
Károly Zsolnai-Fehér's links:
Patreon → https://www.patreon.com/TwoMinutePapers
Facebook → https://www.facebook.com/TwoMinutePapers/
Twitter → https://twitter.com/karoly_zsolnai
Web → https://cg.tuwien.ac.at/~zsolnai/

Neural Networks Explained - Machine Learning Tutorial for Beginners

If you know nothing about how a neural network works, this is the video for you! I've worked for weeks to find ways to explain this in a way that is easy to un...

If you know nothing about how a neural network works, this is the video for you! I've worked for weeks to find ways to explain this in a way that is easy to understand for beginners.
Past Videos:
Intro to Machine Learning with Javascript:
https://www.youtube.com/watch?v=9Hz3P1VgLz4&list=PLoYCgNOIyGABWLy_XoLSxTVRe2bltV8GM&index=2&t=0s
Machine Learning 2 - Building a Recommendation Engine:
https://www.youtube.com/watch?v=lvzekeBQsSo&list=PLoYCgNOIyGABWLy_XoLSxTVRe2bltV8GM&index=3&t=0s
Machine learning and neural networks are awesome. This video provides beginners with an easy tutorial explaining how a neural network works - what math is involved, and a step by step explanation of how the data moves through the network.
The example used will be a feed forward neural network with back propagation. It explains the difference between linear and non linear data, the importance of the activation function, learning rate, and momentum configurations.
-~-~~-~~~-~~-~-
LearningWeb Development? Watch the FREE COURSE:
"Web Development for Absolute Beginners"!
https://www.youtube.com/watch?v=gQojMIhELvM
-~-~~-~~~-~~-~-

If you know nothing about how a neural network works, this is the video for you! I've worked for weeks to find ways to explain this in a way that is easy to understand for beginners.
Past Videos:
Intro to Machine Learning with Javascript:
https://www.youtube.com/watch?v=9Hz3P1VgLz4&list=PLoYCgNOIyGABWLy_XoLSxTVRe2bltV8GM&index=2&t=0s
Machine Learning 2 - Building a Recommendation Engine:
https://www.youtube.com/watch?v=lvzekeBQsSo&list=PLoYCgNOIyGABWLy_XoLSxTVRe2bltV8GM&index=3&t=0s
Machine learning and neural networks are awesome. This video provides beginners with an easy tutorial explaining how a neural network works - what math is involved, and a step by step explanation of how the data moves through the network.
The example used will be a feed forward neural network with back propagation. It explains the difference between linear and non linear data, the importance of the activation function, learning rate, and momentum configurations.
-~-~~-~~~-~~-~-
LearningWeb Development? Watch the FREE COURSE:
"Web Development for Absolute Beginners"!
https://www.youtube.com/watch?v=gQojMIhELvM
-~-~~-~~~-~~-~-

But what *is* a Neural Network? | Deep learning, chapter 1

Subscribe to stay notified about new videos: http://3b1b.co/subscribe
Support more videos like this on Patreon: https://www.patreon.com/3blue1brown
Or don't. It's your call really, no pressure.
Special thanks to these supporters: http://3b1b.co/nn1-thanks
Additional funding provided by Amplify Partners. For any early-stage ML entrepreneurs, Amplify would love to hear from you: 3blue1brown@amplifypartners.com
Full playlist: http://3b1b.co/neural-networks
Typo correction: At 14:45, the last index on the bias vector is n, when it's supposed to in fact be a k. Thanks for the sharp eyes that caught that!
For those who want to learn more, I highly recommend the book by Michael Nielsen introducing neural networks and deep learning: https://goo.gl/Zmczdy
There are two neat things about this book. First, it's available for free, so consider joining me in making a donation Nielsen's way if you get something out of it. And second, it's centered around walking through some code and data which you can download yourself, and which covers the same example that I introduce in this video. Yay for active learning!
https://github.com/mnielsen/neural-networks-and-deep-learning
I also highly recommend Chris Olah's blog: http://colah.github.io/
For more videos, Welch Labs also has some great series on machine learning:
https://youtu.be/i8D90DkCLhI
https://youtu.be/bxe2T-V8XRs
For those of you looking to go *even* deeper, check out the text "Deep Learning" by Goodfellow, Bengio, and Courville.
Also, the publication Distill is just utterly beautiful: https://distill.pub/
Lion photo by Kevin Pluck
Music by Vincent Rubinetti:
https://vincerubinetti.bandcamp.com/album/the-music-of-3blue1brown
------------------
3blue1brown is a channel about animating math, in all senses of the word animate. And you know the drill with YouTube, if you want to stay posted on new videos, subscribe, and click the bell to receive notifications (if you're into that).
If you are new to this channel and want to see more, a good place to start is this playlist: http://3b1b.co/recommended
Various social media stuffs:
Website: https://www.3blue1brown.com
Twitter: https://twitter.com/3Blue1Brown
Patreon: https://patreon.com/3blue1brown
Facebook: https://www.facebook.com/3blue1brown
Reddit: https://www.reddit.com/r/3Blue1Brown

Evolution of Neural Networks using Genetic Algorithm for a 3D car made in Unity

Just a simple implementation of neural net for evolution of a car to finish the track. The neural network itself doesn't evolve in shape, but in the neuron connection weights.
Made inUnityMusic: "Righteous" by Silent Partner

Read the complete tutorial about how to implement a machine learning algorithm for the Flappy Bird video game here:
http://www.askforgametask.com/tutorial/machine-learning-algorithm-flappy-bird
This video shows a creation of an artificial intelligence controller for the Flappy Bird game using neural networks and a genetic algorithm.
The program is written in HTML5 using Phaser framework (http://phaser.io/). There is also used Synaptic Neural Network library (https://synaptic.juancazala.com/) to implement entire artificial neural network instead of making a new one from the scratch.
Download source code here:
https://github.com/ssusnic/Machine-Learning-Flappy-Bird
-----------------------------------------------------------------
According to Arthur Samuel, machine learning is the science of getting computers to act without being explicitly programmed. It is a fine tuning process of learning that incrementally improves an initial random system.
The form of machine learning implemented in this program uses a genetic algorithm to train artificial neural networks.
The birds are learning how to flap optimally in order to fly safely through barriers as long as possible.
The main concept is based on these 3 steps:
1. creating the initial population of 10 birds randomly
2. learning as the game is being played
3. applying natural evolution to form the next improved population
-----------------------------------------------------------------
To play the game, each bird has its own neural network consisted of the next 3 layers:
1. an input layer with 2 neurons representing what a bird sees:
- horizontal distance to the closest gap
- height difference to the closest gap
2. a hidden layer with 6 neurons
3. an output layer with 1 neuron to perform flap if its value is greater than 0.5
-----------------------------------------------------------------
To measure the quality of birds and select the best ones, for each bird is calculated its fitness function in this way:
- reward a bird by its total traveled distance
- penalize a bird by its current distance to the closest gap
When the entire population is dead, the fittest four birds are selected to breed a new population by using genetic algorithm operators: selection, crossover and mutation.
-----------------------------------------------------------------
Visit us:
http://www.askforgametask.com
Like us:
https://www.facebook.com/askforgametask
Follow us:
https://twitter.com/ssusnic
Music:
"BedtimeTune" by Jay Man
http://www.ourmusicbox.com

Neural Network Tries to Generate English Speech (RNN/LSTM)

By popular demand, I threw my own voice into a neural network (3 times) and got it to recreate what it had learned along the way!
This is 3 different recurrent neural networks (LSTM type) trying to find patterns in raw audio and reproduce them as well as they can. The networks are quite small considering the complexity of the data. I recorded 3 different vocal sessions as training data for the network, trying to get more impressive results out of the network each time. The audio is 8-bit and a low sample rate because sound files get very big very quickly, making the training of the network take a very long time. Well over 300 hours of training in total went into the experiments with my voice that led to this video.
The graphs are created from log files made during training, and show the progress that it was making leading up to immediately before the audio that you hear at every point in the video. Their scrolling speeds up at points where I only show a short sample of the sound, because I wanted to dedicated more time to the more impressive parts. I included a lot of information in the video itself where it's relevant (and at the end), especially details about each of the 3 neural networks at the beginning of each of the 3 sections, so please be sure to check that if you'd like more details.
I'm less happy with the results this time around than in my last RNN+voice video (https://www.youtube.com/watch?v=FsVSZpoUdSU), because I've experimented much less with my own voice than I have with higher-pitched voices from various games and haven't found the ideal combination of settings yet. That's because I don't really want to hear the sound of my own voice, but so many people commented on my old video that they wanted to hear a neural network trained on a male English voice, so here we are now! Also, learning from a low-pitched voice is not as easy as with a high-pitched voice, for reasons explained in the first part of the video (basically, the most fundamental patterns are longer with a low-pitched voice).
The neural network software is the open-source "torch-rnn" (https://github.com/jcjohnson/torch-rnn/), although that is only designed to learn from plain text. Frankly, I'm still amazed at what a good job it does of learning from raw audio, with many overlapping patterns over longer timeframes than text. I made a program(*) that substitutes raw bytes in any file (e.g. audio) for valid UTF-8 text characters and torch-rnn happily learned from it. My program also substituted torch-rnn's generated text back into raw bytes to get audio again. I do not understand the mathematics and low-level algorithms that go make a neural network work, and I cannot program my own, so please check the code and .md files at torch-rnn's Github page for details. Also, torch-rnn is actually a more-efficient fork of an earlier software called char-rnn (https://github.com/karpathy/char-rnn), whose project page also has a lot of useful information.
I will probably soon release the program that I wrote to create the line graphs from CSV files. It can make images up to 16383 pixels wide/tall with customisable colours, from CSV files with hundreds of thousands of lines, in a few seconds. All free software I could find failed hideously at this (e.g. OpenOffice Calc took over a minute to refresh the screen with only a fraction of that many lines, during which time it stopped responding; the lines overlapped in an ugly way that meant you couldn't even see the average value; and "exporting" graphs is limited to pressing Print Screen, so you're limited to the width of your screen... really?).
(*)Here is the code rewritten from VB6 in a C++-like pseudocode:
http://robbi-985.homeip.net/information/bintoutf8_pseudo.txt
Also, here is an English explanation of the idea behind how it works:
http://robbi-985.homeip.net/information/bintoutf8_info.txt
EDIT: I have released my BinToUTF8 program to the public! Please have a look here:
http://robbi-985.homeip.net/blog/?p=1845

What is backpropagation really doing? | Deep learning, chapter 3

What's actually happening to a neural network as it learns?
Next video: https://youtu.be/tIeHLnjs5U8
Training data generation: http://3b1b.co/crowdflower
Find the full playlist at http://3b1b.co/neural-networks
The following video is sort of an appendix to this one. The main goal with the follow-on video is to show the connection between the visual walkthrough here, and the representation of these "nudges" in terms of partial derivatives that you will find when reading about backpropagation in other resources, like Michael Nielsen's book or Chis Olah's blog.
Thanks to everyone supporting on Patreon.
http://3b1b.co/nn3-thanks
http://3b1b.co/support
For more on backpropagation:
http://neuralnetworksanddeeplearning.com/chap2.html
https://github.com/mnielsen/neural-networks-and-deep-learning
http://colah.github.io/posts/2015-08-Backprop/
Music by Vincent Rubinetti:
https://vincerubinetti.bandcamp.com/album/the-music-of-3blue1brown
------------------
3blue1brown is a channel about animating math, in all senses of the word animate. And you know the drill with YouTube, if you want to stay posted on new videos, subscribe, and click the bell to receive notifications (if you're into that).
If you are new to this channel and want to see more, a good place to start is this playlist: http://3b1b.co/recommended
Various social media stuffs:
Website: https://www.3blue1brown.com
Twitter: https://twitter.com/3Blue1Brown
Patreon: https://patreon.com/3blue1brown
Facebook: https://www.facebook.com/3blue1brown
Reddit: https://www.reddit.com/r/3Blue1Brown

Snakes, Neural Networks and Genetic Algorithms

So I made a program that trains snake AIs with a genetic algorithm (neuroevolution).
Code can be found here: https://github.com/emgoz/Neural-network-snake
Coding Rainbow: Genetic algorthms: https://www.youtube.com/watch?v=9zfeTw-uFCw
Computerphile: Neural Networks:
https://www.youtube.com/watch?v=py5byOOHZM8
Sorry for the offset video, somehow my screen capture software messed it up.
I wrote the whole thing in Java. Don't get angry, I know it's not the best in performance.

Deep Neural Network Learns Van Gogh's Art | Two Minute Papers #6

Artificial neural networks were inspired by the human brain and simulate how neurons behave when they are shown a sensory input (e.g., images, sounds, etc). They are known to be excellent tools for image recognition, any many other problems beyond that - they also excel at weather predictions, breast cancer cell mitosis detection, brain image segmentation and toxicity prediction among many others. Deep learning means that we use an artificial neural network with multiple layers, making it even more powerful for more difficult tasks.
This time they have been shown to be apt at reproducing the artistic style of many famous painters, such as Vincent Van Gogh and Pablo Picasso among many others. All the user needs to do is provide an input photograph and a target image from which the artistic style will be learned.
______________________
I promised some links, so here they come!
The paper "A Neural Algorithm of Artistic Style" is available here:
http://arxiv.org/abs/1508.06576v1
Disclaimer: I was not part of this research project, I am merely providing commentary on this work.
Recommended for you - Two MinutePapers episode on Artificial Neural Networks:
https://www.youtube.com/watch?v=rCWTOOgVXyE&index=3&list=PLujxSBD-JXgnqDD1n-V30pKtp6Q886x7e
Picasso meets Gandalf:
http://mashable.com/2015/08/29/computer-photos/
A nice website with many results:
https://deepart.io/
More examples with Picasso and some sketches:
http://imgur.com/a/jeJB6
Google DeepMind's Deep Q-learning algorithm plays Atari games:
https://www.youtube.com/watch?v=V1eYniJ0Rnk
The first implementations / source code packages are now available:
1. http://gitxiv.com/posts/jG46ukGod8R7Rdtud/a-neural-algorithm-of-artistic-style
2. https://github.com/kaishengtai/neuralart
3. https://github.com/jcjohnson/neural-style
A great read on Deep DreamingNeural Networks:
http://googleresearch.blogspot.co.uk/2015/06/inceptionism-going-deeper-into-neural.html
Many of you have asked for the code. Some people were experimenting with it in the Machine Learning reddit. Check it out:
https://www.reddit.com/r/MachineLearning/comments/3imx1m/a_neural_algorithm_of_artistic_style/
Subscribe if you would like to see more of these! - http://www.youtube.com/subscription_center?add_user=keeroyz
Splash screen/thumbnail design: Felícia Fehér - http://felicia.hu
Music:
Epilog - Ghostpocalypse by Kevin MacLeod is licensed under a Creative Commons Attribution license (https://creativecommons.org/licenses/by/4.0/)
Source: http://incompetech.com/music/royalty-free/index.html?isrc=USUAN1100666
Artist: http://incompetech.com/
______________________
Károly Zsolnai-Fehér's links:
Patreon → https://www.patreon.com/TwoMinutePapers
Facebook → https://www.facebook.com/TwoMinutePapers/
Twitter → https://twitter.com/karoly_zsolnai
Web → https://cg.tuwien.ac.at/~zsolnai/

Neural Networks Explained - Machine Learning Tutorial for Beginners

If you know nothing about how a neural network works, this is the video for you! I've worked for weeks to find ways to explain this in a way that is easy to understand for beginners.
Past Videos:
Intro to Machine Learning with Javascript:
https://www.youtube.com/watch?v=9Hz3P1VgLz4&list=PLoYCgNOIyGABWLy_XoLSxTVRe2bltV8GM&index=2&t=0s
Machine Learning 2 - Building a Recommendation Engine:
https://www.youtube.com/watch?v=lvzekeBQsSo&list=PLoYCgNOIyGABWLy_XoLSxTVRe2bltV8GM&index=3&t=0s
Machine learning and neural networks are awesome. This video provides beginners with an easy tutorial explaining how a neural network works - what math is involved, and a step by step explanation of how the data moves through the network.
The example used will be a feed forward neural network with back propagation. It explains the difference between linear and non linear data, the importance of the activation function, learning rate, and momentum configurations.
-~-~~-~~~-~~-~-
LearningWeb Development? Watch the FREE COURSE:
"Web Development for Absolute Beginners"!
https://www.youtube.com/watch?v=gQojMIhELvM
-~-~~-~~~-~~-~-

For example, a neural network for handwriting recognition is defined by a set of input neurons which may be activated by the pixels of an input image. After being weighted and transformed by a function (determined by the network's designer), the activations of these neurons are then passed on to other neurons. This process is repeated until finally, an output neuron is activated. This determines which character was read.

This makes FaceMe® one of the world’s leading facial recognition engines powered by deep learning and a neural network algorithm... Powered by deep learning algorithms, FaceMe® delivers the reliable, high-precision, and real-time facial recognition that is critical to AIoT ......

Emotion is an integral element to humans and it can be detected via facial expression, voice synthesis and neural response ... All of these data can be fed to the machine learning algorithm and deep neural networks those power the modern AI system to become emotionally aware....

In the surveillance camera market, we see strong interest for single chip solutions with integrated computer vision processing to support the latest deep neural network based algorithms... But from an AI particular point of view, I want to point out that we don't see AI algorithm particularly....

Neural Networks Explained - Machine Learning Tutor...

Latest News for: neural algorithm

This makes FaceMe® one of the world’s leading facial recognition engines powered by deep learning and a neural network algorithm... Powered by deep learning algorithms, FaceMe® delivers the reliable, high-precision, and real-time facial recognition that is critical to AIoT ......

Emotion is an integral element to humans and it can be detected via facial expression, voice synthesis and neural response ... All of these data can be fed to the machine learning algorithm and deep neural networks those power the modern AI system to become emotionally aware....

In the surveillance camera market, we see strong interest for single chip solutions with integrated computer vision processing to support the latest deep neural network based algorithms... But from an AI particular point of view, I want to point out that we don't see AI algorithm particularly....