Your byte size news and commentary from Silicon Valley the land of startup vanities, coding, learn-to-code and unicorn billionaire stories.

Ad

Tuesday, June 26, 2018

Recurrent Neural Network RNN Basics

Recurrent Neural Network (RNN) is useful for processing sequence data like sound, words, and time series data . RNN remembers a bit of the state from before. It can predict what will come next. Time series is good for heart disease over time, hormone level, blood sugar.

RNN weakness: sometimes gradients too close to 0 or too computationally large. It can also be bad at tracking long term memories - need to use LSTM instead, which has a forget gate, input gate, update layer, output layer