Recurrent neural network

A recurrent neural network (RNN) is a special type of artificial neural network (a computer program) that can remember things it has seen before. RNNs do this by connecting information from the past (called a hidden state) with new information. Because of this, RNNs are good at processing data in a sequence, like understanding language, translating between languages, or predicting what might come next in a sequence.

Recurrent Neural Network Media

• 

  A close-loop cross-coupled perceptron networkTemplate:Pg

• 

  Compressed (left) and unfolded (right) basic recurrent neural network

• 

  Illustration for RNN with encoder and decoder.

• 

  Illustration for RNN with encoder and decoder.

• 

  A decoder without an encoder

• 

  Encoder-decoder RNN without attention mechanism

• 

  Encoder-decoder RNN with attention mechanism

• 

  A fully connected RNN with 4 neurons

• 

  A simple Elman network where \sigma_h = \tanh, \sigma_y = \text{Identity}

• 

  The Elman Simple Recurrent Neural Network. The context layer (u1 ... ul) provides a temporal memory to the network, allowing it to "remember" previous hidden layer states. Generated in LaTeX via PGF/Tikz.