
=========================
Recurrent Neural Networks
=========================

Papers
======

* [1990] `Finding Structure in Time <http://www.sciencedirect.com/science/article/pii/036402139090002E>`_
* [1990] `Backpropagation through time: what it does and how to do it <http://ieeexplore.ieee.org/
  xpls/abs_all.jsp?arnumber=58337>`_
* [1995] `Hierarchical recurrent neural networks for long-term dependencies <http://citeseerx.ist.p
  su.edu/viewdoc/download?doi=10.1.1.38.7574&rep=rep1&type=pdf>`_
* [1997] `Serial Order: A Parallel Distributed Processing Approach <http://www.sciencedirect.com/s
  cience/article/pii/S0166411597801112>`_
* [1997] `Long short-term memory <http://www.bioinf.at/publications/older/2604.pdf>`_
* [1997] `Bidirectional recurrent neural networks <http://ieeexplore.ieee.org/xpls/abs_all.jsp?a
  rnumber=650093>`_
* [2000] `Recurrent nets that time and count <http://ieeexplore.ieee.org/document/861302/>`_
* [2001] `Recurrent neural networks <ftp://91.220.220.11/linux-support/books/computer%20science/Ar
  tificial%20Intelligence/Neural%20networks/Recurrent%20Neural%20Networks%20Design%20And%20Appl
  ications%20-%20L.R.%20Medsker.pdf>`_
* [2007] `Echo state network <http://www.scholarpedia.org/article/Echo_state_network>`_
* [14.02] `A clockwork RNN <https://arxiv.org/abs/1402.3511>`_
* [14.12] `Learning longer memory in recurrent neural networks <https://arxiv.org/abs/1412.7753>`_
* [15.02] `Gated feedback recurrent neural networks <https://arxiv.org/abs/1502.02367>`_
* [15.06] `A Critical Review of Recurrent Neural Networks for Sequence Learning <https://arxiv.or
  g/abs/1506.00019>`_
* [16.11] `shuttleNet: A biologically-inspired RNN with loop connection and parameter sharing <ht
  tps://arxiv.org/abs/1611.05216>`_

Codes
=====

* `Backpropagation.ipynb <https://github.com/craffel/theano-tutorial/blob/master/Backpropagation.
  ipynb>`_
* `theano-rnn <https://github.com/gwtaylor/theano-rnn>`_ : Demonstration of recurrent neural network
  implemented with Theano
* `On the difficulty to training recurrent neural networks <https://github.com/pascanur/traini
  ngRNNs>`_, theano implementation

Blogs
=====

* `Understanding LSTM Networks <http://colah.github.io/posts/2015-08-Understanding-LSTMs/>`_
* `RNN TUTORIAL, PART 1 – INTRODUCTION TO RNNS <http://www.wildml.com/2015/09/recurrent-neur
  al-networks-tutorial-part-1-introduction-to-rnns/>`_
* `RNN TUTORIAL, PART 2 – IMPLEMENTING A RNN WITH PYTHON, NUMPY AND THEANO <http://www.wildml.
  com/2015/09/recurrent-neural-networks-tutorial-part-2-implementing-a-language-model-rnn-with
  -python-numpy-and-theano/>`_
* `RNN TUTORIAL, PART 3 – BACKPROPAGATION THROUGH TIME AND VANISHING GRADIENTS <http://www.wil
  dml.com/2015/10/recurrent-neural-networks-tutorial-part-3-backpropagation-through-time-and-va
  nishing-gradients/>`_
* `RNN TUTORIAL, PART 4 – IMPLEMENTING A GRU/LSTM RNN WITH PYTHON AND THEANO <http://www.wildml
  .com/2015/10/recurrent-neural-network-tutorial-part-4-implementing-a-grulstm-rnn-with-python-
  and-theano/>`_
* `Sequence Classification with LSTM Recurrent Neural Networks in Python with Keras <http://machi
  nelearningmastery.com/sequence-classification-lstm-recurrent-neural-networks-python-keras/>`_
* `Predicting sequences of vectors (regression) in Keras using RNN - LSTM <http://danielhnyk.cz
  /predicting-sequences-vectors-keras-using-rnn-lstm/>`_
* `Written Memories: Understanding, Deriving and Extending the LSTM <http://r2rt.com/written-mem
  ories-understanding-deriving-and-extending-the-lstm.html>`_
* `Anyone Can Learn To Code an LSTM-RNN in Python (Part 1: RNN) <https://iamtrask.github.io/201
  5/11/15/anyone-can-code-lstm/>`_
