Address
 2

Romuald Boné

* Director of the the National Engineering School of Loire Valley &

* Member of the Computer Science Laboratory of the University of Tours (France)

Recurrent Neural Networks

We review existing approaches in using neural networks for solving one step and multi-step-ahead prediction problems.
Recurrent neural networks possess interesting universal approximation capabilities, making them good candidates for time series modeling. Unfortunately, long term dependencies are difficult to learn if gradient descent algorithms are employed. We will present four approaches for improving the accuracy of recurrent neural networks (RNNs) for time series forecasting. Adding well chosen connections with time delays to the RNNs often reduces learning times and allows gradient descent algorithms to find better solutions.

The fist two heuristic based algorithms we present here allow one to choose the right locations and delays for such connections. In the third algorithm, we apply principle of time delay learning by gradient descent. In the fourth algorithm, improvement is achieved by combining a large number of RNNs, each of them is generated by training on a different set of examples. This algorithm is based on the boosting algorithm and allows concentrating the training on difficult examples but, unlike the original algorithm, by taking into account all the available examples. Finally, we compare the performances obtained with other regression methods on benchmark time series.