Statistical approximation learning of discontinuous functions using simultaneous recurrent neural networks

In this paper, we develop an architecture for a novel type of neural network which is known as simultaneous recurrent neural networks (SRNNs). Using this novel neural architecture, we propose a statistical approximation learning (SAL) method. The SRNNs have the capability to approximate non-smooth functions which cannot be approximated by using conventional multi-layer perceptrons (MLPs). However, the most of the learning methods for the SRNNs are computationally expensive due to their inherent recursive calculations. To solve this problem, as an approximation learning method, SAL method is proposed by using a statistical relation between the time-series of the network outputs and the network configuration parameters. Simulation results show that SRNN's trained by the proposed SAL method can learn a strongly nonlinear function efficiently within a practical computation time.