Possibilities of Piecewise-Linear Neural Network Training Using Levenberg-Marquardt Algorithm and Hybrid Differential Evolution

Abstract

This article is focused on the comparison of the learning of an artificial neural network with a hyperbolic tangent activation function and an artificial neural network with a linear saturated activation function in hidden layers. The learning is performed by a Levenberg-Marquardt algorithm and hybrid differential evolution. For evaluating of learning characteristics, there is calculated a comprehensive set of statistical variables. The results are analysed and shown as a table for each experiment. An empirical result discussed at the end of the paper is, that the approximation qualities of both networks under examination are similar.