Feedforward neural networks with ReLU activation functions are linear splines
Magnus Hansson, Christoffer Olsson · Lund University Publications Student Papers (Lund University) · 2017
In this thesis the approximation properties of feedforward articial neural networks with one hidden layer and ReLU activation functions are examined. It is shown that functions of these kind are linear splines and the number of spline knots depend on the number of nodes in the network. In fact an upper bound can be derived for the number of knots. Furthermore, the positioning of the knots depend on the optimization of the adjustable parameters of the network. A numerical example is given where the network models are compared to linear interpolating splines with equidistant positioned knots.