Applications of ST distributions to neural networks and regression models
Pavel Stoynov · AIP conference proceedings · 2022
Neighborhood function, neighborhood radius and learning rate are terms used in neural networks theory to denote area of dependence between neurons and the degree of adaptation to new information correspondingly.For example, in Kohonen Self-Organizing Maps (SOM) – feed-forward neural networks which apply unsupervised learning - the neuron-winner (Best Matching Unit – BMU or Best Performing Unit - BPU) and his neighbors change their weights in order to be closer to the corresponding input vector. Neighborhood here is defined through neighbor function and neighbor radius. The learning rate is used when changing the weights of the BPU and its neighbors. It allows introducing different degree of change of the weights of the neurons in the neighborhood. In this way, different quantity of new knowledge is ensured for different neighbors.In this article, a new type of neighborhood function and learning rate function is proposed based on Switch-Time (ST) distribution.In neural networks, as well as in non-linear regression (which may be considered as a very simple case of neural network), different kinds of non-linear activation functions are used.In this article a new type of non-linear activation functions is proposed based on Switch-Time (ST) distribution and the term stopit regression is defined.ST distributions are also considered as regime capturing functions in neural networks with a smooth transition regime switching.