Multi‐class Classification and Adam Optimization
Resve Saleh, Sohaib Majzoub, A.K. Md. Ehsanes Saleh · 2025
This chapter addresses multi-class classification problems using neural networks and the details of their optimization during training. Loss functions for multi-class problems are described which require categorical versions of the cross-entropy and log-cosh loss functions. The sigmoid function is extended to incorporate multiple outputs, leading to the softmax activation function. The implementation of the softmax activation function is described in detail. The MNIST dataset is used as a demonstration vehicle to identify the advantages of using a robust loss function for problems with outliers in the dataset. In addition, a number of different algorithms that comprise the Adam optimization technique are described, including momentum and rmsprop.