An Improved Random Forest for Classification and Regression Using Dynamic Weighted Scheme
Vikas Kumar Jain, Ashish Phophalia · 2023
Random forest (RF) is an ensemble-based machine learning algorithm that builds a large collection of decision trees. It is used for classification and regression tasks. During testing, the RF assigns equal weights to all decision trees and predicts the final output by aggregating the predictions of all decision trees. However, the weights are static; consequently, it fails to capture the relationship between test samples and learned decision trees. Therefore, in this chapter, instead of assigning equal and static weights to all decision trees, a dynamic weight scheme is proposed. To this end, we compute the similarity of the test sample with respect to every decision tree. The computation of weight uses exponential distribution; hence the proposed approach is called Exponentially Weighted Random Forest (EWRF). The efficacy of the proposed approach is tested at three different levels; First, the classification of hyperspectral datasets over Indian Pines, Pavia University, and Kennedy Space Center is performed. Second, soil moisture prediction is performed as a regression application over the hyperspectral data. Third, object and digit classification is performed over Caltech-101, Caltech-256, and MNIST datasets. Experimental results demonstrate the effectiveness of the proposed approach compared to competitive state-of-the-art approaches, including classification ensemble method such as regression with hyperspectral imaging weighting mechanism.