Random Forest Classification of Remote Sensing Data
Sveinn R. Joelsson, Jón Atli Benediktsson, Jóhannes R. Sveinsson · 2007
CONTENTS 3.1 Introduction ......................................................................................................................... 61 3.2 The Random Forest Classifier........................................................................................... 62 3.2.1 Derived Parameters for Random Forests ........................................................... 63 3.2.1.1 Out-of-Bag Error ...................................................................................... 63 3.2.1.2 Variable Importance................................................................................ 63 3.2.1.3 Proximities ................................................................................................ 63 3.3 The Building Blocks of Random Forests......................................................................... 64 3.3.1 Classification and Regression Tree...................................................................... 64 3.3.2 Binary Hierarchy Classifier Trees........................................................................ 64 3.4 Different Implementations of Random Forests ............................................................. 65 3.4.1 Random Forest: Classification and Regression Tree ........................................ 65 3.4.2 Random Forest: Binary Hierarchical Classifier ................................................. 65 3.5 Experimental Results.......................................................................................................... 65 3.5.1 Classification of a Multi-Source Data Set ........................................................... 65 3.5.1.1 The Anderson River Data Set Examined with a Single CART Tree................................................................................. 69 3.5.1.2 The Anderson River Data Set Examined with the BHC Approach ........................................................................................ 71 3.5.2 Experiments with Hyperspectral Data ............................................................... 72 3.6 Conclusions.......................................................................................................................... 77 Acknowledgment......................................................................................................................... 77 References ..................................................................................................................................... 77 Ensemble classification methods train several classifiers and combine their results through a voting process. Many ensemble classifiers [1,2] have been proposed. These classifiers include consensus theoretic classifiers [3] and committee machines [4]. Boosting and bagging are widely used ensemble methods. Bagging (or bootstrap aggregating) [5] is based on training many classifiers on bootstrapped samples from the training set and has been shown to reduce the variance of the classification. In contrast, boosting uses iterative re-training, where the incorrectly classified samples are given more weight in successive training iterations. This makes the algorithm slow (much slower than bagging) while in most cases it is considerably more accurate than bagging. Boosting generally reduces both the variance and the bias of the classification and has been shown to be a very accurate classification method. However, it has various drawbacks: it is computationally demanding, it can overtrain, and is also sensitive to noise [6]. Therefore, there is much interest in investigating methods such as random forests.