PCA Embedded Random Forest
Charles W. Gardner, Dan Chia-Tien Lo · 2021
This study proposes a new architecture for the random forest algorithm. This architecture attempts to improve Random Forest's ability to recognize feature interdependencies. A performance improvement is achieved by creating a PCA model within each tree. This PCA model creates new additional features containing information from multiple input features. These new features are not used for feature reduction but instead appended to the existing feature vectors. With this approach, the tree can make separations based on information from more than one of its input features. We will outline an approach using PCA feature construction to improve the Random Forest algorithm's performance. The improvement is not universal but instead useful when false positives are more detrimental than false negatives. RF-PCA produces models which favor a high precision with nearly equivalent F1-scores to a traditional random forest algorithm.