CFC-ATE: Causal Feature Construction via Average Treatment Effect
Asmae Lamsaf, Hugo Proença, João C. Neves · 2024
Dimensionality reduction is a crucial step in data preprocessing, particularly for high-dimensional datasets, where the excessive number of features increases the risk of overfitting in machine learning models. Traditional dimensionality reduction methods rely on statistical associations or the relative position of the feature embeddings in the hyper-space to map original features to a compact subspace that preserves the most relevant information of the data. However, these methods fail to capture the causal relationships among variables during the transfor-mation process, leading to a loss of structural coherence of the data in low-dimensional spaces. By employing causal discovery and causal inference, it is possible to simplify these problems, effectively merging critical features while reducing both complex-ity and dimensionality. Our paper introduces a novel approach, Causal Feature Construction via Average Treatment Effect (CFC-ATE), which leverages causal discovery and inference to create more interpretable and reliable features for predictive modeling. Our methodology consists of the following phases: i) leveraging the causal structure of data through the inference of the causal graph. ii) transforming features through the use of the average treatment effect conditioned on the causal structure of the data. The experiments on diverse real-world datasets and synthetic datasets demonstrate the effectiveness of CFC-ATE in improving model performance by comparing it with three methods of feature selection and three benchmark dimensionality reduction techniques.