A Resampling Technique for Learning the Fourier Spectrum of Skewed Data.
Rajeev Ayyagari, Hillol Kargupta · 2002
Function induction using the widely studied Walsh or Multidimensional Discrete Fourier Transform (MDFT) coefficient estimates has several benefits, including the fact that decision trees can be constructed efficiently from the spectrum. While these estimates are accurate for uniform data, highly skewed data is the norm. This paper gives a way of improving the accuracy of the MDFT coefficient estimates in the case of skewed data. An adaptive resampling algorithm for learning the MDFT coefficients is presented and verified experimentally. An equivalent estimator that can be learned using a single pass algorithm is defined. The effectiveness of the technique is demonstrated using controlled experiments.