Efficient Unicode Ordinal Values for Text Embedding with FastText and Word2Vec
Agnij Moitra · 2024
Accurately representing categorical data is crucial for enhancing machine learning model performance, especially when dealing with ordinal data, where the order of categories is significant. This work provides a novel use of ordinal encoding as an alternative to traditional one-hot encoding in word embedding methods. We highlight key theoretical results that demonstrate the benefits of ordinal encoding: improved model interpretability, robustness to noise, and effective out-of-vocabulary handling. Through a series of theorems and empirical benchmarks on publicly available datasets, we show that ordinal encoding can significantly enhance predictive accuracy in text classification tasks. Our findings underscore the importance of preserving ordinal relationships in categorical data and position ordinal encoding as a vital methodology for practitioners aiming for optimal performance. This contribution enriches the ongoing discussion in data representation, offering insights into using FastText and Word2Vec with ordinal values instead of one-hot-encoded vectors.