Random Positive-Only Projections: PPMI-Enabled Incremental Semantic Space Construction
Behrang QasemiZadeh, Laura Kallmeyer · 2016
We introduce positive-only projection (PoP), a new algorithm for constructing semantic spaces and word embeddings.The PoP method employs random projections.Hence, it is highly scalable and computationally efficient.In contrast to previous methods that use random projection matrices R with the expected value of 0 (i.e., E(R) = 0), the proposed method uses R with E(R) > 0. We use Kendall's τ b correlation to compute vector similarities in the resulting non-Gaussian spaces.Most importantly, since E(R) > 0, weighting methods such as positive pointwise mutual information (PPMI) can be applied to PoP-constructed spaces after their construction for efficiently transferring PoP embeddings onto spaces that are discriminative for semantic similarity assessments.Our PoP-constructed models, combined with PPMI, achieve an average score of 0.75 in the MEN relatedness test, which is comparable to results obtained by state-of-the-art algorithms.