Anchor-Guided Semantic Double-Clustering: Scalable, High-Recall Contradiction Detection for Massive Knowledge Bases
Vladyslav Holubiev, Ihor Simashko, Bohdan Ihnatiuk, Vasyl Sheketa, Mykola Koromysel, A. K. Dankiv · 2025
Detecting contradictions in large knowledge bases like Wikipedia requires high recall and efficiency, but traditional hashing or pairwise language model checks don't scale well. We present Anchor-Guided Semantic Double-Clustering (ASDC), a robust, deterministic, and scalable contradiction detection pipeline. ASDC uses linguistic anchors from question–answer semantics to drive a dual-clustering approach: first, anchor-driven blocking prunes candidate pairs with bounded recall loss; second, semantic vector similarity and lexical filtering further reduce candidates. An innovative Union-Find Filtering mechanism rapidly validates contextual coherence, cutting redundant language model checks by nearly sevenfold. We evaluate ASDC on real and synthetic corpora with thousands of embedded contradictions. The pipeline slashes comparisons from 1.2 million to 15.6k (>98% reduction), boosts recall from 92% (LSH baseline) to ~99% and completes clustering in under a second on standard hardware. At Wikipedia scale, ASDC reduces detection time from a year to under two months. ASDC marks a major advance in scalable, high-recall contradiction management, making industrial-scale semantic coherence maintenance practical.