Detecting Machine-Generated Long-Form Content with Latent-Space Variables
Yufei Tian, Zeyu Pan, Nanyun Peng · 2024
The increasing capability of large language models (LLMs) to generate fluent long-form texts is presenting new challenges in distinguishing machine-generated outputs from human-written ones, which is crucial for ensuring authenticity and trustworthiness of expressions.Existing zero-shot detectors primarily focused on token-level distributions, which are vulnerable to real-world domain shifts including different prompting and decoding strategies, and adversarial attacks.We propose a more robust method that incorporates abstract elements-such as event transitions-as key deciding factors to detect machine vs. human texts, by training a latent-space model on sequences of events or topics derived from human-written texts.On three different domains, machine generations which are originally inseparable from humans' on the token level can be better distinguished with our latent-space model, leading to a 31% improvement over strong baselines such as DetectGPT (Mitchell et al., 2023;Bao et al., 2024).Our analysis further reveals that, unlike humans, modern LLMs like GPT-4 generate event triggers and their transitions differently, an inherent disparity that help our method to robustly detect machine-generated texts.