Shannon's Theory of Communication - A Key to Understanding AI Model Collapse?
Pavel Straňák · 2025
Improving generative AI models through the process of learning from syntheticdata could offer significant benefits for AI development. Could the use of synthetic,model-generated data for its own refinement be a solution? Model collapse is an observabledeterioration in the accuracy and relevance of responses from a model trained on synthetic,model-generated data. The phenomenon is not yet fully understood. I propose an explanationbased on the application of Shannon's theory of communication to AI models. Model collapseemerges directly as a necessary consequence of this explanation.