Entropy, information, and computation
Jonathan Machta · American Journal of Physics · 1999
The relation between entropy, information, and randomness is discussed. Algorithmic information theory is introduced and used to provide a fundamental definition of entropy. The relation between algorithmic entropy and the usual Shannon–Gibbs entropy is discussed.