Algebraic Representations of Entropy and Fixed-Sign Information Quantities
Keenan J. A. Down, Pedro A. M. Mediano · Entropy · 2025
Many information-theoretic quantities have corresponding representations in terms of sets. Many of these information quantities do not have a fixed sign—for example, the co-information can be both positive and negative. In previous work, we presented a signed measure space for entropy where the smallest sets (called atoms) all have fixed signs. In the present work, we demonstrate that these atoms have natural algebraic behaviour which can be expressed in terms of ideals (characterised here as upper sets), and we show that this behaviour allows us to make bounding arguments and describe many fixed-sign information quantity expressions. As an application, we give an algebraic proof that the only completely synergistic system of three finite variables X, Y and Z=f(X,Y) is the XOR gate.