An introduction to information theory and entropy
Tom Carter · 2007
system we are looking at is actually a complex system? (i.e., should we even be studying this system? :-) Of course, in practice, we will study the systems that interest us, for whatever reasons, so the problem identied above tends not to be a real problem. On the other hand, having chosen a system to study, we might well ask \\How complex is this system?" In this more general context, we probably want at least to be able to compare two systems, and be able to say that system A is more complex than system B. Eventually, we probably would like to have some sort of numerical rating scale. 5 Various approaches to this task have been proposed, among them: 1. Human observation and (subjective) rating 2. Number of parts or distinct elements (what counts as a distinct part?) 3. Dimension (measured how?) 4. Number of parameters controlling the system 5. Minimal description (in which language?) 6. Information content (how do we dene/measure information?) 7. Minima