Semantic Channel and Shannon Channel Mutually Match and Iterate for Tests and Estimations with Maximum Mutual Information and Maximum Likelihood
Chenguang Lu · 2018
It is very difficult to solve Maximum Mutual Information (MMI) or Maximum Likelihood (ML) for all possible Shannon Channels so that we have to use iterative methods. According to the Semantic Information Measure (SIM) and R(G) function proposed by Chenguang Lu (1993) (where G is the lower limit of the SMI, and R(G) is an extension of rate distortion function R(D)), we can obtain a new iterative algorithm of solving the MMI and ML for tests, estimations, and mixture models. A group of truth functions constitute a semantic channel. Letting the semantic channel and Shannon channel mutually match and iterate, we can obtain the Shannon channel that maximizes mutual information and average log likelihood. This iterative algorithm is called Channels' Matching (CM) algorithm. The convergence can be intuitively explained and proved by the R(G) function. Several iterative examples show that the CM algorithm for tests and estimations with larger samples is simple, fast, and reliable. Multi-label Logical classification is introduced in passing.