Information structures of capacity achieving distribution for channels with memory and feedback
Christos K. Kourtellaris, Charalambos D. Charalambous · 2016
The information structures of the optimal channel input distributions P[0,n]=Δ{PAi|Ai-1, Bi-1: i = 0,1,..., n}, which correspond to the extremum problem of feedback capacity CAn→BnFB=Δsup P[0,n]Σi=0nI(Ai;Bi|Bi-1) are identified, for any class of channel distributions {PBi|Bi-1,Ai: i = 0,1,...,n} and {PBi|Bi-Mi-1,Ai: i = 0,1,...,n}, where Bn=Δ{Bj: j = 0,1,...,n} are the channel output RVs, An=Δ{Aj: j = 0,1,...,n} are the channel inputs RVs, and M is a finite nonnegative integer. The methodology utilizes stochastic optimal control theory, to identify the control process, the controlled process, and a variational equality of directed information, to derive upper bounds on I(An→ Bn)=ΔΣi=0nI(Ai;Bi|Bi-1), which are achievable over specific subsets of P[0,n], which satisfy conditional independence. The main theorem states, that for any channel with memory M, the optimal channel input conditional distribution occur in the subset P[0,n]=Δ{PAi|Bi-Mi-1: i = 1,...,n} ⊂ P[0,n], and the corresponding extremum problem simplifies to the following characterization CAn→BnFB,M=Δsup P[0,n]Σi=0nI(Ai;Bi|Bi-Mi-1).