On Palimpsests in Neural Memory: An Information Theory Viewpoint
Lav R. Varshney, Julius Kusuma, Vivek K Goyal · IEEE Transactions on Molecular Biological and Multi-Scale Communications · 2016
The finite capacity of neural memory and the reconsolidation phenomenon suggest it is important to be able to update stored information as in apalimpsest, where new information overwrites old information. Moreover, changing information in memory is metabolically costly. In this paper, we suggest that information-theoretic approaches may inform the fundamental limits in constructing such a memory system. In particular, we definemalleable coding, which considers not only representation length but also ease of representation update, thereby encouraging some form of recycling to convert an old codeword into a new one. Malleability cost is the difficulty of synchronizing compressed versions, and malleable codes are of particular interest when representing information and modifying the representation are both expensive. We examine the tradeoff between compression efficiency and malleability cost, under a malleability metric defined with respect to a string edit distance. This introduces a metric topology to the compressed domain. We characterize the exact set of achievable rates and malleability as the solution of a subgraph isomorphism problem. This is all done within the optimization approach to biology framework.