SparseRepresentation and SynapticAdaptation of theVisualSensory System
Liqing Zhang · 2005
Thispaperinvestigates thesparse representation of visual neural information anditslearning algorithm. First we introduce agenerative statistical modelforinternal representa- tionofvisual neural information. Thentheneural computing mechanism forrepresenting sensory information inthegener- ative modelisdiscussed, andlearning algorithm isdeveloped fortraining theparameters inthegenerative model.Finally computer simulations areprovided toillustrate thesparseness oftheinternal representation ofthevisual information. I.INTRODIJCTION Internal neural representation ofsensory information isa basic issue incomputational neuroscience andneural net- workmodelling. Thereareanumberofneurophysiological evidences that primary visual/auditory cortexes represent the internal neural information assparse aspossible. Thussparse representation becomes oneofmostimportant mechanisms in theadaptation ofneural networks. There areanumber ofvisual neural coding theory related to thesparse representation, suchasefficient coding, sparse Cod- ingandredundancy reduction. Theefficient coding, proposed byAttneave (4), suggests that thegoal ofvisual perception is toproduce anefficient representation ofthesensory signals. Thetheory ofredundancy reduction indicates that therole of early sensory neurons istoremovestatistical redundancy inthe sensory signals. Still another theory, sparse coding (13), (14), (6)(16), (12) suggests that thereceptive field istoproduce asparse distribution ofoutput activity inresponse tonatural images. Itisknownthattheadaptation ofthesensory neural systems areclosely related tothestatistical properties ofthe environment. Thetaskofinternal neural representation isto reshape thestatistical properties ofthesensory signals into the neural network. Theoretically, theestablishment ofaprecise quantitative relation between environment statistics andneural representation isahardproblem. Inthis paper, we propose thegeneral mathematical for- mulation ofvisual neural representation. First we introduce theinternal representation modelanddiscuss thecomputing mechanism fortheadaptation ofthevisual neural network. Inorder toderive learning algorithms fortraining thevisual neural network, wepropose alocal independent decomposition modelandformulate thelearning modelinthestatistical parametric model. Thena learning algorithm isdeveloped based onthelearning model. Furthermore, weprovide com- puter simulations todemonstrate theperformance oflearning process andhowtheinternal basis functions areorganized by thestatistical properties oftheexternal environment.