Statistical learning machines from ATR to DNA micro arrays: design, assessment, and advice for practitioners
Waleed A. Yousef · The International Conference on Electrical Engineering/The International Conference on Electrical Engineering · 2008
Statistical Learning is the process of estimating an unknown probabilistic inputoutputrelationship of a system using a limited number of observations; and a statisticallearning machine (SLM) is the machine that learned such a process. While their rootsgrow deeply in Probability Theory, SLMs are ubiquitous in the modern world.Automatic Target Recognition (ATR) in military applications, Computer AidedDiagnosis (CAD) in medical imaging, DNA microarrays in Genomics, OpticalCharacter Recognition (OCR), Speech Recognition (SR), spam email filtering, stockmarket prediction, etc., are few examples and applications for SLM; diverse fields butone theory.The field of Statistical Learning can be decomposed to two basic subfields, Designand Assessment. We mean by Design, choosing the appropriate method that learns fromthe data to construct an SLM that achieves a good performance. We mean byAssessment, attributing some performance measures to the designed SLM to assess thisSLM objectively. To achieve these two objectives the field encompasses different otherfields: Probability, Statistics and Matrix Theory; Optimization, Algorithms, andprogramming, among others.Three main groups of specializations—namely statisticians, engineers, and computerscientists (ordered ascendingly by programming capabilities and descendingly bymathematical rigor)—exist on the venue of this field and each takes its elephant bite.Exaggerated rigorous analysis of statisticians sometimes deprives them fromconsidering new ML techniques and methods that, yet, have no “complete”mathematical theory. On the other hand, immoderate add-hoc simulations of computerscientists sometimes derive them towards unjustified and immature results. A prudent approach is needed that has the enough flexibility to utilize simulations and trials anderrors without sacrificing any rigor. If this prudent attitude is necessary for this field it isnecessary, as well, in other fields of Engineering.In the spirit of this prelude, this article is intended to be a pilot-view of the field thatsheds the light on SLM applications, the Design and Assessment stages, necessarymathematical and analytical tools, and some state-of-the-art references and research.