Computational Intelligence: Principles, Techniques and Applications, Prof. Dr Amit Konar, Springer, the Netherlands, 2005. No. of pages: 705. ISBN: 3‐540‐20898‐4
Steve C. Chiu · International Journal of Robust and Nonlinear Control · 2008
The book Computational Intelligence: Principles, Techniques and Applications by Dr Amit Konar provides comprehensive and excellent coverage of current topics in computational intelligence. In-depth treatments are given to many relevant topics of interest, including fuzzy sets and relations, fuzzy logic in process control, fuzzy pattern recognition, fuzzy databases and possibilistic reasoning, neural networks, genetic algorithms (GAs), belief calculus, fuzzy moment descriptors, distributed machine learning with fuzzy cognitive maps, computational intelligence in tele-communication networks, mobile robotics, among others. Also included are open-ended research problems that can be helpful in formulating M.S. or Ph.D. theses. Computational intelligence, the inception of which dates back to the early 1990s, initially centred around the logic of fuzzy sets, neural networks, GAs and probabilistic reasoning. More recently it has evolved, having been heavily influenced by biologically inspired models of machine intelligence. Currently this research includes granular computing, neural computing and evolutionary computing, as well as their interactions with artificial life, chaos theory and others. To address this vast area of study (and the volume of this book), the author has provided a dependence graph to aid in the studying of the chapters, a very useful feature. This chapter provides an introduction to the study of computational science. It begins with a thorough review of the underlying principles of artificial intelligence (AI), and examines the scope of computational intelligence in overcoming the limitations of the traditional AI. Then, the chapter briefly introduces various tools of computational intelligence, e.g. fuzzy logic, neural network, GA, belief network, chaos theory, computational learning theory and artificial life. The synergistic behaviour of these tools on many occasions is shown to far exceed their individual performance. As such, this chapter also includes a discussion on the synergistic behaviour of neuro-fuzzy, neuro-GA, neuro-belief and fuzzy-belief networks. This chapter provides an introduction to fuzzy sets, fuzzy relations and some elementary fuzzy operators such as t-norm, s-norm, max–min composition, max-product composition operators, etc. The extension principle of fuzzy sets and the concept of projection and cylindrical extension are also outlined in this chapter with examples. A brief introduction to fuzzy linguistic variables and fuzzy hedges is given at the end of the chapter. A production system generally embodies a set of rules, called knowledge base, a set of facts called database and an inference engine (i.e. interpreter) for interpretation of the database with the help of the knowledge base. In the process of interpreting the database, the inference engine may generate new inferences. This mechanism of inference generation is well known as reasoning in the treaties of knowledge-based systems. Reasoning in predicate logic is usually performed by three fundamental rules, e.g. modus ponens, modus tollens and syllogisms. Chapter 3 extends the scope of reasoning in knowledge-based systems by way of generalization of the above three rules using the logic of fuzzy sets. Various forms of fuzzy reasoning with single and multiple antecedent clauses are introduced in this chapter and the scope of one such reasoning scheme on a VLSI engine is examined. This chapter ends with a discussion on the principles of fuzzy abductive reasoning. Given that the logic of fuzzy sets and its application in approximate reasoning are introduced in Chapters 2 and 3, this chapter further extends the scope of approximate reasoning of fuzzy logic to process control systems. Two distinct models of fuzzy control, namely Mamdani's model and Takagi–Sugeno's model are discussed in this chapter with numerical illustrations. One important aspect of controller design for smart processes is to ensure the stability of the closed-loop control system. This chapter gives an introduction to stability analysis for the Takagi–Sugeno model, as it is widely used in designing industrial fuzzy controllers. The principle of defuzzification is also discussed with an example. The chapter ends with a case study of fuzzy control of the nuclear reactor. Classical models of pattern recognition partition a set of patterns into classes depending on the similarity in features of the patterns. When the distinctive features of the patterns are correctly identified, the classes can easily be distinguished in the feature space. Unfortunately, features in most pattern recognition problems are selected on an ad hoc basis, consequently causing the pattern classes to overlap, thereby leading to an ambiguity in object recognition. This chapter presents a well-known technique for fuzzy pattern recognition, which is capable of partitioning the patterns by soft boundaries. Thus, a pattern may be classified into one or more classes with a certain degree of membership to belong to each class. The algorithm for fuzzy pattern recognition is numerically illustrated, and its application in object recognition from real-time video frames is presented. This chapter provides a possibilistic interpretation of fuzzy relational databases containing imprecise and noisy data. It proposes fuzzy equality relations, and represents fuzzy functional dependency using such relations. It also outlines a novel scheme for testing the lossless join decomposition of fuzzy relational databases. This chapter employs these two concepts in designing fuzzy relational databases. Chapter 7 gives an introduction to machine learning using artificial neural networks. It reviews biological neural networks, and presents a general framework to construct their mathematical models with a view to study their applications in machine learning. This chapter talks about five different types of machine learning, including supervised learning, unsupervised learning, competitive learning, reinforcement learning and Hebbian learning. Stability and convergence are two fundamental issues in studying machine learning algorithms. The interrelation between stability of a dynamical learning system and convergence of a learning algorithm is presented in detail in this chapter as well. This chapter presents supervised learning algorithms for training feed-forward neural networks. It begins with McCulloch–Pitts model and demonstrates its application in realization of binary logic functions. Rosenblatt's perceptron learning algorithm designed for the McCulloch–Pitts neuronal model is presented. Application of the perceptron learning model in both linear and nonlinear classification problems is introduced. The chapter also covers Widrow-Hoff's ADALINE model and discusses its application in translation and rotation invariant pattern recognition. The most important aspect of this chapter is the derivation of the classical back-propagation learning algorithm from the principles of gradient descent learning. The chapter ends with discussions on radial basis function neural nets and modular neural nets. Chapter 9 provides a thorough review of the classical algorithms on unsupervised neural learning. It begins with a brief introduction to recurrent neural topology and then presents in detail both binary and continuous Hopfield nets, their stability analysis and applications. The chapter also presents a detailed overview to adaptive resonance theory and its application in solving the so-called stability plasticity conflict problem in classical pattern recognition. Finally, the chapter introduces fuzzy associative memory neural nets, and outlines algorithms designed for pattern classification by these proposed neural nets. This chapter presents different models of competitive learning using neural networks. The first model is concerned with a two-layer competitive learning network having a noise-free input realized with an on-centre off-surround configuration. An analysis of the model has been presented in detail. The scope of realization of competition by Hebbian learning and the way-out to handle the limitation of Hebbian learning by Oja's principle are also discussed in detail. This chapter also introduces principal component analysis and self-organizing feature models, and their application in human face recognition problems. This chapter introduces the principles of reinforcement learning that rests on the foundation of the penalty-reward mechanism of our natural learning process. It begins with Q-learning and its variants and discusses the scope of realization of Q-learning on neural networks. Two distinct models of neural topologies have been considered for on-line adaptation of weights in the neural nets, following the dynamics of the Q-learning law. The principles of the Q-learning algorithm are illustrated with the well-known grid-world problem of mobile robots. The convergence analysis of the Q-learning algorithm is presented, and the scope of extension of the Q-learning algorithm in multi-agent learning systems is addressed. A new kind of classical algorithms is presented, which emulates the biological evolutionary process in intelligent search, machine learning and optimization problems. This chapter provides a detailed discussion on one evolutionary algorithm, named GA. An analysis of the GA by the Schema theorem and Markov Chains is presented. The latter part of the chapter covers the possible applications of GA in machine learning, intelligent search and derivative-free optimization problems. This chapter ends with a discussion on another evolutionary algorithm, popularly known as Genetic Programming. This chapter discusses two different techniques for probabilistic reasoning known as Dempster–Shafer theory and Pearl's evidential model for belief propagation. The former technique is employed to reduce uncertainty in decisions when the relevant information needed to arrive at the decision is obtained from multiple sources with non-uniform levels of authenticity. The latter technique is an extension of classical Bayesian literature. It inputs both causal and evidential information of an event to determine its belief. Pearl's belief propagation model is applied on a causal tree or a graph where nodes denote events and the directed arcs denote cause–effect relationship between each two events. This model has extensive applications in diagnostic systems, where the probabilistic sensory data are fed at the leaves of the causal tree, and the root causes of system failure, which are denoted by non-terminal nodes in the network, are identified through an algorithm for belief propagation. Chapter 14 deals with uncertainty management in expert systems for two generic class problems using fuzzy Petri net that represents logical connectivity among a set of imprecise propositions. One class of problems deals with the computation of fuzzy belief of any proposition from the fuzzy belief from the fuzzy beliefs of a set of independent initiating propositions in a given network. The other class of problems relates to the computation of steady-state fuzzy beliefs of the propositions embedded in the network, from the initial fuzzy beliefs through a process called belief revision. During belief revision, a fuzzy Petri net with cycles may exhibit ‘limit-cycle behaviour’ of fuzzy beliefs for some propositions in the network. No decisions can be arrived at from a fuzzy Petri net with such behaviour. To circumvent this problem, techniques have been developed for the detection and elimination of limit cycles. Further, an algorithm for selecting one evidence from each set of mutually inconsistent evidences, referred to as non-monotonic reasoning, is also presented in connection with the problems of belief-revision. Finally, the concepts proposed for solving the problems of belief-revision is applied to tackle imprecision, uncertainty and non-monotonicity of evidences in an expert system for criminal investigation. This chapter proposes to design a new methodology for matching of digital grey images using fuzzy membership–distance products, called moment descriptors. These are estimated for three common kinds of image attributes, namely edge, shade and mixed range. The existing methods for matching of digital images, which concern the comparison of the positions of directed edges, shades and mixed range in an image with the same of another image, are often prone to error, due to noise and/or variation in illumination. Fuzzy moment descriptors being less sensitive to noise, makes the matching process invariant to the above stray external disturbances. Further, the normalization and sorting of the moment descriptor vectors keep the matching process invariant to size and rotation of images. The general scheme for image matching presented in this chapter is applied to facial image database for personnel identification. This chapter also explores the scope of template matching and human mood detection from facial images using fuzzy logic. Chapter 16 introduces the synergistic aspects of different computational tools of machine intelligence, including the logic of fuzzy sets, artificial neural networks, GAs and belief networks. Each of these tools has its inherent merits and demerits. However, a judicious mixture of these tools may sometimes improve the performance of the overall system to a great extent. This chapter explores some of the possible applications, where the integral effect of two or more computational models far exceeds their individual effects. A case study indicating the synergism of two different neural nets and GA is undertaken to study its application in motion planning of mobile robots. Chapter 17 extends the scope of application of Widrow-Hoff's ADALINE model from binary to grey level (fuzzy) pattern recognition. The condition of stability for the extended ADALINE model is derived. The algorithm for training the multi-layered feed-forward neural network consisting of ADALINE neurons is presented. The time required for training the neural net is insignificantly small. The scheme for the recognition of objects from their grey-level images, with fuzzy ADALINE model is translation, rotation and size invariant. Mammals perform spatial reasoning by a specialized structure called cognitive maps that are located in the hippocampus region of their forebrain. In the field of computational intelligence, however, the phrase cognitive maps has a wider meaning. It includes encoding of knowledge about causal events and their automated recall. Modelling of cognitive maps by fuzzy logic is apparent because of the inherent fuzziness of most real-world knowledge bases. This chapter provides a thorough overview of various models of cognitive maps and their learning behaviour. The dynamics of the learning models have been analysed to determine the condition for their stability. This chapter ends with a discussion on the application of these models in engineering systems. This chapter presents a model for unsupervised learning and reasoning on a special type of cognitive maps, realized with Petri nets. The unsupervised learning process in the present context adapts the weights of the directed arcs from transition to places in the Petri net. A Hebbian-type learning algorithm with a natural decay in weights is employed here to study the dynamic behaviour of the algorithm. The algorithm is conditionally stable for a suitable range of the mortality rate. A pre-trained network with stable weights may be used in reasoning phase for computing beliefs of the desired propositions from the supplied beliefs of the axioms (places with no input arcs). Because of the conditional stability of the learning algorithm, it may be employed in complex decision-making and learning such as automated car driving in an accident-prone environment. This chapter also presents a new scheme for knowledge refinement by adaptation of weights in a fuzzy Petri net using a different form of Hebbian learning. The scope of computational models of machine intelligence in telecommunication networks is discussed in this chapter. It begins with a brief introduction to computer networks, and outlines two popular reference models of network architecture. The chapter then presents three interesting problems that the design engineers face in the network layer of the reference models. The problems are centred around network routing, congestion control and call admission control. Classical control and optimization techniques are not suitable for real-time solution to these problems. Thus, a computational intelligence approach to solve the problems in real-time is proposed. In this chapter, routing, congestion control and call admission control are addressed by GA, fuzzy logic and artificial neural networks, respectively. This chapter deals with mobile robots and their engineering applications. The chapter begins with a brief introduction to the anatomy of mobile robots, and explores the scope of intelligent models in building automation for the robots. This chapter includes a comparative study of different neural topologies in path-planning application of the robots. It also outlines image segmentation and localization of a moving target in connection with the discussion on target-tracking application of the robots. The scope of extended Kalman filter in the proposed application is also studied in detail. Chapter 22 provides introduction to new members of the computational intelligence family that are currently gaining importance for their applications in both science and engineering. The list of these new members includes artificial life, particle swarms, artificial immune systems, chaos theory, rough set theory and granular computing. The biological concepts involved in the first three topics are briefly explained to enable the readers to construct their mathematical models for specific engineering applications. The chaos theory is introduced to demonstrate the behaviour of fuzzy dynamical systems. The chaotic behaviour of fuzzy dynamics is illustrated with typical models of fuzzy liars. Rough sets and granular computing are presented in a nutshell to familiarize the readers with these growing disciplines of knowledge. Last chapter of the book, Chapter 23, addresses selected research problems in computational intelligence. The problems are introduced informally so that anyone without any background in the specific domain easily understands them. The problems require either a mathematical formulation or a computer simulation for their solutions. An outline to the solution of the problems is suggested. This book represents a comprehensive and excellent survey of current topics in computational intelligence. An in-depth text, the book is highly suitable for use by pre-Ph.D. students, researchers, as well as practitioners in science and engineering. This book also comes with a CD ROM that contains a C/C++ simulation toolbox to allow interested readers to develop application programs. The book cites numerous references at the end of each chapter. Complementary and representative publications related to the areas covered by this book include Reference 1, which provides a forum for the publication of both experimental and theoretical research in computational intelligence, as well as surveys and impact studies. The work in 2 provides an instructional treatment to the subject of computational intelligence both as a text and a reference for real-world applications. The IEEE publication 3 is another note-worthy compilation of current topics in this area.