Noise Removal and Structured Data Detection to improve search for personality features
Muhammad Fahim Uddin · 2016 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) · 2016
This paper discusses part one of the main work in field of data science, mining and analytics. Family of algorithms is developed to predict the educational relevance of individuals' talents through lens of personality features (unstructured and semi-structured) and academic/career data. The big data (unstructured and semi-structured) contains lots of valuable information that can be mined and analyzed. However, such processing and utilization of data introduce challenges of dealing with noise (irrelevant, unnecessary and redundant data). Regardless of the nature of data processing and utilization for a given problem or an application, noise adds unnecessary time and cost. This paper briefly discusses the overall research work and then presents Noise Removal and Structured Data Detection (NR-and-SDD) algorithm and related math construct. NR-and-SDD detects the noise to reduce the processing cost and improve structured data detection in relevance of personality features. The given results show improved reliability and efficiency of NR and SDD processes. Related study is provided and paper is concluded with final remarks and future works.