A Distributed Approach for Mining Moroccan Hashtags using Twitter Platform
Abdeljalil El Abdouli, Larbi Hassouni, Houda Anoun · 2019
Twitter is a social networking service, on which users can share thoughts and interact with events. In this paper, the authors propose a distributed approach to combine the multilingualism analysis of hashtags generated by Moroccan users in the social network Twitter, to discover and understand the hot subjects that attract the community. The analysis of Moroccan twitter hashtags is a challenge for two main reasons. Firstly, since the Moroccan society is characterized by linguistic diversity, hashtags are expressed in several languages. Secondly, the hashtags of Moroccan users may include spelling errors and abbreviations and do not contain delimiters between words, which leads to misinterpretation. In this paper, we propose a distributed approach using Apache Hadoop Framework and Natural Language Processing Techniques for processing and mining Moroccan hashtags by a program we developed using open source libraries. The result is a clean corpus, which is stored in Apache Hive to allow applying analytic queries. Finally, we apply K-means algorithm to cluster all hashtags into general topics, and then plot them on the Moroccan map to specify their sources of publication by using the coordinates extracted from the tweets.