Hadoop based Network Traffic Analysis to Count Domain Hits from Large Internet Logs
Hasmukh B. Domadiya, Girish C. Bhimani · International Journal of Research in Advent Technology · 2019
The Internet has millions of web resources each of which with unique URL -Uniform Resource Locator.Nowadays every website has a large number of web resources under one domain name.The recent era of computerization and digitalization is seeing a rapid increase in the number of users as well as number of web resources accessed by these users.It is indeed a necessity to analyze user activities by finding some information from their Internet logs.Conventional methods like sniffers, firewalls are suitable for small to medium scale networks due to limited processing capabilities.Hadoop is the present solution which can be used to analyze Internet logs generated by a large or very large scale networks.This piece of work is based on analysis network traffic by collecting URLs visited by users.Domain hits are counted based on Hadoop.The implementation has been done with Cloudera Apache Hadoop Ecosystem.