Restaurant attribute classification using deep learning
Daksh Varshneya, Pawan Dhananjay, Dinesh Babu Jayagopi · 2016
Automatic restaurant attribute classification is an instance of multi-label learning. Users upload hundreds of photos along with textual reviews on websites like Yelp. The users also have the option of labelling the businesses with specific attributes such as if it is good for kids or if it has table service. In our work, we explore a variety of methods to label businesses with attributes using just the photos of the businesses. This data of user uploaded photos is publicly available on the Kaggle website as part of the Yelp Restaurant Photo Classification Challenge, containing 200,000 training and testing images. In our work we compare the results of multiple instance multi label learning and multiple instance single label learning for attribute classification. We show that the former formulation is more accurate. We also bench mark our results with traditional vision based features, which gives intuition into our task. Transfer learning with pre-trained CNNs followed by an SVM Classifier and Extreme Gradient Boosting resulted in an F1 score of 0.79 and 0.80 respectively. We also explore some ensemble models and other techniques to take into account the correlation between labels which result in our highest F1 score of 0.82. Future work includes fine tuning the transfer learning and striving for a better mix of traditional and deep image features.