“Never fry carrots without cutting.” Cooking Recipe Generation from Videos Using Deep Learning Considering Previous Process
Tatsuki Fujii, Yuichi Sei, Yasuyuki Tahara, Ryohei Orihara, Akihiko Ohsuga · 2019
Research on captioning that modifies the contents of images and moving images with natural language using deep training has had considerable results and attracted attention in recent years. In this research, we aim to generate recipe sentences from cooking videos acquired from YouTube. We treat this as image captioning and propose methods suitable for the task. We propose a method that adds a vector of a sentence already generated in the same recipe to the input of a captioning model. Then, we compare generated sentences and correct sentences to calculate scores. We also propose a data-processing method to improve accuracy. We employ several widely used metrics to evaluate image-captioning problems. We then train the same data with the simplest encoder-decoder model, compare it with correct recipe sentences, and calculate the metrics. The results indicate that my proposal methods help increase accuracy.