Hierarchically-Attentive RNN for Album Summarization and Storytelling
Licheng Yu, Mohit Bansal, Tamara L. Berg · 2017
We address the problem of end-to-end visual storytelling.Given a photo album, our model first selects the most representative (summary) photos, and then composes a natural language story for the album.For this task, we make use of the Visual Storytelling dataset and a model composed of three hierarchically-attentive Recurrent Neural Nets (RNNs) to: encode the album photos, select representative (summary) photos, and compose the story.Automatic and human evaluations show our model achieves better performance on selection, generation, and retrieval than baselines.