Memorability Beyond Semantic Features: Probing Memorability of Semantically Similar Images Through Generative Diffusion Model
Hye-Won Han, Yalda Mohsenzadeh · Journal of Vision · 2025
What makes an image more memorable? Recent findings suggest that semantic features exert a stronger influence than perceptual features on the memorability of object images (Kramer et al., (2023)). However, semantically similar images can still exhibit a diverse range of memorability. What features of an image make it more memorable among semantically identical images? To address this question, we propose a novel framework that generates cloned stimuli from original images, replicating both the image features and the memorability of the originals. We utilize the THINGS dataset (Hebart et al., (2023)), a naturalistic image collection comprising diverse kinds of object concepts and their associated memorability scores. First, we develop a memorability predictor model tailored to the THINGS dataset using CLIP (Radford et al., (2021)) and leverage recent advancements in large-scale generative diffusion models (Rombach et al., (2022)) to generate clones of these images. We demonstrate that the cloned images successfully preserved image attributes and features from the originals and exhibited comparable memorability scores. Subsequently, we further investigate what makes an image more memorable with the most forgettable and most memorable cloned images by analyzing groups of artificial neurons responsible for specific image features through the decomposition of image representations (Gandelsman et al., (2024)). Our findings reveal that cloned images not only successfully replicate the memorability of their originals but also exhibit variability in their memorability. By analyzing these variations, we gain deeper insights into perceptual features that drive image memory in addition to semantic features. This study highlights the potential of generative diffusion models to explore the cognitive and computational attributes underlying image memorability.