Imagine spending a weekend afternoon with friends at an art museum: nodding arms crossed, desperately searching for something insightful to say. The vast majority of the paintings you walk past are immediately forgotten, but some stick in your mind. It turns out that the paintings you remember are probably the same ones that everyone else does.
There is a scientific term for that: image memorability. “It’s the idea that, essentially, there are some intrinsic patterns that make some content more memorable than others,” says Camilo Fosco, a computer science doctoral student at MIT and CTO of memorable AI, a startup that uses machine learning to test how attractive content will be to advertisers and creators. In other words, certain works of art have that je ne sais quoi, and now a team of scientists is using AI to figure out what it is.
in a study published earlier this month in the Proceedings of the National Academy of Sciences, University of Chicago researchers Trent Davis and Wilma Bainbridge show that the memorability of artworks is not only consistent across people, but also predictable by AI. In an online experiment, they pulled some 4,000 paintings from the Art Institute of Chicago’s database, excluding anything the institute labeled “driven” or especially famous. Over 3,200 people viewed hundreds of images, so each painting was viewed by about 40 people. The volunteers were then shown the paintings they had seen mixed with others they had not seen and asked whether or not they remembered them. People were really consistent: they all tended to remember (or forget) the same images.
Using a deep learning neural network called ResMemDesigned by data scientist Coen Needell as part of his master’s thesis at the Bainbridge Psychology Lab, the research team was able to predict the probability that each painting was memorable. ResMem roughly mimics how the human visual system passes information from the retina to the cortex, first processing basic information like edges, textures, and patterns, and then escalating to more abstract information, like the object’s meaning. Their memorability scores were highly correlated with those provided by the people in the online experiment, even though the AI knew nothing about the cultural context, popularity, or meaning of each piece of art.
Counterintuitively, these findings suggest that our memory for art has less to do with subjective experiences of beauty and personal meaning, and more to do with the artwork itself, which may have important implications for artists, advertisers, educators, and anyone who wants their content to stick in their brains. “You might think that art is a very subjective thing,” says Bainbridge, “but people are surprisingly consistent in what they remember and forget.”
Although the online experiment was an intriguing start, he continues, “it’s more interesting if we can predict memory in the real world.” So, along with Davis, then a college student double-majoring in neuroscience and visual arts, Bainbridge recruited 19 more people to wander through the American Art wing of the museum as if exploring with friends. The only requirement was that they see each piece at least once. “Especially as an artist, I wanted the results to apply to the real world,” says Davis, who is now the lab’s manager. “We wanted it to be a natural and enjoyable museum experience.”
