Computers predict people’s tastes in art – Pasadena Now

0
Abstract painting by Joseph Schillinger.
Credit: Smithsonian American Art Museum, gift of Mrs. Joseph Schillinger

Do you like the thick brushstrokes and soft color palettes of an impressionist painting like those of Claude Monet? Or do you prefer the bright colors and abstract shapes of a Rothko? Individual artistic tastes have a certain mystique, but now a new study from Caltech shows that a simple computer program can accurately predict which paintings a person will like.

The new study, published in the journal Nature Human Behavior, used Amazon’s Mechanical Turk crowdsourcing platform to recruit more than 1,500 volunteers to evaluate paintings in the genres of Impressionism, Cubism, Abstract, and Color. The volunteers ‘responses were fed into a computer program and then, after this training period, the computer was able to predict the volunteers’ artistic preferences much better than would happen by chance.

“I used to think that art evaluation was personal and subjective, so I was surprised by this result,” says lead author Kiyohito Iigaya, a postdoctoral researcher who works in the lab. Caltech psychology professor. John O’Doherty.

The results not only demonstrated that computers can make these predictions, but also led to a new understanding of how people judge art.

“The main point is that we get some insight into the mechanism that people use to make aesthetic judgments,” says O’Doherty. “That is, people seem to be using basic image features and combining them. This is a first step in understanding how the process works.

In the study, the team programmed the computer to break down the visual attributes of a painting into what they call low-level characteristics – traits like contrast, saturation, and hue – as well as features high level, which require human judgment and include features such as as if the painting is dynamic or still.

“The computer program then estimates how well a specific characteristic is taken into account when making a decision about the degree of appreciation for a particular work of art,” Iigaya explains. “Low and high level functionality is combined when making these decisions. Once the computer has estimated this, it can then successfully predict a person’s taste for another work of art never seen before.

The researchers also found that the volunteers tended to fall into three general categories: those who like paintings with real objects, such as an impressionist painting; those who like colorful abstract paintings, like a Rothko; and those who like complex paintings, such as Picasso’s cubist portraits. The majority of people fell into the first category “real objects”. “A lot of people liked Impressionist paintings,” says Iigaya.

Additionally, the researchers found that they could also train a Deep Convolutional Neural Network (DCNN) to learn to predict the artistic preferences of the volunteer with a similar level of precision. A DCNN is a type of machine learning program, in which a computer receives a series of training images so that it can learn to classify objects, such as cats versus dogs. These neural networks have units that are connected to each other like neurons in a brain. By changing the strength of the connection from one unit to another, the network can “learn”.

In this case, the deep learning approach did not include any of the selected high or low level visual characteristics used in the first part of the study, so the computer had to “decide” which characteristics to analyze. by himself.

“In deep neural network models, we don’t know exactly how the network solves a particular task, because the models learn on their own, much like real brains do,” Iigaya explains. “It can be very mysterious, but when we looked inside the neural network we could tell it was building the same categories of functionality that we ourselves selected.” These results suggest the possibility that the characteristics used to determine aesthetic preference could emerge naturally in a brain-like architecture.

“We’re now actively looking to determine if this is indeed the case by examining people’s brains as they make these same types of decisions,” says O’Doherty.

In another part of the study, the researchers also demonstrated that their simple computer program, which had already been trained on artistic preferences, could accurately predict which photos the volunteers would like. They showed the volunteers photographs of swimming pools, food and other scenes, and saw results similar to those involving paintings. What’s more, the researchers showed that reversing the order also worked: after first training volunteers on the photos, they could use the program to accurately predict the subjects’ artistic preferences.

While the computer program was successful in predicting the artistic preferences of the volunteers, the researchers say there is still a lot to be learned about the nuances that go into an individual’s taste.

“There are aspects of preferences unique to a given individual that we haven’t been able to explain using this method,” says O’Doherty. “This more idiosyncratic component may relate to semantic characteristics, or the meaning of a painting, past experiences and other individual personal traits that might influence the assessment. It may still be possible to identify and learn about these characteristics in a computer model, but doing so will require a more detailed study of each individual’s preferences in a way that may not generalize across individuals. as we found it here.

The study, entitled “Aesthetic preference for art can be predicted from a mixture of low and high level visual characteristicsWas funded by the National Institute of Mental Health (through Caltech’s Conte Center for the Neurobiology of Social Decision Making), the National Institute on Drug Abuse, the Japanese Society for promotion of science, the Swartz Foundation, the Suntory Foundation and the William H. and Helen Lang Undergraduate Summer Research Fellowship. Other Caltech authors include Sanghyun Yi, Iman A. Wahle (BS ’20), and Koranis Tanwisuth, who is now a graduate student at UC Berkeley.

Get our Pasadena Daily Newspaper delivered to your inbox. Release.

Get all the latest Pasadena news, over 10 new stories a day, 7 days a week at 7 a.m.

Leave A Reply

Your email address will not be published.