ING's Next Rembrandt project proves god is in the machine

In Neal Stephenson’s sci-fi novel The Diamond Age, there is a device called the Matter Compiler, which assembles molecules to create clothing and food from scratch. 

 

Does human genius matter when it can be replicated by – even taught to – a machine? The question has been raised by The Next Rembrandt, a project run by J Walter Thompson Amsterdam for ING. 

 

To celebrate ING’s sponsorship of the Rijksmuseum, the agency came up with the idea of creating a "new" Rembrandt portrait, based on information gathered from the artist’s existing works. The resulting image won’t fool art experts, or knowledge-able enthusiasts, but it has rattled the art world.

 

"Do you need to have a soul to touch the soul?" Bas Korsten, executive creative director at JWT Amsterdam, muses to Campaign, shortly before picking up two Grands Prix at Cannes for the project.

 

Through a scanner darkly

 

In collaboration with Microsoft, Delft University of Technology, the Mauritshuis in The Hague and the Rembrandt House Museum, JWT Amsterdam started by choosing 346 Rembrandt paintings to scan. The team wrote neural network software to even out these scans, since not all were of the same quality, and analyse and upscale the fragments. The software mapped characteristics such as eyes, noses, widths, depths, brush strokes and canvas quality. 

 

Microsoft stepped in to handle the sheer amount of data processing, while the university and museums helped choose which Rembrandt portraits to scan.

 

The team then used the scanned data to determine what kind of person to paint – a male, aged 30 to 40, with facial hair, a hat, dark clothing and seated facing right. (A Google image search for "Rembrandt portraits" confirms the old master frequently painted middle-aged men in hats.)

 

The team narrowed its portrait options to the 40 that fitted this description, then examined them in greater depth to determine how to create something "new". It took 18 months and nine iterations but, eventually, The Next Rembrandt was born.

 

According to innovation director Emmanuel Flores, it is important to remember that the final portrait does not comprise averaged-out features. Flores explains that rather than taking, say, the sum of all the lips in Rembrandt’s portraits and creating an average version, the software identified recurring patterns and generated new features. 

 

Finally, the team used X-ray data to glean information on Rembrandt’s brush strokes. Unlike a painted portrait, The Next Rembrandt was printed on a completely smooth surface, meaning the brush strokes had to be created by layering ink during the 3D-printing process.

 

The surface of the final portrait has a thickness of 0.1mm. Initially the team experimented with 0.3mm, an effect Flores says made the portrait look wild and bold, "like a Van Gogh".

 

Whatever creatives think of this pseudo-creativity, there are two aspects of the process they will 

recognise. One was that "it was very painful" and the other was that the team only decided the product was finished because the project ran out of time.

 

For Flores, the painting has already passed a kind of Turing test. "There are two conversations happening around this," he says. "One is the institutional conversation, which is that the institution doesn’t accept this is a Rembrandt. Then there’s the human conversation of the people involved in the process, who say that this is very confusing."

 

Flores describes a visit to the painting by Ernst van de Wetering, the world’s pre-eminent Rembrandt expert. The 78-year-old critic was apparently "the biggest hater" of the project from its inception. But on seeing The Next Rembrandt, he began critiquing the inconsistencies of the brushwork.

 

"He started scouting around and saying that a brush stroke here corresponded to 1652, not 1632," Flores explains. "And I was thinking to myself: ‘You’re now in the subtlety. The higher level has been passed.’"

 

Validating the concept

 

At the Cannes Lions festival, Grey for Good’s I Sea app, designed to send help to migrants stranded in boats, was pulled from Apple’s App Store store the day before it won a bronze Lion. Security experts claimed aspects of the app were fake. This was not the first alleged fraud at Cannes but there may be more as entries become more technical and difficult to vet.

 

Flores is unwilling to point fingers when considering I Sea, which remains under investigation by the Cannes organisers. Asked by Campaign about the importance of transparency for technically driven creative, he says: "This was one of the challenges we faced in our own project. Making the code available was one of the things we considered with Microsoft."

 

As yet, Flores cannot promise the team will open-source the project, although this would answer 

immediate questions about credibility. "Is [the code] ready for that? Not yet," he says. "One of my goals is to prepare that to be available to the public. It’s very important to be transparent on this."

 

Perhaps the question of the machine’s creative superiority over humans is the wrong one. More pertinent is the value of creativity. In common with The Next Rembrandt, Stephenson’s fictional Matter Compiler only creates the new according to predetermined patterns set by the old. But what’s the point of owning a costly original work of art when you might one day own a convincing "new" fake that is less familiar to the eye? It’s no wonder the guardians of culture feel flummoxed.

 

(This article first appeared on CampaignLive.co.uk)