Object Style Transfer

Image style transfer is a technique where a neural network extracts the elements of a photo’s style and then renders a different photo in the style of the first one. It is computationally intensive, but it’s now become one of the standard party tricks of computer vision work. There are several places you can go to online to give it a whirl; I’ve been using https://deepdreamgenerator.com/ to do style transfer onto the same image over and over again. The idea is that this way, I begin to learn slowly what the machine is actually looking at. I began with the well known ancient portrait, woman with a stylus:

And ran it through several different style transfers, including a photograph of our department library:

a nineteenth century photograph of four men:

A zombie:

and the Euphronios Krater:

It seems clear to me that the computer’s attention is very much attracted to the spackling/ visual noise on the left hand side of the painting, as well as her hands. Incidentally, I asked the Visual Chatbot what it thought was going on:

Image recognition algorithms always will tell you how many giraffes there are in an image, because they make guesses – the major training datasets always have giraffes in them, even though giraffes are unlikely to be in your photographs, so if you are asking about giraffes, there must be giraffes in there…

This was all quite fun. But the last example, of the woman-as-styled-by-krater got me to thinking about object style transfer. So many archaeological objects are photographed against neutral backgrounds, it occurred to me that the style transfer algorithm would just apply the ‘style’ of one thing onto the other thing. So I took a photo of a bust of Vespasian…

and transfered the style from the Euphronios Krater …

and achieved this:which is extremely pleasing. I also wondered about mosaics – as the original pixel art – and what that might achieve. Given that it was Super Bowl Sunday and people were sharing pictures of #superbOwls I borrowed an image that Sarah Bond tweeted of a 3rd century owl mosaic:and crossed it against the Laocoon:

and achieved this:So what’s the lesson here? I don’t really know yet. Well, in the first instance, I was just trying to break things, to see what would happen by feeding the algorithm things it wasn’t expecting. What attracts the attention of the machines? But it’s got me thinking more about how archaeological photographs are composed, and what happens when things get remediated. Tara Copplestone once had a blog post up on photobashing as archaeological remediation that was about turning photos into watercolour paintings using filters in Gimp. Remediation forces a readjustment of your archaeological eye and imagination. Here, the process offloaded to an active thing that has its own way of seeing – notice how it focused on the inscription in the mosaic, or the red figure foliage on the krater – that are independent of us and independent of its training. The automated aesthetic?