This past week, I’ve been playing around with more image processing and generation techniques. In particular, I implemented the neural style transfer algorithm by Gatys, Ecker, and Bethge in PyTorch following this tutorial. The paper and technique have been around for a few years, but it wasn’t until now that I have access to a GPU here at Recurse. This was so much fun to implement and experiment with!
My GitHub repo contains instructions on setup and usage, as well as a directory containing many results, if you would like to try it out and explore for yourself!
The approach builds off of the VGG-19, a convolutional neural network pretrained on millions of images. It’s 19 layers deep and built by the Visual Geometry Group, hence VGG-19.
For neural style transfer, we modify the network architecture as such: we insert a content loss layer, using mean squared error, after the fourth convolutional layer; and insert style loss layers, using mean squared error on normalized gram matrices, after the first five convolutional layers.
These are some of my favorite images that resulted from my explorations. Note that this is a curated collection of results.