Digital Curation
Research Lead / Experience Design / Development (2020 - Present)
At what point do people start to find discerning between human- and GAN-created artwork challenging, and does this point vary for different art styles? Does appreciation of GAN-generated artwork vary by art style and change with the complexity of the GAN output?
Digital Curation is a research project and web-based interactive experience that seeks to answer these questions. The experience allows participants to curate a gallery comprised of both human-made and GAN-generated images, in order to understand what a GAN is, how GAN output evolves as it trains, and what artistic outputs GANs are capable of.
What is a GAN?
Generative adversarial networks (GANs) are a type of machine learning algorithm used to create the generative artwork for this experience. They are comprised of dueling neural networks whose competition leads to the creation of complex outputs, such as images that can mimic artistic styles and content.
Two neural networks make up a GAN — a generator that generates images, and a discriminator that attempts to distinguish real images from generated ones when shown them. The better the discriminator's performance, the more the generator will learn to improve its images in the subsequent rounds, a give-and-take method capable of arriving at surprising results.
Experimental Design
The web experience is designed to conduct a mixed-methods HCI research study on how human appreciation of GAN-produced artwork is affected by style and fidelity of composition. The study is designed to be a between-subjects evaluation of discernment and preference of GAN-generated artistic output, with the art styles — Realism, Post Impressionism, and Abstract Expressionism — acting as conditions.
Datasets + Training
The generative models in this experience were trained on data from the WikiArt project. Three art styles were chosen for this experience — Realism, Post Impressionism, and Abstract Expressionism. These styles were selected due to their visual distinctness, their varying levels of photo-realism, and because they contained enough images for adequate training.
I developed the experience using two different models, a custom CNN GAN model and a pre-trained StyleGAN2 model developed by Bailees et al., in order to visualize the change from static noise to fully rendered artwork.
Experience Design & Development
During the experience, participants act as “curators” for an art gallery that showcases work by both human artists and generative algorithms. Each round presents them with a selection of both types of artwork.
As the curator, their task is to distinguish human-created from GAN-generated artworks, as well as to select up to three pieces of any kind for your gallery. As time goes by and more rounds are completed, the generative algorithm learns to improve its output and produce higher fidelity, if sometimes surprising, results.
I designed and developed a web prototype of this experience, hosted on the online platform Glitch. It has the internal logic to record discernment and preference selections at each round, and includes data persistence, opt-in data submission to a database, performance statistics that scale with dataset size, and dynamic performance visualizations via Chart.js.
Final Web Experience
After a participant has completed all of the rounds in the experience, they are presented with a summary of how well they were able to distinguish between human- and GAN-created images, as well as a personalized digital gallery (below) featuring all of the images they selected at the end of each round. The experience is instrumented to collect and (optionally) submit user data to a Firebase database, to enable its future use in a mixed-methods HCI study to be run in late 2021.
To learn more and try the experience for yourself, please visit digital-curation.glitch.me.
