Death Masks : An AI Bricolage
Type : AI Pipeline and Artifacts
Role : Senior Researcher, Project Manager
Tools : Nvidia GauGAN, RunwayML, Blender
AI tools for generating digital art are in their infancy. While some AI digital creation tools are genius such as the famous Dall-E 2, the majority are indecipherable or derivative such as many neural style transfer programs. This is because AI does not generally engage with the visual language with which humans construct meaning. Instead it extrapolates patterns from data in often unpredictable ways. One means to improve the production of meaningful AI - generated art is bricolage. Bricolage is defined by the Oxford Dictionary as construction or creation from a diverse range of available things. In the case of AI, these things are the combination of human intervention in art and decision-making, and the construction of AI pipelines, feeding results from one model into another. Death Masks, a project for which I was the lead researcher, demonstrates the capacity of AI bricolage to produce evocative 3D forms from a generic selfie.
The Problem
Pictured Above : Selfies analyzed by an AI facial feature parser
Death Masks began with the development of an interdisciplinary team including 3D Artists, Engineers, and Designers. We first investigated available software for art production (such as Nvidia's GauGAN, and the RunwayML suite) for its capacity to rapidly prototype and produce manipulateable results. Then, with a goal to create artifacts that were personal, evocative, and relatable we decided to pursue developing user avatars and portraits. We arrived at the idea of Death Mask as an art form and cultural practice taking inspiration from two primary sources. Neri Oxman produced a series of 3-D generative masks and geometries in Vespers [https://www.media.mit.edu/projects/vespers-iii/overview/ ] that highlight how generative processes can be used as a tool for exploration and design. The web application This Person Does Not Exist [https://thispersondoesnotexist.com/ ], which creates realistic images of people who in fact are not real, demonstrates the articulate skills of extrapolation and awe-inspiring ability to question our digital reality possessed by AI. Having established a pipeline of tools, goals, and parameters we embarked on AI bricolage
The Research
Pictured Above : Textures / abstract landscapes by GauGAN
Pictured Below : Importing and manipulating the AI generated 3D model in blender
A Death mask begins with a typical selfie (the first example of human intervention)that is processed in two different ways by AI. The first AI pipeline parses facial features into regions and then feeds these results into GauGAN for the creation of a unique texture image. The second pipeline extrapolates a 3D model of the user's face from the provided selfie. The results of these two processes are then combined to create a personal Mask unique to the landscape and digital analysis of a specific face. This rough-hewn marble of a face is then refined through 3-D manipulation techniques to produce a more compelling image. More specifically, 3D manipulation techniques are used to alter volumes based on the texture, animations are added, and materials such as cloth are overlaid. This third step involving human intervention is critical for producing an aesthetically pleasing outcome. The Death Mask project culminated in the production of 5 unique digital artifacts - ethereal masks - representing the members of the team.
The Results
Pictured Below : 3 resulting masks with animation, materials., and texture applied