Author: Jorge Salgado – Data Engineer
Video Game development is a multidisciplinary challenge, since it typically requires knowledge in different areas: programming, music, design, and art, among others. Art, in particular, is especially challenging for some people, since it requires an extensive amount of training to do it proficiently. In the following article, we will explore how we can use IA Generativa to create art assets for video games.
To illustrate this point, the following graph shows a timeline of YouTube searches in the UK, where we compare the amount of searches for “Blender tutorial”, “Unity tutorial” and “Pixel Art tutorial” in the last year. As we can see, although Unity is the most recognised game engine in the market, most people are looking up how to create art assets with Blender.
Search Google trends for various Game Development topics. Made with: https://trends.google.com/trends/
Generative Adversarial Networks
So how can IA Generativa help us to create assets for videogames? One popular approach is Generative Adversarial Networks (or GANs). It is a deep learning model with two neural networks with two objectives. In this architecture, we have a Generator and a Discriminator.
The Discriminator’s goal is to correctly classify images as fake or real, while the Generator’s goal is to produce images that could trick the Discriminator into thinking they are real. After the discriminator labels the image as fake or real, it learns the correct answer and learns from the outcome. If the Generator did not trick the Discriminator, the Discriminator gives feedback to the Generator.
This is an unsupervised learning model, meaning you only need a dataset with tangible assets to train the Discriminator.
Conditional GAN
As described before, we do not have a way to influence the Generator’s output. Here is where conditional GANs come in handy. In addition to the real images, we could label the real images, so the Discriminator learns the label alongside the images.
The point of this is to pass the label to both the Discriminator and the Generator, so the Discriminator labels images as real only if the image and the label match, which will incentivise the Generator to create certain types of images for certain labels.
As an example, if we train our model with a set of images of numbers and apply conditional GAN to the model’s training, the Generator could produce a very convincing “6”, but if it is labeled as “3”, the Discriminator will deem it as fake. If we give this label of “6” to our Discriminator as a condition, the Generator will produce images of the number six, since it will learn that this is what the Discriminator wants.
RESULTS EXAMPLES
Diagram of Stable Diffusion process. Retrieve from: How to Make Game Asset Art with AI (Free and Easy) – Stable Diffusion Tutorial 2022
Midjourney output example. Retrieved from: https://aituts.com/midjourney-pixel-art/
Tools need a hand to wield them
While GANs are not the only (or best) tool to generate videogame art assets, no tool made so far will be able to replace real artists. These technologies do not intend to replace artists or diminish the value of human creativity but rather provide a means to expedite certain aspects of art asset generation. By leveraging Generative AI, game designers can streamline their workflow, generate preliminary assets, and focus on the more intricate and unique aspects of artistry that require their specialised skills and creativity. Game designers who may lack the skills to create scenarios or music can complement their skills with these tools, allowing them to prototype some version of what they envision. Artists and musicians may also use these skills to obtain inspiration for their artwork before committing to a piece.
References
Jorge Salgado – Data Engineer
