Generative AI for Animation

A different kind of Generative AI

Way before GPTs, LLMs, RAGs, Agents, and other such wild AI creatures were all the fuzz and about to take over the world, there were focused, niche applications of Generative AI, and one of them was character animation.

Based off Daniel Holden’s excellent Phased Function Neural Network architecture, I implemented a whole pipeline to train an AI to predict the next pose of a character based on the previous pose and its trajectory. This allowed us to simplify the thousands of lines of code of our previous State Machine based animation system to just a query into the Network.

The heart of the system was a Supervised Neural Network written in Theano -this library has since been discontinued in favor of other CUDA based libraries such as PyTorch, TensorFlow, etc.-, incluing an Adam Gradient Descent Optimizer, that needed extensive labelling such as previous and future trajectories of the character motion, foot steps, as well as the walking phase (please refer to the original paper for details).

The project involved capturing several hours of motion capture data from live actors, converting them to NumPy format, and labelling the data accordingly with ad-hoc automation tools built by me. Leveraging AWS cloud GPUs we trained the Neural Network several times until the results were stable and good, clipping and debugging multiple data and pipeline errors. Finally I had to integrate the Neural Network Inference system into our application and scale it to thousands of virtual humans.

Ultimately this effort contributed to the successful acquisition of my company AXYZ Design by Chaos Group in 2023.