We built a hybrid AI-human workflow and then produced a song that won the popular vote in the international AI Song Contest 2021, as well obtained a mention in the New York Times and the 11th overall place in the contest.
As part of a large multi-disciplinary team of artists and engineers, I was in charge of creating the hybrid AI-human workflow with MusicVAE, Google Magenta and other transformer-based models. Our goal was creating tools that were easy to use for our musicians to interact with, and iteratively create the song at every layer, from sounds, chord progressions, structure and arrangement.
Some work that didn’t make it in the final release involved a way to use evolutionary computation to mutate and recombine melodies and chords in latent space, which led to very natural-sounding phenotypes compared to applying random mutatons to the latent space.
Watch the music video here: