Summary

Our customer offered a service that allowed users to commission artists for custom songs. They wanted us to create an add-on that included a lyric video generated from the lyrics of the song. We trained a custom speech-to-text model, fine-tuned to perform the task on human singing with forced alignment for character level timestamps. We combine the output with Blender rendering to create a synchronized lyric video with the custom song.