YouTube’s first AI-generated music tools can clone artist voices and turn hums into melodies

YouTube has announced new experimental AI services, including a feature called Dream Track in YouTube Shorts. It creates up to 30-second soundtracks using AI-generation versions of artists’ voices. Though musicians have mostly pushed back on AI (and their voices being used for training models without permission or compensation), YouTube got nine big names from the music industry to participate, including John Legend, Troye Sivan, CharliXCX and T-Pain. The company hoped to announce the feature at its Made on YouTube event in September but has been in negotiation with recording companies over rights and payments.

Users can access Dream Track by typing an idea into the creation prompt and choosing from one of the participating artists. It uses Google DeepMind’s Lyria — a new, powerful music generation model designed specifically for creating high-quality vocals and instrumentals while giving the user more control over the final product. Any content Lyria produces will also have a SynthID watermark, denoting it as such.

Charlie Puth and T-Pain created sample Dream Tracks, which YouTube has shared as inspiration. However, many of the artists involved expressed their apprehension about AI but hoped that collaborative work could create positive, non-exploitative opportunities. “When I was first approached by YouTube I was cautious and still am, AI is going to transform the world and the music industry in ways we do not yet fully understand,” singer CharliXCX said. “This experiment will offer a small insight into the creative opportunities that could be possible and I’m interested to see what comes out of it.”

Music AI Tools are also coming to YouTube, in collaboration with its Music AI Incubator. These tools can create guitar riffs from a hummed melody or turn a pop track into a reggaeton anthem. Producer and songwriter, Louis Bell, created a sample video to showcase it.

YouTube is walking a tightrope as it navigates the careful balance of introducing AI tools and protecting against misuse. The video platform recently announced new policies for labeling videos made using AI and letting public figures, such as musicians, report deepfakes.

Dream Track is currently only available to a select group of creators and artists, whereas participants of the Music AI Incubator should be able to test the tools out later this year.