How to make a music video with AI

Key takeaways

• An AI music video is built shot by shot. Plan a shot list against the song, generate each clip, then cut them together over the track.
• On Melius, attach the song to a brief to Mel, the Melius agent, along with photos of the artist and the look you want. Mel builds the shots, the lip sync, and the edit, and you'll see all of it on one canvas.
• Reference images keep the artist recognizable from the first shot to the last, so they're the first thing to make, before any video.
• Build the singing from short performance shots. Kling Lipsync takes 2 to 10 seconds of video at a time, so a verse is made from several clips.
• To remake a music video that already exists, swap yourself in with Ditto. It replaces the people, props, and backgrounds in a finished video from your reference photos and keeps the original audio by default.
• Check the rights before you post. Songs made on Suno's Free plan can't be used commercially, and YouTube and TikTok both require a label on realistic AI content.

Making a music video with AI

To make a music video with AI, you start from the finished track, plan a shot for each section of the song, generate those shots with a video model, add lip-synced performance clips for the vocals, and cut everything together on the beat. Most of the work comes down to choosing what each shot shows and keeping the artist looking the same across all of them. If the video you want already exists, you can skip the shot list and remake it with yourself in the lead role instead.

Melius is an AI image, video, and audio generation tool where creatives work alongside AI agents. You brief the Melius agent Mel with your job, and it wires a node-based canvas to build it. The song, the artist references, every shot, and the final edit sit on one canvas, so changing the look of one scene doesn't mean starting over.

Before you start

• Have the final mix. Cuts and lip sync are timed to the audio, so a new mix later means retiming. Upload your own track, or generate one with ElevenLabs Music or MiniMax Music 3.
• Write a shot list. Mark the song's sections (intro, verse, chorus, bridge) with timestamps, and give each one a location, an action, and a camera move. Two lines per shot is enough.
• Pick the format first. Vertical 9:16 suits YouTube Shorts, TikTok, and Reels; a horizontal 16:9 cut suits a standard YouTube upload. If you reframe a finished edit, you'll lose part of every shot.
• Gather the artist's references. Three or four clear photos of the singer or character, plus one or two images of the visual style, do more for consistency than any amount of prompt text.

How to brief Mel with your song

Attach the track and the reference photos in the Mel chat. The chat takes MP3, WAV, M4A, AAC, OGG, and FLAC files, and an attached song becomes a file node on the canvas that Mel connects under the video as its soundtrack. Then describe the video the way you'd brief a director:

"Attached is the final mix of 'Low Tide' (3:12) and four photos of the singer. I want a 9:16 video for Reels: dusk on an empty beach, warm film grain, slow camera moves. Intro (0:00 to 0:18): wide shots of the water, no people. Verse 1: the singer walking the shoreline, lip-syncing the first four lines. Chorus: close-ups singing, cut with shots of a bonfire. Keep her face, braids, and yellow coat the same in every shot. Build the first verse and chorus first so I can check the look."

Mel lays out the canvas from that: the song as a file node, a character sheet from your photos, a node for each shot, and a stitch at the end. You review the first section, give notes, and Mel carries them through the rest.

What Mel builds, and where you take over

• The artist. Mel turns your photos into reference images that every video node uses, so the singer's face and wardrobe hold from shot to shot. If a detail drifts, fix it on the reference image once and rerun the shots that use it. Our guide to keeping AI images and video on brand with reference images goes deeper on this step.
• The shots. Each shot is its own video node, run on a model suited to it, such as Seedance 2.5, Veo 3.1, or Kling 4.0. Kling 4.0 can also build a scene of up to 30 seconds in one generation, guided by a prompt for each shot in it, and uses references to keep the singer and the setting consistent. Change the model or edit one shot's prompt and rerun only that node.
• The performance clips. For the sung lines, Mel connects a short video of the singer and the vocal for those lines to a lip-sync model, so attach those vocal sections alongside the full mix. Kling Lipsync accepts 2 to 10 seconds of video with 2 to 60 seconds of audio, and our video model list has other lip-sync options, such as Sync Lipsync v3, VEED Lipsync v2, and HeyGen Lipsync.
• The edit. A stitch node joins the clips in order, and its editor is our multi-track video editor: trim or crop a clip, set each clip's volume, and ask Mel to mute the clips' own sound so only your song plays. To swap a take, rerun that shot, then run the stitch again to export the finished video. See how to trim, arrange, and export video for the editor in detail.

Remaking an existing music video with you in it

To drop yourself into an existing music video, you don't have to rebuild it shot by shot. Ditto replaces the characters, props, and backgrounds in an existing video using reference images, so you can put yourself where the original performer stood. Ditto is in beta and rolling out gradually, and teams outside the beta can join the waitlist from Ditto in the app.

1. Upload the video and your photos. Ditto takes MP4 or MOV footage plus a reference image for each thing you're replacing.
2. Say who you're replacing. With one person on screen, Ditto picks the main performer. With a band, add a photo for each member you're swapping: Ditto matches distinct people in the order of your photos, or you can describe who each photo replaces, such as "the guitarist on the left."
3. Swap more than the people. A single run can carry up to 30 replacements, so you can change the stage, the backdrop, or a prop like the microphone or the car in the same pass.
4. Keep the song. Ditto keeps the original audio by default. If you'd rather put a different mix under the result, set the audio to mute.
5. Check the cost first. Ditto gives you a free credit quote before the paid run starts, and you can run it on one section of the video to test the look before doing the whole thing.

Cutting to the beat

Cuts that land on the beat are what make an AI video feel edited. To find the beat in frames, divide 60 by the song's tempo and multiply by the frame rate: at 120 BPM and 30 frames per second, one beat lasts 15 frames, and a four-beat bar lasts 60. Cut on the bar for verses and on the beat for a fast chorus, and don't cut on every beat throughout, or the edit becomes predictable.

Plan clip lengths to match. A shot meant to hold for two bars at 120 BPM needs four seconds of usable footage, plus a little extra to trim.

Rights and AI labels

Check the license on the song before the video goes anywhere public. Suno's terms limit songs made on its Free plan to personal, non-commercial use, and subscribing later doesn't extend commercial rights to those songs. ElevenLabs' music terms allow commercial use on all its plans, but only Creator and above can distribute to streaming services.

A remake of an existing music video brings two sets of rights into play: the footage and the song both belong to their owners, which may be a label or a director even when it's your band's video. Our terms require that you have the rights to anything you upload, so work from footage you own or have permission to use, and check that the permission covers the song and an edited version before you post.

YouTube requires creators to disclose realistic AI-generated or altered content, and its help page lists AI music that is the main focus of a video as an example; you set it under "AI use" when you upload. TikTok requires a label on AI-generated content that shows realistic-looking scenes or people.

For more on adding sound to generated clips, see our roundup of AI video generators with audio.

Frequently asked questions

Got a track and a concept? Open Melius, attach the song to a brief to Mel, and describe the first shot.