VOCAVID

STEP-BY-STEP GUIDE

From the first upload
to the final film.

This guide walks through a complete VocaVid run. The order is deliberately practical: first structure and timing, then look and movement, then editing and reels.

Before your first project

VocaVid runs locally on Python 3.11 or newer. You also need a running ComfyUI instance, matching workflow files and ffmpeg. A CUDA-capable GPU is recommended for Whisper and generation.

Tip: Start ComfyUI first and verify that the models and custom nodes required by the workflows are available. Then start VocaVid with python -m VocaVid serve and open http://127.0.0.1:8000.

1. Create a new project

  1. Open “New Project”Create a new music-video project from the dashboard.
  2. Name the projectA clear title helps later when locating the Kdenlive file, MP4 and reel exports.
  3. Set the base formatChoose resolution, FPS and transition handles for the intended final format.

2. Fill in values and source material

  1. Upload WAV and lyricsThe WAV file is the timing foundation. Add SUNO-style lyrics with section markers such as [Verse], [Chorus] or [Bridge].
  2. Define the styleDescribe genre, visual language, mood and recurring motifs. This is the creative guardrail for scene planning and prompts.
  3. Add referencesWhen characters or a recurring look matter, add suitable reference images. Avatar defaults can be prepared globally and adapted per project.
  4. Choose segment sizesSet separately for verses and choruses how many lyric lines should form one scene.

3. Segment automatically — then correct if needed

  1. Start “Analyze + Split”VocaVid aligns lyrics to the song with Whisper and creates the first render segments.
  2. Check timingReview lines with low transcription confidence. If Whisper is not precise enough, VocaVid uses a transparent timing fallback.
  3. Refine manuallyUse extended table mode to correct a line’s start and end, group lines differently or adjust segments.
  4. Set instrumentals deliberatelyCreate an interlude for an intro, break or transition in the manual timing editor. These passages become their own renderable scenes.

Get timing and segmentation right first. Every later scene, clip and the final edit follows them.

4. Scene plan and prompts

  1. Create the scene plan“Scene Plan” creates the narrative thread across the whole song. On long songs, VocaVid carries context between batches.
  2. Edit the planSet recurring places, characters, colours and symbolic motifs before generating images.
  3. Generate image promptsCreate image prompts for the selected segments. Individual prompts can — and should — be sharpened manually where needed.
  4. Generate motion promptsThen create image-to-video prompts: camera movement, pace, action and the intended energy of the scene.

5. Images, avatars and clips

  1. Generate imagesCreate base images for selected storyboard cards and compare them in preview.
  2. Create avatars only where they carry the sceneTypical use is a chorus, hook or recurring appearance. Avatar variants help retain one character across scenes.
  3. Choose the image sourceFor each segment, decide whether the clip starts from the base image or an avatar variant.
  4. Generate clipsRender selected scenes as video. The global render queue remains visible.
  5. Review and protectMark good videos OK. Protected clips are not overwritten by later batch actions; regenerate weak moments selectively.

6. Finish, Kdenlive and MP4

  1. Review the entire storyboardCheck order, media source, timing and OK status one final time.
  2. Run “Assemble Final”VocaVid creates an editable Kdenlive project from approved clips, with video and audio tracks plus transitions.
  3. Open the Kdenlive projectUse the .kdenlive file for the last edit, titles, colour correction or manual transitions.
  4. Render MP4Use “Render MP4” to create the final export. If an MP4 already exists, VocaVid opens its preview.

7. Reels from the finished video

  1. Open “Make reels”Switch to the reel workflow after MP4 export.
  2. Start analysisWhisper alignment plus beat and energy analysis suggest musically interesting vertical excerpts.
  3. Review candidatesRender the best moments as previews and assess framing and selection.
  4. Export final reelsOutput approved candidates as vertical exports — ready for Shorts, Reels or TikTok.