This repository provides a two-stage pipeline for converting video footage of people into clean, animation-ready spritesheets using Robust Video Matting (RVM).

The pipeline is intentionally split so that:

• Stage A (slow, GPU-heavy) performs high-quality person matting once
• Stage B (fast, CPU-only) can be rerun many times to experiment with cropping, frame skipping, pixel-art quantisation, and spritesheet layout

input/                   # Raw MP4 videos
intermediate/            # Stage A output (RGBA frames + metadata)
output_frames/           # Stage B output (spritesheets + frames)

stage_a_extract.py       # RVM-based person extraction (GPU recommended)
stage_b_postprocess.py   # Spritesheet generation + quantisation
requirements.txt

This project is licensed under GPL-3.0 to comply with the license of Robust Video Matting. Assets generated by this tool (spritesheets, images, metadata) are not subject to the GPL and may be used freely in commercial or non-commercial projects.

• Python 3.9+ (3.10 or 3.11 recommended)
• Git
• (Recommended) NVIDIA GPU with CUDA for Stage A

Stage B runs entirely on CPU.

This project depends on git submodules (e.g. Robust Video Matting), so you must initialise and update them.

Clone this repo then:

cd video2spritesheet
git submodule update --init --recursive

Create and activate a virtual environment (recommended):

python -m venv venv
source venv/bin/activate   # Linux / macOS
# venv\Scripts\activate    # Windows

Install dependencies from requirements.txt:

pip install -r requirements.txt

Stage A:

• Uses Robust Video Matting
• Produces full-resolution RGBA frames
• Computes global bounds and foot position
• Runs once per source video

NB. A sample mp4 is already provided so you can go ahead and run and experiment.

Place MP4 videos in:

input/

python stage_a_extract.py

intermediate/<video_name>/
├── frames_rgba/
│   ├── frame_00000.png
│   └── ...
└── bounds.json

Stage B:

• Crops using Stage A metadata
• Normalises to fixed sprite size
• Performs motion-based frame skipping
• Supports target FPS + ping-pong safety
• Applies pixel-art quantisation
• Builds spritesheets

NB. Stage B is safe to rerun multiple times. Adjusting these parameters does not require re-running Stage A.

(Windows)

python stage_b_postprocess.py ^
  --input-dir intermediate ^
  --output-dir output_frames ^
  --sprite-size 128 ^
  --frames-per-row 8 ^
  --padding-ratio 0.12 ^
  --pixel-scale 0.25 ^
  --palette-colors 32 ^
  --motion-threshold 0.01 ^
  --target-fps 12 ^
  --max-frames 64

(Linux)

python stage_b_postprocess.py \
  --input-dir intermediate \
  --output-dir output_frames \
  --sprite-size 128 \
  --frames-per-row 8 \
  --padding-ratio 0.12 \
  --pixel-scale 0.25 \
  --palette-colors 32 \
  --motion-threshold 0.01 \
  --target-fps 12 \
  --max-frames 64

output_frames/<video_name>/
├── frames/
├── <video_name>_spritesheet.png
└── <video_name>_meta.json

To experiment with visual styles or animation timing, edit stage_b_postprocess.py and rerun only Stage B.

• Videos should be fixed-camera and fully visible
• Even lighting improves segmentation quality
• Spritesheets are packed row-major, top-to-bottom

This project uses Robust Video Matting as a submodule. Please ensure you comply with its license when redistributing or offering this as a service.

Happy animating ✨