AI Skill Report Card

Producing Roblox Animated Videos

A-88·Nov 17, 2026·Source: Web
14 / 15

Given a script + narration audio + reference images, produce a rendered video:

Bash
# 1. Analyze audio timing (drives all scene timing) node scripts/analyze-audio.ts narration.mp3 --out timing.json # 2. Generate storyboard from script + timing node scripts/storyboard.ts script.md timing.json --out storyboard.json # 3. Resolve assets (search CC0 libraries, fall back to procedural) node scripts/resolve-assets.ts storyboard.json --out assets.json # 4. Generate Remotion scene compositions (one .tsx per shot) node scripts/generate-scenes.ts storyboard.json assets.json --out src/scenes/ # 5. Render chunks with checkpointing npx remotion render src/index.ts MainVideo out/video.mp4 --concurrency=1 # 6. Composite final (narration + SFX + subtitles) node scripts/compose-final.ts out/video.mp4 narration.mp3 sfx.json --out final.mp4

Never hand-author full scenes without first checking assets.json for a reusable CC0 GLB — only generate procedural geometry when nothing suitable exists.

Recommendation▾
Add a concrete bad-output example (e.g., a failed QA case with wrong fix) to show contrast, not just good outcomes
15 / 15

Progress checklist for every production:

- [ ] 1. Ingest: parse script, narration audio, reference images/videos
- [ ] 2. Audio analysis: extract duration, pauses, emphasis, word timings
- [ ] 3. Visual reference analysis: camera angles, lighting, palette, framing style
- [ ] 4. Storyboard: break script into scenes → shots, each with duration from audio
- [ ] 5. Asset resolution: match each shot's needs to CC0 assets (Kenney first), else flag for procedural generation
- [ ] 6. Scene coding: write R3F/Three.js scene per shot (environment, lighting, camera rig)
- [ ] 7. Character animation: keyframe R6 rigs (bones) per action beat, synced to timing.json
- [ ] 8. Camera & blocking: define camera paths, cuts, character positions per shot
- [ ] 9. SFX/VFX pass: attach sound cues and particle/visual effects to narration beats
- [ ] 10. Render: chunked Remotion render with checkpoint files per shot
- [ ] 11. Compose: FFmpeg mux video chunks + narration + SFX + subtitles
- [ ] 12. QA: frame-sample inspection, audio-sync check, asset consistency check

1. Ingest & Analysis

  • Parse script into scene/beat units (dialogue, action lines, emotional cues).
  • Run forced alignment or waveform analysis on narration to get per-word/per-sentence timestamps and silence gaps. This file (timing.json) is the single source of truth for all durations — never hardcode shot length; derive it from audio.
  • For reference images/video: extract camera angle (low/eye/high, wide/medium/close), lighting mood (key/fill/rim presence, color temperature), and composition (rule of thirds, framing) as structured tags, not prose — they get consumed programmatically by scene generation.

2. Storyboarding

  • One shot = one continuous camera setup + one time range from timing.json.
  • Each shot entry: {sceneId, shotId, startFrame, endFrame, characters[], environment, cameraMove, action, emotionalBeat, referenceTags}.
  • Keep shots short (2–6s) by default; longer only if narration pacing demands it.

3. Asset Resolution

Priority order per asset need:

  1. Exact or near match in Kenney/CC0 library already cataloged locally.
  2. Search external CC0 repositories (Kenney, Poly Pizza, OpenGameArt — filter strictly to CC0/CC-BY).
  3. Adapt a close-match asset (recolor, rescale, recombine) rather than building new.
  4. Only generate procedural geometry (primitives, simple parametric meshes) if no asset fits — keep poly count low.

Record every chosen asset's source, license, and local path in assets.json for reproducibility and reuse across scenes (consistency).

4. Character Animation (R6 rigs, GLB)

  • Animate via deterministic keyframes on named bones (Head, Torso, LeftArm, RightArm, LeftLeg, RightLeg) — no physics simulation, no randomness, so any frame can be re-rendered in isolation and match exactly.
  • Build a small reusable library of base actions (idle, walk-cycle, run-cycle, turn, wave, jump, sit, point, emotive head-tilt) as parametrized keyframe generators, not one-off hand-animated clips. Compose shot-specific animation by blending/sequencing these.
  • Drive action timing from timing.json beats (e.g., gesture starts 150ms before the word it emphasizes).
  • Lip-sync approximation: map phoneme-ish amplitude envelope from audio to a simple jaw/mouth-open blend value — don't attempt full viseme accuracy unless assets support it.

5. Cinematic Direction

  • Camera as code: define moves as named primitives (push-in, orbit, pan, static-wide, over-shoulder, cut) parametrized by start/end position, target, duration, and easing. Reuse these across shots for visual language consistency.
  • Default to subtle, motivated camera movement — static or slow push unless the story beat demands more.
  • Lighting rig per environment type (interior/exterior/night/day) defined once, reused across all scenes in that environment for consistency.

6. Rendering

  • Each shot renders as an independent Remotion composition/chunk, named deterministically (scene03-shot02.mp4).
  • Checkpoint after each shot renders successfully; a crash mid-production should only require re-rendering unfinished shots.
  • Use SwiftShader (CPU WebGL) as the default renderer — do not assume GPU availability. Keep geometry/texture budgets modest accordingly.

7. Composition & QA

  • FFmpeg concatenates shot chunks in storyboard order, then mixes narration + SFX tracks, then burns/attaches subtitles.
  • QA pass: sample 1 frame per second, check for (a) missing/placeholder assets, (b) character popping/discontinuity between shots, (c) audio drift vs. timing.json, (d) black frames or render artifacts. Flag and re-render only the failing shots.
Recommendation▾
Include sample JSON schema snippets for timing.json/storyboard.json/assets.json to remove ambiguity about exact structure
17 / 20

Example 1: Input: Script line "Max waves goodbye and walks toward the sunset" with narration audio timestamped 4.2s–7.8s, reference image showing warm backlit wide shot. Output: Shot entry {action: "wave→turn→walk-cycle", cameraMove: "static-wide-backlit", duration: 3.6s, environment: "park-sunset (Kenney nature kit, recolored warm)", lighting: "key=warm-orange low-angle, rim=strong"} → keyframed wave (0–1.2s) blended into turn (1.2–1.6s) into walk-cycle (1.6–3.6s) on the R6 rig, camera static matching reference framing.

Example 2: Input: Need a "market stall" prop for a background scene. Output: Search Kenney's CC0 "Mini-Market" pack first → found matching stall mesh → import GLB, log in assets.json with license, scale to scene proportions. No procedural generation needed.

Example 3: Input: QA detects character model scale differs between scene02 and scene05. Output: Root cause: asset reused from two different source packs with different unit scale → normalize both references to the canonical rig scale stored in assets.json, re-render both affected shots only.

Recommendation▾
Clarify fallback behavior when narration audio or reference images are missing/incomplete, since the pipeline assumes all inputs are present
  • Audio timing drives everything; generate timing.json before any scene/shot duration decisions.
  • Build and reuse animation/camera/lighting primitive libraries rather than one-off code per shot — this is what makes consistency and speed possible.
  • Keep a persistent assets.json registry across the whole project so the same character/prop/environment assets are reused everywhere they recur.
  • Default to CC0 (Kenney etc.) assets; only go procedural when nothing fits, and keep procedural geometry simple.
  • All animation must be deterministic/frame-addressable so any shot can render independently and resume after failure.
  • Keep poly counts and shader complexity CPU-render friendly (SwiftShader), not GPU-assumed.
  • Validate with automated frame sampling before declaring a render complete.
  • Don't hardcode shot durations — always derive from narration audio timing.
  • Don't regenerate assets that already exist in CC0 libraries; search first.
  • Don't use physics-based or random animation — it breaks deterministic re-rendering and checkpoint recovery.
  • Don't mix asset packs with inconsistent scale/style without normalizing — causes visual discontinuity between scenes.
  • Don't over-animate the camera; motivated, subtle movement reads better than constant motion.
  • Don't treat narration as a subtitle track bolted onto static scenes — every visual beat should respond to the audio.
  • Don't render the whole video as one monolithic job — chunk per shot for recoverability.
0
Grade A-AI Skill Framework
Scorecard
Criteria Breakdown
Quick Start
14/15
Workflow
15/15
Examples
17/20
Completeness
18/20
Format
15/15
Conciseness
13/15