Producing Roblox 3D Animation
Markdown--- name: producing-roblox-3d-animation description: >- Produces Roblox-style 3D animated storytelling videos with a deterministic, code-driven pipeline (Node/TypeScript + Remotion + Three.js + @remotion/three + SwiftShader + FFmpeg). Use when the user asks to make, remake, or continue a Roblox/R6 3D story animation; analyze a Roblox or game-engine video reference; turn a script or narration into a storyboard and 3D scenes; rig or animate the roblox_r6_rigged.glb character (walk, run, turn, gestures, expressions); build procedural or CC0 Roblox-style environments; sync visuals to narration timing; render frames in chunks; or debug broken 3D output such as missing limbs, limbs swinging sideways, inverted faces, blank frames, or headless WebGL crashes. Also use for evidence-based visual QA of rendered frames, asset license checks (CC0 only), and render-cost/quality gate decisions on a GPU-less 4-core machine. Do not start the production pipeline (P1) without explicit approval — benchmark harness work is P0 only. --- # Producing Roblox 3D Animation
Before touching anything, classify where you are:
Tested → rendered + visually verified in p0/
Designed → documented in production-design/, zero implementation
Not started → e.g. P1 production pipeline
Then run the reference analysis → audio timeline → storyboard → asset check → implement → render preview → inspect for defects → fix root cause → re-render → final render → report loop described below. Never skip the "inspect against rendered frames" step — code executing is not evidence of correctness.
Hard rule, always: keep @remotion/three + SwiftShader, 1280×720 @30fps, CC0
assets only, roblox_r6_rigged.glb as the character, and f(sceneId, absoluteFrame)
determinism (no useFrame(), Date.now(), unseeded Math.random()). Do not
start P1 without explicit written approval.
Progress:
- [ ] Step 0: State check — read checkpoint doc, classify tested/designed/not-started, confirm scope (P0 only unless approved)
- [ ] Step 1: Reference analysis (rea-video) — read VIDEO.md → timeline.md → shots/*.md only if needed; override mislabeled AI/live-action verdicts with visual review
- [ ] Step 2: Audio-first timeline — ffprobe duration, 16kHz mono copy, VAD, forced alignment, emphasis detection, totalFrames = ceil(duration*30)
- [ ] Step 3: Storyboard + chunk plan — motion budget ≥0.9, no subtitle-only scenes, final scene resolves, chunk boundaries snap to VAD gaps
- [ ] Step 4: Asset research — Kenney/KayKit CC0 search first, verify license, log registry (source, license, author, SHA-256), build procedurally if no CC0 fit
- [ ] Step 5: Implement — explicit GLTFLoader+delayRender, preserve rest quaternions, pre-multiply animation deltas in parent frame, instance repeated meshes
- [ ] Step 6: Preview QA — render frames, compare to reference, diagnose with bind-pose control render / side-camera trick, fix root cause, re-render, re-inspect
- [ ] Step 7: Cost gate — measure s/frame and RAM, compare to baseline table, apply 1.5x regression gate, never reduce resolution/fps to pass
- [ ] Step 8: Final render — chunked with guard frames, frame ledger sums to totalFrames, overlap hash check, continuous master-mix.wav mux
- [ ] Step 9: Production report — tested vs designed vs not-started, defects found and fixed with evidence, cost numbers, outstanding approvals needed
Do not advance a step until the previous one has rendered evidence, not just code that runs.
Example 1 — Diagnosing missing limbs
Input: Rendered frames 0–60 of R6 walk cycle show legs missing and arms jutting sideways.
Output: Render a bind-pose control frame with all bone mutation disabled. Character is intact → animation code is at fault, not loading/materials. FK inspection shows leg rest rot.x ≈ π; code was overwriting it with an absolute 0.22, pointing legs horizontally. Fix: store rest quaternion, add delta as deltaParentX(angle).multiply(rest), never assign rotation.x directly. Re-render and confirm legs/arms present before moving on.
Example 2 — Diagnosing in-out vs front-back swing
Input: After the rest-quaternion fix, legs now swing in/out instead of front/back; tracking camera makes it ambiguous.
Output: Temporarily switch to a pure side camera on the X axis. Legs move in screen depth → confirms torso-local X motion (wrong). FK check: rest × localX → [-0.46,-1.76,0] (X=in-out) vs parentX × rest → [0.5,-1.76,-0.96] (Z=front-back, correct). Switch multiplication order to pre-multiply in torso frame. Restore original camera. Re-render and verify front-back motion before declaring fixed.
Example 3 — License conflict Input: A "New York Cartoon City" GLB looks visually perfect for the street scene but is CC-BY-4.0. Output: Reject it. Log the conflict in ASSET_REGISTRY.md (asset name, why rejected, license found). Build a procedural CC0 street instead, or source an equivalent from Kenney/KayKit. Never silently use the CC-BY asset even if swapping later "looks fine."
- Treat
VIDEO.mdas the entry point for any reference analysis; only drill intoshots/*.mdwhen a specific shot is ambiguous. Never readbreakdown.jsondirectly. - Visually override tool verdicts on game-engine footage — YouTube/transcode artifacts cause false "AI-generated" or "live action" labels. This has happened twice; always sanity-check with your own eyes.
- Audio is the master clock. Compute
totalFramesonce from duration; never time-stretch (atempo,asetrate,playbackRate,rubberbandare all forbidden). - Keep a hard separation between Tested / Designed / Not started in every status report. Never let a working prototype imply production readiness.
- Use the measured baseline table (not guesses) for every cost decision. Apply the 1.5× regression gate and 12 GB RAM ceiling strictly.
- Optimize CC0 props for CPU rendering: ≤5,000 tris, flat Lambert materials, ≤256px textures, instance repeated meshes.
- When 3D motion direction is ambiguous under a tracking camera, swap to a pure side/axis camera temporarily to disambiguate, then restore the original camera.
- Prefer fake contact shadows / additive glow over real shadow maps; only pay for a 512 shadow map when the visual gain is measured and justified.
- Record every defect found and the root cause fix in the production report — do not silently accept a flawed frame.
- Overwriting
rotation.x/y/zwith absolute values instead of pre-multiplying a delta onto the preserved rest quaternion → missing/distorted limbs. - Post-multiplying rotation deltas on the bone's local axis instead of pre-multiplying in the parent (torso) frame → limbs swing in the wrong plane.
- Assuming GLTF bone names match the source file — GLTFLoader replaces spaces with underscores (
Left Arm_02→Left_Arm_02). - Using
useGLTF+Suspenseinside Remotion, whereframeloop='never'prevents Suspense from ever resolving — use explicitGLTFLoader+delayRender/continueRender. - Calling
continueRender()before React has committed loaded-state, causing blank frames on short ranges — trigger it from auseEffectkeyed on the loaded state. - Leaving
CanvasTexture.flipYat its defaulttrue→ inverted or invisible faces. - Setting luminance threshold too high when regenerating face textures → strokes vanish after recolor; use 0.45.
- Async GLB loads completing out of order → character colors/sides swap between renders; sort by intended order before
setState. - Using
-vsyncwith FFmpeg 9 (removed) — use-fps_mode vfr. - Computing global SSIM on full frames → near-zero meaningless scores; use windowed 8×8 SSIM and crop bottom ~18% (UI/subtitle area).
- Dividing render-time projections by concurrency twice → wildly optimistic ETAs. Formula:
minutes = seconds × 30 × s_per_frame / 60. - Deriving prop material colors from scratch instead of
src.color.getHexString()→ washed-out recolored assets. - Assuming a bug fixed in one scene file is fixed everywhere —
anim/,r6/,v3/may all contain the same overwrite-rotation pattern; audit siblings before reuse. - Starting or expanding P1 (production pipeline) without explicit approval, even if P0 benchmarks look promising.
- Accepting a CC-BY or unlicensed asset because it looks better than the CC0 alternative.