Automating Video Clipping
Python# Minimal clipping pipeline: source -> detect moments -> cut -> export import subprocess import json def extract_clip(source_path, start, end, output_path): """Cut a single clip using ffmpeg (stream copy when possible for speed).""" duration = end - start cmd = [ "ffmpeg", "-y", "-ss", str(start), "-i", source_path, "-t", str(duration), "-c:v", "libx264", "-c:a", "aac", # re-encode for frame-accurate cuts "-preset", "fast", output_path ] subprocess.run(cmd, check=True) def batch_clip(source_path, moments, out_dir="clips"): """moments: list of dicts [{'start': 12.5, 'end': 42.0, 'label': 'funny_moment'}]""" import os os.makedirs(out_dir, exist_ok=True) for i, m in enumerate(moments): out_path = f"{out_dir}/{i:03d}_{m.get('label','clip')}.mp4" extract_clip(source_path, m["start"], m["end"], out_path) print(f"Saved: {out_path}") # Example usage moments = [ {"start": 120.0, "end": 155.5, "label": "intro_hook"}, {"start": 890.2, "end": 932.0, "label": "key_insight"}, ] batch_clip("input_video.mp4", moments)
Progress:
- Step 1: Identify the source (local file, livestream VOD, YouTube URL, screen recording)
- Step 2: Choose a moment-detection method (manual timestamps, transcript keyword search, audio energy/silence detection, or AI scene/highlight scoring)
- Step 3: Generate a list of candidate clip ranges with start/end times and labels
- Step 4: Cut clips with ffmpeg (or moviepy), applying padding/buffer around each timestamp
- Step 5: Post-process each clip (crop to vertical 9:16, add captions/burn-in subtitles, normalize audio)
- Step 6: Export with consistent naming and metadata (title, source timestamp, platform target)
- Step 7: (Optional) Auto-upload or queue to destination (TikTok/Shorts/Reels folder, cloud storage, scheduler)
Step details
Step 1 — Ingest source
- Local file: use path directly.
- YouTube/stream: download with
yt-dlp -f bestvideo+bestaudio -o source.mp4 <URL>. - Live capture: record with OBS/ffmpeg screen capture to a file first; don't clip live.
Step 2 — Detect moments Pick one strategy based on available signal:
- Transcript-based: Transcribe with
whisperor an ASR API, then search for keywords, laughter tags, questions, or sentiment spikes. Map matched text segments back to timestamps. - Audio-based: Use
librosa/pydubto find loudness spikes (crowd reaction, laughter) or silence gaps (natural cut points). - AI highlight scoring: Pass transcript chunks to an LLM with a scoring prompt ("rate 0-10 how clip-worthy this moment is, return JSON") and keep top N above a threshold.
- Manual: User supplies a list of timestamps directly — skip detection.
Step 3 — Build clip list
Always normalize to a list of {start, end, label} dicts. Add 1-2s padding before/after natural speech boundaries — snap to nearest silence if possible to avoid mid-word cuts.
Step 4 — Cut
- Use
-c copy(stream copy) for speed when exact frame accuracy isn't critical and cut points land on keyframes. - Use re-encode (
-c:v libx264) when cut points are arbitrary — guarantees accurate trims.
Step 5 — Post-process
- Vertical reformat:
ffmpeg -i in.mp4 -vf "crop=ih*9/16:ih,scale=1080:1920" out.mp4 - Burn captions: generate
.srtfrom transcript, then-vf "subtitles=clip.srt". - Normalize loudness:
ffmpeg -i in.mp4 -af loudnorm out.mp4.
Step 6 — Export & name
Use pattern: {index}_{label}_{platform}_{duration}s.mp4. Store a manifest JSON alongside clips logging source timestamps for traceability/re-cutting.
Step 7 — Automate end-to-end Wrap steps 1-6 in a single CLI script or scheduled job (cron, GitHub Action, or watch-folder trigger) so new source videos auto-produce clips without manual intervention.
Example 1:
Input: A 2-hour podcast MP4 + instruction "clip the 5 most insightful moments as vertical shorts with captions"
Output: Transcript generated → LLM scores 20 candidate segments → top 5 selected → each cut with padding → cropped to 1080x1920 → captions burned in from SRT → files named 001_insightful_moment.mp4 ... 005_insightful_moment.mp4 in clips/ with a manifest.json listing original timestamps.
Example 2:
Input: User provides manual timestamp list: [(00:01:30, 00:02:10), (00:45:00, 00:46:30)]
Output: Two clips cut directly via ffmpeg stream copy (fast, no re-encode since timestamps are approximate and speed is prioritized), saved as clip_1.mp4, clip_2.mp4.
Example 3: Input: Twitch VOD with "make highlight clips from chat spam/hype moments" Output: Parse chat log for message-rate spikes → map spike timestamps to VOD time → generate candidate ranges with ±10s padding → cut and export as horizontal 16:9 clips (no reformat since target is YouTube, not Shorts).
- Always pad clip boundaries by 1-3 seconds and snap to silence/scene breaks — raw timestamps rarely align with natural speech.
- Keep a manifest (JSON/CSV) mapping each output clip back to source + timestamps for re-editing later.
- Default to re-encoding unless performance is critical; stream-copy cuts can be off by seconds due to keyframe alignment.
- Separate "detection" and "cutting" into distinct, reusable functions — detection logic changes often, cutting logic rarely does.
- For batch jobs, process clips in parallel (multiprocessing or GNU parallel) since ffmpeg encoding is CPU-bound.
- Validate clip duration limits for target platform (e.g., ≤60s for Shorts/Reels, ≤140s for TikTok) before export.
- Don't use
-c copywhen precise cut points matter — it snaps to the nearest keyframe, causing drift of up to several seconds. - Don't skip audio normalization — clips pulled from different sections of a source often have inconsistent loudness.
- Don't hardcode aspect ratio conversion — check source dimensions first; naive center-crop can cut off faces/text.
- Don't rely solely on silence detection for cut points in noisy audio (music, games) — combine with transcript/energy signals.
- Don't forget to clean up temp files (downloaded sources, intermediate transcodes) in automated/scheduled runs — disk fills up fast.