Writing Technical Model Reports

YAML
---
name: writing-technical-model-reports
description: Structures and writes technical reports for AI/ML model releases covering architecture, training infrastructure, benchmarks, and safety evaluations. Use when documenting a new model family, writing a technical report or model card, or communicating research/engineering advances to a mixed technical audience of researchers, engineers, and decision-makers.
---

[3-5 sentences: what's being introduced, key differentiator (SoTA on X), one standout capability (e.g. "processes 3 hours of video"), how the family spans a cost/capability tradeoff, closing sentence on what this unlocks.]

[Restate the family lineage: "Building upon Model N, this generation..."] [One paragraph per variant: name — one-sentence role — key differentiator] [Reference a comparison table and a Pareto frontier figure here.]

Comparison Table (do this first — it anchors everything)

Always include: input modalities, input context length, output modalities, output length, "thinking"/reasoning mode, tool-use support, knowledge cutoff. Mark experimental/preview features with an asterisk and a footnote.

Architecture Section

• State the core architecture class in one sentence with citations (e.g., "sparse mixture-of-experts transformers with native multimodal support").
• Explain why the architectural choice matters mechanically (e.g., "decouples total model capacity from per-token compute cost") — one sentence, not a tutorial.
• Report known failure modes of the architecture class honestly (e.g., training instabilities) before stating how this generation mitigated them.
• Tie every architecture claim to a downstream, measurable benefit ("resulting in a considerable boost in performance straight out of pre-training").

Dataset Section

Keep this short. State: composition (domains/modalities), cutoff date, and what changed since the last version (new filtering, dedup, quality methods). Don't re-describe the entire pipeline if it's unchanged.

Training Infrastructure Section

Lead with hardware (chip generation, pod/cluster size, parallelism strategy). Then describe 1-3 named engineering advances, each with:

1. The problem it solves (in one sentence)
2. The mechanism (short paragraph)
3. A concrete quantitative outcome (%, time saved, error rate)

Example pattern:

[Named Technique]: [Problem in one sentence]. [Mechanism, 2-3 sentences]. [Quantified result]: "throughput recovers to ~97% while X% of the fleet is degraded."

Close with an aggregate efficiency statistic (e.g., "93.4% of time was spent on useful computation").

Post-training Section

Structure around the stages you touched (SFT, RM, RL). For each, state: what changed, why (what capability gap it closes), and reference the resulting benchmark gain with a specific number (e.g., "+122 Elo points").

Thinking/Reasoning or Special-Capability Sections

If the model has a distinguishing mechanism (e.g., inference-time reasoning, tool-use), give it its own subsection: describe the training method (usually RL), show a benchmark chart citation demonstrating scaling with the mechanism, and explain user-facing controls (e.g., "thinking budget").

Benchmark Results

• One table per capability cluster (reasoning, code, multimodal/long-context, agentic/tool-use, factuality).
• Always benchmark against both the model's own predecessor and best external competitor.
• Cite benchmark sources with dates ("Source: LMArena, imported on 2025-06-16") — freshness matters for reproducibility claims.

Qualitative Demonstrations

Show, don't just tell: describe a task that required combining multiple capabilities (e.g., long-context + multimodal + reasoning = agentic workflow) with a specific named example.

Safety Section

Follow this exact pattern:

1. State metrics stayed robust while helpfulness improved (avoid the false tradeoff framing).
2. Name the specific critical capability areas evaluated (CBRN, cyber, ML R&D, deceptive alignment, etc.).
3. Explicitly state whether any threshold/Critical Capability Level was crossed — say so plainly even if the answer is "no."

• Quantify everything. Never say "much faster" — say "97% throughput" or "+122 Elo."
• Frame as deltas, not full re-explanations. Assume the reader knows the prior generation; describe what changed and why.
• Pair every capability claim with a benchmark citation and import/access date.
• Use the Pareto frontier framing when a family has multiple size/cost tiers — position models as covering a capability-vs-cost curve, not as strictly ranked.
• Be honest about known failure modes of techniques used (e.g., MoE instability) before explaining the fix — this builds credibility.
• State safety findings plainly, including explicit non-events ("did not reach any Critical Capability Level").
• Footnote experimental/preview features rather than omitting caveats.

• Don't bury the comparison table — put it in the introduction, not an appendix.
• Don't describe unchanged infrastructure/architecture in full; only describe deltas.
• Don't make capability claims without a named benchmark and source date.
• Don't use vague safety language ("we made it safer") — name the specific evaluation categories and thresholds.
• Don't let the qualitative demo section merely restate the benchmark numbers — it should show a combined capability (e.g., multimodal + long-context + reasoning) not a single-axis win.
• Don't omit engineering cost tradeoffs (e.g., distillation increasing storage/compute for training) — state them alongside the benefit.