Designing Agentic Systems

Markdown
---
name: designing-agentic-systems
description: Guides architectural decisions for LLM-based systems, choosing between workflows and agents and selecting appropriate composition patterns (prompt chaining, routing, parallelization, orchestrator-workers, evaluator-optimizer). Use when designing new LLM applications, deciding whether a task needs an agent versus a simpler workflow, refactoring an overcomplicated agentic system, or evaluating whether to adopt an agent framework.
---

# Designing Agentic Systems

Progress:

• Step 1: Define the task and success criteria concretely
• Step 2: Try a single augmented LLM call (retrieval + tools + good prompt)
• Step 3: If insufficient, determine if the task decomposes into fixed steps → workflow
• Step 4: If subtasks are unpredictable/dynamic → consider an agent
• Step 5: Pick the specific pattern matching the task shape (see table above)
• Step 6: Implement with direct API calls; add framework only if it saves real effort
• Step 7: Add guardrails — gates between steps, max iteration limits, sandboxing
• Step 8: Test extensively, measure against success criteria, iterate

Key distinction to hold onto throughout:

• Workflows = predefined code paths orchestrating LLMs and tools (predictable, consistent).
• Agents = LLM dynamically directs its own process and tool use (flexible, autonomous, costlier, harder to predict).

Don't reach for "agent" by default — most production wins come from workflows or even a single augmented call.

Prompt chaining — decompose into sequential LLM calls, each processing the prior output; insert programmatic "gate" checks between steps. Use when a task cleanly decomposes into fixed steps and you're trading latency for accuracy. Example: generate outline → validate outline against criteria → write full doc.

Routing — classify input, dispatch to a specialized downstream prompt/model. Use when input categories are distinct and benefit from separate handling. Example: route simple queries to a cheap/fast model, complex ones to a stronger model.

Parallelization — either sectioning (split task into independent parallel subtasks) or voting (run same task N times, aggregate). Use when subtasks are truly independent or when diverse independent judgments increase confidence. Example: one call generates a response, a separate parallel call screens for policy violations (don't overload one call with both jobs).

Orchestrator-workers — a central LLM dynamically decides what subtasks are needed and delegates to worker LLM calls, then synthesizes. Use when the number/nature of subtasks can't be predicted in advance. Example: a coding agent that doesn't know ahead of time which files need editing.

Evaluator-optimizer — one LLM generates, another evaluates and gives feedback, loop until criteria met. Use when there's a clear evaluation rubric and iteration measurably improves output. Example: literary translation with an evaluator LLM critiquing nuance.

Full agent loop — LLM plans, calls tools, observes real environment feedback (tool results, code execution output), and continues autonomously, pausing for human input at checkpoints or a stopping condition (max iterations). Use only for open-ended problems where step count can't be predicted and you can trust the model's judgment in a semi-sandboxed environment.

• Start with direct LLM API calls; only adopt a framework (Claude Agent SDK, Strands, Rivet, Vellum, etc.) once you understand what it abstracts — misunderstanding the underlying mechanics is the most common source of bugs.
• Maintain simplicity: add a new pattern/loop/agent only when it demonstrably improves measured outcomes, not preemptively.
• Prioritize transparency — make the agent's planning/reasoning steps visible for debugging and trust.
• Invest in the agent-computer interface: clear, well-documented, thoroughly tested tool definitions matter as much as the prompt. Poor tool docs are a common root cause of agent failure.
• For agents, always ground each step in real environment feedback (tool output, execution results) rather than letting the model reason unchecked.
• Add stopping conditions (max iterations, cost/time budgets) and human checkpoints for any autonomous loop.
• Test agents extensively in sandboxed environments before granting real-world side effects (refunds, file writes, deployments).
• For parallelization "sectioning," don't make one call do double duty (e.g., both respond and self-screen) — split into separate focused calls.

• Reaching for a multi-agent framework when a single optimized prompt with retrieval/tools would suffice.
• Using orchestrator-workers when parallelization would do — check whether subtasks are truly predictable (use parallelization) or dynamically determined (use orchestrator-workers).
• Letting an agent run unbounded with no iteration cap, cost ceiling, or human checkpoint.
• Adopting a heavy framework without understanding what it does under the hood, then being unable to debug prompt/response issues.
• Skipping the "can a single augmented LLM call handle this?" check and jumping straight to agentic complexity.
• Treating "agent" as a marketing term rather than an architectural choice — always be explicit about whether you actually need dynamic, model-driven control flow.