AI Skill Report Card

Scaffolding Agentic AI Projects

A85·Sep 5, 2026·Source: Web
Markdown
--- name: scaffolding-agentic-ai-projects description: Generates production-grade folder structures and boilerplate for Agentic AI and Generative AI applications built with OOP Python, LangChain/LangGraph, and FastAPI. Use when starting a new AI agent project, refactoring a prototype into production structure, or when asked "how should I structure this LangChain/LangGraph project." ---
15 / 15

Given a business requirement, generate this structure immediately (adapt names to domain, don't ask clarifying questions unless requirements are truly ambiguous):

project-name/
├── src/
│   └── project_name/
│       ├── __init__.py
│       ├── main.py                  # FastAPI app entrypoint
│       ├── core/
│       │   ├── __init__.py
│       │   ├── config.py            # Pydantic Settings (env vars)
│       │   ├── logging.py           # Structured logging setup
│       │   └── exceptions.py        # Custom exception classes
│       ├── api/
│       │   ├── __init__.py
│       │   ├── v1/
│       │   │   ├── __init__.py
│       │   │   ├── router.py        # Aggregates all v1 routes
│       │   │   └── endpoints/
│       │   │       ├── __init__.py
│       │   │       ├── chat.py
│       │   │       └── health.py
│       │   └── deps.py              # FastAPI dependency injection
│       ├── agents/
│       │   ├── __init__.py
│       │   ├── base_agent.py        # Abstract base class (ABC)
│       │   ├── graphs/
│       │   │   ├── __init__.py
│       │   │   └── main_graph.py    # LangGraph StateGraph definition
│       │   ├── state.py             # TypedDict/Pydantic state schemas
│       │   └── nodes/
│       │       ├── __init__.py
│       │       ├── planner_node.py
│       │       ├── executor_node.py
│       │       └── critic_node.py
│       ├── chains/
│       │   ├── __init__.py
│       │   └── rag_chain.py         # LangChain LCEL chains
│       ├── tools/
│       │   ├── __init__.py
│       │   ├── base_tool.py
│       │   └── search_tool.py
│       ├── prompts/
│       │   ├── __init__.py
│       │   ├── templates/
│       │   │   └── system_prompt.jinja2
│       │   └── loader.py
│       ├── llm/
│       │   ├── __init__.py
│       │   ├── factory.py           # LLM provider factory (OpenAI, Anthropic, etc.)
│       │   └── embeddings.py
│       ├── memory/
│       │   ├── __init__.py
│       │   └── checkpointer.py      # LangGraph checkpoint/persistence
│       ├── retrievers/
│       │   ├── __init__.py
│       │   └── vector_store.py
│       ├── models/
│       │   ├── __init__.py
│       │   ├── schemas.py           # Pydantic request/response models
│       │   └── domain.py            # Domain entities
│       ├── services/
│       │   ├── __init__.py
│       │   └── chat_service.py      # Business logic orchestration layer
│       └── utils/
│           ├── __init__.py
│           └── helpers.py
├── tests/
│   ├── __init__.py
│   ├── conftest.py
│   ├── unit/
│   │   ├── test_agents.py
│   │   └── test_chains.py
│   └── integration/
│       └── test_api.py
├── scripts/
│   ├── seed_vectorstore.py
│   └── evaluate_agent.py
├── notebooks/
│   └── experiments.ipynb
├── data/
│   ├── raw/
│   └── processed/
├── .env.example
├── .gitignore
├── pyproject.toml                   # Poetry/uv dependency management
├── Dockerfile
├── docker-compose.yml
├── Makefile
└── README.md
Recommendation
Add a third example showing a multi-agent handoff scenario to demonstrate more complex graph structures
14 / 15

Progress:

  • Step 1: Clarify domain and pick project name (snake_case)
  • Step 2: Determine if it's LangChain-only (linear RAG/chains) or LangGraph-required (multi-step agentic loops, cyclic reasoning, human-in-loop)
  • Step 3: Generate folder tree using template above, pruning unused modules
  • Step 4: Scaffold core/config.py with Pydantic BaseSettings
  • Step 5: Scaffold base_agent.py as an ABC with run()/arun() abstract methods
  • Step 6: Scaffold main_graph.py with StateGraph, nodes, edges, and checkpointer
  • Step 7: Wire FastAPI router → service layer → agent/chain layer (never call agents directly from endpoints)
  • Step 8: Add pyproject.toml with pinned deps: langchain, langgraph, fastapi, uvicorn, pydantic-settings, pytest
  • Step 9: Add Dockerfile (multi-stage build) and docker-compose.yml
  • Step 10: Write README.md with architecture diagram description and run instructions
Recommendation
Include a concrete code snippet for base_agent.py or main_graph.py rather than only describing them
  • Use LangChain (chains only) when the flow is linear: retrieve → augment → generate, no branching/looping.
  • Use LangGraph when there's cyclic reasoning, multi-agent handoff, human-in-the-loop approval, or conditional routing based on state.
  • Always separate agents/ from chains/ — chains are stateless pipelines, agents are stateful graphs with memory/checkpointing.
  • Service layer is mandatory — API endpoints must never instantiate LLMs or graphs directly; they call services/, which call agents/ or chains/.
  • Config via Pydantic Settings only — no os.getenv() scattered in code.
16 / 20

Example 1: Input: "Build a customer support agent that can search a knowledge base, escalate to human, and remember conversation history across sessions." Output: Full structure above, with agents/graphs/main_graph.py containing a StateGraph with nodes retrieve_kb, answer, escalate_human, conditional edge based on confidence score, and memory/checkpointer.py using SqliteSaver or PostgresSaver for cross-session persistence.

Example 2: Input: "Simple RAG chatbot over PDF documents, no multi-step reasoning needed." Output: Trimmed structure — drop agents/ and memory/checkpointer.py entirely; keep chains/rag_chain.py using LCEL (prompt | llm | parser), retrievers/vector_store.py for FAISS/Chroma, and standard FastAPI endpoint calling the chain via chat_service.py.

Recommendation
Add guidance on evaluation/observability tooling (e.g., LangSmith) integration since scripts/evaluate_agent.py is referenced but not elaborated
  • Use ABCs (abc.ABC) for base_agent.py and base_tool.py to enforce consistent interfaces across implementations.
  • Keep state schemas (agents/state.py) as TypedDict (LangGraph requirement) or Pydantic models, versioned separately from API schemas.
  • Use LLM factory pattern (llm/factory.py) so switching providers (OpenAI ↔ Anthropic ↔ local) is a config change, not a code change.
  • Externalize prompts as Jinja2/text templates, never hardcode multi-line prompt strings in Python files.
  • Add async support (arun, ainvoke) everywhere since FastAPI is async-first.
  • Include checkpointing/persistence from day one for any LangGraph agent — retrofitting memory later is painful.
  • Write unit tests for nodes/tools individually, integration tests for the full graph/chain.
  • Use dependency injection (api/deps.py) to provide LLM clients, vector stores, and services to endpoints — enables easy mocking in tests.
  • Don't put business logic in api/endpoints/ — keep endpoints thin, delegate to services/.
  • Don't hardcode API keys or model names — always route through core/config.py.
  • Don't mix LangChain LCEL chains and LangGraph graphs in the same module — keep chains/ and agents/ distinct.
  • Don't skip the state.py schema definition — passing raw dicts between graph nodes causes silent bugs.
  • Don't forget checkpointer config — without it, LangGraph agents lose memory on every restart.
  • Don't version-lock only langchain — LangGraph and LangChain release independently and can drift; pin both explicitly.
0
Grade AAI Skill Framework
Scorecard
Criteria Breakdown
Quick Start
15/15
Workflow
14/15
Examples
16/20
Completeness
18/20
Format
14/15
Conciseness
13/15