AI Skill Report Card
Architecting Production AI Systems
Markdown--- name: architecting-production-ai-systems description: Builds complete, production-ready codebases for AI-integrated backend systems (APIs, data extraction pipelines, microservices) with zero placeholders, full infrastructure-as-code, and enterprise security/observability baked in. Use when a request involves generating a deployable application, integrating an LLM into a production service, or producing infra (Docker, Terraform, CI/CD) alongside application code. --- # Architecting Production AI Systems
Quick Start13 / 15
Given a request like: "I want a production-ready Python API on Cloud Run that uses Gemini to extract structured data from raw text, with rate-limit handling," immediately produce:
- A full directory tree (not a description of one)
- Every file, complete — config, schema, client, entrypoint, Dockerfile, CI/CD, IaC
- No
// TODO, no..., no "implement this later" — every function body is real and runnable
Default stack unless the user specifies otherwise:
- Language: Python (FastAPI) for AI/data services; TypeScript (NestJS/Next.js) for full-stack apps; Go for latency-critical services
- LLM integration: Vertex AI SDK + Pydantic schema enforcement (
response_schema) so the model cannot return malformed JSON - Resilience:
tenacityexponential backoff on all external API calls - Cloud: GCP (Cloud Run, Secret Manager, Cloud Logging) unless another provider is named
- Containerization: Multi-stage Dockerfile, non-root user, minimal base image
- IaC: Terraform for provisioning; Cloud Build or GitHub Actions for CI/CD
Recommendation▾
Add a second, contrasting example (e.g., a smaller-scope or non-GCP request) to show the skill generalizes beyond the single Cloud Run/Gemini scenario
Workflow13 / 15
Progress checklist for every build request:
- 1. Establish context: Reread conversation for prior constraints (language, cloud provider, budget, existing schema). Identify whether this is new-build, debug, or asset-generation. Load the relevant domain knowledge (cloud architecture, ML serving, finance, etc.)
- 2. Verify facts / calculate: If the request touches live APIs, current model names/versions, pricing, or precise numbers — search or compute (mentally run through the math/logic step-by-step) rather than guessing. Never invent a version number, price, or API signature.
- 3. Architect the structure: Design the file tree first. Map dependencies between files (schema → client → routes → main). Show the tree before or alongside the code.
- 4. Generate complete code: Write every file in full. Inject: structured logging, error handling, env-based config (no hardcoded secrets), input validation via Pydantic/Zod.
- 5. Package & provision: Add Dockerfile (multi-stage, non-root), CI/CD pipeline config, and IaC (Terraform) with least-privilege IAM.
Recommendation▾
Include an actual code snippet (even abbreviated) rather than only the directory tree, to demonstrate the 'no placeholders' standard concretely
Example
Input: "Production-ready Python API on Cloud Run, uses Gemini 1.5 Pro to extract structured data from prescription transcripts, handle rate limits automatically."
Output structure:
prescription-extractor/
├── config.py # env-driven settings via pydantic_settings
├── schemas.py # Pydantic models defining exact extraction schema
├── client.py # Vertex AI wrapper with @retry (tenacity, exponential backoff)
├── main.py # FastAPI app, structured JSON logging, /healthz endpoint
├── Dockerfile # multi-stage, non-root, minimal base
├── cloudbuild.yaml # build → push → deploy pipeline
└── main.tf # Cloud Run SA, least-privilege IAM, API enablement
Each file delivered in full — no truncation. Schema enforcement means the LLM response is guaranteed parseable JSON. Retry decorator specifically catches 429/throttling. Logging emits structured JSON with latency per request for observability.
Best Practices
- Schema-first for any LLM integration: define the Pydantic/Zod model before writing the prompt or client code. Pass it as
response_schemaso parsing never fails. - Least privilege by default: every service account gets only the IAM roles it needs (e.g.,
roles/aiplatform.user, notroles/owner). - Secrets via environment/Secret Manager only — never hardcode keys or project IDs; use
pydantic_settings.BaseSettingswith sane defaults for local dev. - Non-root containers: always create and switch to a dedicated user in the Dockerfile.
- Structured logging over print statements: every log line should be JSON with an
eventfield and relevant metrics (latency, tokens, error). - Show the tree before the code so the user can see the architecture before parsing implementation details.
- Lead with the direct answer: give the scannable summary/output first, elaborate after if needed.
- Run the numbers, don't estimate: for token counts, cost projections, or data transforms, work through the calculation explicitly rather than approximating.
Common Pitfalls
- Don't leave
// TODOor...anywhere in generated code — incomplete code defeats the purpose. - Don't skip retry/backoff logic on any external API call (LLM or otherwise) — assume rate limits will happen.
- Don't hardcode project IDs, regions, or secrets — always route through config/env.
- Don't use
latestor unpinned base images in Dockerfiles — pin versions for reproducibility. - Don't grant broad IAM roles (
editor,owner) when a scoped role suffices. - Don't guess at current model names, API signatures, or pricing — verify before asserting them as fact.
- Don't bury the directory structure inside prose — present it as an explicit tree.