AI Skill Report Card
Designing AI Memory Architecture
Quick Start14 / 15
Given a system needing memory (e.g., "an AI assistant that remembers user preferences across sessions"), produce five sections:
Recommendation▾
Add an example covering a failure/anti-pattern scenario (e.g., a design that over-stores or under-governs memory) to show contrast between good and bad outcomes explicitly.
Memory Layer
[Which memory types are needed: episodic, semantic, long-term, contextual, and why]
Storage Flow
[How data moves from input → processing → storage, including format and triggers]
Recall Strategy
[How memory is retrieved at query time — indexing, ranking, filtering]
Retention Policy
[What is kept, for how long, decay/eviction rules]
Memory Governance
[Access control, privacy, auditability, user control over memory]
Workflow14 / 15
Progress:
- Step 1: Identify what needs to be remembered (facts, events, preferences, conversation state)
- Step 2: Map each item to a memory layer (episodic/semantic/long-term/contextual)
- Step 3: Define storage flow — capture trigger, encoding format, write path
- Step 4: Define recall strategy — retrieval trigger, ranking/similarity method, injection into context
- Step 5: Define retention policy — TTL, decay, consolidation, deletion rules
- Step 6: Define governance — who can read/write/delete, consent, audit trail
- Step 7: Output the five sections in order
Memory Layer Definitions
- Episodic memory: specific past events/interactions ("user asked about X on date Y"). Time-stamped, sequential, tied to a session or interaction.
- Semantic memory: generalized facts and knowledge distilled from episodes ("user prefers concise answers"). Not tied to a specific event.
- Long-term memory: persists across sessions indefinitely (or until explicit removal). Usually holds consolidated semantic knowledge + key episodic anchors.
- Contextual memory: short-lived, scoped to current session/task (working memory). Discarded or summarized after session ends.
Storage Flow Pattern
- Capture — raw input/event captured at interaction time
- Filter — decide if it's worth storing (relevance/importance scoring)
- Encode — convert to storable form (embedding, structured record, summary)
- Write — persist to appropriate layer's store (vector DB for semantic, log/DB for episodic, cache for contextual)
- Consolidate — periodically merge episodic → semantic (batch summarization)
Recall Strategy Pattern
- Trigger — new query/turn initiates recall
- Retrieve — pull candidates via similarity search (semantic), recency+relevance (episodic), or session state (contextual)
- Rank — score by relevance, recency, importance
- Filter — apply governance/privacy constraints
- Inject — merge into prompt context within token budget
Recommendation▾
Include guidance on scaling considerations (e.g., vector DB choice tradeoffs, cost implications of embedding storage) to deepen completeness for production use.
Examples18 / 20
Example 1: Input: "Design memory for a customer support AI that should remember past tickets and user tone preference."
Output:
Recommendation▾
Consider trimming the 'Storage Flow Pattern' and 'Recall Strategy Pattern' sections slightly since they partially restate the Quick Start template content, reducing redundancy.
Memory Layer
- Episodic: individual past tickets (issue, resolution, timestamp)
- Semantic: distilled preferences ("user prefers formal tone", "user is a premium tier customer")
- Long-term: semantic facts + ticket summaries, retained indefinitely unless user requests deletion
- Contextual: current conversation turns within active session
Storage Flow
- Capture: each ticket closure logs full transcript
- Filter: importance scorer flags tone/preference signals
- Encode: transcript summarized into semantic facts via LLM summarization; embeddings generated for semantic search
- Write: raw ticket → episodic log (SQL); summary + embedding → semantic vector store
- Consolidate: weekly batch job merges repeated signals into stable preference profile
Recall Strategy
- On new ticket, retrieve top-k similar past tickets via embedding similarity
- Retrieve semantic profile (preferences) directly by user ID, no ranking needed
- Merge contextual (current session) + episodic (past 3 tickets) + semantic (profile) into prompt
- Cap injected memory at 500 tokens, prioritize semantic > recent episodic
Retention Policy
- Contextual: cleared at session end, summarized into episodic if significant
- Episodic: retained 2 years, then archived/compressed to semantic-only
- Semantic: retained indefinitely, updated on new evidence
- Decay: preference confidence score decays if contradicted by newer episodes
Memory Governance
- User can view/export/delete all stored memory (GDPR-style right to erasure)
- Write access restricted to system pipeline; no direct manual edits without audit log
- PII fields encrypted at rest; access logged per read
- Consent required before storing semantic profile beyond current session
**Example 2:**
Input: "Simple chatbot, no persistence needed beyond current chat."
Output:
Memory Layer
- Contextual only: rolling window of current conversation turns
- No episodic, semantic, or long-term layer required
Storage Flow
- Capture: each turn appended to in-memory buffer
- Encode: raw text, no summarization needed
- Write: held in process memory / session cache, not persisted to disk
Recall Strategy
- Full buffer passed into context window each turn
- If buffer exceeds token limit, truncate oldest turns or summarize into a running note
Retention Policy
- Cleared immediately on session end
- No archival, no cross-session recall
Memory Governance
- No PII persistence risk since nothing is stored beyond session
- No user action needed for deletion (automatic on session close)
Best Practices
- Always match memory type to actual need — don't add long-term storage if contextual suffices.
- Separate raw episodic logs from distilled semantic facts; never let unfiltered raw data pollute recall ranking.
- Cap recall injection by token budget explicitly; specify prioritization order (semantic > recent episodic > contextual).
- Include a consolidation step (episodic → semantic) — this is the difference between "logging" and "learning."
- Governance is not optional: always define retention limits and user control, even for internal systems.
- Default to time-decay or confidence-decay for semantic facts so contradictions get resolved over time.
Common Pitfalls
- Don't treat all memory as one flat store — retrieval quality collapses without layer separation.
- Don't skip retention policy — unbounded episodic storage grows costs and leaks stale/irrelevant context.
- Don't recall everything relevant; recall the highest-value subset within budget — over-injection dilutes the prompt and buries key facts.
- Don't conflate contextual (session-scoped) memory with long-term memory — contextual should be ephemeral by design.
- Don't omit governance — memory systems that store user data without access control or deletion capability are a compliance liability.