Building Agent Memory Systems

Implement the three-layer memory architecture:

1. Observation  → /diary command   → writes raw session notes to disk
2. Reflection   → /reflect command → analyzes N diary entries for patterns
3. Retrieval    → CLAUDE.md        → read into every future session

Minimal file layout:

commands/
  diary.md      # instructs Claude how to write a diary entry
  reflect.md    # instructs Claude how to analyze entries + update CLAUDE.md
hooks/
  pre-compact.sh   # auto-triggers diary before context is lost
~/.claude/memory/diary/YYYY-MM-DD-session-N.md   # observation store
CLAUDE.md          # synthesized long-term memory
processed.log      # tracks which diary entries have been reflected on

Progress:

• Define the observation format (what a diary entry captures)
• Define the trigger points (manual command + automatic hook)
• Define the reflection algorithm (pattern threshold, categories, dedup)
• Define the retrieval mechanism (how memory re-enters context)
• Wire up storage location and idempotency (processed.log)
• Test end-to-end: session → diary → reflect → CLAUDE.md update → next session

1. Design the observation (diary) layer

• Trigger context-first: reflect on what's already in the conversation, not just logs. Fall back to parsing transcripts (e.g., JSONL) only when needed.
• Fixed sections force completeness: task summary, work done, design decisions, user preferences, review feedback, challenges/solutions, code patterns.
• Instruct the model explicitly to be factual and specific, capture the "why" behind decisions, document all stated preferences (commits, PRs, linting, testing), and include failures — failures are the highest-signal learning material.
• Name files deterministically: YYYY-MM-DD-session-N.md so ordering and dedup are trivial.

2. Design triggers

• Automatic: hook into a lifecycle event that already signals "context about to be lost" (e.g., PreCompact for long sessions). This guarantees no silent loss of learnings.
• Manual: always allow an explicit command for the user to force capture at any time.
• Prefer hooking into an existing lifecycle event over inventing a new polling mechanism.

3. Design the reflection layer

• Reflection must be incremental: check a processed-log to skip entries already analyzed. Never re-derive the same pattern twice unless explicitly asked to reprocess.
• Use an explicit occurrence threshold for what counts as a pattern vs. a strong pattern (e.g., 2+ = pattern, 3+ = strong pattern). This prevents overfitting to a single anecdote.
• Prioritize scanning for violations of existing rules above discovering new rules — reinforcing broken rules is higher value than adding new ones.
• Synthesize across fixed categories so nothing is missed: external feedback (PR review), stated preferences, design decisions that worked, anti-patterns to avoid, efficiency lessons, project-specific patterns.
• Support filters on the reflection command (date range, project path, keyword, count) so users can scope analysis instead of always running full-corpus reflection.
• Support explicit reprocess/include-all override for re-analysis, separate from the default incremental behavior.

4. Design the retrieval layer

• Output of reflection must be written directly into the memory file that's auto-loaded every session (e.g., CLAUDE.md) — retrieval should require zero user action.
• Enforce a strict output format for new rules: one-line bullets, imperative tone, no verbose explanation. Long-form reasoning belongs in diary entries, not in the always-loaded memory file.