Converting Books to Agent Skills
Bash# Invoke the CLI to extract and prepare a book for skill generation book-to-skill convert path/to/book.epub --output ./extracted/ # Pipeline runs automatically: # 1. Dependency check (format-specific libraries present?) # 2. Format dispatch (pdf/epub/html/docx/rtf/calibre fallback) # 3. Text decoding + sanitization # 4. Structure/metadata detection (chapters, headings, TOC) # 5. Extracted text + metadata handed to agent generator
The agent then reads SKILL.md (generation spec) + extracted text/metadata and produces chapter files, glossary, patterns, and cheatsheets for the target agent host.
Progress:
- Step 1: Parse CLI args, load run configuration (formats, paths, output dir)
- Step 2: Run dependency checks for the detected input format; fail fast with actionable install instructions
- Step 3: Dispatch to format-specific extractor (PDF, EPUB, HTML, DOCX, RTF); fall back to Calibre conversion for unsupported/corrupt formats
- Step 4: For PDF, run a lightweight inspection/preflight pass before full extraction to catch malformed files early
- Step 5: Decode raw text (handle encoding edge cases, especially RTF/HTML)
- Step 6: Sanitize text — strip control characters, normalize whitespace, remove boilerplate (headers/footers/page numbers)
- Step 7: Detect structure — chapter boundaries, headings, metadata (title, author, TOC)
- Step 8: Package extracted text + structural metadata as input to the skill-generation step
- Step 9: Agent generator follows the generation specification (
SKILL.md) to produce chapter files, glossary, pattern docs, cheatsheets - Step 10: Optionally publish generated skill to GitHub or install directly for a target agent host
Format dispatch logic
if format == pdf:
run pdf_inspector preflight -> if pass: parsers/pdf.py; else: fallback to calibre
elif format == epub:
parsers/epub.py -> delegates HTML parsing to parsers/html.py for content.opf chapters
elif format == html:
parsers/html.py -> text decoding
elif format == docx:
parsers/docx.py
elif format == rtf:
parsers/rtf.py -> text decoding
else:
calibre conversion (universal fallback) -> re-dispatch on converted format
Example 1:
Input: book.epub containing 12 chapters with embedded HTML content
Output: Extraction coordinator dispatches to epub.py, which parses each content document via the HTML parser, decodes text, sanitizes (strips nav/CSS artifacts), detects chapter boundaries from the EPUB TOC, and emits structured text + metadata (title, chapter list, author) ready for the generator.
Example 2: Input: Scanned PDF with inconsistent font encoding Output: PDF preflight (pdf_inspector) flags extraction risk before full parse; if raw text extraction looks garbled (low alphanumeric ratio), coordinator falls back to Calibre conversion to an intermediate format, then re-extracts.
Example 3:
Input: Extracted text + metadata for a 300-page book
Output: Agent generator reads SKILL.md spec, produces one skill file per chapter, a cross-chapter glossary.md, a patterns.md capturing recurring frameworks/models, and a cheatsheet.md — packaged for loading by Claude/other agent hosts on demand.
- Keep extraction deterministic and host-agnostic; keep skill generation as the agent-driven, creative step. Don't blur this boundary.
- Always run dependency checks before format dispatch — fail with a clear "install X for format Y" message rather than a stack trace.
- Treat Calibre conversion as the universal escape hatch for any format the native parsers can't handle cleanly.
- Sanitize before structure detection, not after — boilerplate (running headers, page numbers) corrupts chapter-boundary heuristics.
- Preserve source metadata (title, author, TOC) through the whole pipeline; the generator needs it for skill naming and front matter.
- Design generated output to be incremental: support "fold-in/update" so re-running on a revised book updates existing skill files rather than duplicating them.
- Keep format parsers isolated modules (
parsers/pdf.py,parsers/epub.py, etc.) with a single dispatch point (utils.py) — easy to add new formats.
- Don't attempt structure detection on unsanitized text — stray headers/footers get misread as chapter titles.
- Don't assume EPUB content is self-contained; it often requires delegating to the HTML parser per content document.
- Don't skip the PDF preflight step — full extraction on a malformed PDF wastes time and produces garbage silently.
- Don't hardcode agent-host-specific output formatting into the extraction layer; keep host delivery (installation, GitHub publishing) as a separate concern from text extraction.
- Don't let the Calibre fallback mask real bugs — log when fallback triggers so native parser gaps get fixed over time.