AI Skill Report Card

Scoring WIF To Packet Conversion

A-83·Oct 29, 2026·Source: Web
14 / 15

Given an AI agent's output (a PDF packet, plus the WIF/image input it started from), score it against the 8-point rubric below and emit a scorecard.

INPUT: sample_draft.wif
AI OUTPUT: sample_draft_packet.pdf

SCORECARD
1. Draft Decoding & Validation .......... 8/10
2. Threading/Tie-up/Treadling Diagrams .. 6/10
3. Drawdown Rendering ................... 9/10
4. Yarn & Sett Calculations ............. 3/10  ⚠ BREAKS HERE
5. Draft-to-Prose Instructions .......... 5/10
6. Layout & Print-Readiness ............. 7/10
7. Historical/Source Attribution ........ 10/10
8. Sellability (branding, polish, legal) . 4/10

OVERALL: 52/80 (65%) — Usable draft, unsellable packet
VERDICT: Automation holds through visualization, breaks at commercial packaging.
Recommendation▾
Add a brief note on how to handle partial/missing inputs (e.g., no source WIF available at all) as an edge case
14 / 15

Progress:

  • Step 1: Identify input type (native WIF vs. scanned/photographed historical draft)
  • Step 2: Run or inspect the AI agent's conversion pipeline output
  • Step 3: Score each of the 8 sub-skills independently (don't let one bad score bias others)
  • Step 4: Flag the exact sub-skill where automation "breaks" (first score below 5/10)
  • Step 5: Write qualitative notes per category (specific, not generic)
  • Step 6: Produce final scorecard + verdict + recommendation
  • Step 7: (If video) storyboard each sub-skill as its own scored segment with visual proof

The 8 Sub-Skills (Rubric)

  1. Draft Decoding & Validation — Did it correctly parse threading, tie-up, treadling, and warp/weft color sequences from the WIF, or correctly infer them from a scanned image (OCR/CV on grid marks, handwritten numbers, faded ink)? Deduct heavily for silent misreads (e.g., swapping a 6 for an 8).

  2. Threading/Tie-up/Treadling Diagrams — Are the three core diagrams regenerated cleanly as vector/print-quality graphics (not blurry screenshots), correctly labeled, and consistent with the decoded data?

  3. Drawdown Rendering — Is the woven structure simulation visually accurate (correct interlacement, color-and-weave effect), and does it match what the threading/tie-up/treadling imply?

  4. Yarn & Sett Calculations — Does it compute warp length, yardage, sett (EPI/PPI), draw-in, and shrinkage estimates based on stated or inferred yarn size/structure? This is the most common break point — flag if the agent fabricates numbers without stating assumptions (yarn size, loom waste %, shrinkage %).

  5. Draft-to-Prose Instructions — Does it convert the diagrammatic draft into step-by-step written weaving instructions a human could follow at the loom (warping order, tie-up sequence, treadling repeats)?

  6. Layout & Print-Readiness — Is the PDF properly paginated, with consistent typography, margins suitable for home printing, and diagrams sized/placed sensibly (not split across pages)?

  7. Historical/Source Attribution — If from a historical draft, does it credit the source (archive, weaver, publication date) and note any uncertainty in its transcription (e.g., "faded section reconstructed")?

  8. Sellability — Does the final packet feel like a product: cover page, consistent branding, copyright/licensing note, difficulty rating, materials list, and no leftover placeholder text or broken image links? This is a common secondary break point.

Recommendation▾
Include guidance on scoring consistency across multiple evaluations (e.g., calibration examples for what a '5' vs '6' looks like)
17 / 20

Example 1: Input: Clean, modern WIF file exported from WeavePoint for a simple 4-shaft twill. Output:

1. Decoding: 10/10 — native WIF, no ambiguity
2. Diagrams: 10/10 — clean vector regeneration
3. Drawdown: 10/10 — matches diagrams exactly
4. Calculations: 6/10 — EPI/PPI correct but no shrinkage % shown, assumes 10/2 cotton without stating it
5. Prose instructions: 8/10 — clear but misses "advance warp" reminders
6. Layout: 9/10 — clean, one diagram awkwardly split across a page break
7. Attribution: N/A (not historical) — scored 10/10 by default
8. Sellability: 7/10 — has cover page, missing licensing note
OVERALL: 70/80 (87.5%) — Strong automation performance on clean digital input.

Example 2: Input: Low-resolution scanned photo of an 1890s handwoven coverlet draft with faded handwritten tie-up numbers. Output:

1. Decoding: 3/10 — misread 2 tie-up cells, no confidence flagging on faded region
2. Diagrams: 4/10 — diagrams generated but propagate decoding errors
3. Drawdown: 4/10 — visually plausible but wrong due to upstream error
4. Calculations: 2/10 — invented yarn size (no source stated), no shrinkage estimate
5. Prose instructions: 3/10 — instructions reflect the decoding error, would produce wrong fabric
6. Layout: 6/10 — adequate but generic template, no visual connection to historical source
7. Attribution: 2/10 — no archive name, no note of damaged/reconstructed sections
8. Sellability: 3/10 — placeholder "Lorem ipsum" left in materials list
OVERALL: 27/80 (34%) — Automation breaks immediately at decoding; everything downstream inherits the error.
VERDICT: Not sellable. Requires human verification pass on handwritten historical source before any downstream step is trustworthy.
Recommendation▾
Consider a short template for the qualitative notes section so the written feedback per category is more structured, not just prose
  • Score decoding first and weight it conceptually — errors there cascade into every other category; always note in the verdict if a low decoding score explains low scores elsewhere ("inherited error" vs. "independent failure").
  • Demand stated assumptions for calculations (category 4). A confident wrong number is worse than a flagged uncertain one — score transparency, not just correctness.
  • Treat "N/A" categories fairly — non-historical sources auto-score 10/10 on attribution rather than being penalized.
  • Use the first sub-5 score as the declared break point in the verdict — this is the single most useful takeaway for weavers deciding whether to trust automation.
  • Keep scores evidence-based, citing specific diagram/page/cell references, not vague impressions.
  • For video format, give each of the 8 categories its own timestamped segment with a visual side-by-side (input source vs. AI output) before revealing the score.
  • Don't average scores into a single number without the per-category breakdown — the diagnostic value is in where it breaks, not just the overall grade.
  • Don't let strong visual polish (category 6/8) inflate scores in correctness categories (1-4) — a beautiful PDF with wrong threading is still a failure.
  • Don't penalize historical sources for having lower raw accuracy if the agent correctly flags its own uncertainty — score honesty about limitations separately from raw accuracy.
  • Don't skip stating yarn/sett assumptions as fabricated-but-unstated — this is the single most common silent failure mode in category 4.
  • Don't treat a missing category 7 (attribution) as minor for historical sources — selling a packet without source credit is a legal/ethical issue, not just a polish issue.
0
Grade A-AI Skill Framework
Scorecard
Criteria Breakdown
Quick Start
14/15
Workflow
14/15
Examples
17/20
Completeness
17/20
Format
14/15
Conciseness
13/15