AI Skill Report Card

Building MNIST Image Classifier

B+78·Sep 1, 2026·Source: Web

Understanding Claude Code Skills First

Before the skill file, here's the context you asked for:

How a Skill differs from manually prompting:

Manual PromptingSkill
You re-explain your requirements every time ("build MNIST classifier, use CNN, save model, write inference script...")You say "use the mnist-classifier skill" (or Claude auto-detects it) — instructions are pre-loaded
Claude may vary approach each session (different architecture, different file structure)Consistent, repeatable methodology every time
No memory of your preferred conventions (train/test split, model format, logging style)Encodes your exact workflow, best practices, and pitfalls once, reused forever
Prompt lives in your head or scattered chat historySkill lives as a versioned SKILL.md file — shareable, editable, greppable
Works only in that conversationWorks across sessions/projects once installed in Claude's skills directory

Mechanism — how Skills actually work under the hood:

  1. Storage: Skills are markdown files (SKILL.md) with YAML frontmatter, stored in a skills directory Claude Code scans.
  2. Discovery: Claude reads the name + description frontmatter of all available skills at session start (lightweight index, not full content).
  3. Triggering: When your prompt matches the description's trigger phrases (e.g., "classify MNIST images"), Claude loads that skill's full content into context.
  4. Execution: Claude follows the skill's workflow step-by-step — using its code snippets, checklists, and pitfalls as grounded instructions instead of improvising.
  5. Composability: Skills can reference helper scripts/files alongside them (e.g., a train.py template) which Claude can read or execute.

Now, the skill itself:


Markdown
--- name: building-mnist-image-classifier description: Builds, trains, and runs inference with a CNN image classification model on the MNIST handwritten digit dataset using PyTorch. Use when the user asks to classify MNIST digits, build a digit recognition model, train an image classifier on MNIST, or run inference on handwritten digit images. ---
15 / 15
Python
# train.py — trains and saves an MNIST CNN classifier import torch, torch.nn as nn, torch.nn.functional as F from torch.utils.data import DataLoader from torchvision import datasets, transforms device = "cuda" if torch.cuda.is_available() else "cpu" class MNISTCNN(nn.Module): def __init__(self): super().__init__() self.conv1 = nn.Conv2d(1, 32, 3, padding=1) self.conv2 = nn.Conv2d(32, 64, 3, padding=1) self.pool = nn.MaxPool2d(2) self.fc1 = nn.Linear(64 * 7 * 7, 128) self.fc2 = nn.Linear(128, 10) self.dropout = nn.Dropout(0.25) def forward(self, x): x = self.pool(F.relu(self.conv1(x))) x = self.pool(F.relu(self.conv2(x))) x = x.flatten(1) x = self.dropout(F.relu(self.fc1(x))) return self.fc2(x) def main(): transform = transforms.Compose([ transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,)) ]) train_ds = datasets.MNIST(root="./data", train=True, download=True, transform=transform) test_ds = datasets.MNIST(root="./data", train=False, download=True, transform=transform) train_loader = DataLoader(train_ds, batch_size=128, shuffle=True) test_loader = DataLoader(test_ds, batch_size=256) model = MNISTCNN().to(device) optimizer = torch.optim.Adam(model.parameters(), lr=1e-3) for epoch in range(5): model.train() for x, y in train_loader: x, y = x.to(device), y.to(device) optimizer.zero_grad() loss = F.cross_entropy(model(x), y) loss.backward() optimizer.step() model.eval() correct, total = 0, 0 with torch.no_grad(): for x, y in test_loader: x, y = x.to(device), y.to(device) pred = model(x).argmax(1) correct += (pred == y).sum().item() total += y.size(0) print(f"Epoch {epoch+1}: test accuracy = {correct/total:.4f}") torch.save(model.state_dict(), "mnist_cnn.pt") print("Saved model to mnist_cnn.pt") if __name__ == "__main__": main()

Run: python train.py → produces mnist_cnn.pt.

Recommendation
Add more diverse examples, e.g. a failure case (low confidence prediction, blurry image) showing how the skill handles edge cases
13 / 15

Progress:

  • Step 1: Set up environment (torch, torchvision)
  • Step 2: Load MNIST via torchvision.datasets.MNIST (auto-downloads)
  • Step 3: Define CNN architecture (2 conv + 2 pool + 2 fc layers is sufficient)
  • Step 4: Train for 5 epochs with Adam optimizer, cross-entropy loss
  • Step 5: Evaluate on test split each epoch, print accuracy
  • Step 6: Save model weights (torch.save(model.state_dict(), ...))
  • Step 7: Write separate infer.py for inference on new images
  • Step 8: Validate inference on a few test samples before trusting on custom images

Inference script

Python
# infer.py — loads trained model and predicts on an image import sys import torch from torchvision import transforms from PIL import Image from train import MNISTCNN # reuse architecture definition device = "cuda" if torch.cuda.is_available() else "cpu" def load_model(path="mnist_cnn.pt"): model = MNISTCNN().to(device) model.load_state_dict(torch.load(path, map_location=device)) model.eval() return model def preprocess(image_path): transform = transforms.Compose([ transforms.Grayscale(num_output_channels=1), transforms.Resize((28, 28)), transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,)) ]) img = Image.open(image_path) return transform(img).unsqueeze(0).to(device) # add batch dim def predict(image_path, model_path="mnist_cnn.pt"): model = load_model(model_path) x = preprocess(image_path) with torch.no_grad(): logits = model(x) probs = torch.softmax(logits, dim=1) pred = probs.argmax(1).item() return pred, probs.squeeze().tolist() if __name__ == "__main__": image_path = sys.argv[1] digit, confidences = predict(image_path) print(f"Predicted digit: {digit}") print(f"Confidence: {confidences[digit]:.4f}")

Run: python infer.py path/to/digit.png

Recommendation
Include guidance on handling non-28x28 or non-grayscale custom images programmatically (auto-resize/convert) rather than only mentioning it as a pitfall
13 / 20

Example 1: Input: "Train an MNIST classifier and tell me the final test accuracy" Output: Runs train.py, reports per-epoch accuracy, e.g. "Epoch 5: test accuracy = 0.9912", saves mnist_cnn.pt.

Example 2: Input: "Predict the digit in my_digit.png using the trained model" Output: Runs infer.py my_digit.png → "Predicted digit: 7, Confidence: 0.9987"

Recommendation
Consider parameterizing epoch count/architecture choice or noting when to deviate from the 5-epoch default (e.g., larger datasets, different accuracy targets)
  • Always normalize inputs with MNIST's known mean/std (0.1307, 0.3081) — matches training distribution.
  • Keep the model definition (MNISTCNN class) in one shared file (train.py) and import it in infer.py — avoids architecture mismatch when loading weights.
  • Use model.eval() + torch.no_grad() during inference to disable dropout and save memory.
  • For custom hand-drawn images: invert colors if background is white/foreground black (MNIST is white digit on black background) — check with a quick visualization before batch inference.
  • 5 epochs with this architecture reaches ~99% test accuracy — no need for deeper nets unless requirements demand higher accuracy.
  • Forgetting model.eval() before inference — leaves dropout active, causing inconsistent predictions.
  • Mismatched preprocessing — using different normalization/resizing at inference than training corrupts predictions silently (no error, just wrong results).
  • Not resizing custom images to 28x28 — model expects fixed input size; will crash or silently mis-predict if reshaped incorrectly.
  • Color inversion mismatch — feeding a black-digit-on-white-background image without inverting colors (real-world scans are often the opposite of MNIST's convention).
  • Re-downloading dataset every run — set download=True only once; reuse the local ./data cache.
0
Grade B+AI Skill Framework
Scorecard
Criteria Breakdown
Quick Start
15/15
Workflow
13/15
Examples
13/20
Completeness
17/20
Format
14/15
Conciseness
13/15