AI Skill Report Card
Building MNIST Image Classifier
Understanding Claude Code Skills First
Before the skill file, here's the context you asked for:
How a Skill differs from manually prompting:
| Manual Prompting | Skill |
|---|---|
| You re-explain your requirements every time ("build MNIST classifier, use CNN, save model, write inference script...") | You say "use the mnist-classifier skill" (or Claude auto-detects it) — instructions are pre-loaded |
| Claude may vary approach each session (different architecture, different file structure) | Consistent, repeatable methodology every time |
| No memory of your preferred conventions (train/test split, model format, logging style) | Encodes your exact workflow, best practices, and pitfalls once, reused forever |
| Prompt lives in your head or scattered chat history | Skill lives as a versioned SKILL.md file — shareable, editable, greppable |
| Works only in that conversation | Works across sessions/projects once installed in Claude's skills directory |
Mechanism — how Skills actually work under the hood:
- Storage: Skills are markdown files (
SKILL.md) with YAML frontmatter, stored in a skills directory Claude Code scans. - Discovery: Claude reads the
name+descriptionfrontmatter of all available skills at session start (lightweight index, not full content). - Triggering: When your prompt matches the
description's trigger phrases (e.g., "classify MNIST images"), Claude loads that skill's full content into context. - Execution: Claude follows the skill's workflow step-by-step — using its code snippets, checklists, and pitfalls as grounded instructions instead of improvising.
- Composability: Skills can reference helper scripts/files alongside them (e.g., a
train.pytemplate) which Claude can read or execute.
Now, the skill itself:
Markdown--- name: building-mnist-image-classifier description: Builds, trains, and runs inference with a CNN image classification model on the MNIST handwritten digit dataset using PyTorch. Use when the user asks to classify MNIST digits, build a digit recognition model, train an image classifier on MNIST, or run inference on handwritten digit images. ---
Quick Start15 / 15
Python# train.py — trains and saves an MNIST CNN classifier import torch, torch.nn as nn, torch.nn.functional as F from torch.utils.data import DataLoader from torchvision import datasets, transforms device = "cuda" if torch.cuda.is_available() else "cpu" class MNISTCNN(nn.Module): def __init__(self): super().__init__() self.conv1 = nn.Conv2d(1, 32, 3, padding=1) self.conv2 = nn.Conv2d(32, 64, 3, padding=1) self.pool = nn.MaxPool2d(2) self.fc1 = nn.Linear(64 * 7 * 7, 128) self.fc2 = nn.Linear(128, 10) self.dropout = nn.Dropout(0.25) def forward(self, x): x = self.pool(F.relu(self.conv1(x))) x = self.pool(F.relu(self.conv2(x))) x = x.flatten(1) x = self.dropout(F.relu(self.fc1(x))) return self.fc2(x) def main(): transform = transforms.Compose([ transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,)) ]) train_ds = datasets.MNIST(root="./data", train=True, download=True, transform=transform) test_ds = datasets.MNIST(root="./data", train=False, download=True, transform=transform) train_loader = DataLoader(train_ds, batch_size=128, shuffle=True) test_loader = DataLoader(test_ds, batch_size=256) model = MNISTCNN().to(device) optimizer = torch.optim.Adam(model.parameters(), lr=1e-3) for epoch in range(5): model.train() for x, y in train_loader: x, y = x.to(device), y.to(device) optimizer.zero_grad() loss = F.cross_entropy(model(x), y) loss.backward() optimizer.step() model.eval() correct, total = 0, 0 with torch.no_grad(): for x, y in test_loader: x, y = x.to(device), y.to(device) pred = model(x).argmax(1) correct += (pred == y).sum().item() total += y.size(0) print(f"Epoch {epoch+1}: test accuracy = {correct/total:.4f}") torch.save(model.state_dict(), "mnist_cnn.pt") print("Saved model to mnist_cnn.pt") if __name__ == "__main__": main()
Run: python train.py → produces mnist_cnn.pt.
Recommendation▾
Add more diverse examples, e.g. a failure case (low confidence prediction, blurry image) showing how the skill handles edge cases
Workflow13 / 15
Progress:
- Step 1: Set up environment (
torch,torchvision) - Step 2: Load MNIST via
torchvision.datasets.MNIST(auto-downloads) - Step 3: Define CNN architecture (2 conv + 2 pool + 2 fc layers is sufficient)
- Step 4: Train for 5 epochs with Adam optimizer, cross-entropy loss
- Step 5: Evaluate on test split each epoch, print accuracy
- Step 6: Save model weights (
torch.save(model.state_dict(), ...)) - Step 7: Write separate
infer.pyfor inference on new images - Step 8: Validate inference on a few test samples before trusting on custom images
Inference script
Python# infer.py — loads trained model and predicts on an image import sys import torch from torchvision import transforms from PIL import Image from train import MNISTCNN # reuse architecture definition device = "cuda" if torch.cuda.is_available() else "cpu" def load_model(path="mnist_cnn.pt"): model = MNISTCNN().to(device) model.load_state_dict(torch.load(path, map_location=device)) model.eval() return model def preprocess(image_path): transform = transforms.Compose([ transforms.Grayscale(num_output_channels=1), transforms.Resize((28, 28)), transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,)) ]) img = Image.open(image_path) return transform(img).unsqueeze(0).to(device) # add batch dim def predict(image_path, model_path="mnist_cnn.pt"): model = load_model(model_path) x = preprocess(image_path) with torch.no_grad(): logits = model(x) probs = torch.softmax(logits, dim=1) pred = probs.argmax(1).item() return pred, probs.squeeze().tolist() if __name__ == "__main__": image_path = sys.argv[1] digit, confidences = predict(image_path) print(f"Predicted digit: {digit}") print(f"Confidence: {confidences[digit]:.4f}")
Run: python infer.py path/to/digit.png
Recommendation▾
Include guidance on handling non-28x28 or non-grayscale custom images programmatically (auto-resize/convert) rather than only mentioning it as a pitfall
Examples13 / 20
Example 1:
Input: "Train an MNIST classifier and tell me the final test accuracy"
Output: Runs train.py, reports per-epoch accuracy, e.g. "Epoch 5: test accuracy = 0.9912", saves mnist_cnn.pt.
Example 2:
Input: "Predict the digit in my_digit.png using the trained model"
Output: Runs infer.py my_digit.png → "Predicted digit: 7, Confidence: 0.9987"
Recommendation▾
Consider parameterizing epoch count/architecture choice or noting when to deviate from the 5-epoch default (e.g., larger datasets, different accuracy targets)
Best Practices
- Always normalize inputs with MNIST's known mean/std (
0.1307,0.3081) — matches training distribution. - Keep the model definition (
MNISTCNNclass) in one shared file (train.py) and import it ininfer.py— avoids architecture mismatch when loading weights. - Use
model.eval()+torch.no_grad()during inference to disable dropout and save memory. - For custom hand-drawn images: invert colors if background is white/foreground black (MNIST is white digit on black background) — check with a quick visualization before batch inference.
- 5 epochs with this architecture reaches ~99% test accuracy — no need for deeper nets unless requirements demand higher accuracy.
Common Pitfalls
- Forgetting
model.eval()before inference — leaves dropout active, causing inconsistent predictions. - Mismatched preprocessing — using different normalization/resizing at inference than training corrupts predictions silently (no error, just wrong results).
- Not resizing custom images to 28x28 — model expects fixed input size; will crash or silently mis-predict if reshaped incorrectly.
- Color inversion mismatch — feeding a black-digit-on-white-background image without inverting colors (real-world scans are often the opposite of MNIST's convention).
- Re-downloading dataset every run — set
download=Trueonly once; reuse the local./datacache.