AI Skill Report Card
Evaluating AI Skill Viability
Quick Start15 / 15
Assessment Questions:
- Delegation: Can you clearly define what tasks this skill should handle vs. what humans should do?
- Description: Can you explain how this skill works and why it produces specific outputs?
- Discernment: Can you identify when the skill's outputs are reliable vs. need human review?
- Diligence: Do you have a plan for maintaining and monitoring this skill over time?
If you answer "yes" to all four, the skill is likely worth developing.
Recommendation▾
Add concrete scoring rubric for the 1-5 ratings mentioned in Step 5 - what makes something a 3 vs 4?
Workflow15 / 15
Step 1: Delegation Assessment
- Define specific tasks the skill will handle
- Identify tasks that must remain human-controlled
- Confirm the AI can realistically perform the delegated tasks
- Estimate time/effort savings vs. development cost
Step 2: Description Assessment
- Can you explain the skill's logic to stakeholders?
- Are the inputs and expected outputs clear?
- Can you document why certain decisions are made?
- Is the skill's purpose easily communicable?
Step 3: Discernment Assessment
- Identify potential failure modes or edge cases
- Define quality thresholds for outputs
- Plan human review checkpoints
- Consider bias or ethical concerns
Step 4: Diligence Assessment
- Plan for regular performance monitoring
- Identify who will maintain the skill
- Consider how requirements might change over time
- Estimate ongoing maintenance effort
Step 5: Final Decision Score each competency 1-5. If total score ≥16, proceed with development.
Recommendation▾
Include a template or framework for documenting the assessment results
Examples18 / 20
Example 1: Code Review Skill Input: "Should I create a skill for reviewing Python code?" Assessment:
- Delegation: ✓ AI can check syntax, style, common patterns
- Description: ✓ Can explain what coding standards are being checked
- Discernment: ✓ Can identify when complex logic needs human review
- Diligence: ✓ Coding standards evolve, but manageable to update Output: Recommended - Strong candidate for skill development
Example 2: Investment Advice Skill Input: "Should I create a skill for giving financial investment advice?" Assessment:
- Delegation: ✗ High-stakes decisions require human expertise
- Description: ✓ Can explain analysis methodology
- Discernment: ✗ Hard to identify when advice might be wrong
- Diligence: ✗ Market conditions change rapidly Output: Not Recommended - Too many red flags
Recommendation▾
Provide more specific guidance on estimating development cost vs. time savings in the delegation assessment
Best Practices
- Start small: Begin with narrow, well-defined use cases
- Plan for iteration: Skills improve through use and feedback
- Document assumptions: Record what the skill can/cannot do
- Set success metrics: Define how you'll measure skill effectiveness
- Consider alternatives: Sometimes existing tools are sufficient
Common Pitfalls
- Over-automating: Delegating tasks that need human judgment
- Under-explaining: Creating "black box" skills nobody understands
- Ignoring edge cases: Not planning for when the skill fails
- Set-and-forget: Building skills without ongoing maintenance plans
- Scope creep: Adding features that dilute the core purpose