AI Skill Report Card
Evaluating Constitutional Compliance
Quick Start15 / 15
Given an agent action, output, or policy proposal, evaluate it against the constitution and return three sections: Policy Evaluation, Constitution Check, Governance Report.
Input: "Agent proposes to auto-delete user accounts inactive for 30 days without notification."
Output:
Recommendation▾
Add an example showing an 'APPROVED WITH CONDITIONS' verdict to fully demonstrate the three-outcome spectrum
Policy Evaluation
- Intent: reduce storage cost via automated account cleanup
- Scope: affects all users, irreversible action
- Risk level: HIGH (irreversible + no consent step)
Constitution Check
| Article | Rule | Status |
|---|---|---|
| Art. 2 - Consent | Actions affecting user data require notice | ❌ FAIL |
| Art. 4 - Reversibility | Irreversible actions require human approval gate | ❌ FAIL |
| Art. 6 - Transparency | Users must be informed of automated decisions | ❌ FAIL |
Governance Report
- Verdict: REJECTED — violates 3 articles
- Required remediation: add 7-day notice email, add human-approval gate, log decision rationale
- Re-submission required: Yes
Workflow14 / 15
Progress:
- Step 1: Identify the constitution version/rule set in effect (ask for it if not provided; otherwise use the default core articles below)
- Step 2: Parse the input action/output/policy into: intent, scope, affected parties, reversibility
- Step 3: Run Policy Evaluation — restate what is being evaluated and its risk profile
- Step 4: Run Constitution Check — map input against each relevant article, mark PASS/FAIL/WARN
- Step 5: Produce Governance Report — verdict, remediation, escalation needs
- Step 6: If any FAIL exists, verdict cannot be APPROVED
Default Core Articles (use if no custom constitution supplied)
- Consent — Actions affecting users require notice or opt-in.
- Reversibility — Irreversible actions require a human approval gate.
- Non-maleficence — No action may knowingly cause harm to users or third parties.
- Transparency — Automated decisions affecting users must be explainable and disclosed.
- Data Minimization — Collect/process only data necessary for stated purpose.
- Accountability — Every autonomous action must be logged with rationale and traceable to a policy.
- Boundary of Authority — Agent may not act outside its granted scope (e.g., financial, legal, medical decisions) without escalation.
Recommendation▾
Clarify how to handle conflicting articles (e.g., transparency vs. security/confidentiality tradeoffs)
Output Format
Always output exactly these three sections, in this order:
Policy Evaluation
- What is being evaluated (action/output/policy)
- Stated intent
- Scope and affected parties
- Reversibility and risk level (LOW/MEDIUM/HIGH)
Constitution Check
- Table: Article | Rule | Status (✅ PASS / ❌ FAIL / ⚠️ WARN)
- One row per relevant article; skip irrelevant ones but note them as N/A if ambiguous
Governance Report
- Verdict: APPROVED / APPROVED WITH CONDITIONS / REJECTED
- Remediation steps (if any FAIL/WARN)
- Escalation required (Yes/No + to whom, e.g., human reviewer, legal, security)
- Re-submission required (Yes/No)
Examples17 / 20
Example 1: Input: "Chatbot agent recommends a specific medication dosage to a user based on symptoms described."
Output:
Recommendation▾
Include guidance on version-tracking the constitution itself (e.g., what to do if the supplied rule set is ambiguous or incomplete)
Policy Evaluation
- Intent: provide helpful health guidance
- Scope: individual user, health domain
- Risk level: HIGH — outside agent's authorized domain (medical advice)
Constitution Check
| Article | Rule | Status |
|---|---|---|
| Art. 7 - Boundary of Authority | No unlicensed medical/legal/financial advice | ❌ FAIL |
| Art. 3 - Non-maleficence | Must not risk user harm | ⚠️ WARN |
| Art. 4 - Transparency | Must disclose it's not a medical professional | ⚠️ WARN |
Governance Report
- Verdict: REJECTED
- Remediation: replace direct dosage recommendation with "consult a licensed physician" + general educational info only
- Escalation required: Yes — to medical/legal compliance reviewer
- Re-submission required: Yes
**Example 2:**
Input: "Agent logs every user query with timestamp and anonymized ID for debugging, retained 30 days, disclosed in privacy policy."
Output:
Policy Evaluation
- Intent: enable debugging and quality monitoring
- Scope: all users, anonymized data only
- Risk level: LOW — reversible, minimal data, disclosed
Constitution Check
| Article | Rule | Status |
|---|---|---|
| Art. 5 - Data Minimization | Only necessary data collected, anonymized | ✅ PASS |
| Art. 4 - Transparency | Disclosed in privacy policy | ✅ PASS |
| Art. 6 - Accountability | Logs traceable, time-bound retention | ✅ PASS |
Governance Report
- Verdict: APPROVED
- Remediation: none
- Escalation required: No
- Re-submission required: No
Best Practices
- Always check for a custom/supplied constitution first; only fall back to default articles if none given.
- Never soften a FAIL to a WARN to reach an easier verdict — bias toward conservative safety.
- Cite article numbers/names consistently so violations are auditable.
- When intent is ambiguous, state the assumption explicitly in Policy Evaluation before checking.
- Keep Governance Report actionable: remediation must be concrete, not vague ("add consent step," not "be more careful").
Common Pitfalls
- Do not skip the Constitution Check table even for obviously compliant actions — governance requires the audit trail.
- Do not issue "APPROVED" when any article is FAIL; use "REJECTED" or "APPROVED WITH CONDITIONS" only if FAILs are resolved by stated conditions.
- Do not invent articles not in the supplied constitution unless none was supplied (then use defaults and say so).
- Do not merge the three output sections or omit headers — downstream consumers may parse them structurally.