AI Skill Report Card

Security

A90·Oct 15, 2026·Source: Extension-selection

Security decisions are made from evidence, not checklists. Depth of investigation scales with data sensitivity, exposure, impact, and reversibility — not with the length of a generic best-practices list. A one-line CSS change gets no security review. An auth change gets a full trust-boundary analysis.

14 / 15

Given a task, answer these before touching code:

  1. What changed or is being built? (feature, endpoint, dependency, config)
  2. Does it cross a trust boundary? (accepts external input, handles auth, touches sensitive data, is externally reachable)
  3. If no → proceed with normal engineering care, no special review.
  4. If yes → run the Workflow below, scoped to the actual boundary touched.

Do not run a "full security audit" for a task that doesn't call for one. Do not skip investigation for a task that does.

Recommendation▾
Name could be more descriptive/gerund form, e.g. 'assessing-security-risk' instead of generic 'security'
15 / 15
Progress:
- [ ] 1. Scope: identify assets, trust boundaries, entry points, sensitive data touched
- [ ] 2. Threat ID: enumerate plausible attack paths an actor could actually take
- [ ] 3. Investigate: confirm exploitability in real code/config, find root cause
- [ ] 4. Remediate: apply the minimal control that addresses the root cause
- [ ] 5. Verify: prove the risk is reduced without breaking legitimate behavior
- [ ] 6. Report: findings, confidence level, residual risk, what's unverified

1. Scope the context

Read the actual code, auth/authz logic, data flow, dependency manifest, and existing controls relevant to the change — not the whole codebase. Identify:

  • What data/operations are sensitive here (PII, credentials, money, privileged actions)
  • Who can reach this code path and with what privileges
  • What already defends this boundary (don't assume absence of a control — check)

2. Identify threats, proportionally

For trivial changes: a sentence is enough ("input is a fixed internal enum, no injection surface").

For meaningful changes: think like an attacker with access equivalent to each trust level (anonymous, authenticated user, authenticated user of another tenant, admin). Ask what each could do that they shouldn't. Use OWASP Top 10 / ASVS as a reference checklist to avoid blind spots, not as a mandatory gate — select only the requirements relevant to this architecture and data.

3. Investigate before fixing

  • Trace the exact code path. Don't fix a symptom you haven't located in source.
  • Determine if it's exploitable under the real config (e.g., a "vulnerable" query might be unreachable because an upstream filter already restricts input — verify, don't assume).
  • Find root cause: a missing authz check at the service layer, not just the one endpoint that happened to expose it.
  • If you cannot reproduce or confirm: say so explicitly. Report it as a credible-but-unverified risk with what evidence would resolve it. Never claim exploitability without evidence; never dismiss a plausible risk just because you couldn't reproduce it in the time available.

4. Apply the minimal effective control

Match control to actual threat — see reference table below. Prefer the framework's built-in secure mechanism over a custom one. Don't bolt on unrelated hardening while fixing a specific bug.

5. Verify risk reduction

Define success per finding before declaring it fixed:

  • Authz bug → test that unauthorized actor is denied AND legitimate actor still works
  • Injection → regression test with malicious payload through the real code path
  • Secret exposure → confirm removed from history/logs/config, not just current file
  • Dependency CVE → confirm vulnerable function is actually reachable/used, then patch and re-scan

A green scanner or passing test is evidence, not proof of total safety. State what remains unverified.

6. Coordinate, don't absorb

Hand off when the fix requires another domain's full workflow:

  • architecture — trust-boundary redesign, new service boundaries
  • backend-data — implementing the actual server-side validation/query code
  • frontend — safe rendering, client-side session UI
  • debugging — root cause of an unexplained failure isn't yet known
  • testing-quality — broad regression suite beyond the security-specific tests
  • performance — a control has material, evidenced performance cost
Recommendation▾
Consider trimming the Control Reference table slightly or linking out, as it borders on checklist-style despite the stated philosophy
AreaRiskControl
AuthZHorizontal/vertical privilege escalation, IDORCheck authorization per-resource, per-operation, server-side, every request — not just at login
InputSQLi, XSS, command injection, SSTI, insecure deserializationParameterized queries, context-aware output encoding, allow-lists, avoid eval/dynamic exec on untrusted data
SecretsPlaintext/leaked credentialsHashed passwords (established KDF, e.g. bcrypt/argon2), secrets manager, never in logs/VCS/config in plaintext
CryptoWeak/custom cryptoEstablished libraries only, vetted algorithms, no home-grown ciphers
WebCSRF, SSRF, open redirect, upload abuseFramework CSRF protection, URL/scheme allow-lists for server-side fetches, validate upload type/size/storage location
ErrorsInfo disclosureGeneric client errors, detailed logs server-side only, no stack traces/secrets to client
DependenciesKnown CVEsConfirm affected version AND vulnerable code path is actually used before patching; check compatibility before upgrading
Config/DeliverySupply-chain, excess permissionsLeast-privilege build/deploy creds, protect artifact integrity, review exposed config only when exposure warrants it
19 / 20

Example 1 — Broken access control Input: "User A can view /api/orders/{id} for order IDs belonging to User B by changing the ID in the URL." Investigation: Trace handler — confirm it fetches by ID with no ownership check against session user. Output: Confirmed IDOR. Root cause: missing order.user_id == session.user_id check in GET /api/orders/:id. Fix: add ownership check at the data-access layer (not just the route), returning 404 (not 403) to avoid existence leakage. Verify: test that User A gets 404 on B's order, and still 200 on their own.

Example 2 — Suspected SQLi Input: "This search endpoint concatenates a string into a query." Investigation: Read actual query construction. Found: f"SELECT * FROM items WHERE name LIKE '%{q}%'" with q from unvalidated query param. Output: Confirmed injectable. Fix: parameterized query (WHERE name LIKE ? with bound param). Verify: regression test with payload ' OR '1'='1 confirms no behavior change / no error-based leak; confirm legitimate search still works.

Example 3 — Secret in repo Input: "Found what looks like an API key in a config file committed to git." Investigation: Confirm it's a real, live-format secret (not a placeholder/example value) and check git history for exposure window. Output: Confirmed leaked secret. Remediation: rotate the credential immediately (code fix alone is insufficient since it's in history), move to secret manager/env var, add to .gitignore, consider history scrub if repo is public. Verify: old key confirmed revoked/rejected by the issuing service.

Example 4 — Dependency CVE Input: "Scanner flags lodash@4.17.15 for a prototype pollution CVE." Investigation: Check whether the vulnerable function (e.g., merge/zipObjectDeep) is actually used in this codebase and whether attacker-controlled input reaches it. Output: If unused/unreachable: document as non-exploitable in this context, low-priority upgrade. If reachable: confirmed risk, upgrade to patched version, run existing test suite for breakage, re-scan to confirm resolution.

Example 5 — Unreproducible report Input: "Pen-test report claims session fixation but steps don't reproduce in staging." Investigation: Check session ID regeneration logic on login server-side. Output: Code shows session ID is regenerated on privilege change — report likely a false positive against an older build, OR: if regeneration is missing on one specific login path (e.g., SSO callback), confirmed partial vulnerability on that path only. State confidence level and which paths were checked vs. not.

Example 6 — Fix vs. breakage tradeoff Input: "Tightening CORS policy to fix an overly permissive * origin." Investigation: Identify which legitimate origins/clients currently rely on the open policy (check API consumers, docs, frontend origins). Output: Replace wildcard with explicit allow-list of known legitimate origins; add a test asserting blocked origins get rejected and known origins still work; flag any origin whose legitimacy is unclear for product/architecture sign-off rather than guessing.

Recommendation▾
Add a brief explicit example of a 'trivial change, no review needed' to reinforce the proportionality principle with a concrete case
  • Investigate the real code/config before asserting a vulnerability exists or is fixed.
  • Match investigation depth to impact — don't full-audit a typo fix, don't skim an auth change.
  • Fix root cause (missing check in shared logic) not just the one symptomatic call site.
  • Prefer framework/library secure defaults over custom security code.
  • State confidence and residual risk explicitly; don't imply total safety.
  • Verify both that the attack is blocked AND that legitimate use still works.
  • Running a full OWASP checklist against every change regardless of relevance.
  • Declaring something "vulnerable" or "fixed" without reading the actual code path.
  • Patching the visible symptom while leaving the same root cause in other call sites.
  • Rotating/upgrading nothing when a secret was only ever in history, not current files (incomplete remediation) — or conversely, upgrading dependencies blindly without checking reachability or compatibility.
  • Overclaiming: "this scan passed" ≠ "this is secure."
  • Breaking legitimate access while closing a hole, without a test proving both.
  • Performing intrusive/destructive security testing against production or third-party systems without explicit authorization.
0
Grade AAI Skill Framework
Scorecard
Criteria Breakdown
Quick Start
14/15
Workflow
15/15
Examples
19/20
Completeness
19/20
Format
13/15
Conciseness
14/15