AI Skill Report Card

Triaging Wazuh Security

>

A-86·Oct 18, 2026·Source: Web

Triaging Wazuh Security

Produce a reproducible, evidence-based security assessment of a Wazuh-managed fleet. Never summarize dashboard totals as fact — recompute from raw evidence, verify every CVE against a live authoritative source, and mark UNKNOWN rather than guess.

13 / 15
Bash
# Phase 1: mandatory fresh extraction (always run first, wait for exit) cd /mnt/1TBSATASSD/IT/wazuh && python3 wazuh_extract.py \ --mcp-url http://192.168.10.151:3000/ --output wazuh_evidence # Find and validate the newest run RUN=$(find /mnt/1TBSATASSD/IT/wazuh/wazuh_evidence -maxdepth 1 -mindepth 1 \ -type d -name 'run_*' -printf '%T@ %p\n' | sort -nr | head -1 | cut -d' ' -f2-) test -n "$RUN" && test -s "$RUN/run_meta.json" || { echo 'ERROR: missing run'; exit 2; }

Inspect run_meta.json for status, agent_count, calls_ok/calls_made, and per-collection pagination (rows_retrieved, api_total, complete) before touching any downstream analysis.

Recommendation▾
Hardcoded paths (/mnt/1TBSATASSD/IT/wazuh) and IP addresses reduce portability—consider parameterizing or noting these are environment-specific placeholders.

Raw evidence → derived calculation → analyst interpretation. Keep these three layers distinct in every claim. UNKNOWN beats a guess. A claim without a cited, opened source or a verified extraction artifact does not go in the report.

14 / 15
Progress:
- [ ] Phase 1: Fresh extraction run + exit-code/metadata validation
- [ ] Phase 2: Environment snapshot (agents, alerts, vuln counts, anomalies)
- [ ] Phase 3: CVE research against authoritative sources
- [ ] Phase 4: SCA/CIS assessment with benchmark-applicability check
- [ ] Phase 5: Findings (3-8 max) + written, verified report

Phase 1 — Extraction (never skip, never reuse old evidence)

Run the extractor in blocking mode. Wait for actual termination — do not infer completion from a quiet terminal or new files appearing. Check exit code, then validate run_meta.json. Decision rules:

  • complete / complete_with_errors → proceed, note any errors.
  • incomplete → proceed only within stated limits; label affected sections.
  • A critical collection returns 0 rows against a high api_total → stop, report that collection as failed, don't draw conclusions from it.
  • Unreconciled counts (rows > api_total, unstable pagination) → flag as data-quality anomaly, don't auto-rerun.

Phase 2 — Environment Snapshot

From overview.md, raw JSON/CSV, logs, and run_meta.json, record: agent inventory (ID/name/OS/version/status/IP), alert volumes (routine vs. investigation-worthy), vulnerability counts by severity with denominator stated, pagination/completeness per collection, extraction errors. Recompute counts from raw data — never trust a prior summary.

Phase 3 — CVE Research

Get the real date first:

Bash
date -u +%F

Group findings by (package, installed_version, os_release) before researching — don't research identical rows repeatedly. For each group:

  1. Open a live authoritative source (Debian Security Tracker, Ubuntu Security Tracker/USN, MSRC, GitHub Advisory DB, OSV, or vendor PSIRT) matching the ecosystem.
  2. Confirm CVE, product/package, and release applicability on that page.
  3. If the page can't confirm applicability → UNKNOWN, not a guess.
  4. Never invent a fixed version. Distinguish: upstream fixed version vs. distro/vendor fixed version for the installed release vs. installed version vs. Wazuh detection status.
  5. "No KEV entry" only means none was found as of the assessment date — not proof of non-exploitability.

Common mismatch traps (treat each as UNKNOWN until resolved, never guess):

  • OS/release/kernel/package-version inconsistency — verify via /etc/os-release, dpkg-query -W/rpm -q, not uname -r alone.
  • Unusual version strings (e.g., Windows build numbers) — verify actual product before matching an advisory.
  • "Resolved/fixed/patched" in a CVE description describes the upstream bug, not proof the fix is in the installed package — verify against the distro tracker.
  • Installed version string equals the advisory's "fixed" version — check it's the same product/release/epoch, not just string equality; a fix in a different release (e.g. sid vs trixie) does not confirm the installed release is fixed.

Phase 4 — SCA / CIS Assessment

Discover actual loaded policy names from evidence — never hardcode an assumed policy. Query per-agent SCA tools directly; cross-check against raw SCA data. A zero-policy response is something to investigate, not proof of no policy.

Before calling a score "CIS compliance," verify the benchmark matches the host's OS edition/release/role. If unconfirmed, label it proxy/partial, not definitive compliance.

Per-agent state must be one of: ASSESSED, NOT_ASSESSED (zero checks ≠ PASS), NO_SUPPORTED_POLICY, DATA_MISSING, NOT_APPLICABLE, PARTIAL.

Before recommending remediation for a failed check, read the SCA policy YAML's detection logic if accessible — flag structural detection issues (undefined variables, missing drop-in directory checks) rather than telling the user to weaken a stricter-than-benchmark setting.

Open Wazuh CSV exports with encoding="utf-8-sig" (BOM) — a plain utf-8 read causes KeyError on the first header column.

Phase 5 — Findings & Report

3–8 findings max, each with: Observation (+ evidence ref), Why it matters, Remediation target (vendor-confirmed or UNKNOWN), Authoritative source (actually opened), Priority (P1/P2/P3/UNKNOWN), Uncertainties.

P1 requires both confirmed vulnerability and confirmed exposure/exploitability/operational impact — severity score or alert volume alone is not P1.

Write to /mnt/1TBSATASSD/IT/wazuh/wazuh_reports/<timestamp>/wazuh-security-report.md with sections: Header, Environment Snapshot, CVE Matrix, SCA/CIS Assessment, Findings, Improvement Plan, Definition of Done, Sources. Verify the directory is writable before writing; read the file back and cross-check claims against evidence before declaring done.

Recommendation▾
Add a third example showing a failure/edge case (e.g., extraction incomplete, critical collection returns 0 rows) to demonstrate the stop-and-report behavior in action.
15 / 20

Example 1: Input: "What's vulnerable on my agents?" Output: Fresh extraction run → snapshot of agent inventory and vuln counts by severity with stated denominator → grouped CVE research with per-group authoritative source citations → report listing only confirmed findings, UNKNOWN for anything unverifiable, saved to wazuh_reports/<ts>/.

Example 2: Input: "Second-pass audit of last week's Wazuh report" Output: Recalculate all counts from raw JSON/CSV/metadata (not prior summaries) → discrepancy ledger (previous value vs. raw-evidence value vs. corrected value) → re-verify any CVE previously called "fixed" or "false positive" against current advisories → discrepancy_ledger.csv, corrected_counts.csv, cve_verification.csv, audit_summary.json → report marked AWAITING_REVIEW.

Recommendation▾
The examples section could include more concrete before/after snippets (e.g., sample discrepancy ledger row, sample CVE matrix entry) rather than describing outputs at a high level.
  • Always distinguish raw-row counts from unique-CVE counts; state the identity key when deduplicating.
  • Recover package versions from raw JSON, never from a spreadsheet-converted CSV (version strings like 2.4.10 can be silently converted to dates).
  • Query live high-severity alert tools (e.g., level ≥12) directly instead of trusting aggregated top-rule tables, which can hide a long tail.
  • For every command: execute in blocking mode, wait for actual termination, check exit code, verify output files are non-empty and complete — never infer success from a quiet terminal.
  • Validate generated Python with python3 -m py_compile before running analysis scripts; fix the cell/file the traceback actually names.
  • Treating a Wazuh/scanner finding as proof of exploitability — it's only proof the scanner reported it.
  • Claiming PASS/fixed/compliant without an opened, confirming source.
  • Reusing a stale evidence bundle instead of running a fresh extraction.
  • Collapsing records that differ by agent/package/version/OS-release when deduplicating CVEs.
  • Padding the findings list to hit a target count instead of reporting fewer, well-supported findings.
  • Auto-rerunning the extractor in a loop instead of reporting a data-quality anomaly or incomplete pagination.
0
Grade A-AI Skill Framework
Scorecard
Criteria Breakdown
Quick Start
13/15
Workflow
14/15
Examples
15/20
Completeness
19/20
Format
14/15
Conciseness
13/15