AI Skill Report Card

Conducting Iterative Research Audits

A-87·Oct 31, 2026·Source: Web
YAML
--- name: conducting-iterative-research-audits description: Runs recursive, gap-targeted research passes that audit a prior report, identify unverified claims and blind spots, execute new grounded searches against only those gaps, and output a coverage delta with corrections, new sources, and remaining-gaps tracking. Use when iteratively deepening a research report across multiple passes, when prior findings need re-verification against current sources, when a knowledge base has known blind spots to close, or when building a self-extending research skill that must avoid repeating prior searches. --- # Conducting Iterative Research Audits
14 / 15

Given a prior report (or a fresh topic with no prior report), run this single pass:

  1. Extract every "unverified," "unknown," "TODO," or "Previously flagged" item from the prior report into a gap list.
  2. Write 5-10 new search queries, each targeting exactly one gap — never re-run a query phrased similarly to one already in the report's source list.
  3. Execute searches, prioritizing: primary sources > maintained awesome-lists/directories > recent (last 12mo) authoritative writeups > forums/Q&A as corroboration only.
  4. For each finding, tag it with the gap ID it closes, a confidence label, and a dated source.
  5. Output using the fixed template (below) — delta only, no restatement of prior content.
  6. If fewer than 2 substantive new findings surface, say so explicitly and recommend stopping iteration.

No preamble, no summary of "what I did" — go straight to the audit.

Recommendation▾
Add a third example showing a fully fresh topic with no prior report, since the Quick Start mentions this case but no example demonstrates it
15 / 15

Progress checklist for each pass:

  • Step 1: Audit prior report for gaps (unverified claims, missing sub-topics, missing expert lenses, uncited domains)
  • Step 2: Convert each gap into 1+ targeted, non-overlapping search queries
  • Step 3: Execute multi-angle searches (direct, adjacent-domain, counter-narrative, retraction/correction check)
  • Step 4: Cross-check currency — discard anything deprecated, superseded, or outdated as of the stated current date
  • Step 5: Verify each candidate finding against ≥2 independent sources where possible; label single-source claims as "unverified"
  • Step 6: Draft New Findings grouped by theme, each mapped to the gap it closes
  • Step 7: Draft Updated/Corrected Claims — explicitly state what prior claim is superseded and why
  • Step 8: Draft Remaining Gaps — be honest if yield is low
  • Step 9: Compile New Sources list (numbered, dated, linked, excluding anything already cited)
  • Step 10: Final self-check pass for sycophancy/hallucination before output (see Verification below)

Step 1 — Gap Extraction

Scan the prior report for:

  • Explicit markers: "unverified," "unknown," "TBD," "not yet confirmed"
  • A "Previously" / "Remaining Gaps" section from the last pass
  • Sub-topics named but not expanded (one-line mentions)
  • Single-perspective coverage (e.g., only vendor docs, no practitioner/critic view)
  • Domains adjacent to the topic that were never touched
  • Sources older than the acceptable freshness window for that domain

Produce a numbered gap list (G1, G2, G3...) before searching anything.

Step 2 — Query Design

Rules:

  • One query ≠ one gap is fine, but never reuse a query signature from the prior pass (check prior report's implied search terms if listed, or infer from citation phrasing).
  • For each gap, generate at least: one direct query, one adjacent/alternate-terminology query, one "criticism OR limitations OR deprecated" query, one "site:github.com awesome" or curated-list query if the domain has active maintained lists, one "2026" or current-year-filtered query for recency.
  • Explicitly search for retractions/updates to previously cited sources: "[prior source name]" update OR correction OR deprecated OR retracted.

Step 3 — Multi-Angle Execution

Run searches in this order per gap, stopping early only if saturation is reached (3+ consistent independent sources):

  1. Direct factual query
  2. Counter-narrative / skeptic query
  3. Primary source / official docs / changelog query
  4. Community consensus query (awesome-lists, top-starred repos, current directories, recent high-signal forum threads)
  5. Currency check query (filter explicitly for current year / "latest")

Step 4 — Currency Filter

Discard or flag as "historical context only":

  • Anything superseded by a newer version, standard, or finding
  • Deprecated tools/APIs/methods unless needed to explain why they were replaced
  • Stats/benchmarks older than the domain's natural refresh cycle (e.g., 6mo for fast-moving tech, 2-3yrs for slower domains)

Step 5 — Verification (anti-hallucination, anti-sycophancy gate)

Before writing any claim:

  • Does it trace to a specific, nameable source? If not → label "unverified" or drop it.
  • Would this claim survive if the user disagreed with it? If a claim only exists because it sounds agreeable or expected, cut it.
  • Is this common knowledge dressed up as a finding? If yes, cut it — it's fluff.
  • Does this contradict the prior report? If yes, it goes in "Updated or Corrected Claims," not "New Findings."
Recommendation▾
Include a concrete bad-output example (e.g., sycophancy leak or restated finding) alongside the good ones to reinforce the pitfalls section
Markdown
### New Findings **Theme: [Theme Name]** (addresses: G[#]) - Claim. [S1] — Confidence: High/Medium/Unverified ### Updated or Corrected Claims - Prior claim: "..." → Correction: "..." [S#] (reason: outdated/contradicted/superseded) ### Remaining Gaps - G[#]: still unresolved because [reason] - If <2 substantive new findings this pass: state explicitly, recommend stopping further iteration on this branch. ### New Sources 1. [Title](URL) — Publisher, YYYY-MM-DD — used for G[#]
16 / 20

Example 1: Input: Prior report on "vector database selection" flagged G3: "pricing at scale — unverified" and G5: "no mention of on-device/embedded vector stores." Output:

### New Findings
**Theme: Pricing at Scale** (addresses: G3)
- Managed vector DB cost per 10M vectors with metadata filtering varies 4x between providers primarily due to storage-tier vs compute-tier billing models, not raw vector count. [S1] — Confidence: Medium

**Theme: Embedded/On-Device Vector Stores** (addresses: G5)
- SQLite-based vector extensions now support HNSW indexing in-process, eliminating the network hop for sub-1M-vector use cases. [S2] — Confidence: High

### Remaining Gaps
- G3 partially resolved; enterprise contract pricing still opaque (unverified).

Example 2: Input: Re-running the same prompt on a report that was already exhaustively covered two passes ago. Output:

### New Findings
None met the 2-source verification bar this pass.

### Remaining Gaps
No substantive new findings (0 passed verification). Recommend stopping iteration on this topic; revisit only if a triggering event occurs (new major release, regulatory change, etc.)
Recommendation▾
Define 'saturation' and 'confidence' labels more precisely (e.g., exact criteria for High vs Medium vs Unverified) to reduce ambiguity across passes
  • Always date-stamp sources; treat undated sources as lower confidence.
  • Prefer primary sources (official docs, changelogs, maintainer statements, original papers) over secondary summaries.
  • Use awesome-lists/curated directories as discovery tools, not as citable evidence — chase the actual source they point to.
  • Explicitly search for disagreement/criticism on every major claim before accepting it — absence of searched-for criticism is itself worth noting.
  • Keep each pass scoped to the gap list; resist the urge to re-explain the whole topic.
  • When a prior source has been updated or retracted, say so loudly in "Updated or Corrected Claims" — don't bury it.
  • Track query signatures across passes (mentally or in a scratch list) so pass N+1 doesn't repeat pass N's search angles.
  • Restating prior findings as if new — if it's not a delta, cut it.
  • Treating forum consensus as verified fact — use it only as a lead to find the primary source.
  • Padding "Remaining Gaps" with vague filler to look thorough — only list gaps that are specific and actionable.
  • Accepting a single source for a load-bearing claim — label it unverified instead.
  • Letting agreeable-sounding claims slip through without checking if they're actually grounded (sycophancy leak).
  • Ignoring currency — citing a 2023 benchmark in a fast-moving domain without checking for a 2026 replacement.
  • Over-querying near-duplicate phrasings that just re-surface the same sources as the prior pass.
0
Grade A-AI Skill Framework
Scorecard
Criteria Breakdown
Quick Start
14/15
Workflow
15/15
Examples
16/20
Completeness
18/20
Format
14/15
Conciseness
13/15