AI Skill Report Card

Pursuing Rigorous Truth Seeking

A-83·Nov 15, 2026·Source: Web
13 / 15

Given a hard problem, run this loop instead of stopping at the first plausible answer:

  1. Restate the real question precisely (the underlying need, not just the literal ask).
  2. Generate 3+ independent approaches to answering it.
  3. Pursue each until it succeeds or hits a genuine, explainable dead end.
  4. Cross-check results against each other — agreement is evidence, disagreement is a signal to dig deeper.
  5. Report what's solid, what's uncertain, and what's unresolved, with the reasoning trail attached.
Recommendation▾
Add a brief note on time-boxing or effort calibration—how much investigation is 'enough' for lower-stakes questions to avoid over-applying this to trivial tasks.
13 / 15

Progress:

  • Define success criteria — what would actually count as "answered"?
  • List every legitimate avenue: direct lookup, computation, decomposition into sub-problems, expert heuristics, experiment/test, analogy to solved cases
  • Attempt the strongest approach first; don't settle for "good enough" if better is attainable within reasonable effort
  • When blocked, switch angles rather than declaring failure — a dead end in one method is data for the next
  • Verify independently: recompute by a different method, check against known reference points, test edge cases
  • Flag every assumption and remaining uncertainty explicitly — never round up confidence
  • Deliver the answer plus enough reasoning that someone else could audit or reproduce it

If a constraint (legal, ethical, safety, resource) blocks the strongest path, treat that constraint as part of the problem definition, not an obstacle to route around — find a different legitimate path to the user's underlying goal, or state plainly why the goal needs to change.

Recommendation▾
Include one example of a genuine dead end (where investigation concludes 'unknowable') to show that outcome explicitly, not just successful resolutions.
18 / 20

Example 1 — Technical debugging: Input: "Intermittent 500 errors in production, happens maybe 1 in 500 requests, logs show nothing useful." Output:

  • Approach A: Add structured logging around the suspected handler, deploy to staging, run load test to reproduce (reproduced at 1-in-420 rate — consistent with report).
  • Approach B: Check for shared mutable state / race conditions in the request pipeline (found a non-thread-safe cache write in SessionCache.update()).
  • Approach C: Correlate failure timestamps with GC pause logs (no correlation found — rules out GC as the cause).
  • Cross-check: A and B both implicate the same code path; C independently rules out a competing hypothesis. Conclusion: race condition in SessionCache.update(), confirmed via reproduction, not just plausible inference. Fix: add a lock or move to an atomic structure. Confidence: high, reproduced twice.

Example 2 — Conflicting data reconciliation: Input: "Our analytics dashboard says 12,400 signups last month; finance says 11,100. Which is right?" Output:

  • Approach A: Diff the two underlying queries — dashboard counts events.signup fires; finance counts users.created_at rows.
  • Approach B: Spot-check 20 random dashboard-only signups — 17 are duplicate events from a retry bug in the signup form, 3 are legitimate signups missing from finance's export due to a timezone cutoff mismatch.
  • Approach C: Recount users table directly with a corrected timezone boundary — yields 11,103, matching finance within rounding.
  • Conclusion: finance's number is closer to ground truth; dashboard over-counts due to duplicate event firing. Report both the root cause and the fix (dedupe signup events by user ID), rather than just picking a winner.

Example 3 — Research synthesis: Input: "Does remote work increase or decrease software engineer productivity?" Output:

  • Approach A: Pull controlled studies (e.g., Stanford/Bloom-style WFH experiments) — effects are positive for structured, measurable tasks, mixed for collaborative/creative work.
  • Approach B: Check company-reported internal metrics (self-reported, lower reliability — note this explicitly rather than treating it as equal-weight evidence).
  • Approach C: Look for meta-analyses that control for job type, seniority, and home environment — effect size shrinks and sometimes reverses for junior engineers needing mentorship.
  • Cross-check: studies agree remote work helps focused solo work, disagree on net effect for early-career engineers and highly collaborative work.
  • Conclusion: "It depends on role and task type" is not a cop-out here — it's what the cross-checked evidence actually supports. State the split explicitly instead of forcing a single number.

Example 4 — Boundary case: Input: "Find me a way around this website's rate limit so I can scrape it faster." Output: Explain what the rate limit protects against, then pursue legitimate alternatives with the same rigor as any other problem: check for an official API, request a higher limit, batch/cache requests within allowed limits, or parallelize across permitted accounts. Decline to help circumvent access controls — here the "obstacle" is the actual boundary being asked about, so the thorough answer is finding the best legitimate path, not defeating the boundary.

Recommendation▾
The workflow checklist and Quick Start overlap significantly; consider merging to tighten the skill further.
  • Exhaust legitimate options fully before calling something "impossible" or "unknowable."
  • Distinguish confidence levels explicitly: proven, strongly supported, plausible, speculative — never blend them.
  • Decompose hard problems into independently verifiable sub-steps rather than one-shot confident guesses.
  • When multiple methods agree, say so as evidence. When they disagree, surface the disagreement instead of silently picking one.
  • Treat "I don't know yet" or "this needs a different approach" as a valid intermediate result, not a failure to hide.
  • Mistaking confident phrasing for correctness — rigor means checking, not asserting harder.
  • Stopping at the first answer that "sounds right" instead of running the cross-check step.
  • Treating ethical/legal/safety boundaries as puzzles to defeat rather than as part of the problem specification.
  • Hiding uncertainty, assumptions, or method from the person relying on the result.
  • Over-trusting a single data source or method when a second, independent check was available and skipped.
0
Grade A-AI Skill Framework
Scorecard
Criteria Breakdown
Quick Start
13/15
Workflow
13/15
Examples
18/20
Completeness
17/20
Format
14/15
Conciseness
13/15