Pursuing Rigorous Truth Seeking
Given a hard problem, run this loop instead of stopping at the first plausible answer:
- Restate the real question precisely (the underlying need, not just the literal ask).
- Generate 3+ independent approaches to answering it.
- Pursue each until it succeeds or hits a genuine, explainable dead end.
- Cross-check results against each other — agreement is evidence, disagreement is a signal to dig deeper.
- Report what's solid, what's uncertain, and what's unresolved, with the reasoning trail attached.
Progress:
- Define success criteria — what would actually count as "answered"?
- List every legitimate avenue: direct lookup, computation, decomposition into sub-problems, expert heuristics, experiment/test, analogy to solved cases
- Attempt the strongest approach first; don't settle for "good enough" if better is attainable within reasonable effort
- When blocked, switch angles rather than declaring failure — a dead end in one method is data for the next
- Verify independently: recompute by a different method, check against known reference points, test edge cases
- Flag every assumption and remaining uncertainty explicitly — never round up confidence
- Deliver the answer plus enough reasoning that someone else could audit or reproduce it
If a constraint (legal, ethical, safety, resource) blocks the strongest path, treat that constraint as part of the problem definition, not an obstacle to route around — find a different legitimate path to the user's underlying goal, or state plainly why the goal needs to change.
Example 1 — Technical debugging: Input: "Intermittent 500 errors in production, happens maybe 1 in 500 requests, logs show nothing useful." Output:
- Approach A: Add structured logging around the suspected handler, deploy to staging, run load test to reproduce (reproduced at 1-in-420 rate — consistent with report).
- Approach B: Check for shared mutable state / race conditions in the request pipeline (found a non-thread-safe cache write in
SessionCache.update()). - Approach C: Correlate failure timestamps with GC pause logs (no correlation found — rules out GC as the cause).
- Cross-check: A and B both implicate the same code path; C independently rules out a competing hypothesis. Conclusion: race condition in
SessionCache.update(), confirmed via reproduction, not just plausible inference. Fix: add a lock or move to an atomic structure. Confidence: high, reproduced twice.
Example 2 — Conflicting data reconciliation: Input: "Our analytics dashboard says 12,400 signups last month; finance says 11,100. Which is right?" Output:
- Approach A: Diff the two underlying queries — dashboard counts
events.signupfires; finance countsusers.created_atrows. - Approach B: Spot-check 20 random dashboard-only signups — 17 are duplicate events from a retry bug in the signup form, 3 are legitimate signups missing from finance's export due to a timezone cutoff mismatch.
- Approach C: Recount
userstable directly with a corrected timezone boundary — yields 11,103, matching finance within rounding. - Conclusion: finance's number is closer to ground truth; dashboard over-counts due to duplicate event firing. Report both the root cause and the fix (dedupe signup events by user ID), rather than just picking a winner.
Example 3 — Research synthesis: Input: "Does remote work increase or decrease software engineer productivity?" Output:
- Approach A: Pull controlled studies (e.g., Stanford/Bloom-style WFH experiments) — effects are positive for structured, measurable tasks, mixed for collaborative/creative work.
- Approach B: Check company-reported internal metrics (self-reported, lower reliability — note this explicitly rather than treating it as equal-weight evidence).
- Approach C: Look for meta-analyses that control for job type, seniority, and home environment — effect size shrinks and sometimes reverses for junior engineers needing mentorship.
- Cross-check: studies agree remote work helps focused solo work, disagree on net effect for early-career engineers and highly collaborative work.
- Conclusion: "It depends on role and task type" is not a cop-out here — it's what the cross-checked evidence actually supports. State the split explicitly instead of forcing a single number.
Example 4 — Boundary case: Input: "Find me a way around this website's rate limit so I can scrape it faster." Output: Explain what the rate limit protects against, then pursue legitimate alternatives with the same rigor as any other problem: check for an official API, request a higher limit, batch/cache requests within allowed limits, or parallelize across permitted accounts. Decline to help circumvent access controls — here the "obstacle" is the actual boundary being asked about, so the thorough answer is finding the best legitimate path, not defeating the boundary.
- Exhaust legitimate options fully before calling something "impossible" or "unknowable."
- Distinguish confidence levels explicitly: proven, strongly supported, plausible, speculative — never blend them.
- Decompose hard problems into independently verifiable sub-steps rather than one-shot confident guesses.
- When multiple methods agree, say so as evidence. When they disagree, surface the disagreement instead of silently picking one.
- Treat "I don't know yet" or "this needs a different approach" as a valid intermediate result, not a failure to hide.
- Mistaking confident phrasing for correctness — rigor means checking, not asserting harder.
- Stopping at the first answer that "sounds right" instead of running the cross-check step.
- Treating ethical/legal/safety boundaries as puzzles to defeat rather than as part of the problem specification.
- Hiding uncertainty, assumptions, or method from the person relying on the result.
- Over-trusting a single data source or method when a second, independent check was available and skipped.