Detecting Cognitive Bias with AI

Three-phase prompt pattern to run on any decision or belief:

PHASE 1 — BIAS DETECTION
"Here is my reasoning: [X].
Analyze it using a Risk–Resource Vulnerability model:
- List destructive/distorting factors (biases, missing data, emotional drivers)
- List protective/resource factors (evidence, expertise, checks already in place)
Give a balance verdict: is this conclusion resource-backed or vulnerability-driven?"

PHASE 2 — RED TEAM
"Now attack this same reasoning as hard as you can.
Assume I'm wrong. Find the 3 weakest assumptions and the single
scenario where this plan fails worst. Don't soften it."

PHASE 3 — ITERATE
"Given the critique above, here's my revised version: [X'].
Re-score the error (how far from solid) like a control-loop residual.
Repeat until the error is acceptably small, not until it's zero."

Progress:

• Step 1: State the belief/plan/decision in one clear paragraph (the "signal")
• Step 2: Run Risk–Resource Vulnerability scan (bias detection)
• Step 3: Run Red Team critique (adversarial stress test)
• Step 4: Revise based on surviving criticisms only (discard noise)
• Step 5: Re-score residual error; repeat Steps 3–4 until convergence
• Step 6: Lock in final version + list of risks knowingly accepted

Step 2 detail — Risk–Resource Vulnerability scan: Ask AI to split findings into two columns:

• Vulnerability factors: confirmation bias, sunk cost, anchoring, overconfidence, missing counter-evidence, emotional investment
• Resource factors: verified data, domain expertise, redundancy/fallback plans, prior successful precedent

The verdict isn't "good/bad" — it's a ratio. High vulnerability + low resource = do not proceed without more work.

Step 3 detail — Red Teaming: Explicitly instruct the AI to argue against you, not evaluate neutrally. Ask for:

1. The weakest assumption
2. The failure mode with worst consequences (not most likely — worst)
3. What a smart skeptic would say in one sentence

Step 5 detail — Iteration like a PID loop:

• Don't chase a perfect answer in one pass (that's the "P"-only mistake: overshoot/oscillation)
• Track recurring errors across rounds (the "I" term — integral/accumulated issues) — if the same flaw resurfaces 3 times, it's structural, not noise
• Make smaller corrections as you approach a good answer (the "D" term — don't overcorrect late in refinement)
• Stop when marginal error reduction per round drops near zero — that's convergence, not perfection

Example 1: Input: "I'm planning to invest my savings into my friend's startup because I trust him completely and the idea sounds great." Output:

• Vulnerability factors: trust-based reasoning substituting for due diligence, no competitor analysis, emotional loyalty bias, single data point ("sounds great")
• Resource factors: personal knowledge of friend's work ethic (weak-to-moderate signal)
• Verdict: high vulnerability, low resource — do not proceed as-is
• Red team: "Worst case — friend's venture fails within a year, you lose savings AND the friendship, because there was no formal agreement."
• Iteration: revised plan adds independent market research, a written agreement, and capping investment to a defined "loss-acceptable" amount. Residual error: lowered but not zero (startups are inherently risky) — acceptable to proceed.

Example 2: Input: "My BPLA chassis design is final, ready for test flights." Output:

• Bias scan flags: confirmation bias from simulation-only data, no adversarial load testing
• Red team: "What load/angle breaks the mount first? What's the single point of failure if that mount fails mid-flight?"
• Iteration: reinforce the identified weak joint, retest, re-ask "what's the new weakest point" — repeat until failure threshold exceeds required margin.