AI Skill Report Card

Challenging Assumptions and Plans

A91·Nov 20, 2026·Source: Web
14 / 15

When given a plan, architecture, feature, or decision, don't say "this looks good" or "this is wrong." Ask first:

"What single assumption is this whole plan standing on — and what breaks if it's false?"

Then run the 7-step audit below, ending with one concrete test and one kill criterion.

Recommendation▾
Add a brief 'bad output' contrast example (e.g., a shallow critique that just validates) to show what NOT to do, reinforcing the harsh-but-constructive tone.
15 / 15

Progress:

  • Step 1: Find the load-bearing assumption — the one claim that, if false, collapses the entire plan. Attack it first, not peripheral details.
  • Step 2: Label every key claim — Fact (verified), Inference (derived from evidence), or Assumption (unverified belief). Flag where evidence is thin.
  • Step 3: Premortem — assume it failed 6 months from now. Walk backward: what sequence of events led there? What small warning sign today are we ignoring? Present as a plausible scenario, not a prediction.
  • Step 4: Claims vs. reality — for software: trace each feature's actual data path (input → processing → output → failure handling); file/function existing ≠ working. For non-software plans: trace each claimed capability to its actual evidence (has it been tested, demoed, or only asserted?).
  • Step 5: Build vs. reuse, complexity audit, root cause — (a) does an existing solution already do this? (b) what parts of the design are unnecessary complexity? (c) for any problem being solved, give 3 alternative root causes and a test to distinguish them.
  • Step 6: Stress and cost check — (a) edge cases: bad input, empty data, concurrency, partial failure, restart, dependency down; (b) compounding risks: which 2-3 small failures combine into a big one; (c) total cost of ownership beyond build: test, debug, document, support, security, migrate; (d) opportunity cost: what doesn't get done, vs. best alternative and vs. doing nothing.
  • Step 7: Decide the next move — propose the smallest test that would validate/invalidate the core assumption (Step 1), with pre-set success/failure criteria, and define kill/pivot/reduce conditions now, before sunk cost accumulates.
Recommendation▾
Consider a short template/checklist block for Step 2's Fact/Inference/Assumption labeling to make it more directly reusable.
19 / 20

Example 1 — Architecture decision: Input: "We're starting with microservices because it'll scale."

Output:

  • Load-bearing assumption: "We'll need large-scale traffic soon." If false, the complexity is pure cost.
  • Fact: Migrating monolith→microservices later is well-documented as painful but doable.
  • Assumption (not fact): "Our 3-person team can handle distributed systems' operational overhead."
  • Premortem: Fails in 6 months because inter-service debugging ate all feature velocity. Today's ignored warning sign: no observability/tracing tooling in place yet.
  • Build vs. reuse: A modular monolith gives the same boundaries without operational overhead.
  • Smallest test: Build one "service boundary" as an in-process module for 2 weeks; measure if it's sufficient.
  • Kill criterion: If single-server load stays under 70% capacity after 3 months, don't split into services.

Example 2 — Plugin/code claim vs. implementation: Input: "Our plugin system's docs say it supports 'automatic retry on failure.'"

Output:

  • Documented claim: "Automatic retry on failure."
  • Code check: retry() exists but is never called by any error handler.
  • Data path: input → API call → exception → logged only. Retry never triggers.
  • Verdict: Feature is documented, not implemented.
  • Test to verify: Force a failing endpoint call; check logs for a retry attempt count > 1.

Example 3 — Business/staffing decision: Input: "We're hiring 2 senior engineers now to 'move faster' on the roadmap."

Output:

  • Load-bearing assumption: "Our current velocity bottleneck is engineering headcount." If the real bottleneck is unclear requirements or slow decision-making, more engineers won't fix it — may worsen it (Brooks's Law).
  • Evidence check: Do we have data on where time is actually lost (waiting on specs vs. waiting on code)? If not, this is an assumption, not inference.
  • Premortem: 6 months later, velocity hasn't improved; new hires spent 2 months onboarding into undocumented systems, and existing seniors lost time mentoring instead of shipping.
  • Root causes to distinguish (pick one test each): (a) headcount shortage → check open ticket backlog vs. engineer-hours available; (b) unclear requirements → audit how many tickets got reworked after ambiguous specs; (c) decision latency → measure average time from "idea proposed" to "approved to build."
  • Opportunity cost: Same budget could fund a PM/tech-lead hire to fix requirement clarity, or fund tooling investment instead.
  • Smallest test: Track the above three metrics for 3 weeks before hiring; let data pick the root cause.
  • Kill/pivot: If backlog-per-engineer is not actually above team average, pause hiring and fix process first.

Example 4 — Feature/product decision: Input: "We should add a dark mode toggle, users will love it."

Output:

  • Load-bearing assumption: "Users have an active, demonstrated need for this," not just "it's a common feature elsewhere."
  • Evidence check: Fact (if true) — X% of support tickets/feature requests mention it. Opinion — "users will love it" is unverified until shown.
  • User-need validation: What's the actual current workaround (OS-level dark mode, browser extension)? If a workaround already satisfies most users, demand may be weak.
  • Reversibility: Low-risk, highly reversible — easy to ship behind a flag and roll back.
  • Smallest test: Ship behind a feature flag to 5% of users for 2 weeks; measure opt-in rate and support ticket deltas.
  • Kill criterion: If opt-in rate is under a predefined threshold (e.g., 10%), deprioritize further investment (custom theming, etc.).
Recommendation▾
The workflow checklist items are dense paragraphs rather than scannable steps — could tighten each bullet to one line with sub-bullets for sub-parts.
  • Start with the foundational, plan-collapsing assumption — not minor details.
  • Label every claim Fact / Inference / Assumption; never blur the line to make your case look stronger.
  • "Code/module exists" ≠ "feature works" — always trace the real end-to-end path.
  • Present premortems as plausible scenarios, never as certain predictions.
  • Give 3 distinct root-cause hypotheses with a differentiating test each — don't settle on one without evidence.
  • Set kill/pivot criteria before running the test, so sunk cost can't bias the decision later.
  • Listing risks in isolation and missing how 2-3 small ones compound into one big failure.
  • Treating "opinion stated confidently" as evidence.
  • Only testing the happy path; skipping edge cases (bad input, concurrency, partial failure).
  • Proposing a new system/hire/process without checking if an existing one already covers it.
  • Counting only build cost, ignoring maintenance, support, migration, and security over time.
  • Letting past investment ("we already spent 3 months on this") justify continuing past a kill criterion.
0
Grade AAI Skill Framework
Scorecard
Criteria Breakdown
Quick Start
14/15
Workflow
15/15
Examples
19/20
Completeness
19/20
Format
15/15
Conciseness
13/15