Generated Skill

Don't start with methodology design. Start by killing bad problems.

1. Generate 8-12 candidate research questions (use researchquestionsgenerator.com or equivalent — seed it with a domain phrase like "real-time feedback loops," "small team AI infrastructure," "model drift detection")
2. For each, run the False Positive Dissection (see Workflow Step 2) — most will die here
3. Survivors go through the 7-column research matrix (see Output Structure)
4. Output only problems that survive BOTH demand validation AND adversarial stress-testing of their proposed methodology

If you can't find a counter-narrative that would kill a demand signal, you haven't looked hard enough — go find one before trusting the signal.

Progress:
- [ ] Step 1: Generate raw candidate questions
- [ ] Step 2: False Positive Dissection (kill synthetic/seasonal signals)
- [ ] Step 3: Rank survivors by demand evidence strength
- [ ] Step 4: Design non-standard methodology per surviving problem
- [ ] Step 5: Decompose methodology into fragmented logic blocks
- [ ] Step 6: Adversarial stress-test each methodology
- [ ] Step 7: Build deployment blueprint for top 3-5
- [ ] Step 8: Assign trend probability with explicit reasoning

Step 1: Generate Raw Candidates

Seed the generator (or manually brainstorm in its style: causal / comparative / evaluative / policy question frames) with current AI friction phrases: "context window cost," "agent reliability," "eval drift," "small team ML ops," "hallucination detection at scale," "RAG retrieval precision," "fine-tuning ROI." Produce 8-12 questions minimum before filtering — filtering fewer than 8 means you're not casting wide enough.

Step 2: False Positive Dissection

For every demand signal (GitHub stars/issues velocity, HN/Reddit thread frequency, arXiv citation recency, job postings, funding rounds), write the strongest counter-narrative that would debunk it. Examples of counter-narratives:

• "GitHub activity is inflated by a single viral tutorial repo, not sustained tooling demand"
• "Reddit discussion is nostalgia/hype cycle from a single product launch, not recurring pain"
• "Job postings mention this term as a buzzword insert, not a functional requirement"

A signal only survives if the counter-narrative is weaker than the evidence. Document both — never present the signal without its counter-narrative attached.

Step 3: Rank by Demand Evidence

Score each survivor 1-5 on: recency (last 6 months weighted highest), cross-source corroboration (does it show up in ≥2 independent signal types?), and practitioner specificity (is this a builder's problem or a pundit's talking point?). Discard anything scoring only on one signal type — single-source signals are the most common false positive.

Step 4: Design Non-Standard Methodology

For each validated problem:

1. State the traditional consensus in one sentence — the thing every practitioner would say first
2. Identify the one core assumption that consensus depends on
3. Invert or bypass that assumption — this is your methodology's thesis
4. If your "non-standard" approach doesn't visibly contradict the stated assumption, it's not non-standard — it's a rebrand. Discard and retry.

Step 5: Fragmented Logic Blocks

Decompose the methodology into 3-5 discrete, independently combinable components. Each block requires all four fields — an incomplete block is not deployable:

Block N: [Name]
- Constraint: [hard limit this block operates under]
- Assumption: [what must be true for this block to work]
- Data Requirement: [what data/access this block needs to function]
- Verification Method: [how you'd know this block is working, not just running]

Step 6: Adversarial Stress-Test

For each methodology:

• Write the counter-argument a traditional expert would raise (steelman it — make it genuinely strong, not a strawman)
• Explain why it's valid
• Explain why the methodology survives anyway (mechanism, not hand-waving)
• Explicitly flag: "theoretically sound, practically unproven" components — anything not flagged this way is implicitly claimed as proven, so be honest here or the whole matrix is untrustworthy

Step 7: Deployment Blueprint

Define minimal viable scope (smallest possible test that would falsify or confirm the methodology), the feedback metrics (leading indicators, not vanity metrics), and the utility multiplier (specific beneficiary + why they switch from status quo — "faster/cheaper" is not specific enough, name the mechanism).

Step 8: Trend Probability

Score confidence (Low/Medium/High) with reasoning tied back to Step 3 evidence scores AND Step 6 adversarial survival. A methodology that didn't survive adversarial testing cannot score High regardless of demand evidence.

Present as a matrix, one row per validated problem:

Example 1:

Input: Seed question — "Under what conditions can small software teams integrate real-time feedback loops to improve model accuracy without increasing infrastructure costs?"

Output row (abbreviated):

• Problem Statement: Small ML teams (<10 eng) lack cost-viable continuous model validation; current tooling assumes enterprise-scale streaming infra.
• Demand Evidence: r/MachineLearning threads on "poor man's MLOps" (weekly recurrence, last 3 months); HN "Show HN" posts for lightweight eval tools (rising frequency); counter-narrative — this could be founder-hype from tool vendors seeding discussion; survives because independent job postings (startup ML roles) cite "cost-efficient monitoring" as a requirement, not just marketing.
• Traditional Consensus: Continuous retraining + real-time stream monitoring (assumes infra cost is secondary to model freshness).
• Non-Standard Methodology: Asynchronous validation queues that prioritize edge-case sampling over exhaustive monitoring. Blocks: (1) Local edge-filter trigger [Constraint: no cloud call until threshold breach; Assumption: most inputs are non-informative; Data Requirement: on-device inference logs; Verification: sampled precision audit against full-stream baseline], (2) Async labeling queue [Constraint: human latency tolerance <24h; Assumption: delayed feedback doesn't degrade model faster than retrain cycle; Data Requirement: edge-case flagged samples; Verification: compare accuracy delta vs. real-time baseline], (3) Cost-threshold retrain trigger [Constraint: fixed compute budget; Assumption: performance degradation is detectable before user-facing failure; Data Requirement: rolling accuracy metric; Verification: A/B against always-on retrain cost/accuracy curve].
• Adversarial Dissection: Counter-argument — "async feedback introduces staleness that compounds in fast-drift domains (e.g., fraud detection)." Valid for high-velocity drift domains. Rebuttal — methodology explicitly scopes to teams where drift velocity < labeling latency (stated as deployment precondition, not universal claim). Unproven: whether edge-filter threshold tuning generalizes across domains without per-domain manual calibration — flagged as requiring validation.
• Deployment Viability: MVP scope — single-model, single-team pilot with existing edge logging; feedback metric — cost-per-accuracy-point vs. baseline always-on monitoring; utility multiplier — solo/small ML teams avoid enterprise MLOps vendor lock-in (named mechanism: eliminates streaming infra bill, not just "cheaper").
• Trend Probability: Medium — demand evidence cross-corroborated (Step 3) but core generalization assumption unproven (Step 6 caps confidence).

• Always attach the counter-narrative to the demand signal in the same breath — never report evidence without its strongest rebuttal
• A methodology is not "non-standard" unless it names and breaks one specific consensus assumption — vague novelty claims are rejected
• Every fragmented logic block needs all 4 fields or it's not executable — incomplete blocks default to "not ready"
• Explicitly separate "theoretically sound" from "practically proven" in every stress-test — conflating them is the most common way these matrices become fake-rigorous
• Cap Trend Probability at Medium for any methodology that hasn't survived a genuinely steelmanned adversarial counter-argument

• Single-source demand signals treated as validated — one viral GitHub repo or one Reddit thread is noise, not demand; require ≥2 independent signal types
• Non-standard methodology that's just a rebrand — if it doesn't visibly contradict a named traditional assumption, it's cosmetic, not novel
• Strawmanning the adversarial counter-argument — a weak counter-argument that's easily dismissed proves nothing; steelman it or the stress-test is theater
• Presenting market assumptions as facts — every demand claim needs a source or an explicit "unverified" flag, no exceptions
• Skipping the minimal viable scope — a deployment blueprint without a falsifiable minimal test is a pitch deck, not an execution plan