Decomposing Research Questions

Given a broad question, don't answer it directly — decompose first:

Original question: "Why is our product's retention declining?"

Decomposition:
1. [Descriptive] What does the retention trend actually look like? (segments, cohorts, timeframes)
2. [Causal-internal] What changed in the product/pricing/onboarding around the inflection point?
3. [Causal-external] What changed in the market/competitors/user behavior?
4. [Measurement] Is this a real decline or a measurement/definition artifact?
5. [Mechanism] Through which user segments or funnel stages is the decline concentrated?

Priority order: 4 → 1 → 5 → 2 → 3
(Rule out measurement error first; then localize before explaining)

Progress:
- [ ] Step 1: Clarify the root question — what decision or action depends on the answer?
- [ ] Step 2: Classify the question type(s)
- [ ] Step 3: Generate candidate sub-questions
- [ ] Step 4: Deduplicate and check coverage (no gaps, no overlaps)
- [ ] Step 5: Map dependencies between sub-questions
- [ ] Step 6: Order by priority (what blocks other answers, what's cheapest to rule out)
- [ ] Step 7: Assign method/data source to each sub-question
- [ ] Step 8: Flag which sub-questions are in-scope vs. parked for later

Step 1 — Clarify the root question. Ask: what will change if we get the answer? If nothing, the question is likely mis-scoped — rewrite it around the actual decision.

Step 2 — Classify question type(s). Most big questions are a mix of:

• Descriptive — what is happening (facts, trends, measurements)
• Causal — why is it happening (mechanisms, drivers)
• Evaluative — is it good/bad relative to a goal or benchmark
• Predictive — what will happen next
• Normative/decision — what should be done

Each type needs different methods; mixing them unexamined is the most common cause of muddled research.

Step 3 — Generate candidate sub-questions. Use these levers:

• Decompose by component: break the subject into parts (segments, stages, geographies, time periods)
• Decompose by cause chain: what must be true at each link between cause and effect
• Decompose by alternative explanation: list competing hypotheses, including boring ones (measurement error, selection bias) before interesting ones
• Decompose by evidence needed: what data/observation would move the needle on belief

Step 4 — Check coverage. Use a MECE pass (mutually exclusive, collectively exhaustive). If two sub-questions would be answered by the same evidence, merge them. If answering all sub-questions still wouldn't resolve the root question, you're missing one.

Step 5 — Map dependencies. Some sub-questions are prerequisites (e.g., "is the data even correct?") before others become meaningful. Draw this as a simple tree or ordered list, not a flat bag of questions.

Step 6 — Prioritize. Rank by:

1. Cheap-to-rule-out alternatives (kill boring explanations first)
2. Questions that gate many downstream questions
3. Questions most relevant to the actual decision from Step 1

Step 7 — Assign method per sub-question. Each sub-question should get a concrete next action: "query database," "run survey," "read N papers," "interview X." If a sub-question has no feasible method, it's either poorly framed or out of scope — flag it.

Step 8 — Scope explicitly. State which sub-questions will be answered now vs. explicitly deferred. Prevents scope creep disguised as thoroughness.

Example 1: Input: "Is remote work good for productivity?" Output:

Root question too value-laden — reframe around decision: "Should we extend our remote work policy?"

Sub-questions:
1. [Descriptive] How is productivity currently measured for remote vs. in-office employees?
2. [Evaluative] By that measure, is there a current difference, and is it statistically meaningful?
3. [Causal] If there's a difference, is it caused by remote work itself or by confounds (role type, tenure, self-selection)?
4. [Descriptive] What do employees report about friction points (collaboration, onboarding, isolation)?
5. [Predictive] Would effects differ if remote work were policy-mandated vs. self-selected (addressing confound in #3)?
6. [Decision] What tradeoffs (cost savings, talent pool, culture) matter besides productivity?

Priority: 1 → 3 → 2 → 4 → 6 → 5
(#5 parked — requires an experiment, flagged as future work)

Example 2: Input: "Why did the experiment fail?" Output:

Sub-questions:
1. [Measurement] Did the experiment actually run as designed (sample size, randomization, duration)?
2. [Measurement] Was the metric correctly instrumented?
3. [Descriptive] Was there a true null effect, or a real effect that didn't reach significance (power issue)?
4. [Causal] If underpowered, was the expected effect size wrong, or was variance higher than assumed?
5. [Causal] If truly null, which part of the causal chain (exposure → behavior → metric) broke?

Priority: 1 → 2 → 3 → 4/5
(Rules out "boring" explanations — bugs, power — before chasing causal theories)