Applying Scientific Cycle to Social Science
Markdown--- name: applying-scientific-cycle-to-social-science description: Applies the scientific cycle, induction, and deduction to economics and social science research. Use when developing research questions, hypotheses, study designs, evidence assessments, or iterative research plans, or when distinguishing causal, descriptive, predictive, and interpretive claims in social and economic research. --- # Applying the Scientific Cycle to Social Science Turn a question about people, markets, institutions, or policy into a testable, revisable research argument. Adapt the cycle to the user's study stage and requested deliverable — do not force completed research back through every step. Treat conclusions as conditional on measurement, context, design, and assumptions.
- Identify topic, research stage, claim type, and available evidence. State working assumptions where details are missing.
- Walk through the scientific cycle (observe → hypothesize → predict → test → analyze → revise), making each link explicit.
- Populate the Study Brief template. Mark unknowns explicitly; distinguish proposed work from completed analysis.
- Before asserting causation, check against the bad example and corrected contrast.
Example invocation: "Use the scientific-cycle skill to develop a research plan on whether public transport reliability improves employment."
Progress:
- Frame the inquiry (population, setting, unit of analysis, claim type)
- Observe and question (source, verify pattern isn't an artifact of measurement)
- Form hypothesis + rival explanations (operationalize key concepts)
- Deduce predictions (direction, horizon, subgroup, falsifying evidence)
- Design test matched to question (check method-specific assumptions)
- Analyze (effect size + uncertainty, not just significance)
- Revise and plan next discriminating test
1. Frame the inquiry
State population, setting, period, unit of analysis, and question type:
- Descriptive — what happens
- Predictive — what is likely
- Causal — what changes because of an intervention
- Interpretive — how people understand/experience something
Separate empirical claims from normative judgments — evidence informs policy but doesn't supply the values used to rank options.
2. Observe and question
Identify a documented pattern, anomaly, or lived experience. Check whether sampling, measurement, or shifting definitions could produce it. Record the source; distinguish established observations from proposed ones.
3. Form a hypothesis
Propose a mechanism and plausible rival explanations. Specify scope and what observation would count against it. Operationalize abstract concepts (trust, inequality, welfare) and flag construct-validity concerns, especially across groups.
4. Deduce predictions
Derive observable implications from the mechanism plus explicit auxiliary assumptions: outcome, comparison, expected direction, horizon, relevant subgroups. State magnitude only if justified. Prefer predictions that discriminate between the hypothesis and its rivals. Pre-register confirmatory predictions where feasible; label anything else exploratory.
5. Design a test and gather evidence
Match method to question: surveys, interviews, ethnography, archival/administrative records, experiments, natural experiments, formal models. A simulation shows implications under assumptions — not empirical validation. For causal claims, define treatment, outcome, target effect, and counterfactual, then apply the method checks below.
Method-specific assumption checks:
- RCT — attrition, compliance, interference between units
- Diff-in-diff — credible parallel trends, timing, anticipation effects
- RDD — continuity around cutoff, no manipulation of the running variable
- IV — relevance, exclusion restriction, independence — argue, don't just assert
6. Analyze and assess
Report evidence, uncertainty, competing interpretations, limitations. Favor effect sizes with uncertainty over significance thresholds. Address missingness, multiple testing, sampling, dependence. For qualitative work: justify case selection, coding, negative cases, reflexivity. Never fabricate data, estimates, citations, or completed tests.
7. Revise and repeat
State whether evidence supports, challenges, or leaves the hypothesis unresolved — within scope. Distinguish a weak/imprecise test from genuine evidence of no effect. Propose the next study that could distinguish remaining rival explanations.
- Induction — move from cases/patterns to a candidate explanation. Strengthens generalization across settings but never proves a universal rule; weigh representativeness and rival mechanisms.
- Deduction — move from theory + assumptions to a specific prediction. Valid logic doesn't mean the premises hold. A failed prediction may indict the mechanism, the auxiliary assumptions, the measurement, or the design — not necessarily the core theory. A successful prediction doesn't uniquely confirm it if rivals predict the same thing.
- Abduction (when useful) — infer the most plausible explanation among alternatives, then seek evidence that discriminates between them.
- Association ≠ causation. Consider reverse causality, omitted variables, selection, simultaneity, measurement error. Adding controls doesn't fix identification — controls that are mediators or colliders can introduce new bias.
- People react to policy and to being observed. Consider strategic behavior, spillovers, feedback, equilibrium adjustment. State whether an effect is local, short-run, or likely to generalize.
- Context limits generalization. Institutions, culture, history, and sample selection bound external validity. Motivate subgroup analysis from theory, not post-hoc search (apply the confirmatory/exploratory distinction).
- Ethics. For proposed data collection: consent, confidentiality, harm, institutional review. Use anonymized/controlled-access data when public sharing would expose participants.
- Reproducibility vs. replication. Reproducing = same data/code, same result. Replication = new, independent evidence. Don't conflate the two.
Markdown# Study Brief: [Title] **Stage and evidence status:** [Exploration / research plan / completed analysis] **Claim type:** [Descriptive / predictive / causal / interpretive]
[Research question; population, setting, period, unit of analysis; empirical vs. normative separation]
[Motivating pattern; sources; verified vs. unverified premises]
[Mechanism, scope, induction trail, competing explanations]
[Observable implications; direction/horizon/comparison; disconfirming evidence; exploratory vs. confirmatory]
[Operational definitions; data source/collection plan; sample selection; validity, missingness]
[Method and fit; identification argument; treatment/outcome/counterfactual for causal claims]
[Planned/completed analysis; uncertainty; sensitivity checks; competing interpretations]
[Pending, or: observed findings, warranted conclusion, unresolved alternatives, generalization limits]
[Participant protections; data/code/versioning needed to reproduce]
[What to keep/revise/leave open; next discriminating test]
Example 1: Input: "Does minimum wage increase unemployment among teens?" Output: A study brief that (a) frames this as a causal question on a specific population/period, (b) notes the long empirical literature's mixed findings as context rather than settled fact, (c) proposes a diff-in-diff or border-discontinuity design comparing adjacent regions with different minimum wage changes, (d) flags parallel-trends and spillover (cross-border employment) threats, (e) specifies that findings are local to the studied period/labor market and shouldn't be generalized without further replication.
Example 2: Input: "I already ran a survey showing trust in local government rose after a transparency portal launched. Did I prove the portal worked?" Output: Flags that this is pre/post observational, not causal — rival explanations (concurrent local events, response bias, regression to the mean) remain uncontrolled; suggests looking for a comparison municipality without the portal, checking for pre-trends, and reporting the finding as "associated with" rather than "caused by" pending a stronger design.
Bad example — do not produce:
Neighborhoods with more reliable public transport have higher employment. Therefore, reliable transport causes higher employment, and expanding bus services will increase employment everywhere.
Why it fails: Ignores confounds (local job availability, income, residential selection), ignores reverse causality (job growth attracting transport investment), asserts a counterfactual with no comparison group, and over-generalizes to "everywhere."
Corrected version:
Transport reliability is associated with employment in the observed neighborhoods. Reduced commuting barriers are one possible mechanism; job availability, residential sorting, and reverse investment flows are rival explanations. To test the mechanism, study a specific reliability improvement using a credible comparison group and an explicit identification strategy; report the resulting estimate, its uncertainty, and the scope of the setting only after that analysis.
Repair principle: Match claim strength to evidence — qualify associations, name unresolved alternatives, propose a test that could discriminate between them.
- Always state the claim type (descriptive/predictive/causal/interpretive) up front — it determines what evidence can support.
- Mark every unverified or hypothetical figure/finding explicitly; never present invented numbers as data.
- Prefer predictions that discriminate between rival hypotheses over predictions both explanations would satisfy.
- Report uncertainty and effect size, not just whether a test "passed."
- Treating correlation in observational data as sufficient for a causal claim.
- Adding control variables as a substitute for an identification argument.
- Generalizing a local, short-run estimate to "everywhere, always."
- Fabricating citations, data, or "completed" analysis that wasn't actually run.
- Skipping the rival-explanations step and presenting one hypothesis as if it were uncontested.