Debunking Pseudoscientific Claims
When asked "is [scientific claim X] actually true?", don't just summarize the consensus view — trace the claim back to its origin:
- Find the earliest/foundational source for the claim
- Check if that source actually demonstrated what it's cited for
- Follow the citation chain forward to see how the claim got amplified
- Identify any commercial, institutional, or career incentives that pushed the claim
- Compare against the rigorous methodology that should have been used
- State clearly what we actually know vs. what's assumed
Progress:
- [ ] Identify the specific claim and its current popular/academic status
- [ ] Trace the claim to its originating study or event
- [ ] Scrutinize the originating source's methodology and evidence
- [ ] Map the citation/amplification chain (who picked it up and why)
- [ ] Identify conflicts of interest or incentive structures (commercial, career, publication bias)
- [ ] Define the rigorous standard/method that would actually prove the claim
- [ ] Contrast popular claim vs. actual evidentiary standard
- [ ] Note any legitimate related findings that got conflated with the myth
- [ ] Suggest what real evidence would look like / where promising work exists
Step details:
- Trace origin: Look for the first conference, paper, or press event where the claim appeared. Check footnotes/acknowledgments for sponsors or suppliers who had a commercial stake — these are often buried in a single line.
- Scrutinize methodology: Ask "were the standard/required steps actually followed?" For any domain, there's an established rigorous pipeline (e.g., extraction → identification → bioassay testing for pheromones). Check whether the foundational study skipped steps.
- Map amplification: A credibility cascade often follows the pattern: obscure/commercial origin → endorsement by a respected independent scientist → explosion of derivative studies → echo chamber (reviewers share the same unexamined premise) → media hype cycles.
- Check incentives: Patents, corporate sponsorship, sexy/publishable topics, publication bias toward positive results, lack of replication requirements.
- Define rigor: Explicitly state the gold-standard methodology from the field, even if hypothetical for this case, to show the gap between what was done and what's needed.
Example 1: Input: "Androstadienone is called the 'human sex pheromone' in countless articles and sold commercially. Is this established science?" Output: Traces origin to a 1991 EROX Corporation-sponsored conference; the foundational "study-zero" paper omitted all extraction/identification/bioassay details and merely footnoted that the molecules were "supplied by EROX Corporation" (a company seeking patents on them). Shows how a 2000 endorsement by a respected scientist (McClintock) gave it legitimacy, spawning dozens of citing studies vulnerable to false positives and echo-chamber peer review. Concludes: no rigorous pheromone identification was ever done; commercial origin tainted the entire literature.
Example 2: Input: "A supplement company claims their product is 'clinically proven' based on citations in several journals." Output: Check whether the cited studies were funded/supplied by the company itself, whether the foundational study followed the field's standard proof pipeline (e.g., RCT with placebo, dose-response, replication), and whether later citations repeat the same uncontrolled claim without independent verification. Flag citation inflation as a red flag distinct from genuine scientific consensus.
- Distinguish "widely cited" from "well established" — citation count is not evidence quality.
- Always check footnotes and acknowledgments, not just abstracts — commercial interests often hide there.
- Name the actual rigorous methodology for the field so readers can see exactly what's missing.
- Point out legitimate adjacent findings that got conflated with the myth (e.g., individual body odor research vs. species-wide pheromones) — nuance builds credibility.
- End with what real, ongoing rigorous research looks like, to show the field isn't dismissed wholesale, just the specific myth.
- Keep a "follow the money / follow the incentive" lens throughout.
- Don't just say "the science is unsettled" — show precisely where and why the evidentiary chain broke.
- Don't conflate skepticism of a specific claim with dismissal of the entire research area.
- Don't ignore the social/institutional mechanics (echo chambers, peer review conflicts, media hype) — the myth's survival often depends on these as much as the original weak evidence.
- Avoid vague claims of "bias" without identifying the concrete commercial or career incentive at play.