Auditing Layered Simulation Verification Systems
YAML--- name: auditing-layered-simulation-verification-systems description: Reviews and validates multi-layer scientific/simulation web applications that combine physics modeling, cryptographic proof chains, and self-verifying test suites (e.g., lensing/astrometry dashboards, engine telemetry with Merkle chains, lemma-compliance tables). Use when auditing a complex interactive simulation page for internal consistency, checking that claimed proofs/invariants actually hold against displayed data, tracing a numeric result back through its derivation chain, or writing/porting a verification script that mirrors an app's self-checks line by line. --- # Auditing Layered Simulation Verification Systems
Given a page/app that mixes physics outputs, algebraic invariants, and crypto-style integrity checks, don't trust the "PASS" badges — reconstruct each one:
- Identify the data model (physical constants, geometry spec, e.g.
TIERS_SPEC, mass, radius). - Identify the derived quantities (Einstein radii, timing residuals, frame rotations).
- Identify the invariants claimed (parity identities, closure theorems, norm preservation).
- Recompute each invariant independently from raw inputs; compare to the displayed value with explicit tolerance (exact integer/rational vs floating-point ≈).
- Flag any check whose "PASS" depends on a vacuous or trivial condition (e.g., "cyclic convention trivially PASS tests nothing").
Progress:
- Step 1: Inventory every distinct subsystem/panel and what claim it makes (definition, computed value, live check, "not used").
- Step 2: For each numeric result, locate its generating formula and units; recompute by hand or script.
- Step 3: For each "PASS", classify it as: (a) exact algebraic identity, (b) floating-point match within stated tolerance, (c) convention/definition (not a real test), (d) untested/"not used".
- Step 4: For cryptographic/integrity layers (Merkle roots, tamper tests), verify the chain recomputation actually changes on tampering and that pre-filter/cache fields (e.g., "South Halo") are explicitly not treated as integrity checks.
- Step 5: Cross-check cross-referenced theorems/lemmas against the actual computation performed — a "Proposition" label does not guarantee the code proves it.
- Step 6: Produce a verification report: counts of real PASS vs definitional vs not-used, and any silent assumptions (e.g., weak-field superposition, order-preserving maps, open vs cyclic convention).
- Step 7: If asked to port the system's self-check script, preserve exact value semantics (BigInt/rational vs binary32 "≈") rather than silently converting types.
Example 1:
Input: A page claims "Residual parity, Proposition G1 PROVEN" with Y = Y_I + Y_II + Y_III + Y_R and shows Y_R = 0.
Output: Recompute Y - Y_I - Y_II - Y_III from the displayed component values for each observable channel (astrometry, timing, velocity). Confirm it equals the displayed Y_R to the stated precision. Note that "recovering Y_II from the other three" is a different, stronger claim than "the residual is zero" — verify both separately, and report that this is an exact BigInt identity, not a statistical fit.
Example 2:
Input: "Frame closure, Proposition G2 PROVEN... open convention PASS (exact identity)... cyclic convention trivially PASS tests nothing."
Output: Confirm the open-convention check (H = q0 * q_last^-1) actually multiplies four independently-derived rotation/quaternion legs and lands on identity within the stated tolerance (here, exact to 0 bits). Explicitly call out that the cyclic convention is a tautology (multiplying a closed loop always returns identity) and should not be counted as independent evidence.
Example 3:
Input: A lemma-compliance table lists 20 live PASS rows and 9 "not used" rows across 15 lemmas plus §6–§7 results.
Output: Re-tally: for each PASS row, identify whether it's (a) an exact integer/rational check (e.g., tier laws Rn = 5n²+15n+15), (b) a floating-point numeric check with explicit epsilon, or (c) a structural/definitional statement mislabeled as a check (e.g., "scheduling convention shown" is not a pass/fail test). Report the real ratio of substantive checks vs. definitions vs. unused lemmas, rather than accepting the page's own count.
- Separate "defined" from "proven." A row saying "not used" or "definition" is not a failure — but it's also not evidence of correctness. Keep these buckets distinct in any summary.
- Track exactness explicitly. Distinguish BigInt/rational exact identities (0 residual) from floating-point ones (tolerance stated in bits, decades, or epsilon). Never let a "≈" value get reported as "PASS exact."
- Treat conventions as conventions. Anything described as "trivially PASS," "vacuous," or "by construction" should be labeled as a convention check, not a test of the underlying physics/math.
- Recompute from raw constants, not from displayed derived numbers. E.g., rebuild Schwarzschild radius and Einstein radii from
M_BH,R0,GMrather than trusting intermediate panel outputs. - Respect units and scale carefully. Mixed units (arcsec, mas, µas; pc vs AU; Hz vs s⁻¹) are common sources of silent errors — always re-derive the conversion factor.
- Verify tamper-evidence claims empirically. If a Merkle/root-chain "Tamper" button exists, actually flip a leaf and confirm the root changes; don't assume integrity from a static root string.
- Call out scope limitations explicitly. E.g., "superposition holds because field is weak at every known pulsar" is conditioned on current data; note it as an assumption, not a universal proof.
- Don't accept "PASS" counts at face value. Pages routinely conflate real numerical tests with structural definitions to inflate a pass tally.
- Don't ignore precision classes. A
≈binary32 payload marked as "exact" in the leaf is a different claim than a literal BigInt exact match — mixing them invalidates an audit. - Don't treat a named theorem/proposition as self-certifying. Labels like "Theorem 7.18 PROVEN" describe what the code claims to check, not that the check is sound — always inspect the actual comparison performed.
- Don't skip the "not applicable" boundary cases. E.g., shared boundary radii between systems meaning a stacking theorem "does not apply" is a meaningful caveat, not a minor footnote.
- Don't conflate cache/pre-filter fields with integrity fields. A field like "South Halo" that's explicitly a pre-filter must never be reported as part of tamper-evidence.
- Don't silently round when porting scripts. If the original keeps exact rationals (e.g.,
14462834225/4194304), a ported version using plain floats will diverge and falsely "fail" or "pass."