Solving Causal Inference with Spiking Networks

Given a generative model μ = U r + noise with non-negative causes r ≥ 0, the MAP inference problem reduces to minimizing a quadratic energy:

L(r) = 0.5 * (μ - U r)^T (μ - U r) + α * 1^T r + 0.5 * β * r^T r,  s.t. r ≥ 0

Expand and collect terms into E(r) = -0.5 r^T W r + h^T r where W = -(U^T U + β I) and h = -U^T μ + α 1. Then:

1. Rate-based network first: build dynamics τ dr/dt = -r + [W r - h]+ (or equivalent), verify it descends E(r) and respects r ≥ 0.
2. Spiking network second: replace rate units with integrate-and-fire neurons whose synaptic input matrix is W, whose threshold/reset encodes the linear (α) and quadratic (β) regularization terms, and whose recurrent inhibition implements "explaining away" dynamically via spikes rather than through population-averaged rates.
3. Validate: show the network recovers the correct sparse combination of causes from a noisy mixture, and stress-test with spike-generation noise, timing jitter, and mistuned weights.

Progress:

• Step 1: Formalize the generative model (features, causes, noise, prior)
• Step 2: Derive the negative log-posterior / energy function
• Step 3: Identify regularization roles of each prior parameter (L1 vs L2)
• Step 4: Construct rate-based network dynamics that provably minimizes the energy
• Step 5: Map rate dynamics onto spiking (LIF) neurons with realistic synaptic kernels
• Step 6: Identify which biophysical mechanism implements each regularization/constraint term
• Step 7: Demonstrate explaining away emerges from tuned recurrent inhibition
• Step 8: Test robustness (internal noise, external jitter, mistuned connectivity)
• Step 9: Characterize where information lives (rates? timing? population covariation?)

Step 1–3: Formalize the problem

• Define feature vectors u_i (columns of U), cause coefficients r_i ≥ 0, and a linear-Gaussian observation model.
• Choose a prior on causes that is itself non-negative-truncated (e.g., truncated Gaussian) so the MAP problem is naturally an inequality-constrained QP.
• Recognize that the Laplace prior term (α) enforces sparsity (L1-like penalty imposing a soft threshold), and the truncated-Gaussian term (β) enforces a smooth quadratic penalty (L2-like, effectively shifts self-inhibition/leak).

Step 4: Rate network

• Write dr/dt = -r + [W r - h]+, with W symmetric and negative (semi-)definite when feature vectors are independent or regularized.
• Prove convergence with a Lyapunov argument: show E(r) is non-increasing along trajectories and that fixed points lying in the non-negative orthant coincide with KKT points of the constrained QP.
• Confirm the non-negative orthant is invariant under the dynamics (rectification prevents escape).

Step 5–6: Spiking implementation

• Map W onto recurrent synaptic weights with realistic time constants (not instantaneous coupling).
• Encode the linear penalty α as a global, uniform inhibitory offset/current (shifts effective threshold for all neurons equally — implements L1-style regularization).
• Encode the quadratic penalty β as a renormalized reset potential or leak (implements L2-style regularization; scales down each neuron's own contribution after each spike).
• Use hard nonlinearities only at spike generation and reset — keep all sub-threshold interactions linear. This is the key simplicity constraint: linear synapses + threshold/reset nonlinearity, nothing else.
• Ensure the network is designed so that the time-averaged spike-count vector approximates the rate-network fixed point, i.e., spiking dynamics should track the same energy-descent trajectory in a coarse-grained sense.

Step 7: Explaining away

• Explaining away = when one cause sufficiently accounts for the observation, competing (correlated) causes are suppressed via inhibitory synaptic terms proportional to their feature-vector overlap (U^T U off-diagonal terms).
• In the spiking regime, this suppression should be dynamic and spike-triggered: a spike from one cause-neuron should transiently inhibit overlapping cause-neurons in a manner that reproduces the population-level competitive dynamics, not merely an averaged/rate description.

Step 8: Robustness tests

• Internally generated spike variability (e.g., Poisson-like irregularity from the LIF dynamics itself).
• Externally imposed timing jitter/noise added post-hoc to spike times.
• Mistuned recurrent weights (perturb W away from the exact U^T U structure) — check graceful degradation rather than catastrophic failure.
• Report performance in terms of task accuracy (correct cause identification) under each perturbation, not just qualitative stability.

Step 9: Locate the code

• Test whether decoding accuracy depends on precise spike timing, single-neuron firing rate, or slow covariations across the population — favor the explanation that survives jitter/mistuning tests (typically: population-level slow covariation, not single-unit rate or precise timing).