REVIEW 5 cited by
ePC: Fast and Deep Predictive Coding in Digital Simulation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
ePC: Fast and Deep Predictive Coding in Digital Simulation
read the original abstract
Predictive Coding (PC) offers a brain-inspired alternative to backpropagation for neural network training, described as a physical system minimizing its internal energy. However, in practice, PC is predominantly digitally simulated, requiring excessive amounts of compute while struggling to scale to deeper architectures. This paper reformulates PC to overcome this hardware-algorithm mismatch. First, we uncover how the canonical state-based formulation of PC (sPC) is, by design, deeply inefficient in digital simulation, inevitably resulting in exponential signal decay that stalls the entire minimization process. Then, to overcome this fundamental limitation, we introduce error-based PC (ePC), a novel reparameterization of PC which does not suffer from signal decay. Though no longer biologically plausible, ePC numerically computes exact PC weights gradients and runs orders of magnitude faster than sPC. Experiments across multiple architectures and datasets demonstrate that ePC matches backpropagation's performance even for deeper models where sPC struggles. Besides practical improvements, our work provides theoretical insight into PC dynamics and establishes a foundation for scaling PC-based learning to deeper architectures on digital hardware and beyond.
Forward citations
Cited by 5 Pith papers
-
Training a Predictive Coding Network on ImageNet using Equilibrium Propagation
A VGG10 predictive coding network is trained on ImageNet via equilibrium propagation to 13.23% top-5 error, close to the 12.2% backpropagation baseline, marking the first such demonstration at this scale.
-
Understanding Sample Efficiency in Predictive Coding
Predictive coding learns more sample-efficiently than backpropagation because its updates align better with output prediction errors in deep linear networks, with exact conditions for optimal alignment derived.
-
Understanding Sample Efficiency in Predictive Coding
Predictive coding achieves higher target alignment than backpropagation in deep linear networks, explaining observed sample efficiency gains.
-
Closed-form predictive coding via hierarchical Gaussian filters
Predictive coding is recast as deep hierarchical Gaussian filters to restore precision-weighted message passing, yielding closed-form inference and online precision learning that matches backpropagation speed on Fashi...
-
Cross-Entropy Is Load-Bearing: A Pre-Registered Scope Test of the K-Way Energy Probe on Bidirectional Predictive Coding
Cross-entropy loss is empirically load-bearing for the K-way energy probe outperforming softmax margins in predictive coding on CIFAR-10, with roughly two-thirds of the gap due to logit scale.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.