Authors
Fishell, W., Honnuraiah, S.
Abstract
Spiking Neural Networks (SNNs) have gained increasing attention for their ability to operate on low-power neuromorphic hardware, offering a pathway toward energy-efficient large language models (LLMs). However, they continue to underperform state-of-the-art artificial neural networks (ANNs) on tasks such as language modeling, limiting their broader adoption. Here, we present an explicit covering-number analysis of SNNs, focusing on the non-leaky integrate-and fire (nLIF) model. Building on recent advances in causal-piece analysis and local Lipschitz continuity, we derive a global Lipschitz constant for nLIF networks and extend this framework from single-token inputs to full sequences. Our analysis reveals that the sample complexity of nLIF architectures grows quadratically with sequence length, in contrast to the linear or logarithmic scaling observed in recurrent and Transformer models. These findings expose a fundamental architectural limitation in SNNs and suggest targeted architectural and biological modifications needed to overcome it. Finally, we show that the same assumptions emerge in biological circuits, emphasizing the critical role of inhibitory interactions in enabling efficient learning of long-range dependencies.
Preprint server:
bioRxiv
The authors list and abstract were imported from bioRxiv on 04 Nov 2025.
Advertisement
Stats
- Recommendations n/a n/a positive of 0 vote(s)
- Views 36
- Comments 0