Weight Importance Is Not Enough: Neural Expressiveness for Model Compression.

Authors

Angelos Christos Maroudis, Sotirios Xydis, George Dimitrakopoulos

Published in

IEEE transactions on pattern analysis and machine intelligence. Volume PP. Sep 17, 2026. Epub Sep 17, 2026.

Abstract

Neural Network Pruning has been established as driving force in the exploration of memory and energy efficient solutions with high throughput both during training and at test time. We propose Neural Expressiveness (NEXP), a new criterion for model compression that shifts the focus from weight importance to the capacity of neurons or groups of neurons to redistribute and preserve information flow, quantified through activation overlap. This perspective establishes a clear conceptual distinction between importance-based (weight-centric) and expressiveness-based (activation-centric) pruning, motivating a broader class of activation-dependent compression strategies. We provide a mathematical formulation of expressiveness that is model-agnostic and can be integrated into any activation-based pruning framework. Our analysis shows that expressiveness captures information complementary to weight importance, revealing a form of partial orthogonality between the two families of criteria. Furthermore, we demonstrate that expressiveness is strongly tied to the network's initialization, enabling a stateless pruning regime that is partially independent of training dynamics and directly informs the "when to prune" question. We also show that NEXP can be reliably approximated using limited arbitrary data, and thus paving the way for data-agnostic compression. Overall, the proposed concept of "Neural Expressiveness" in this paper expands on the prevalent yet unexplored notion of expressiveness in efficient deep learning and aims to forge a distinct paradigm, transitioning from the conventional weight-centered importance assessment to an emphasis on activations in model compression.

PMID:
42758944
Bibliographic data and abstract were imported from PubMed on 19 Sep 2026.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.