BERT4NID: An intra and inter packet representation with pre-training transformers for IoT network intrusion detection.

Authors

Zeyu Li, Wenbin Yao

Published in

Neural networks : the official journal of the International Neural Network Society. Volume 205. Issue Pt A. Pages 109398. Jul 17, 2026. Epub Jul 17, 2026.

Abstract

It's crucial for IoT network intrusion detection models to learn discriminative traffic representations from observable network traffic. Existing methods primarily rely on manually crafted features to represent network traffic. However, these manually constructed features depend heavily on the knowledge and experience of network experts and are difficult to generalize to diverse and constantly changing network environments. Therefore, leveraging a large amount of unlabeled raw network traffic to learn traffic representations from observable traffic patterns has become a research hotspot in the field of network intrusion detection. This paper proposes a new IoT network intrusion detection model called Bidirectional Encoder Representations from Transformer for Network Intrusion Detection (BERT4NID), which learns both intra-packet and inter-packet features by proposing two novel proxy tasks and pre-training on a large amount of unlabeled data to obtain traffic representations. The pre-trained model can be fine-tuned with a small amount of labeled data for downstream classification tasks and achieve excellent performance. With only 1% of labeled training data, the model's F1 scores on MedBIoT and Kitsune datasets are 93.46% and 92.85%.

PMID:
42480161
Bibliographic data and abstract were imported from PubMed on 22 Jul 2026.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.