A Hybrid Deep Learning Architecture For Malicious Traffic Detection In Https Environments

Detecting malicious activities in HTTPS traffic remains a crucial challenge in cybersecurity, as encryption simultaneously enhances privacy and conceals malicious behaviors from conventional inspection methods. This study addresses the research question: how can a hybrid deep learning architecture effectively detect malicious encrypted traffic while preserving user privacy (using flow-level features only) and ensuring generalization across modern datasets? For this, we propose a hybrid architecture combining Convolutional Neural Networks (CNN), Efficient Channel Attention (ECA-Net), and Transformer Encoders to jointly capture spatial, statistical, and temporal dependencies within encrypted traffic flows. The model was trained and validated using two public datasets, CIRA-CIC-DoHBrw-2020 and HIKARI-2021, under a data leakage-free experimental protocol. When evaluated on the HIKARI-2021 dataset, the proposed architecture achieved an AUC of 92.7%, recall of 99.5%, precision of 87.6%, and F1-score of 93.2%, outperforming the baseline model reimplemented and reexecuted under equivalent conditions, demonstrating robustness and scalability in detecting malicious HTTPS traffic without relying on Deep Packet Inspection (DPI).