A Robust LightGBM-Based Machine Learning Framework for Network Outlier Detection
Authors: Rohit Chandra, Krishna Murari
Published in: IJRRETAS, Volume 12, Issue 7, pp. 1-14 (03 Jul 2026)
Impact Factor: 7.95 (2026)
DOI: https://doi.org/10.65630/ijrretas.2026.v12i71-14
Citations: 0 (Crossref)
License:
CC BY 4.0 (Open Access)
Certificate: View Certificate
Abstract
The growing volume and complexity of network traffic have made manual and signature-based monitoring insufficient for detecting malicious activity. This paper presents a robust machine learning framework, based on the Light Gradient Boosting Machine (LightGBM), for detecting outliers in network traffic data. The framework integrates data cleaning, one-hot encoding, logarithmic transformation, mutual-information-based feature selection, standardisation and a regularised LightGBM classifier trained with early stopping. It was evaluated on the NSL-KDD benchmark and compared with Logistic Regression, Random Forest and a Support Vector Machine under identical conditions. On a held-out test set of 25,193 connections, LightGBM achieved an accuracy of 99.933%, a precision of 99.957%, a recall of 99.898%, an F1-score of 99.927% and a ROC-AUC of 0.99999, making only 17 errors. It outperformed all baseline models while training about four times faster than Random Forest. SHAP-based interpretation showed that byte volume, connection rate, service concentration and SYN-error features drive detection. Robustness experiments confirmed stable performance across data splits and random seeds, and resilience to severe class imbalance, with an F1-score of 97.74% when outliers formed only 1% of traffic. On an independent test set containing unseen attack types, accuracy fell to 78.54%, identifying generalisation to novel attacks as the main limitation. The results show that LightGBM offers an accurate, efficient and interpretable foundation for network outlier detection.
Keywords
- network outlier detection
- anomaly detection
- intrusion detection
- LightGBM
- gradient boosting
- SHAP
- NSL-KDD
References
- Ahmad, Z., Shahid Khan, A., Wai Shiang, C., Abdullah, J., & Ahmad, F. (2021). Network intrusion detection system: A systematic study of machine learning and deep learning approaches. Transactions on Emerging Telecommunications Technologies, 32(1), e4150.
- Arp, D., Quiring, E., Pendlebury, F., Warnecke, A., Pierazzi, F., Wressnegger, C., Cavallaro, L., & Rieck, K. (2022). Dos and don'ts of machine learning in computer security. In Proceedings of the 31st USENIX Security Symposium (pp. 3971–3988). USENIX Association.
- Chicco, D., & Jurman, G. (2020). The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC Genomics, 21, Article 6.
- Engelen, G., Rimmer, V., & Joosen, W. (2021). Troubleshooting an intrusion detection dataset: The CICIDS2017 case study. In 2021 IEEE Security and Privacy Workshops (SPW) (pp. 7–12). IEEE.
- Grinsztajn, L., Oyallon, E., & Varoquaux, G. (2022). Why do tree-based models still outperform deep learning on typical tabular data? In Advances in Neural Information Processing Systems 35 (pp. 507–520).
- Jin, D., Lu, Y., Qin, J., Cheng, Z., & Mao, Z. (2020). SwiftIDS: Real-time intrusion detection system based on LightGBM and parallel intrusion detection mechanism. Computers & Security, 97, 101984. https://doi.org/10.1016/j.cose.2020.101984
- Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q., & Liu, T.-Y. (2017). LightGBM: A highly efficient gradient boosting decision tree. In Advances in Neural Information Processing Systems 30 (pp. 3146–3154).
- Khraisat, A., Gondal, I., Vamplew, P., & Kamruzzaman, J. (2019). Survey of intrusion detection systems: Techniques, datasets and challenges. Cybersecurity, 2, Article 20.
- Kilincer, I. F., Ertam, F., & Sengur, A. (2021). Machine learning methods for cyber security intrusion detection: Datasets and comparative study. Computer Networks, 188, 107840.
- Liu, H., & Lang, B. (2019). Machine learning and deep learning methods for intrusion detection systems: A survey. Applied Sciences, 9(20), 4396.
- Liu, J., Gao, Y., & Hu, F. (2021). A fast network intrusion detection system using adaptive synthetic oversampling and LightGBM. Computers & Security, 106, 102289. https://doi.org/10.1016/j.cose.2021.102289
- Lundberg, S. M., Erion, G., Chen, H., DeGrave, A., Prutkin, J. M., Nair, B., Katz, R., Himmelfarb, J., Bansal, N., & Lee, S.-I. (2020). From local explanations to global understanding with explainable AI for trees. Nature Machine Intelligence, 2(1), 56–67.
- Lundberg, S. M., & Lee, S.-I. (2017). A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems 30 (pp. 4765–4774).
- Resende, P. A. A., & Drummond, A. C. (2018). A survey of random forest based methods for intrusion detection systems. ACM Computing Surveys, 51(3), Article 48.
- Ring, M., Wunderlich, S., Scheuring, D., Landes, D., & Hotho, A. (2019). A survey of network-based intrusion detection data sets. Computers & Security, 86, 147–167.
- Seth, S., Singh, G., & Kaur Chahal, K. (2021). A novel time efficient learning-based approach for smart intrusion detection system. Journal of Big Data, 8, Article 111. https://doi.org/10.1186/s40537-021-00498-8
- Sharafaldin, I., Habibi Lashkari, A., & Ghorbani, A. A. (2018). Toward generating a new intrusion detection dataset and intrusion traffic characterization. In Proceedings of the 4th International Conference on Information Systems Security and Privacy (ICISSP) (pp. 108–116). SciTePress.
- Talukder, M. A., Islam, M. M., Uddin, M. A., Hasan, K. F., Sharmin, S., Alyami, S. A., & Moni, M. A. (2024). Machine learning-based network intrusion detection for big and imbalanced data using oversampling, stacking feature embedding and feature extraction. Journal of Big Data, 11, Article 33. https://doi.org/10.1186/s40537-024-00886-w
- Vinayakumar, R., Alazab, M., Soman, K. P., Poornachandran, P., Al-Nemrat, A., & Venkatraman, S. (2019). Deep learning approach for intelligent intrusion detection system. IEEE Access, 7, 41525–41550.
- Yin, C., Zhu, Y., Fei, J., & He, X. (2017). A deep learning approach for intrusion detection using recurrent neural networks. IEEE Access, 5, 21954–21961.