TENNER: intrusion detection models for industrial networks based on ensemble learning

INTRODUCTION

Some incidents such as the interruption of Google services in 2020 and the crash of social media in 2021 brought questions about the importance of the Internet, especially in pandemic situations, where companies and employees depend on this resource for remote work. This technology has been inside people's lives also with the rise of the Internet of Things (IoT), which shows us that devices such as TVs, fridges, and even cars can connect to the Internet^[1]. The Industrial IoT (IIoT) is also growing, allowing manufacturing and energy management machines to connect to an industrial network and Industrial Control Systems (ICSs) that manage the behavior of those machines^[2,3].

With the progress of the Internet, cybersecurity has become a concern because once a device or system is connected to it, it turns into a potential target for cyberattacks ^[4]. Some attacks that we can cite in this paper are the reconnaissance attack and the Denial of Service (DoS) attack^[5]. The reconnaissance attack is usually used to learn about the target, so the cybercriminal studies its behavior such as vulnerabilities the target may have ^[6,7]. On the other hand, the DoS makes services unavailable by overloading the target network, as presented by Murini^[8]. It has a variant called Distributed DoS (DDoS) that takes over multiple machines to overload even more the network and crash services, as shown by Zargar et al.^[9].

Thinking about these attacks and the IIoT scenario, Intrusion Detection Systems (IDSs) have been developed to detect potential attacks on networks. Discussing the differences between IoT and IIoT networks, it is possible to say that the main difference between an industrial and a home network is the traffic patterns, since the industrial traffic is more deterministic, meaning that a small variation in the usual network flow can define if it is a malicious or normal traffic. This differentiation can be done by observing the network features such as the destination port, total packets sent or received, etc. This can make the detection easier, especially if Machine Learning (ML) and Deep Learning were used in the model training. Besides, using Ensemble Learning (EL) can also increase the detection because it can combine different ML algorithms to run the same task.

To train a machine learning (ML) model, it is necessary to utilize a dataset that reflects the current scenario. This dataset can consist of either real or simulated data. In real-world IIoT network data, the occurrence of attacks and intrusions is typically low compared to the volume of normal traffic, as observed in the study by Dalarmelina et al.^[10].

Safhin et al. highlight the increasing frequency and sophistication of attacks targeting industrial systems, establishing attacks on the hardware limitations of devices that involve IIoT but also in relation to the entire flow of data and information exchanged by IIoT elements ^[11]. The cyber criminals aim to avoid detection; this often results in imbalanced data, where over 50% of the dataset comprises a single sample. This imbalance can pose challenges, particularly when training an IDS, as it may lead to a significant number of false alarms.

To address the imbalance issue, popular solutions such as Over Sampling and Under Sampling are employed. These approaches take different paths to achieve the same objective. Under Sampling involves reducing the majority class, while Over Sampling involves increasing the minority class, thereby equalizing the number of instances for each class within the dataset. The advantage of Over Sampling is that no data is discarded, but it comes at a high computational cost. In contrast, Under Sampling may result in data loss but offers a lower computational burden.

Thinking about the performance of models trained using ML, the use of datasets with "unnecessary" features can show critical challenges in the data modeling ^[12]; besides, it is needed to care about the data balancing at the time we train the model, since an imbalanced dataset can generate biased models^[13].

Considering the model performance, electing dataset features and addressing data imbalance during training are critical challenges. This paper aims to develop an industrial intrusion detection model named TENNER, based on EL. The model intends to exhibit high predictive accuracy and efficiency in learning time, utilizing datasets containing real attacks from an industrial water storage tank control network^[13]. The objective is to contribute to the field by achieving satisfactory performance in industrial intrusion detection, considering the unique challenges posed by imbalanced datasets and the need for efficient model training. Therefore, the contributions of this paper are:

· Novel Intrusion Detection Model for Industrial Networks: The primary contribution of this work lies in the development of a novel intrusion detection model named TENNER, specifically tailored for industrial networks. By leveraging EL, the model aims to achieve heightened predictive accuracy and efficient learning time, addressing the unique challenges posed by the characteristics of industrial network traffic.

· Addressing Imbalance Challenges in IIoT Datasets: This research addresses the prevalent issue of imbalanced datasets in the context of IIoT and investigates and implements solutions such as Over Sampling and Under Sampling to mitigate the challenges associated with imbalanced data, ensuring the robustness of the intrusion detection model by equalizing class instances within the dataset.

· Practical Application of ML and EL in Cybersecurity: By utilizing ML and, specifically, EL techniques, the study contributes to practically applying advanced technologies in the domain of industrial cybersecurity. The integration of these methodologies aims to enhance the efficiency of IDSs, demonstrating their potential utility in addressing the evolving threat landscape of cyberattacks in industrial networks.

RELATED WORKS

In the literature, some works focus on ML for cybersecurity. Below, we will describe some of them.

Apruzzese et al. analyzed technologies for problems encountered in cybersecurity, such as malware analysis, intrusion and spam detection, in order to discover if these techniques would supply the prevention and identification needs of these problems^[14]. According to other authors, the results showed that the technologies analyzed were still affected by some shortcomings that can reduce the security effectiveness in the cyberspace. The approaches showed vulnerability to attacks and needed constant retraining and non-automated parameters adjustments. The authors also highlight that when the same classifier is applied to identify different threats, the spam and intrusion detection decline.

Shafin et al. proposed a robust and efficient Android malware detection and classification system capable of detecting popular malicious attacks of recent times such as banking malware and riskware^[15]. For this, the authors assessing the importance of features gives us an insight into contributions of individual features in detecting malware. They accomplished this by combining ensemble methods and stacking algorithm.

An intrusion detection model based on decision tree was developed by Sarker et al., where the features of the network flow are initially ranked according to their importance, which, according to the authors, minimizes the computational complexity of the model^[16]. The approach proved to be effective for the objective of the work. Teixeira et al. developed datasets of real attacks in an industrial environment. The authors developed a Supervisory Control And Data Acquisition (SCADA) testbed which offers resources that can be used for training, validating and comparing results in cyber security research^[17]. They also highlight that the testbed provides a satisfactory understanding about the consequences and effects of attacks in real SCADA environments.

Using the K-Nearest Neighbors (KNN) classifier, suspect behaviors were identified in IoT network backbones by Pajouh et al.^[18]. Component and linear discriminant analyses of the dimension module were performed to select features. According to the authors, the approach overcame previous models in the User to Root (U2R) and Remote to Local (R2L) attack detection. In the data balancing context for IDS training, Zolanvari et al. presented a study about the effects of an imbalanced dataset in training models using ML in the IIoT area^[13]. An approach using EL is demonstrated by Arya and Gupta^[19], where they leveraged Pearson Correlation Coefficient (PCC) and Isolation Forest to select the most appropriate features, testing the built IDS performance using the Random Forest classifier and Bot-IoT e NF-UNSW-NB15-v2 datasets. Additionally, Mohy-Eddine et al. propose EL utilization for feature selection employing four reducing irrelevant features techniques^[20]. Consequently, two reduced feature groups were generated using union and intersection techniques. Verma Abhishek and Ranga Virender^[21] investigated the prospects of using ML classification algorithms for securing IoT against DoS attacks. A comprehensive study is conducted on the classifiers that can advance the development of anomaly-based IDSs. Performance of classifiers is assessed in terms of prominent metrics and validation methods. Popular datasets CIDDS-001, UNSW-NB15, and NSL-KDD are used for benchmarking classifiers. Friedman and Nemenyi tests are employed to analyze the significant differences among classifiers statistically. Khoei et al. investigate the performance of three different ensemble learning techniques: bagging-, boosting-, and stacking-based^[22]. Their results are compared to those of three traditional ML techniques, namely K nearest neighbor, decision tree, and Naive Bayes; for this, the authors used two features

Liang et al. introduced an industrial network intrusion detection algorithm rooted in a multi-feature data clustering optimization model^[23]. The algorithm lays down the foundation of a data clustering optimization model tailored explicitly for identifying intrusion attacks within industrial networks. Subsequently, the authors proposed a cluster center selection algorithm complemented by an intrusion detection algorithm designed to handle multi-feature data. Notably, within the algorithm, despite the trace procedure augmenting the training center of the model, there is a significant enhancement in detection accuracy, particularly for attacks characterized by high levels of overlap and disguise.

Khan et al. investigated an IDS model termed federated-simple recurrent units (SRUs) to safeguard IoT-based security of ICSs^[24]. More precisely, the federated-SRUs IDS model employs an enhanced SRUs architecture to diminish computational expenses and mitigate the gradient vanishing challenge in recurrent networks.

The works found in the literature, described in this section, mostly aim to develop an IDS model capable of meeting the needs of each specific scenario. Unlike these works, the goal of the present work is to analyze the better approach in terms of feature selection, data balancing and the use of EL to generate an efficient IDS model for industrial networks. Table 1 compares works found in the literature and the proposed solution. We can observe that TENNER addresses issues such as feature selection, which is not considered by Apruzzeseet al.^[14], Teixeira et al.^[17], Verma Abhishek and Ranga Virender^[21], Arya and Gupta ^[19], and Khan et al.^[24] and the data balancing that, among the cited works, is treated by Zolanvari et al.^[13] and Liang et al.^[23].

TENNER aims to develop a model that IDS can use - represented in the IDS column -, as proposed by Sarker et al.^[16], Teixeiraeet al.^[17], and Pajouh et al.^[18]. This model is also centered on industrial environment where IIoT is used, which can only be seen in the research of Zolanvari, Teixeira and Jain^[17], Khan et al.^[24], and Teixeiraeet al.^[13]; despite dealing with the industrial scenario, these studies do not have the development of the model as their main proposal. The present proposal also uses ML, as can be seen in the works of Apruzzese et al.^[14], Teixeira et al.^[17], Pajouh et al.^[18], Khoei et al.^[22], Liang et al.^[23], and Shafinet al.^[15], but specifically, using EL to train a more robust model can only be seen in the study of Mohy-Eddine et al.^[20]. To reduce the model training time, there is still a reduction in the feature size while analyzing the optimization of model training time and possible model retraining without losing performance in predicting attacks on network flows.

TENNER: INTRUSION DETECTION MODELS FOR INDUSTRIAL NETWORKS BASED ON ENSEMBLE LEARNING

TENNER is an industrial IDS leveraging EL to optimize the performance of IDSs used in industrial networks [Figure 1]. TENNER is composed of:

Figure 1. TENNER. TENNER: intrusion detection models for industrial networks based on ensemble learning framework.

· (A) Database

– To assemble the model proposed in this work, it was necessary to use databases containing flows from industrial networks, which were subjected to DDoS and Reconnaissance attacks, as found in the work of Teixeira et al.^[17].

· (B) FSAnalysis

– With the databases containing flows - normal and under attack - from industrial networks, it was necessary to implement the preprocess module FSAnalysis (selection of features and analysis, Feature Selection Analysis) ^[25] for selecting the best features for model prediction, in addition to balancing the data, so that there would be no bias in model training.

· (C) Ensemble Learning

– Two different models were trained using ensemble learning, called Ensemble Learning, to compare the performance of trained models with a reduction in the size of each feature against those without this reduction. At the end of these tests, the best-performing model was selected, deciding whether the input stream was malicious and classifying it as "normal" or "attack".

To build TENNER, initially, the dataset was submitted to the preprocessing module FSAnalysis; this determined the best approach for selecting features and deciding whether or not to balance the data to create the model. The features selected include DstPort, SrcLoss, DstLoss, and PLoss, and the model training carried out with the balanced database also obtained better results.

It was observed that the database containing reconnaissance attacks used 3.4GB of memory, having five features of type object, eight of type float64, and 12 of type int64. With this information, a way to reduce the size of this database without losing any data was analyzed. Therefore, the pandas function, "to_numeric", was used to convert the types of these features, so that they could occupy less memory; so, features of type float64 were converted to float32, and features of type int64 were converted to int32 and int8, according to the size of its value. This way, the memory usage of this database dropped to 2.8GB. To compare whether this reduction in dataset would make a significant difference in model training, the data processing step was segmented into a sub-step divided into two modes, the fast mode (FM) and the default mode (DM), as illustrated in Figure 2. In this setup, the databases, (D) the database that contains the DDoS attacks and (R) includes the Reconnaissance attacks, were submitted separately.

Figure 2. Database processing stage.

Fast Mode (FM) consists of (a) reducing the features using the method described previously to reduce the size they occupy. After lowering the features, the data is balanced (b) using the Under Sampling method so that the computational cost in generating the model is reduced. Then, EL is used, more specifically, the Stacking Classifier - using the learnersDecision Tree Classifier, Random Forest Classifier, KNeighbors Classifier, and XGB Classifier -, to create the model (c) from the treated database. Finally, the completed model has its performance evaluated (d) using the LogisticRegression algorithm, from the "linear_model" module of the open source library "sklearn" ^[26], extracting the accuracy and the confusion matrix so that other calculations can be carried out. Once you have the trained model, it is possible to move on to the testing stage, where the model can identify malicious network flows, classifying them as attack or normal. However, Default Mode (DM) comprises conducting the same processes as FM, without process (a), so that the features continue to occupy the same amount of memory. At the end of the DM and FM experiments, the FM approach was selected as it obtained better results than DM using calculations such as accuracy, sensitivity, False Alarm Rate (FAR), and Undetected Rate (UR) extracted from the confusion matrix, along with learning and prediction time.

FSAnalysis

This stage of the project involved conducting experiments to select the best approach in the context of choosing the best features and setting the dataset approach (balanced or unbalanced). To select the best features, the Random Forest algorithm and PCC were used. The FSAnalysis architecture flowchart, as illustrated in Figure 3, has the database (A) as its centralizing element. Two databases that contain samples of expected flows and attacks in an industrial network will be used. Each database has a different type of attack, namely reconnaissance and DDoS attacks.

Figure 3. Flowchart of FSAnalysis adapted from Dalarmelina et al. ^[25].

Using the database, experiments will be performed using the Random Forest algorithm (B) and, in parallel, the PCC (C) to find the features of the most significant relevance for each method.

To use the Random Forest algorithm, the RandomForestClassifier method from the "scikit-learn" library was used. Even with the same training data, the best prediction combination found from this method may vary. In the present work, this could not happen as one of the objectives was to see the best features to be used in training an IDS for industrial networks, and experiments must be able to be replicated later. Therefore, to control the randomness of the sample initialization when building the decision trees, the value "0" was defined for the "random_state" parameter of the method, and the value "10" in the " n_estimators" indicated that the model would generate only ten decision trees for training.

After the experiments using Random Forest, it was observed that the featuresDstBytes, DstLoad, and DstRate are the ones that bring the most important characteristics when evaluating the database containing DDoS attacks (D), as these presented scores above 0.15. In the database containing Recognition (R) attacks, it was observed that featurespLoss, Dport, SrcPkts and TotPkts were the most relevant, obtaining scores above 0.15. In the literature, it is possible to find several metrics for evaluating the best scores resulting from the PCC calculation. Cohen ^[27] considers that values between 0.50 and 1 can be interpreted as of great magnitude. At the same time, other authors believe that this range is a little smaller, from 0.70 or even 0.80 to 1. The value range for choosing the best score can vary according to each context and purpose. Therefore, in this work, for the selection criterion of the best features, it was determined that only features that obtained score greater than 0.55 would be selected. With this, it was noted that the best features using the database (D) were DstLoss, pLoss, Mean and SrcLoss, and using the database (R), DstLoss, Dport, SrcLoss and pLoss, achieving scores greater than 0.55. Details of the results can be found in the work by Dalarmelina et al.^[25].

With the most relevant features for each attack, the data present in the databases used were normalized using the StandardScaler class from the "preprocessing" module of the "sklearn" library ^[26] and divided into two parts, 80% for training and 20% for testing. After this preprocessing, the LogisticRegression class, from the "linear_model" module of the open source library "sklearn" ^[26], created the network flow prediction model using the most important features, selected as previously described.

To conduct the prediction tests to choose the best approach, using the bases in complete and the balanced commands, three scenarios were applied: (1) application of the features selected using the algorithm Random Forest; (2) application of the selected features using PCC; (3) application of all features present in both databases. The same training routine was performed using datasets (D) and (R). Some calculations, including the Logistic Regression prediction algorithm, learning time, accuracy, sensitivity, FAR, non-detection rate (UR), and Matthews correlation coefficient (MCC) are employed, which are extracted from the confusion matrix that can be interpreted as follows:

· Input stream: Value to be analyzed by the algorithm. This value can be 0 or 1, with 0 being the numerical representation of the normal network flow and 1 being the flow under attack.

· True Negative: Represents the number of normal flows correctly classified as normal.

· False Positive: Indicates the number of normal flows wrongly classified as flow under attack.

· False Negative: Means the number of flows under attack mistakenly classified as normal flow.

· True Positive: Denotes the number of abnormal flows correctly classified as attacks.

The calculations used were FAR, Accuracy, MCC, Sensitivity or True Positive Rate, and the UR. The FAR calculates the normal traffic misclassified as attack percentage, which means the FP divided by FP plus TN, expressed as

(1) $$ FAR = \frac{FP}{FP+TN} \times 100 $$

The accuracy calculation results in the percentage of the model hits, determined by

(2) $$ Accuracy = \frac{TP+TN}{TP+TN+FP+FN} \times 100 $$

The MCC measures the classification quality, denoted as

(3) $$ MCC = \frac{TP \times TN - FP \times FN}{\sqrt{(TP+FP) \times (TP+FN) \times (TN+FP) \times (TN+FN)}} \times 100 $$

To measure the effectiveness in predicting attacks correctly classified as attacks, the sensitivity is used, as given in

(4) $$ Sensitivity = \frac{TP}{TP+FN} \times 100 $$

Finally, the UR results in the predictions and real values correlation, established as

(5) $$ UR = \frac{FN}{FN+TP} \times 100 $$

The results have shown that, although the difference is not considerably large, the model trained with balanced databases performed better, reaching superior results, especially in terms of learning time, MCC and sensibility. Table 2 describes the confusion matrix.

Table 2

Confusion matrix

		Prediction
		Normal	Under attack
Input	Actual normal	True Negative (TN)	False Positive (FP)
	Actual under attack	False Negative (FN)	True Positive (TP)

TESTS AND RESULTS

The main focus of this work is to validate Tenner against EL. For that, it was made a reduction of the database where tests were made for measuring the model performance trained without the reduction (DM) and with the model trained with the reduction (FM) using the best approach found on FSAnalysis module; in other words, the models were trained using the selected features using PCC and balanced datasets. Further details about FSAnalysis module and the features selection can be found in the research of Dalarmelina et al.^[10,25]. From the confusion matrix, some metrics were used in the present work to evaluate the trained model performance. Specifically, the metrics used are FAR, UR and sensibility.

The calculating accuracy is formulated in Equation 2 and the results are illustrated at 4(a), where it is possible to observe the results having a difference of 0.03% e 0.01%, respectively, for datasets (D) and (R). The FAR calculation used is determined by Equation 1 and the results are presented in 4(b), which also obtained a difference of 0.03% for the database (D) and 0.02% for the database (R). In the industrial network context, the UR can be considered more important than FAR, since this metric represents the under attack traffic portion that was wrongly classified as normal, and can be calculated using Equation 5. Analyzing the chart illustrated in 4(c), it is possible to observe the results having a bigger difference; despite still a little difference, FM obtained 0.1% and 0.09% more than the DM in the non-detection rate.

Figure 4. Solution performance regarding accuracy and FAR. FAR: False alarm rate.

The sensibility models were evaluated by calculating the True Positive Rate to measure abnormalities correctly classified as attacks. For this task, Equation 4 was employed. The results with this formula are shown in 4(d), where it is possible to note that FM performed 0.03% better than DM with dataset (D). The MCC was also considered using Equation 3; the result is shown in 5(a); using this metric, the DM performed 0.09% and 0.15% better.

As can be seen in 4(d), both modes obtained similar results in all calculations performed, presenting a difference below 0.1% in three of the four calculated equations; both the learning time and the prediction time were also compared to evaluate the performance of the modes (DM and FM). 5(b) illustrates the time, in hours, covered by the models in the two modes presented. In this scenario, FM had an advantage, as it completed model training in 1.34 h, equivalent to approximately 80 min, and in 0.92 h, equivalent to approximately 55 min, in databases (D) and (R), respectively, while DM took 156 h, equivalent to approximately six days for (D) and 128 h, equivalent to approximately five days for (R). Considering a proposal in which a model can be retrained with each new attack presented, FM can be a good choice due to its ability to train in a reduced time. Furthermore, analyzing the comparison of the prediction time of the two modes, as presented in 5(c), the FM also stands out, since it concluded the prediction in 0.0067% and 0.0065% less than the DM using (D) and (R), respectively; these values may seem minimal, but they can be crucial in a critical network, such as an industrial one.

Figure 5. Solution performance regarding models Sensitivity and models MCC. MCC: Matthews correlation coefficient.

CONCLUSION

This study elucidates the methodologies employed to ascertain the optimal approach for crafting a resilient and effective model tailored for deployment in industrial networks using ML. Leveraging datasets from existing literature, pivotal aspects such as feature selection for model training and the comparative analysis of training methodologies using unbalanced and balanced datasets were thoroughly explored.

The culmination of our research efforts has led to the development of a specialized intrusion detection model designed for industrial networks, harnessing the power of EL. A significant focus was placed on anticipating potential scenarios that would necessitate model retraining in response to newly identified attacks, considering the time investment involved in model training. The model that emerged as the most promising in this study demonstrated an impressive accuracy rate of 99.93%, achieving training completion in a remarkably brief duration of 1 h and 34 min. In contrast, the model trained without the implemented treatments exhibited a slightly higher accuracy of 99.94% but required an extensive 156 h for training. Moreover, regarding prediction time, the superior model executed predictions in a swift 0.0009 seconds, while the alternative model required 0.0076 seconds for the same task. The marginal difference in accuracy between our proposed model and the comparison model is small and we recognize this difference. However, it is important to highlight that accuracy is just one of many metrics to consider when evaluating an industrial network intrusion detection model. To help understand the performance and interpretability of the model, we also used other metrics to evaluate it, such as FAR, UR, Sensitivity, and MCC. The proposed model showed better results in all used metrics. The FSAnalysis selected the best features to build the proposed model. These findings underscore the effectiveness and practical applicability of the EL-based approach in developing a resilient intrusion detection model for industrial networks.

DECLARATIONS