Diving Into End-to-End Deep Learning for Cybersecurity

We look into developments in end-to-end deep learning for cybersecurity and provide insights into its current and future effectiveness.
August 20, 2020

By Spark Tsao (Data Scientist)

The application of artificial intelligence (AI) across various industries has undeniably made significant improvements in the digital era. With the capability to interpret and make complex decisions based on data, AI technologies have enabled tasks or processes to function with human-like intelligence, enhancing the speed of and innovating business operations and adding valuable user experiences.

The cybersecurity industry is one of the many sectors that have greatly benefited from AI. If done efficiently, the application of AI can provide cybersecurity solutions an improved capability to detect a wide range of threats, including brand-new or unclassified ones. The process of using AI efficiently usually involves state-of-the-art models, an iterative method to improve the model’s accuracy, and accurately labeled data, among others. For many cybersecurity companies that use AI, the said requirements – particularly the process of accurately labeling data – are supported by threat experts who preprocess data and extract and engineer features, among other manual tasks or processes that produce handcrafted input. In essence, these expert-handcrafted input enable models to perform with less ambiguity by allowing the underlying structure of the data to be accurately represented, thereby enhancing threat detection capabilities.

Faster and More Accurate Malware Detection Through Predictive Machine Learning
To further push the boundaries of how machine learning can be used for a more effective cybersecurity solution, we developed a machine learning model that uses two training phases — pre-training and training — to improve detection rates and reduce false positives.

However, the emergence of new methods for detecting threats using AI challenges the need for expert handcrafted input. Specifically, these methods involve end-to-end deep learning solutions, which are being touted by some as the next big thing in malware detection. In the pipeline of such solutions, expert handcrafted input are out of the loop and replaced with ones provided by automated processes. While this is arguably becoming more acceptable in some industries that use AI for various purposes, the absence of expert handcrafted input gives rise to the question of whether or not expert handcrafted input are still relevant in the process of developing an efficient AI-powered cybersecurity solution.

End-to-end deep learning as a solution for detecting malware

Some research on end-to-end deep learning propose a methodology that doesn’t rely on expert handcrafted input in the process of testing and training samples that are fed to AI models. One approach inspected malware binaries that were plotted as grayscale images, which revealed the textural and structural similarities and differences either between binaries from the same and different malware families, or between malware and benign software. This avoids the process of manual feature engineering, saving time and reducing workload for cybersecurity companies. Another approach involves a process where the engine was fed raw input made up of raw byte values and produced output which showed the classification of a malicious or benign file.

Due to confidentiality reasons, further details about the files used for training and testing the end-to-end deep learning models are undisclosed. Unfortunately, this makes it hard for one to gain a deeper insight into the performance results of the AI models they used. In particular, the said research have been unable to divulge the percentage of easily detected unknown malware samples as well as the percentage of ones that are difficult to detect. Such information is crucial as this could determine if such solutions are sufficient for proactively detecting complex, unknown threats that cybercriminals are continuously developing in present time.

Measuring the detection rates of an end-to-end deep learning model

In an attempt to validate one of the early end-to-end models, Trend Micro experts conducted an experiment in 2017. They used experimental datasets that comprised of carefully collected samples. The datasets used were for training and testing of two approaches: one with end-to-end deep learning and the other with a machine learning approach that includes expert handcrafted feature. The latter will be called “Expert AI” in the later sections. It should be noted that the datasets were organized in two ways according to the Trend Micro Locality Sensitive Hashing (TLSH) clustering results. This was done to compare and paint a clearer picture of the detection performance of the two approaches.

The datasets were organized into Type A dataset and Type B dataset. For Type A dataset, Trend Micro experts selected one sample from each TLSH cluster to form the training set, while the rest of the samples were selected from each cluster to form the testing set. This means every sample in the testing set had a similar sample in the training set. For Type B dataset, the experts experimented with select clusters as a whole in the training dataset and the testing dataset. This means that it was possible for the samples in the testing data to be significantly different from the samples in the training data. Theoretically speaking, Type A testing data might have been easier to detect if the model had already seen the Type A training data because it is very similar to the testing data.

The findings of the experiment showed that for Type A, end-to-end deep learning achieved 99.39% accuracy, while Expert AI achieved 99.87%.


Figure 1. Experimental datasets, labelled Type A and Type B, were trained and tested using Expert AI and end-to-end deep learning approaches.

For Type B, end-to-end deep learning achieved 92.49% accuracy. On the other hand, Expert AI achieved 95.21%. While there is only an extremely small difference in the performance results between end-to-end deep learning and Expert AI for Type A, the results for Type B showed that Expert AI fared better – with a 2.72% advantage – than end-to-end deep learning in terms of detecting difficult, unknown samples. The experiment concluded that Expert AI, which made use of expert handcrafted input, performed better than the approximated end-to-end deep learning model in terms of detecting unknown samples.

Model Type A Type B
Expert AI 99.87% 99.39%
End-to-end deep learning 95.21% 92.49%

Table 1. The results of the experiment showed that Expert AI performs better than end-to-end deep learning in terms of detecting unknown samples.

A research conducted in 2018 that experimented with a dataset of 20 million unpacked, half-megabyte Portable Executable (PE) files echoed the findings of the 2017 Trend Micro experiment. The former concluded that an end-to-end learning approach was still slightly behind a traditional machine learning pipeline that made use of handcrafted features.

End-to-end deep learning is innovative, but slightly behind an expert-supported AI solution

In the previously mentioned Trend Micro experiment, findings revealed that end-to-end deep learning is an innovative, not to mention impressive, approach to malware detection in terms of noise canceling. However, the experiment also showed that when detecting unknown threats, an end-to-end deep learning solution is less accurate compared to an AI solution that is supported by expert handcrafted input.

On another note, cybersecurity entities who are interested in end-to-end deep learning should invest in new network architecture or learning schemes to enhance their malware detection technologies. Some of these entities, for instance, might be vendors who use transfer learning techniques from the computer vision domain to reduce the amount of consumption data needed for training a model. From Trend Micro’s perspective, when deep learning is used to successfully detect an object in a raw image file, it should be highlighted that the raw image file in discussion is in fact the bitmap array values extracted from image files (for example, JPEG, TIFF, and PNG) that are formatted via expert-designed parsers. If deep learning is applied on complex file formats such as JPEG, TIFF, and PNG directly, it is likely that the performance of the model will drop. Therefore, when detecting malware in a Windows PE file format or a Linux ELF executable file that has various variants in different OS versions, simply using deep learning to learn from raw files will result in a less sufficient detection capability. This is just one of the issues that vendors need to look closely at if expert handcrafted input were to be removed from the pipeline. 

For now, it can be surmised that a more effective cybersecurity strategy involves the collaboration between AI and threat experts, especially in the current threat landscape where threat actors are increasingly employing modularized malware threats. In advanced persistent threat (APT) attacks, for example, a piece of malware is divided into many components that can be spread out across different attack vectors. There is a possibility that each component could be detected as a benign file by traditional or insufficient cybersecurity solutions. In such a scenario, a deep learning end-to-end solution might have its limitations.

AI and expert security analytics in Trend Micro™ XDR

Trend Micro XDR, a multilayered solution that employs both AI and expert handcrafted input, can be relied on by organizations to break down the silos of detection to assemble and contextualize such scattered malware components. In particular, it does so by applying expert analytics to the activity data collected across email, network, endpoint, and servers, thereby producing correlated, actionable alerts. XDR combines global threat intelligence and continuously updated expert detection rules to maximize the power of analytical models.

In addition, XDR provides a single platform for investigation to get the full context needed for an attack-centric view of an entire chain of events across security layers. XDR also enables organizations to run a root cause analysis, look at the execution profile, identify the scope of the impact, and take response from a single place.
HIDE

Like it? Add this infographic to your site:
1. Click on the box below.   2. Press Ctrl+A to select all.   3. Press Ctrl+C to copy.   4. Paste the code into your page (Ctrl+V).

Image will appear the same size as you see above.

Posted in Security Technology, Machine Learning

We Recommend