AI vs AI: DeepFakes and eKYC

By Philippe Lin, Fernando Mercês, Roel Reyes, and Ryan Flores
With help from Vincenzo Ciancaglini, Bakuei Matsukawa, Fyodor Yarochkin, Vladimir Kropotov, and Sean Park

Know Your Customer (KYC) is a critical process — first employed by financial organizations and then adopted by other industries — that is used to verify the identity of clients, along with their financial knowledge, risks, and investment profiles. It is a mandatory regulatory process which is applied when a person opens a banking account and is designed to prevent fraud and money laundering. Today, a variety of organizations, including banks, lenders, gambling websites, cryptocurrency exchanges, and even smaller businesses such as online alcohol vendors, are increasingly adopting electronic KYC (eKYC) for online customer identity verification. The Covid-19 pandemic further boosted its adoption rate as a standard protocol for online customer onboarding.

Typically, eKYC requires the scanning of an ID card such as a driver’s license or passport to verify its holder’s authenticity. Alongside this, the user’s face is also scanned to match the document. Sometimes, facial analysis is used to determine the customer's age and gender, while third-party checks cross-reference external anti-fraud databases for additional validation. To counter the threat of deepfake technology, eKYC providers may employ passive liveliness checks that require users to perform simple actions, such as turning their heads and blinking, making it more difficult for for deepfake videos to pass as genuine.

This blog entry aims to study the eKYC supply chain and test the technology’s resilience against deepfakes within a legal framework, along with a glimpse at underground markets. Given that major providers dominate the market, bypassing a provider's security measures could potentially compromise all customers using that provider's technology. By understanding these vulnerabilities, we can better protect ourselves and our customers from emerging cybersecurity threats.

The eKYC supply chain

In our study of the eKYC landscape, we identified around ten prominent providers that serve as key players in the field of electronic identity verification and document procedure automation. They serve a global customer base and work with system integrators and OEM software companies. In addition, eKYC vendors also support government services in several countries, such as the eGovPH mobile app used in the Philippines. The providers we studied include Microblink, IDScan, Shufti Pro, Onfido, Jumio, GBG (which acquired Acuant and Idology), Smart Engines, Veriff, Sumsub, and Au10tix. Another major eKYC provider, Anyline, focuses primarily on the automotive industry, thus is not covered in this entry.

A significant observation based on our study is that these eKYC providers sometimes collaborate with each other by delegating part of the tasks, such as reference checks and fraud detection, to another provider. For instance, IDScan uses Shufti Pro for cross-validation, iProov handles liveliness detection for Jumio, and Sumsub partners with Smart Engines to detect document forgeries. While these partnerships can enhance the robustness of eKYC systems, they also raise concerns about data privacy. Specifically, it becomes unclear where sensitive data, such as faces and passports, might flow through and end up (although this issue is beyond the scope of our current research).

We also noted that two of the providers operate primarily out of Russia, which can be a concern as these providers process personal and biometric data, which might be an issue for some governments — as seen in an executive order issued by US president Joe Biden in February 2024. Understanding the data transfer chain between service vendors and their countries of origin will become increasingly relevant in light of legislation and policies similar to the executive order.

System integrators also play a crucial role in the eKYC ecosystem, often rebranding and using multiple service providers to offer comprehensive solution packages. For example, Scalable Solutions partners with both Sumsub and GBG, Jack Henry packaged IDScan SDK into JHA OpenAnywhere to provide online onboarding solutions for dozens of US banks, and Asseco used MicroBlink’s library in mobile apps that they developed for dozens of European banks. Interestingly, some financial institutions may opt for different eKYC providers for different branches, adding another layer of complexity to the landscape.

The intricate web of relationships among providers, integrators and their customers is visualized as a detailed diagram found in this link, which shows an interconnectedness that highlights the importance of understanding the supply chain. The diagram has been created for illustration purposes only using readily available public information collected from KYC provider websites, and the information presented may be inaccurate or outdated, depending on the reliability of the sources used. Inclusion on the graph does not imply that an entity is vulnerable to eKYC or deepfake attacks. Some of the providers shown here also provide document scanning, optical character recognition (OCR) and document automation, in addition to KYC and online onboarding, which may be the only services for which some of the entities use the provider. We do not claim implicitly or explicitly that any company listed on the graph are using KYC services.

The role of AI models in eKYC

The eKYC process can employ local AI models as a first-level filter on the scanning of ID cards, passports, and faces. We have observed an increasing usage of ONNX and Tensorflow Lite models on web interfaces and mobile apps. ONNX, or Open Neural Network Exchange, is an open-source format for machine learning models with a public runtime with which people can interface with the models cross-platform or even in a browser. Meanwhile, Tensorflow is a widely adopted machine-learning platform primarily developed by Google.

Various eKYC providers use AI models in ONNX or Tensorflow formats to perform initial checks on scanned documents and faces to ensure they meet basic criteria before sending them to the provider's backend servers for more intensive computation. This approach not only saves bandwidth, but also speeds up the verification process for the end user. Among the providers we studied, only Smart Engines claims to operate entirely locally without sending data to backend servers, although we did not verify this claim.

To ensure that these models run efficiently on slower computers and ordinary mobile devices, they are designed to be lightweight in terms of resource use. For example, some providers detect face presence at a resolution of 128x128 pixels and face orientation at 224x224 pixels. This low resolution is advantageous in scenarios where the user's front camera may not be of high quality, or the ambient lighting is poor. However, the downside of this design choice is that lower resolutions can make it easier for deepfake technologies to bypass initial checks. Having access to the local models also means that researchers and hackers can study how they work (as shown in Figure 2).

Figure 1. Netron (a neural network, deep learning and machine learning visualizer) enables all users to study the structure and adjust the parameters of an AI model (part of a KYC facial recognition model is show in the image)

There is a noticeable trend of local models being served on websites using ONNX runtime written in WebAssembly (WASM). Whether they use Tensorflow Lite or ONNX, these machine learning models output probabilities for various labels, such as detected objects or the acceptability of a face. We have discussed the importance of protecting their confidentiality and integrity in more detail in a previous blog post.

There is a noticeable trend of local models being served on websites using ONNX runtime written in WebAssembly (WASM). Whether they use Tensorflow Lite or ONNX, these machine learning models output probabilities for various labels, such as detected objects or the acceptability of a face. We have discussed the importance of protecting their confidentiality and integrity in more detail in a previous blog post.

Upon the successful pre-filtering of ID cards and faces by local AI models, the data is then typically uploaded to the provider's servers that stores raw documents for audition purposes — lasting as long as three years in some countries — for further computation. However, it remains unclear whether the data is properly secured. A report by 404 Media highlighted a significant security breach where the credentials of a senior manager were stolen, leading to the exposure of a large number of driver's licenses. While this issue is critical, it falls beyond the scope of our current research because it is an issue of the eKYC provider’s data security. Nevertheless, this highlights how eKYC’s are a critical data store of customer personally identifiable information (PII).

The bypass-as-a-service model

Deepfakes use AI models to create convincing fake images and videos, posing a significant threat to eKYC. Malicious actors can buy and sell bypass-as-a-service in underground forums, involving criminals using malware and/or data leaks to obtain photo IDs, faces and other PII.

We have observed a steady supply and demand for bypassing account verification systems that use webcams in underground forums, even before the term eKYC was also adopted there. Scanned document copies have been sold for at least a decade in the underground at an affordable price of US$2-$5, serving as a fundamental enabler for bypass-as-a-service transactions.

Several incidents highlight the growing risk of deepfakes. For example, in 2022, Sensity published a report, where they claimed the total success of spoofing attacks against eKYC providers. Another notable incident involved tax fraudsters that hacked a Chinese government-run facial recognition system to claim fake invoices and evade taxes worth approximately US$70 million. GroupIB discovered an Android and iOS trojan that stole facial recognition data, which hackers then used to empty pension accounts. This scamming technique was further explained in a YouTube video to raise awareness of such attacks.

Additionally, tutorials on how to bypass KYC verification using deepfake technology are readily available online, some of which are published by malicious actors as an advertisement. These resources make it easier for even less technically skilled individuals to exploit eKYC systems.

Our team monitored underground forums from June 27 to July 10, 2024 to understand the prevalence and pricing of the bypass-as-a-service model. We found that underground service providers advertised daily their ability to bypass systems from providers such as Onfido, Jumio, Sumsub, and Veriff. These services often echoed our concern that bypassing one provider's security measures could potentially compromise all customers using that provider's technology.

The typical cost for these services was around 3,000 Russian Rubles (RUB) —approximately US$30. More lucrative services, such as bypassing Binance's KYC, ranged from US$180 to US$200. On an even high end of the scale, “unlocking” a Binance account costs between US$400 and US$600. During our monitoring period, we observed four individuals willing to pay for Onfido bypass services, two for Sumsub, and one inquiry about bypassing Sumsub that went unanswered. Screenshots of bypass services and tutorial sellers are shown in Figure 3.

We have also learned that the knowledge being spread in underground forums is not always feasible. For example, a user taught other users how to use OBS Studio along with deepfake apps such as DeepFaceLab, FaceSwap, or Zao. However, DeepFaceLab and FaceSwap recommends using GPUs to train models offline with video clips of the faker and the faked, making them unfeasible for real-time online verification. Meanwhile, Zao was designed to swap faces found in clips from famous TV shows and thus is not useful for eKYC bypassing techniques.

Figure 2. Malicious actor sells bypassing services (top), with another malicious actor selling a tutorial (bottom).

A deepfake-based attack requires a criminal either have a machine with a compatible GPU and some seasoned IT skills, pay for an online deepfake service, or purchase the service of a Bypass-as-a-Service vendor from an underground source. Another cheaper alternative Face fraud factories, which sell photos and videos of real people who are willing to sell their ID cards for small sums of money (enabling criminals to bypass selfie checks for online services), offer a cheaper alternative if no specific identity needs to be impersonated.

Deepfakes bypassing eKYC

To determine whether the deepfakes being peddled in underground forums are as good as claimed, we conducted a detailed study on the methods hackers use to perform their attacks and tested these techniques against chosen eKYC providers. Our goal was to understand the effectiveness of current eKYC defenses two years after Sensity’s report.

Deepfake is nearly a fixed procedure, with each component able to be implemented with different technologies, as shown in Figure 4.

Figure 3. Procedure to bypass eKYC

Figure 4. People asking how to bypass KYC on Reddit

With easier access to GPUs or virtual GPUs on the cloud, deepfakes have become so affordable that there are even users on sites like Reddit requesting help for KYC bypassing (Figure 5). One common method of showing faked faces in front of a virtual camera involves the use of third-party Android firmware to offload the deepfake processing to a faster computer with a GPU, allowing for the creation of high-quality deepfake videos. Another method involves using Android Cloud services that support remote cameras — for example, Leidianyun and Genymotion are legitimate services that can be exploited for such purposes. Note that while cloud services enable hackers to run deepfake applications remotely, in practice, however, bandwidth can be a limitation.

Interestingly, we have not observed any hacking attempts targeting the local AI models used in eKYC systems. This is likely because most providers still send the data to their backend servers for further computation, making it unnecessary to bypass the local models. We anticipate that we will see such attacks when providers make more use of local models.

To assess the effectiveness of deepfakes against eKYC systems, we decided to test the Deepfake Offensive Toolkit (DOT) developed by Sensity on a laptop with an NVIDIA MX550 GPU (2GB VRAM, hence not very powerful). After making some minor tweaks to the toolkit, we found that it produced outputs that, while not perfect, were sufficiently persuasive for our tests. In addition to DOT, we also tested a Deep-Live-Cam, which uses another machine learning model to generate deepfake images. Both DOT and Deep-Live-Cam achieved very good results.

We scanned our own passports and copied the faces from the passport to create a deepfake of another researcher. Since the face is directly taken from the passport, the resolution can be as low as 200x260 pixels with official stamps on it. However, our tests revealed that the deepfake could successfully bypass Veriff, a provider that does liveliness checks known for requiring verification exclusively on cellphones. Despite these measures, we managed to pass the verification process by streaming our deepfaked faces onto an Android Cloud service, as shown in Figure 6.

Figure 5. Veriff accepting the deepfaked faces

Next, we tested the deepfake against IDScan. This proved to be more challenging, as they require high resolution webcams, preventing the use of the de-facto virtual camera (a software-based camera simulating a real camera) in Linux, a kernel module called v4l2loopback. However, we found that akvcam (another Linux-based virtual camera) worked perfectly.

IDScan uses several local ONNX models and a realFaceMode that performs additional local checks before uploading the images to their backend servers. We did not find an exposed entry point to the webpack from the source code of their frontend, preventing us from changing the camera settings with the JavaScript debugger. Despite these security measures, IDScan does not perform liveliness checks, allowing us to upload spoofed pictures to achieve good results, as seen in Figure 7.

Figure 6. Bypassing IDScan using the Deepfake Offensive Kit

According to IDScan’s API Manual, passing the test requires an antiSpoofingFaceImageConfidence score of 70 or higher and a faceMatchConfidence score of 70 or higher. In our tests, we achieved an antiSpoofingFaceImageConfidence score of 99 and a faceMatchConfidence score of 87. While we could not bypass the system in a single attempt, a few tries yielded satisfactory scores, demonstrating the feasibility of deepfake attacks.

Our study confirms that deepfake poses a significant threat to eKYC. Despite various security measures, the verification processes of prominent eKYC providers can still be bypassed using off-the-shelf deepfake techniques. These findings also confirmed the feasibility of underground malicious actors and showed that the landscape is still worrying two years after Sensity’s test report.

Mitigations and best practice suggestions

An overall assessment of the cybersecurity risks brought about by bypassed KYC systems is shown in Table 1.

Table 1. Cybersecurity risk assessment of bypassed KYC systems

Our research on deepfakes being used against eKYC services reveals significant risks that the industry needs to address. Several measures can be immediately implemented to enhance the security posture and protect against ordinary faking attempts.

Organizations can implement several techniques to enhance their defenses against spoofing attacks. For example, scanning a document multiple times at real-time can help detect the high amounts of consistency that indicates spoofing attempts. If all scans are identical, it is likely that the document is not being held by hand. This technique is already adopted by a few KYC providers that we tested. Furthermore, although passive liveliness checks can be circumvented, they still raise the bar for basic security and can deter less sophisticated attackers.

Adjusting webcam resolution by changing MediaTrackConstraints is another effective strategy for defending against deepfake-based attacks. Many virtual camera implementations lack the capability of changing resolutions on the fly, so switching between 640x480 and 1280x860 can prevent their use in several circumstances.

Finally, we recommend that eKYC providers hire a penetration testing service to evaluate their eKYC systems. These testers can identify weak spots and provide insights into potential improvements, although the specific testing service will depend on the provider’s requirements.

Conclusion

Ransomware double extortion models and repetitive data breaches have led to a massive exposure of ID cards, driver’s licenses and passports, fueling identity theft on a global scale. The advancement of AI technology further lowers the cost and complexity of generating deepfakes that can be used for malicious purposes. We have observed the use of deepfakes in creating non-consensual pornography, disinformation, targeted impersonation schemes like virtual kidnapping and business email compromise (BEC), and KYC bypassing in the past few years.

Underground markets have capitalized on this trend by offering bypass-as-a-service, with prices ranging from US$30 to US$600 USD, lowering the barrier for entry significantly. We believe this trend is going to hurt financial institutions and online businesses if enhanced countermeasures are not taken.

Our tests confirmed that open-source tools like the Deepfake Offensive Toolkit (DoT) and Deep-Live-Cam are easy to obtain, user-friendly and have proven to be effective against at least two major KYC providers. We believe that the current AI-based automatic KYC is broken, in many cases, when facing cybercriminals. We suggest that organizations implement a mandatory check of commercially available fraud databases and the use of deepfake-detection models to protect their verification processes and safeguard against the growing menace of deepfake technologies.