Kong API Gateway Misconfigurations: An API Gateway Security Case Study

by Alfredo Oliveira and David Fiser

In this article, we continue our journey through the security issues of the API Gateway landscape. Our new research focuses on another popular API gateway — Kong.

The API gateway consists of three main components: the gateway itself, the database, and the administration API. Kong allows running an API gateway without a database, which means that the configuration is directly accessed from memory. However, running Kong without a database prohibits the use of certain features.

The Kong API Gateway is available in two versions, namely community and enterprise.
Kong’s default database engine is PostgreSQL, and in its earlier version, Cassandra. Additional data stores, such as Redis, can be used for caching using community plug-ins. Security-wise, the community version lacks encryption and vault support compared to the enterprise version.

Figure 1. API gateway architecture example

Like the APISIX, the Kong API Gateway is also built on NGINX.

Figure 2. API request chain within Kong deployment

We highlight the architecture here to emphasize the fact that the system is only as secure as its weakest part. Any vulnerability or misconfiguration in any of these components could lead to information disclosure, API gateway compromise, back-end compromise, or a supply-chain attack, putting the whole organization at risk.

The Kong API Gateway allows deployment in multiple environments. Each deployment affects the security based on its application components and configuration. This is applicable regardless of where the API gateway is deployed — whether on-premises, entirely in the cloud, or in a hybrid cloud. Users must ensure that only authorized entities can access crucial security components — the Administration API and the database.

Securing the Kong API Gateway administration API must be a top priority as access to this component allows users to read gateway configuration in plain text and modify the configuration in various ways, such as adding routes or tampering with the authorization mechanism. Exposing the administration API to the public or even within a local network puts back-end services at risk.

Unauthorized visitors can usually access protected routes, and information obtained from the gateway configuration could be abused. For instance, malicious actors could harvest user credentials, which can be used and reused to gain access to services. Such leakage allows threat actors to impersonate API gateway users successfully.

Unfortunately, we can still observe some publicly exposed Kong administration API instances that are open to the internet, as shown by the Shodan search engine.

 Figure 3. Global distribution of publicly exposed Kong API gateways

The first indexations started in late 2021, and from then we have observed a rising trend in the number of exposed gateways.

Figure 4. Publicly exposed Kong API gateways arranged by time

It should be noted that not all exposed gateways necessarily represent actual Kong API deployment, as some might be honeypots. Following our analysis, we noticed an unsafe deployment pattern consisting of multiple misconfigurations used within the cloud deployments.

Figure 5. Example of exposed admin API showing OpenID Connect configuration

Figure 6. Multiple misconfiguration pattern – API gateway pattern

The Kong API Gateway runs inside a container with the administration API listening on port 8001 or 8443 for SSL. This port is then forwarded to the host, or the container shares the same network. Lastly, the absence of firewall rules on cloud instances leaves an open door for everyone.

In the following sections, we look further into common misconfigurations.

Misconfiguration No. 1: Forwarding the Admin API

In more secure deployments, the admin API should be assigned to a localhost only, not forwarding administration ports nor sharing the network with the host.

Gateway administrators can set a new route as mitigation, requiring credentials for access. The route can point to the administration API, making it accessible outside a container. Similarly, with the enterprise version of API gateway, role-based access control (RBAC) that requires token authentication can be used.

Analyzing this misconfiguration, we can see that we can easily follow this scenario by using default configurations or examples found within container image repositories.

The Kong API Gateway listens on the local network interface only by default. However, the example listed on the Docker Hub image repository (seen in Figure 7) listens on all network interfaces, overriding the default protection and using weak passwords bypassing environment variables.

We can see the danger of copy-pasting without additional security thinking. We also note settings posing security risks when a database endpoint becomes available due to misconfiguration or adjacent system compromise. This provides a simple avenue to access confidential data, similar to scenarios where no credentials were used.

Kong's documentation provides detailed instructions on securing the Admin API.

Figure 7. Docker hub official image example exposing port admin 8001 and 8444

Misconfiguration No. 2: Missing firewall rules and exposing the whole computing instance to the public

Imagine an IP address as a door. Having a public door means that anyone — even intruders — can have access to it at any time. It goes without saying that to promote safety, steps such as locking the door and providing keys to only trusted people are a must.

Similarly, make sure that only authorized entities can access your cloud instance. Limiting access to exposed ports only for specific IP addresses or subnets, together with an authentication mechanism, helps mitigate this scenario.

In addition, take extra care in deployments where the applications are accessed with persons using dynamic IP addresses. Instead, use additional access vectors such as virtual private networks (VPNs).

API gateway security concern: Confidential information/secrets storage issues

Usually, accessing protected resources such as administration planes, API back ends, and serverless endpoints requires an authentication mechanism.

One of the API gateway's use cases is simplifying the authentication using an organizational identity provider issuing valid tokens accepted by the API gateway, which then forwards a request to the protected resource using a pre-saved secret. The configuration affects the way the secrets are stored, as they might be saved in the memory of the deployed application or inside the database.

Securing these secrets is thus crucial in overall system security, and there are several steps that can help with this. First, the data store should be accessible only from the API gateway. Credentials for accessing the database shouldn't be easy to guess or copied from default configurations and examples. Gateway administrators should apply TLS as protection to prevent network sniffing in on-premises deployments.

In scenarios where it is unnecessary to forward a secret — for instance, when configuring API key or token access on gateway routes — confidential information shouldn't be stored in only plain text or encrypted plain text, as this makes it possible to recover the secret upon leakage. Instead, hashing mechanisms and salting should be applied, making it almost impossible to recover the original secret upon database leakage. Encryption or external vault storage should be used in scenarios when it is not applicable.

Kong API Gateway stores every secret in the database as plain text by default. Specific plug-ins support additional encryption only when using the enterprise version with a configured keyring. During our research, we discovered that only specific plug-ins support the encryption of sensitive information, and users should be extra careful, especially when using third-party plug-ins.

Figure 8. Example of API key for Azure Functions saved in plain-text database

Encryption support can be verified by checking the encrypted parameter within the plug-in's source code if the source code is available or manually by querying the database when encryption is used.

Figure 9. Encryption and vault support inside Kong API gateway plugins

The encryption is available only in the enterprise version of the Kong API Gateway, leaving the community version secrets stored in plain text. The default key-auth plug-in does not support encryption even in the enterprise version, and the key-auth-sec plug-in should be used instead.  

Consequently, the API Key authentication plug-ins store the secret instead of hashing, which contradicts the basic-auth plug-in, in which hashing is implemented.

Cases where API keys are stored in plain text and used for authentication on the gateway only and not forwarded to the back ends should be classified as a vulnerability.

Delegating the storage of secrets to solutions specially designed for that purpose is the best practice. Vaults can provide the benefits of single secret storage using a safe mechanism, making rotating it easy.

Previously, Kong listed environment variables as a vault (as seen in Figure 10). After sharing our findings with them, this has been changed.

Figure 10. The environmental variables are a vault, though this has been changed.

However, it shows the misconception in the vault terminology presented within official documentation, as environmental variables do not comply with the vault definition. We explained why storing secrets within environmental variables is a bad practice in our previous article.

Figure 11. Updated documentation with correct vault definition

Similar to encryption support, not every plug-in supports a vault for storing sensitive information, as a referenceable argument must be set to true in the plug-in source code. Only the enterprise version of the API gateway supports external vaults.

Plug-ins provide additional functionality the community demands, often providing security challenges and bringing vulnerabilities. As most of the functionality of the Kong API gateway is provided by plug-ins, it makes it a serious concern in terms of security. In previous paragraphs, we described plug-ins that might miss encryption or vault support when required attributes are missing within the schema definition.

Figure 12. Example of missing vault and encryption support of key-auth plugin

Whenever a user inputs data or code, it should be considered a security risk, including plug-ins. For example, request- or response-modifying policies potentially lead to a denial of service (DoS) or worse, remote code execution (RCE).

Kong API Gateway also supports "serverless" code execution; in the Kong language, the administrator could configure the gateway to execute a custom Lua code upon processing a request using the serverless plug-in. The good news for security is the code is run inside a Lua sandbox by default.

However, the user can still bring an issue when the configuration is altered, the sandbox is disabled, or dangerous imports are allowed within the configuration file.

These misconfigurations might allow vulnerable or malicious execution on the Kong API Gateway. The threat actors will get complete control of the API gateway consequently, leaking all the secrets, eventually leading to a successful supply chain attack and putting the whole organization at risk.

Users should also be careful with an older gateway version that might be allowing sandbox escapes. We suggest turning off the functionality when not explicitly needed and using the up-to-date version of the Kong API Gateway.

Figure 13. Example of possible RCE when untrusted Lua is enabled

Figure 14. Example of unsafe Lua imports even when sandboxed

Accessing APIs and web applications in general are tightly linked with authentication and authorization. OAUTH 2.0 and Open ID Connect have become industry standards, and their support has been implemented in API gateways. Correct implementations of these flows are crucial elements of API security.

In that regard, Kong API Gateway supports issuing its tokens to third parties and using the third-party OAUTH 2.0 mechanism as an authorization mechanism for routed resources.

Security-wise, API gateway administrators should validate that a secure configuration is applied, avoiding redirection attacks and allowing only trusted identity providers (IdPs).

Failing in those elements might lead to successful user impersonation and unauthorized access, which could be hardly detectable. Issued tokens and authorization flow settings are saved within the database or in memory configuration; failing to secure the administration API or database could lead to the same consequences.

Conclusion

Technologies and their adoptions are double-edged swords; while they provide benefits, they also often inadvertently bring new security challenges. Tools that aggregate access into multiple different environments, such as API gateways, pose a security risk for all these environments upon breach.

Users should be aware of all security implications and product security features before choosing their solution, as security often comes at a cost and might not be perfectly implemented. Aggregating multiple secrets under a single user or namespace brings security risks, whether inside the API gateway or vault, and we should evaluate these risks by modeling threat scenarios to design more secure architecture and guidelines.