Artificial Intelligence (AI)
Capacity is Critical in Riskier Threat Landscape
International cooperation, reporting, and capacity building are critical to enhance cybersecurity defenses. Effective governance in an increasingly risky landscape requires visibility as well as coordinated vulnerability disclosure.
Save to Folio
The Pall Mall Process was launched Feb 2024 by the UK and France, building multistakeholder dialogue on the proliferation of Commercial Cyber-Intrusion Capabilities (CCICs). Work on this followed from other group initiatives including CyberTech Accord and the Paris Call for Trust and Security in Cyberspace, with from the Paris Peace Forum. Yet, spyware remains a growing concern, with increasing incidents reported globally. Combined with a growing rate of vulnerability discovery and increased fragmentation in disclosure reporting, policy makers are faced with the necessity to turn their commitments into action. On the eve of the Second Pall Mall Process Conference, it’s important to consider how the current state of the threat landscape informs the actions that will most impact the trade of CCICs.
The Pall Mall Process focus on spyware is understandable given the harms these malwares create – to individual privacy, human rights, and national security. In spite of their malingering demise, NSO Group malware Pegasus is still detected in the wild. The repercussions to its use are ongoing, with a US District Court finding them liable for breaking US and California laws when targeting WhatsApp servers. Predictions of this ruling chilling the spyware market sound much like the chilling sound from initial sanctioning of spyware vendors. Spyware still abounds, with Mexico and Italy each recently implicated in spying on their citizens. At its heart, spyware monetizes exploitation of target users’ devices to steal sensitive information. New Yorker magazine estimated the global trade of spyware in 2021 to be around $12B, and since then the spyware market has continued to adapt and there are no signs of it slowing down.
2024 exceeded expectations, for those hoping for a banner year of vulnerabilities and zero-days. While there’s no indication that AI is increasing the rate of discovery and exploitation, the technology is moving that way. On open-source software, Google researchers have demonstrated how LLMs lower the bar for vulnerability discovery. With agentic AI hooking tool use, such as reverse engineering and generative fuzzing tools, AI will be increasingly capable of taking this to closed software of greater and more interesting complexity. Packaging these into widely deployable exploits is still research in progress, often requiring detailed descriptions not available for zero-day exploit developers. The race to find and close vulnerabilities has long been one of the ongoing battles in cyber-security, pitting purple teams against the gray market trading spyware, exploits and vulnerabilities. Narrowly defining and restricting one class of software, e.g. spyware, will move the problem to where the problems begin – trading vulnerabilities and exploits.
While software exploit risks rise, our current threat intelligence ecosystem is not rising to meet the challenge. The primary authority of vulnerability severity and description is the US National Vulnerability Database (NVD), maintained by NIST. The NVD concedes they are not able to keep up with the rate of vulnerability discovery, with the analysis backlog a known and growing problem in cybersecurity. CISA maintains the Known Exploited Vulnerabilities (KEV) list, indicating what vulnerabilities – recorded in NVD – are being exploited in the wild. When vendors independently assess vulnerabilities in their software or through their own bug bounty programs in obscurity and then release patches indicating “all good,” much often remains vulnerable. This problem applies too to vendors assessing security of their new AI, indicating more urgency in promoting third-party flaw disclosure. While vendors benefit from these third-party coordinated disclosures by proactively improving their product, the market is sub-optimal for ethical vulnerability research and disclosure. Currently the market looks like an iterated prisoner’s dilemma where every nation has chosen to defect, to keep alive their own access to spyware, exploits and vulnerabilities.
Perhaps to fill some of this void, reporting and analysis is becoming more fragmented. Last week, Wiz announced an online vulnerability database for cloud vulnerabilities that really matter. When we look at emerging AI software, new reporting mechanisms have been created to deal with the changing attack surface from the incidents that occur to the novel risks created with new software capabilities. Circling back to the original focus, spyware too gets its own tracking, with the CyberPeace Institute collecting reports of spyware affecting civil society and Freedom House releasing a template for reporting spyware use. Clearly the level of current reporting and information sharing hasn’t satisfied the calls to do more, and we can view these as evolutions in the third-party evaluations of bug reporting. Nonetheless, the reporting mechanisms are proliferating alongside the vulnerabilities, and some coordination would be help. Important questions in security posture management – where do I source my vulnerabilities? how do I know what’s affecting my peers? - are harder to answer when reporting mechanisms fragment. Enter capacity building...
Many of the remedies suggested by The Pall Mall Process: Consultation on Good Practices Summary Report will go far towards addressing proliferation of CCICs, with perhaps exception of export controls. International cooperation to establish norms on use and trade of dangerous technology is required, and codifying responsible government use and procurement addresses parts of the CCIC lifecycle. Coordinated reporting fills most of the other parts of the CCIC lifecycle, not just reporting of spyware use and trade, but of vulnerabilities, exploits and incidents. Coordination among defenders, secure information sharing, takes down attackers. To get ahead in the race to discover and exploit flaws in software, defenders make use of responsible vulnerability disclosure to defend against flaws before zero day. This takes research, and requires a healthy market for coordinated disclosure, so research is driven away from the grey market of CCICs. Increased export controls can stifle this ethical red teaming, preventing responsible disclosure and proactive security. Also, with changing controls, the spyware market adapts and forms new corporations in new jurisdictions to circumvent these restrictions. The Pall Mall Process should steer into cooperation efforts, especially reporting.
More could be done to improve information sharing, and here is where capacity building is really needed. Incentivizing vendor-independent bug bounty programs which use a coordinated disclosure process would improve the market conditions that drive research out of the grey market. These programs need to be committed to working with vendors like HackerOne, BugCrowd and Trend Micro's Zero Day Initiative to fix bugs. Otherwise, vulnerabilities can be hoarded for later use in exploits, or caught and killed by vendors who don't intend to release a meaningful patch. Furthermore, funding vulnerability analysis programs is also needed to keep up with the increasing pace of vulnerability discovery. This year we need to empower defenders with AI tools to triage AI discovered bugs. Secure information sharing programs drive protections in advance of attacks, and more vendors could follow Microsoft’s Advanced Protection Program (MAPP) example in sharing vulnerability patch information. These programs could be enhanced with coordinated vulnerability analysis, providing defenders the tools they need to stay ahead. Finally, standardization and coordination on incident reporting would improve defenses with increased visibility into the threat landscape.
Vulnerabilities are where every CCIC starts, and capacity building for vulnerability disclosure and analysis gets to the heart of the problem. Incident coordination can also be improved. While it’s unrealistic to expect common governance across borders or industry sectors, more can be done to promote secure information sharing. Regulating is more challenging than building secure collaborative research programs and improving the cyber threat intelligence ecosystem will lift all defenders’ ships.
Governance begins with visibility; you cannot govern what you cannot see. From there policy should improve security first and foremost. Promoting coordinated vulnerability disclosure gets to the root of the spyware problem; vulnerabilities enable exploits that make spyware work. You must deal with the ecosystem, and not a single symptom, and make ethical disclosure of vulnerabilities the low bar which everyone must clear. With the increasing rate of discovery and decreasing ability for current systems to keep pace with this rate, support for vulnerability and incident reporting, analysis, and sharing structures is needed now.