Collaborative DDoS Mitigation: From Research to Operational Practice

This blog summarises our recent article in IEEE Communications Magazine, which can be read in full here.

Traditional DDoS mitigation

A common threat in the modern Internet is a so-called Distributed Denial of Service (DDoS) attack. In a DDoS attack, malicious actors send a flood of network traffic toward a victim, disrupting the legitimate traffic and services running on the target system. Network operators often employ DDoS mitigation solutions on-premises or outsource their mitigation to third-party providers.

Handling DDoS attacks this way has two downsides. First, network operators tackling DDoS attacks themselves must do so with their narrow view of the DDoS landscape and their knowledge of DDoS attacks, which may be difficult to keep current. In addition, handling DDoS attacks this way might lead to delays and mistakes because operations teams need to go through a learning curve for each new attack to understand its properties while being under pressure of finding the best way to mitigate the attack. Second, the outsourcing of DDoS mitigation has led to centralisation in DDoS protection services, with currently a handful of large providers, such as Akamai and Cloudflare, dominating the market, which results in a reduced knowledge of DDoS mitigation in other sectors and an increased dependency on such providers.

Collaborative DDoS mitigation

We envision an alternative approach; one where network operators collaboratively handle DDoS attacks and learn from each other. In such setups, network operators form (cross-sectoral) communities in which they share (meta)data and expertise on DDoS attacks. This allows them to increase the resilience of their networks because they learn of (new types of) DDoS attacks and mitigation procedures from their peers before the attacks hit them, which reduces the pressure on their incident response teams along with operational costs.

Our main motivation to start our research on this topic was a series of DDoS attacks that plagued the Dutch banking industry and government institutions in 2018, and the continuous torment caused by DDoS since then. If the targeted organisations had been able to exchange DDoS intelligence, subsequent targets could have better prepared their infrastructure and mitigation measures for these repeated attacks.

Previous research on collaborative approaches to DDoS mitigation mostly leaves non-technical implications of their contributions out of scope. We found that the organisational, legal, and governance frameworks are fundamental to this problem and must be considered in unison with the technical challenges. Hence, we approach the problem (and opportunity!) of collaborative DDoS mitigation as a multidisciplinary one. We provide a technical solution, as well as blueprints for a coalition governance model and legal agreements.

Our concept: Anti-DDoS Coalitions

The goal of our work is to address the limitations of traditional soloistic DDoS mitigation techniques. To accomplish this, we propose the concept of an Anti-DDoS Coalition (ADC): a group of network operators collaborating to proactively increase the resilience of their infrastructures and services against DDoS attacks. We identify three types of activities for members of an ADC (see Fig. 1):

• Fingerprinting DDoS attacks and sharing these “DDoS fingerprints” through a so-called “DDoS Clearing House” (DDoS-CH),
• Conducting realistic large-scale collaborative DDoS exercises to evaluate the resilience of ADC members against DDoS attacks, and
• Sharing knowledge around DDoS.

Legal and governance structures

An ADC needs several agreements to manage various “assets”, such as the DDoS fingerprints. Examples are data sharing agreements for sharing DDoS fingerprints through the DDoS Clearing House (more on that later), membership rules that network operators need to adhere to (e.g., regarding the confidentiality of data), and agreements to waive liability for potential damage caused in DDoS exercises.

One of the challenges in cross-operator collaborations is to establish mutual trust. With mutual trust, governance can be based on unanimous decision-making, but this may become ineffective as ADCs grow to dozens of members. As ADCs grow, personal trust among coalition members must shift to impersonal trust in the procedures and governance mechanisms of the coalition.

The Dutch ADC co-developed and used the templates for ADC governance and legal agreements, which are publicly available in the appendices 2-8 of the DDoS Clearing House Cookbook.

ADCs can be cross-sectoral communities of network operators, or sector-specific providers of critical infrastructure such as financial institutions.

Sharing DDoS fingerprints

Network operators in an ADC can generate fingerprints of the DDoS attacks targeted at them and share them with other network operators in the ADC. A DDoS fingerprint is a JSON file that summarizes the key features of a DDoS attack such as source IP addresses, source and destination ports, protocols and services, time, and duration of the attack, thus describing the different attack vectors that make up a DDoS attack. Fingerprints help ADC members learn of new (types of) DDoS attacks that have not hit them yet and evaluate whether their deployed defensive measures are suitable to handle these attacks. This allows network operators to widen their perspective on the DDoS landscape, which is important because DDoS attacks are very dynamic, and attackers continually change their attack vectors to bypass countermeasures. Also, in the case of sequential DDoS attacks, other ADC members receive an early warning for attacks that might hit them next.

Collaborative DDoS exercises

An ADC facilitates large-scale collaborative DDoS exercises, during which its members practice their resilience to DDoS attacks in realistic attack scenarios, such as volumetric attacks (Tbps-levels of traffic), and application-layer attacks.

In the Dutch anti-DDoS Coalition, such exercises are carried out twice a year, coordinated by the Dutch Tax and Customs Administration. The motto for the exercises is “test your anti-DDoS measures before others do it for you”. Often, at least one participant experiences real downtime during the exercises. To limit the impact on users, the exercises are carried out at night on the weekend. Providers of large network infrastructure capable of sending out such large volumes of traffic volunteer their services to act as a DDoS source to enable the exercises for the coalition.

After an exercise, participating network operators discuss results and improve their DDoS resilience measures to be better prepared for real attacks. For example, the Dutch Tax and Customs Administration has lately been targeted with DNS PRSD (Pseudo Random Sub Domain) DDoS attacks, which they could mitigate effectively because they had practiced this in one of the exercises.

Technical platforms

Here we provide an overview of the two technical systems that we designed for ADCs: the “DDoS Clearing House”, used to share DDoS fingerprints between network operators, and a “DDoS Cyber Range” for carrying out small-scale DDoS exercises. The technical details of both systems are described in the DDoS Clearing House Cookbook. All our software is open source and available on Github.

DDoS Clearing House

The DDoS Clearing House (DDoS-CH) is a platform through which ADC members share fingerprints of DDoS attacks that hit their infrastructure, as depicted in the schematic overview in Fig. 2.

The DDoS-CH consists of two core components: the Dissector and the DDoS Database (DDoS-DB). It also comes with four supplementary components, meant to enrich the services of the DDoS-CH, which we don’t discuss here for brevity. We designed the DDoS-CH to be a modular application, meaning organizations can choose which components they want to use or leave out of their setup.

The Dissector generates DDoS fingerprints by inspecting samples of DDoS network traffic. It looks for outliers in the traffic traces by inspecting various features such as source port and network protocol and summarises them in distinct attack vectors. DDoS fingerprints can be uploaded to an instance of DDoS-DB and/or MISP.

DDoS-DB is the repository that stores DDoS fingerprints. We designed DDoS-DB so that it can be run in a centralised or decentralised setup, with either one central database, or a network of linked databased hosted by each participating network operator (or a hybrid form with both). Here, DDoS fingerprints are shared and synchronised between coalition members.

DDoS Cyber range

The DDoS Cyber range enables network operators to run smaller-scale DDoS exercises at their own convenience, next to the cross-operator large-scale exercises organised by an ADC.

The Cyber Range is a web-based dashboard through which users can customise their attack traffic (e.g., by choosing from a few well-known predefined DDoS traffic patterns) and launch these attacks from a few Virtual Machines (VMs) distributed across the world on a cloud platform. The cyber range is limited to sending very small samples of DDoS attacks solely to the party requesting the traffic. This setup made the deployment less legally challenging, disposing of the need for waiver agreements.

From research to operations

We carried out the research and development of the applications and governance models in the European-funded CONCORDIA project, expanding our collaboration with Telecom Italia, FORTH, and University of Zurich. At the start of CONCORDIA, we found ourselves in the unique position to establish a continuous dialog between research (in CONCORDIA) and operations (the NL-ADC). We could therefore co-develop with and get feedback directly from our intended user group.

In a pilot of the technical systems with the Dutch anti-DDoS coalition we had coalition member NBIP, which runs a DDoS scrubbing service, generate DDoS fingerprints of over 260 DDoS attack. That allowed us to test the applications in a semi-operational setting and finetune the dissector further to make more accurate fingerprints.

Since March of 2024 the DDoS Clearing House runs in production at the Dutch Anti-DDoS Coalition and is available for its members for free!

Lessons learned and future work

In this trajectory of 6 years involving vision formation, research, software development, and practical collaboration, we have learned 3 important lessons:

1. More than a technical problem: Joint DDoS mitigation requires more than just a technically functioning DDoS clearinghouse. It is crucial to design governance structures and simple data sharing agreements simultaneously before and during the development of such a technical system. Like a collection of musical instruments, various aspects (technical, organisational, and legal) must collaborate to make the whole thing work. The instruments themselves do not form the orchestra.
2. Multidisciplinary teams: Joint DDoS mitigation is a problem grounded in multiple different disciplines (such as network management, privacy, and security operations) and requires corresponding teams. For example, we extensively involved legal experts in the Anti-DDoS coalition in the technical concepts of DDoS attacks and the sharing of fingerprints and their importance. Conversely, they could bring technical teams into legal or privacy considerations. With this mutual understanding, we were able to design the necessary data-sharing agreements to be simpler and less strict than before.
3. Start small and grow: By starting with a pilot of the DDoS clearinghouse with a small number of participants, we could quickly iterate over software versions and process new requirements from the NL-ADC rapidly. This built further trust within the coalition and allowed us to develop agreements for sharing DDoS fingerprints relatively quickly. Subsequently, it was a much smaller step toward a production-ready service.

Although the Clearing House is now deployed in production and CONCORDIA ended in May 2023, work on the DDoS clearinghouse within the NL-ADC continues. With the DDoS Clearing House, we will identify DDOS sources and perform trend analysis on fingerprints in the newly formed working group “intel and attribution” in the NL-ADC. We will also further improve the system technically based on feedback and use the generated DDoS fingerprints to develop attack scenarios for large-scale DDoS exercises that the NL-ADC organises annually.

Acknowledgements

We thank all members of the Dutch anti-DDoS coalition for making possible all activities and collaborations within the coalition. We also thank our CONCORDIA partners for their contribution to the technical system: Telecom Italia, FORTH and University of Zurich.

This blog only briefly summarises our long-running efforts on this topic and merely scratches the surface. You can find our IEEE Communications Magazine article here. For a comprehensive account of all (implementation) details, we refer to the DDoS Clearing House Cookbook, which we developed under the EU’s Horizon-2020 project CONCORDIA. Questions and feedback are always welcome at info@nomoreddos.org.