ARTEMIS: an Open-source Tool for Detecting BGP Prefix Hijacking in Real Time

General

ARTEMIS (Automatic and Real-Time dEtection and MItigation System) is an open-source tool that provides the following services to an operational network that deploys it:

1. Real-time monitoring of BGP updates, using BGP streaming services from the RIPE NCC's Routing Information System (RIS) (RIS live), RouteViews and CAIDA BMP feeds, as well as monitors that are deployed locally in the network that ARTEMIS protects (e.g., using exaBGP interfaces to the network's BGP border routers).
2. Accurate and comprehensive detection of BGP prefix hijacking attacks, within seconds from their initiation.
3. Flexible and automated mitigation of BGP prefix hijacking attacks, using practical mechanisms (such as prefix de-aggregation), within seconds to minutes from the initiation of the attacks.

Users can choose to enable only some of these services (each requiring the previous one to be enabled).

ARTEMIS contributes to a more secure Internet, since:

• It offers a network operator an easy-to-use open-source tool to detect and counter, in real-time, BGP hijacking attacks (e.g., sub-prefix, fake origin) against its own prefixes.
• It is complementary to RPKI. By working in concert, the two approaches can offer more complete proactive (RPKI) and reactive (ARTEMIS) protection against BGP prefix hijacking attacks.
• It surpasses the state of the art (i.e., third party detection services) in terms of detection speed, comprehensiveness, and accuracy, by leveraging both global (BGP monitors) and local (network operator contextual knowledge) information and scalable architectures for collecting and analysing incoming BGP feeds.

In the following, we introduce the architecture and logic of ARTEMIS and its main technical features, we illustrate examples of how to configure it, and we highlight the characteristics of the BGP prefix hijacks it detects. We then conclude this article with information about current users and future plans.

This RIPE Labs article is a follow-up to our previous one, which focused more on an overview of ARTEMIS as a defense approach, and elaborates on the details of the open-source tool that is currently available on GitHub. The ARTEMIS software is open-sourced under the BSD-3 license.

Basic Architecture & Logic

ARTEMIS is built as a multi-container docker application implementing the microservices shown in Fig. 1. It can run on top of any Linux server with minimal requirements. 

The basic philosophy behind the ARTEMIS software architecture is the use of a message bus (MBUS), routing messages (RPC, pub/sub, etc.) between different microservices within and across containers. The microservices interface with the MBUS using message producers and consumers.

The user (i.e. the network operator) fills in a configuration file and interacts with the system through a web application (UI). Through the UI, the administrators can control the monitoring, detection and mitigation microservices, monitor detected hijacks (including checking their current status), and mark events as resolved, false alert (i.e. the configuration file needs updating), etc. The configuration is imported in all microservices, since it is used for filtering the monitoring feed, tuning detection, and configuring mitigation and other functions.

The feed from the monitoring microservice (which can stem from multiple BGP monitoring sources around the world, including local monitors) is validated and transmitted to the detection and database access microservices. The detection microservice reasons about whether what it sees is a hijack and, if positive, generates a hijack entry that is stored in the DB together with the corresponding monitoring entries (i.e. full information for each BGP update that triggered the hijack alert). Finally, through the UI, the operator can instruct the mitigation microservice to mitigate a hijack or mark it as resolved/ignored; hijack events are also automatically marked as outdated, withdrawn or dormant depending on their status. All information (configuration, BGP updates, hijacks and microservice state) is persistently stored in the DB, which is accessed by the web application. Clock, listener/supervisor and observer microservices are auxiliary, and take care of periodic clock signaling, microservice status change events and configuration change notifications, respectively. For more information please check the GitHub Wiki.

Figure 1: ARTEMIS system overview: components and microservices (blue: back-end, purple: data storage and message routing, dark green: front-end, light green: CI/CD, grey: new auxiliary microservices).

Features

The ARTEMIS tool currently supports the following features:

• Real-time monitoring of changes in the BGP routes towards address space owned by the operator running ARTEMIS.
• Real-time detection and notifications (via the web UI, email, and custom loggers) of BGP prefix hijacking attacks/events of the following types (please refer to the attack taxonomy in our IEEE/ACM ToN paper):
  • Exact-prefix, type 0/1, any data plane manipulation
  • Sub-prefix, any type (0/1/-), any data plane manipulation
  • Squatting attacks, type 0 (others are non-applicable), any data plane manipulation
• Automatic or custom tagging of detected BGP hijack events (ongoing, resolved, ignored, under mitigation, withdrawn and outdated).
• Manually-controlled (with the push of a button) mitigation of BGP prefix hijacking attacks.
• Web-based User Interface (UI) with system information, statistics, BGP updates, BGP hijacks, etc. supporting both mobile and desktop environments.
• Configuration file editable by the operator (directly or via the web UI), containing information about: prefixes, ASNs, monitors and ARTEMIS rules (e.g., "ASX originates prefix P and advertises it to ASY").
• Support for both IPv4 and IPv6 prefixes.
• Modularity/extensibility by design.
• Support for Kubernetes setups.
• CI/CD.

For a comprehensive list of features that have been added during the course of development (and are still added to the tool upon new releases), please check the "Added" sections in this file. Examples are: replaying historical BGP updates, managing users, interfacing with Postgres database via GraphQL, etc.

Detected BGP Prefix Hijacks

ARTEMIS classifies potential hijacking events in four distinct dimensions:

Prefix dimension

How a hijacker manipulates a prefix. Hijacks can be:

• Sub-prefix (S): the attacker announces a sub-prefix of a configured super-prefix.
• Exact-prefix (E): the attacker announces a prefix that matches exactly a configured prefix.
• sQuatting (Q): the attacker announces a prefix that is not supposed to be seen publicly in BGP.

(AS-)Path dimension

How a hijacker manipulates the path to a prefix. Hijacks can be:

• Type-0 (0): the attacker announces a path with an illegal origin.
• Type-1 (1): the attacker announces a path with a legal origin, but illegal first hop.
• Type-N (N): the attacker fakes a link deep in the path (N=2: 2nd hop is illegal, N=3: 3rd hop is illegal, etc.).
• Type-U (U): the attacker does not change the path at all (can be combined with a sub-prefix hijack).

Currently, ARTEMIS issues '-' for Type-N/U attacks (not supported).

Data plane dimension

How an attacker manipulates the traffic leading to a prefix. Hijacks can be:

• Blackholing (B): the attacker drops packets en-route.
• Imposture (I): the attacker impersonates the services of a victim.
• Man-in-the-Middle (M): the attacker intercepts (and potentially alters) traffic en route.
  
  

Currently, ARTEMIS issues '-' for these types of attacks (as a pure control-plane tool).

Policy dimension

How an attacker manipulates BGP policies related to a prefix. Hijacks can be:

• Route leak due to no-export violation (L): the attacker announces a no-export route to another (non-monitor) AS.
  
  

Currently, besides 'L' (no-export violations), ARTEMIS issues '-' for other types of policy violations.

Combining dimensions

In general, any potential hijack can be represented as a combination of the aforementioned dimensions:
```
Prefix | Path | Data plane | Policy
```
ARTEMIS currently detects the following combinations:

• S|0|-|-: sub-prefix announced by illegal origin.
• S|1|-|-: sub-prefix announced by seemingly legal origin, but with an illegal first hop.
• S|-|-|-: not S|0|- or S|1|-, potential type-N or type-U hijack.
• E|0|-|-: exact prefix announced by illegal origin.
• E|1|-|-: exact prefix announced by seemingly legal origin, but with an illegal first hop.
• Q|0|-|-: squatting hijack (is always '0' on the path dimension since any origin is illegal).
• *|*|*|L: no-export policy violation.
• E|-|-|-: not a hijack.
  
  

For further information on BGP hijack states and actions we refer the reader to this Wiki page.

The ARTEMIS Configuration File

The file is composed of 4 sections (Prefixes, Monitors, ASNs, and Rules) and is written in YAML for easy parsing both by machines and humans. In the following examples, keywords are in bold fonts.

Prefixes

This section allows the user to declare references to prefixes of interest that can be used in the rest of the configuration file. Note that even if a prefix is declared here, it does not mean it will be monitored by ARTEMIS: only prefixes that are present in at least one rule (see Rules section) are actually monitored and checked for hijacks. Note that RFC2622 operators on prefix definitions are supported.

prefixes:
    # A reference for a single prefix
    simple_prefix: &my_prefix
        <IPv4|IPv6_prefix>
    # A reference for a list of prefixes
    simple_prefix_list: &my_prefixes
        - <IPv4|IPv6_prefix_1>
        - ...
        - <IPv4|IPv6_prefix_N>

Monitors

This section contains all monitors that will be used to provide feeds of BGP updates to the monitoring and detection components of ARTEMIS. Currently, the following feeds are supported:

monitors:
    riperis: ['']
    bgpstreamlive:
        - routeviews
        - ris
    betabmp: betabmp
    exabgp:
        - ip: <ip_of_exabgp_1>
          port: <port_1>
        - ...
        - ip: <ip_of_exabgp_N>
          port: <port_N>
    bgpstreamhist: <csv_dir_with_formatted_BGP_updates>

ASNs

In this section the user can declare ASNs as references to be used in the rules section. Individual ASN, lists, or ranges can be specified and referenced. Users can choose meaningful names for references in order to make their rules more readable and maintainable.

asns:
    my_asn: &my_asn
        1234
    my_asns: &my_asns
        - 321
        - 432
    my_neighbor: &my_neighbor
        222
    my_neighbors: &my_neighbors
        - 333
        - 444
    my_asn_range: &my_asn_range
        - 1234-1240

Rules

Rules are the "heart" of the ARTEMIS configuration logic, since they constitute the operator-supplied ground truth against which ARTEMIS checks incoming BGP updates to detect hijacks.

An example Rules section looks like this:

rules:
# First rule
- prefixes:
 - *my_ipv4_prefix_group
 - *my_ipv6_prefix_group
 origin_asns:
 - *my_asns
 neighbors:
 - *my_neighbors
 # E.g., assume you rely on DDoS mitigation that announces your attacked prefix to scrub it and re-tunnel it back to you
 - *bgp_based_ddos_mitigation_service_asn
 # E.g., a special service ASN (such as goBGP) you use for monitoring and operations
 - *special_asn
 mitigation: manual

# Second rule
- prefixes:
 - *my_special_prefixes_originated_by_others
 origin_asns:
 - *my_friend_asn
 neighbors:
 - *my_friend_asn_neighbors
 mitigation: manual

# More rules
- prefixes: 
[...]

Specifically, a rule is composed of the following subsections:

prefixes: the list of prefixes the rule applies to (they can be lists themselves). Example:

prefixes:
    - *my_prefix_1
    - ...
    - *my_prefix_N
    - *my_prefix_list

origin_asns: the list of origin ASNs of the networks that are allowed to advertise the aforementioned prefixes. Example:

origin_asns:
  - *my_asn_1
  - ...
  - *my_asn_N
  - *my_asn_list

neighbors: the list of the ASNs of the direct neighbors of the aforementioned networks (owning the origin ASN(s)) that are supposed to propagate routes towards the aforementioned prefixes. Example:

neighbors:
  - *neighbor_asn_1
  - ...
  - *neighbor_asn_N
  - *neighbor_asn_list

mitigation: the action that the user intends to be executed when instructing the mitigation of a hijack. By default it is set to "manual", which resorts to doing essentially nothing (ARTEMIS as a passive monitoring and detection tool). However, the user can also set here the location of a script that runs the desired code (e.g., using ExaBGP to fragment a prefix announcement).

 mitigation: manual

policies: the policies that apply to this rule. Currently we support only the no-export policy, which can be configured as follows:

policies:
   - 'no-export'

More fun with rules

The rule syntax is flexible enough (i) to reflect your network configuration when it varies depending on the prefix or (ii) to allow the user to specify different alert/mitigation options depending on the characteristics of the attack and on the victim prefix. We present some indicative examples below:

Exact prefix advertised by fake origin AS: "E|0|-|-"

• ARTEMIS rule:

- prefixes:
    - prefix_A
  origin_asns:
    - ASN_A
  neighbors:
    - ASN_B
  mitigation: manual 

• Example of BGP update triggering a hijack alert (with ASN_C the hijacker):
  

  [..., ASN_C]:prefix_A

• Example of legal BGP update:
  

  [..., ASN_B, ASN_A]:prefix_A

More specific (sub-)prefix advertised via fake first hop neighbor AS: "S|1|-|-"

• ARTEMIS rule:

- prefixes:
    - prefix_A
  origin_asns:
    - ASN_A
  neighbors:
    - ASN_B
  mitigation: manual

• Example of BGP update triggering a hijack alert (with ASN_C the hijacker):

  [..., ASN_C, ASN_A]:sub_prefix_A

• Example of legal BGP update:

  [..., ASN_B, ASN_A]:prefix_A 

Squatting: "S|0|-|-"

• ARTEMIS rule:

 - prefixes:
     - prefix_A
   mitigation: manual

• Example of BGP update triggering a hijack alert:

[...]:prefix_A

• Example of legal BGP update:

Non-Applicable

No-export policy violation for exact prefix by unknown party: "E|-|-|L"

• ARTEMIS rule:

- prefixes:
    - prefix_A
  origin_asns:
    - ASN_A
  neighbors:
    - ASN_B
  policies:
    - 'no-export'
  mitigation: manual

• Example of BGP update triggering the hijack:
  

  [..., ASN_D, ASN_B, ASN_A]:prefix_A

• Example of legal BGP update:

  [ASN_D, ASN_B, ASN_A]:prefix_A

  (Note: ASN_D is considered OK for export as the used RV/RIS/etc. monitor, so the latter 3-hop path does not trigger an alert by design.)

Web UI and Demo

Please check this page for web UI screenshots (login, registration, overview, BGP update page, BGP hijack page, system configuration, user management, rule learning, etc.). For a running demo of ARTEMIS based on the configuration of the FORTH research institute please check this page.

Current Users

Besides simulation, emulation and controlled real-world environments, we have tested ARTEMIS in the following real-world deployments:

1. AMS-IX, one of the biggest European Internet eXchange Points.
   
2. A major Greek ISP with hundreds of active peerings.
3. Internet2, the biggest R&E network in the US providing network services to thousands of R&E institutes.
4. FORTH, a stub dual-homed academic network serving one of the largest research institutes in Greece.

Other users (not mentioned here for anonymity reasons) have also deployed ARTEMIS in their own environments, either in production or in testing setups. Some quotes from users follow:

"ARTEMIS has proven to be a fast, accurate and very reliable solution to monitor our prefixes and identify hijacks in timely manner. After passing successfuly the testing period, we have started integrating it with the rest of our systems and consider it as an important piece of our infrastructure. The INSPIRE team proved to be a great team player in this project and provide us the necessary features and support when needed."

Stavros Konstantaras, NOC engineer at AMS-IX

"ARTEMIS has been a very informative and helpful tool in examining potential BGP misconfigurations. It provides more information, and the team is responsive and easy to work with."

Adair Thaxton, Cyberinfrastructure Security Engineer at Internet2

Future Plans

In the immediate future, we plan the following for the ARTEMIS open-source tool:

1. Test ARTEMIS in the two biggest European IXPs, i.e. AMS-IX and DE-CIX, as well as other R&E networks based in the US.
2. Continue maintaining and extending the open-source tool by adding new features. Specifically, we plan to add modules that perform verification of the impact of a hijack as seen on the data plane (e.g., via RIPE Atlas traceroutes towards the affected prefixes).
3. Be present at RIPE events, or other network operator meetings, and disseminate the open-source tool through mailing lists and social media.

Acknowledgements and Funding Sources

This work is supported by the following funding sources on the European side (FORTH):

• RIPE NCC Community Projects Fund
• European Research Council (ERC) grant agreement no. 790575 (PHILOS Project)
• European Research Council (ERC) grant agreement no. 338402 (NetVolution Project)

The following funding sources supported the collaboration with CAIDA, UC San Diego, on the US side:

• National Science Foundation (NSF) grants OAC-1848641 and CNS-1423659
• Department of Homeland Security (DHS) Science and Technology Directorate, Cyber Security Division (DHS S&T/CSD) via contract number HHSP233201600012C
• Comcast Innovation Fund

We would like to note that the RIPE NCC Community Projects Fund was the primary funding mechanism that enabled the development and evaluation of the ARTEMIS open-source tool. In particular, it helped us to:

• Test ARTEMIS on three real networks with different characteristics.
• Co-design the software with network operators (instead of isolated in a research lab), to better fit their needs and be useful in everyday network operations.
• Hire two full-time developers for the ARTEMIS software implementation (front-end and back-end, respectively), and one part-time supervisor/co-developer for the project.
• Travel to present the ARTEMIS project.
• Make the project visible among circles of network operators affiliated to the RIPE NCC.