Geoff Huston

Detecting IP Address Filters

Geoff Huston
0 You have liked this article 0 times.

Until recently IP network operators were encouraged to set up so-called "bogon address filters" at the edge of their networks. These filters were intended to discard all incoming traffic where the source address in the IP header was from a block of addresses that was known to be unallocated. The inference was that a matching packet was either an unintentional leak from some privately addressed network domain or was generated using source address spoofing. In either case there is no point in delivering the packet, since it comes from a demonstrably fictitious source.

The initial use of these bogon filters was to mark those /8 address blocks in the IPv4 address space that were held by the IANA, although some tools were subsequently developed (and deployed in more limited contexts) to also mark as bogons address blocks held as unallocated by the Regional Address Registries (RIRs), and other tools were developed to mark address blocks that were consistently used to originate SPAM or malicious traffic, based on collected statistics and other heuristics.

A persistent issue with the use of these bogon filters is that they can lose currency. When addresses change from "unallocated" to "in-use" (such as an IANA allocation action, as happened in the past, or a RIR assignment or allocation of addresses to a Local Internet Registry) or when an address block already in use changes hands, then the rationale for maintaining the entry in a bogon filter almost certainly changes.

Another major issue with bogon filters is that, in general, they are not centrally managed. Many ISPs use local procedures to maintain their bogon filters. So when an address block changes hands, how can the new address holder work out where there are bogon filters that is listing the address block, and how can they get the filters changed?

Some of the Regional Internet Registries, notably the RIPE NCC, have been operating a " Debogon Project " where addresses from selected address blocks are advertised, and end users can test reachability into these address blocks via conventional use of ping and traceroute. Additional active tests can be carried out from the announced blocks to known test networks and devices, performing a test in the opposite direction.

The approach described here is somewhat different, in that it does not rely on the use of cooperating users, nor on the deployment of probes or other network edge devices. Instead, we have used online advertisement placement to co-opt end users to undertake basic reachability tests as a background test in their browsers. The remainder of the document describes the methodology used in these tests and the results.

Reachability Test Setup

The approach used here is to employ the Internet at large to perform the reachability tests. The basic method is to get the client's browser to fetch a number of a 1 pixel image files under the control of a script that was obtained from a known working IP address. All image files, except the final image, are sourced from domain names that map to IP addresses drawn from the address blocks being tested. The final image is sourced from the same address block as that from which the experiment was delivered from in the first place, so it acts as an end marker to indicate that the user has not interrupted the browser's actions mid way through the tests.

This test methodology can be achieved in a number of ways, including in javascript or in flash code embedded into a web page. However with such an approach, then the clients who undertake the reachability test are restricted to the set of clients who visit a web page that has the test included in the source of the page. As we were wanting to test a collection of clients from across the entire Internet, we were interested in a way to present these tests in a broader fashion than could be possible with a small number of appropriately equipped web sites.

The approach taken here is to use Google's Image Advertisements as the vehicle for presenting the reachability tests to end client browsers. Embedded in the ad material is a segment of flash code. The flash code is executed by the client's browser whenever the ad is delivered to the client, and the code will execute irrespective of whether the ad is "clicked" or not. The flash code fetches a control file from the experiment controller, using a control IP address. This control file includes a number of image URLs, which form the core of the experiment. An example of the control URL and the generated file is shown in Figure 1.



    Figure 1. Example of an Experiment Control File

In this example there are two address reachability tests being conducted ("rqaa" and "rqab"), and a control test ("rqac"). The parameters to the test include the identification of the test version, the time the test was conducted and a random number that allows us to recombine the individual experiments back into a single test set when performing analysis of the web server's log files. The random number in the domain name also prevents pre-cached experiments from being invoked, since every test-set has a different unique DNS name using this method.

The images are 1x1 pixel images, and are fetched asynchronously from page rendering, and additionally are not included in the displayed page. This is to minimize any additional delay on the page view from the testing, and largely takes place without any visible impact on the users browser. The same mechanism is employed by Google Analytics and other page view collection methods.

In this experiment three tests use three wildcard domain names, namely *, *, and * These domain names map to the following IP addresses, from within the associated network address blocks:

Test Address Address Prefix
rqac (control network)

The server environment was configured such that these test address prefixes were being advertised as reachable from this server, and the server itself was configured with secondary IP addresses that match the three rqa addresses.

The Apache web server of the system was configured with a wildcard virtual server host of the form *, so that all the images are served from the same server

The Apache web server logs were used for reachability analysis.

If a client successfully performs the entire test then the web logs will contain a sequence of records, an example of which is shown in Figure 2.

   - - [02/Jan/2012:00:01:38 +0000] "GET /crossdomain.xml
    HTTP/1.1" 200 726 "-" - - [02/Jan/2012:00:01:38 +0000] "GET /measureipv6.cgi?&hash=3103003790
    HTTP/1.1" 200 140
    "" - - [02/Jan/2012:00:01:39 +0000] "GET /crossdomain.xml
    HTTP/1.1" 200 726 "-" - - [02/Jan/2012:00:01:39 +0000] "GET /1x1.png?t2.u940792609.
    HTTP/1.1" 200 157
    "" - - [02/Jan/2012:00:01:40 +0000] "GET /crossdomain.xml
    HTTP/1.1" 200 726 "-" - - [02/Jan/2012:00:01:41 +0000] "GET /1x1.png?t2.u940792609.
    HTTP/1.1" 200 157
    "" - - [02/Jan/2012:00:01:41 +0000] "GET /crossdomain.xml
    HTTP/1.1" 200 726 "-" - - [02/Jan/2012:00:01:41 +0000] "GET /1x1.png?t2.u940792609.
    HTTP/1.1" 200 157

    Figure 2 - The Web Server's Log of a Reachability Test

Because the ad itself is served from a remote site, all fetches from the control server are in fact two fetches, the first for the crossdomain policy file to determine if fetches from this site are permitted, and the second being the data file itself.

In the example shown here the experiment generated 8 fetches. The first pair of fetches retrieved the list of URIs to fetch ( measurev6.cgi ). The next 3 pairs of fetches performed the reachability test itself.

What we are looking for are origin AS's where there is a systematic failure to retrieve the first, second or both of the first two images, yet a consistent ability to fetch from the control address.

Reachability Test Results

The test was run over 22 days. During that time 513,141 individual client IP addresses performed the reachability test. These IP addresses were sourced from 9,665 originating Autonomous Systems.

In order to eliminate potential problems with individual tests not completing (due to the client moving to a different webpage before the text was complete, for example), we have filtered out all originating AS's where there are only 1 or 2 individual tests performed. This left 507,121 tests, spanning 4,839 originating AS's.

3 AS's were unable to reach the "a" and "b" target (indicating some form of filter on both the and address blocks) These AS's were:

AS Number of Tests Country AS Description
23771 26 CN SXBCTV-AP SXBCTV ,Internet Service Provider
38283 3 CN CHINANET-SCIDC-AS-AP CHINANET SiChuan Telecom Internet Data Center

There were no origin AS where more than 2 individual experiments were able to reach the "b" target, but unable to reach the "a" target.

16 AS's were able to reach the "a" target, but unable to reach the "b" target (the prefix appears to be filtered). These AS's were:

AS Number of Tests Country AS Description
4835 10 CN CHINANET-IDC-SN China Telecom (Group)
6983 9 US ITCDELTA - ITC Deltacom
9051 17 LB IncoNet Data Management sal
14868 55 BR Companhia Paranaense de Energia - COPEL
19332 7 MX Marcatel Com, S.A. de C.V.
23252 3 CA IK - WTC Communications
25248 77 CZ BLUETONE-AS Ceske Radiokomunikace a.s.
28378 4 MX TV Rey de Occidente, S.A. de C.V.
28678 3 PL KOSMAN-EDU-AS Technical University of Koszalin
39566 12 PL TRUSTNET-PL-AS / hosting datacenter
41088 13 CZ CZNSYS N-SYS s.r.o.
43153 9 PL SFERANET-AS SferaNET Sp. z o.o.
44651 12 HU COMUNIQUE Com.unique Telekommunikacios Szolgaltato Kft.
49289 11 PL NITKA-NET ELPRO - Elektronika Profesjonalna Waldemar Nitka
197100 4 ES Esystel Servicios Multimedia, SL

If one were to assume that this is a representative sample of reachability by origin AS, then of the total population of 39,908 AS's some 66 AS's could be assumed to be filtering, and 12 AS's could be assumed to be filtering both and

Consideration of Experimental Results

The experiment raises a number of questions, which we will briefly discuss here.

The first question is, can these experimental results be verified by other means?

A simple means of validation of the result is by using ping, with the source address option. We were able to validate a small number of addresses chosen from random from the client set, that where the end systems were responsive to ping packets sources from the "control" address of, they were unresponsive to pings when using the tested addresses as a source address. It is noted that not all addresses are responsive to ping probes, due to firewalls and filters, so this by no means a precise form of validation.

    $ ping 194.126.xx.xx
    PING 194.126.xx.xx (194.126.xx.xx): 56 data bytes
    64 bytes from 194.126.xx.xx: icmp_seq=0 ttl=41 time=372.199 ms
    64 bytes from 194.126.xx.xx: icmp_seq=1 ttl=41 time=372.317 ms
    64 bytes from 194.126.xx.xx: icmp_seq=2 ttl=41 time=372.270 ms
    --- 194.126.xx.xx ping statistics ---
    3 packets transmitted, 3 packets received, 0.0% packet loss
    round-trip min/avg/max/stddev = 372.199/372.262/372.317/0.049 ms

    $ ping -S 194.126.xx.xx
    PING 194.126.xx.xx (194.126.xx.xx) from 56 data bytes
    --- 194.126.xx.xx ping statistics ---
    4 packets transmitted, 0 packets received, 100.0% packet loss

    Figure 3. Example of ping probes

The second question relates to the question of where the client may be positioned in the inter-AS routing space. In this case we are interested to establish whether there is a common AS path segment for a number of AS's that show evidence of a bogon filter, or whether there is no common path segment.

AS AS Path from Server to Client
Common Upstream Path AS Transit Path
23771 4608 1221 4637 4134 4837 18118 23771
38283 4608 1221 4637 4134 38283
45143 4608 1221 4637 7473 9506 45143
4835 4608 1221 4637 4134 4835
6983 4608 1221 4637 2828 6983
9051 4608 1221 4637 3356 42020 9051
14868 4608 1221 4637 3549 14868
19332 4608 1221 4637 174 19332
23252 4608 1221 4637 3356 23252
25248 4608 1221 4637 3549 5588 25248
28142 4608 1221 4637 3549 14868 28142
28378 4608 1221 4637 3356 19332 28378
28678 4608 1221 4637 3356 8501 28678
39566 4608 1221 4637 6453 24724 15694 39566
41088 4608 1221 4637 174 41088
43153 4608 1221 4637 3320 20804 43153
44651 4608 1221 4637 3356 8928 47169 44651
49289 4608 1221 4637 3549 5588 8246 49289
197100 4608 1221 4637 174 197100

There is no clear evidence that there is the systematic deployment of packet filters in the AS transit paths to these origin AS's.


There is no obvious evidence that either of the tested prefixes ( and are the subject of widespread bogon filtering in the Internet.

There was some observed evidence of filtering of these prefixes, and of a total of 507,450 tests, some 329 tests, or 0.065% of the test population, showed some evidence of filtering of these prefixes.

There is also some evidence that the prefix has a slightly greater level of filtering than the prefix). This could be due to the fact that was allocated to APNIC in November 2008, while was allocated to APNIC in February 2011. This may be the result of irregularly maintained bogon filters where the filter has not been updated to reflect the current IANA address allocation status.

The experimental technique of using advertisement placement to perform a large number of client-based reachability tests appears to be a relatively effective test methodology. It does not require investment in deployment of a large set of test units across the Internet, and because it uses a client-initiated test methodology it bypasses the limitations of NAT traversal which is associated with server-side initiated reachability tests. The ad placement approach also circumvents the common forms of packet filtering of ICMP, as all the test traffic sits within TCP port 80 transactions.

There is further work to be done to consider how representative this set of tested clients is in terms of the entire population of Internet clients, but as an initial exercise in showing how advertisement placement can be used as a useful mechanism to support some forms of network measurement, this has been a highly informative exercise.

0 You have liked this article 0 times.

You may also like

View more

About the author

Geoff Huston AM is the Chief Scientist at APNIC, where he undertakes research on topics associated with Internet infrastructure, IP technologies, and address distribution policies. From 1995 to 2005, Geoff was the Chief Internet Scientist at Telstra, where he provided a leading role in the construction and further development of Telstra's Internet service offerings, both in Australia and as part of Telstra's global operations. Prior to Telstra, Mr Huston worked at the Australian National University, where he led the initial construction of the Internet in Australia in the late 1980s as the Technical Manager of the Australian Academic and Research Network. He has authored a number of books dealing with IP technology, as well as numerous papers and columns. He was a member of the Internet Architecture Board from 1999 until 2005 and served as its Executive Director from 2001 to 2005. He is an active member of the Internet Engineering Task Force, where he currently chairs two Working Groups. He served on the Board of Trustees of the Internet Society from 1992 until 2001 and served a term as Chair of the Board in 1999. He has served on the Board of the Public Internet Registry and also on the Executive Council of APNIC. He chaired the Internet Engineering and Planning Group from 1992 until 2005.

Comments 1