NAME

ipup - ipdb - ipdb-update.sh — Tools for generating IP based Geo-blocking and Geo-routing tables in order to configure the system's firewall and/or routing facilities

SYNOPSIS

DESCRIPTION

In general, access control by the firewall is established by selectors that can be attributed to incoming and outgoing IP packets, like physical interfaces on which the packets are going, source and destination IP addresses, protocol types, port numbers, content types and content, etc., and routing is determined by destination IP addresses. The Geo-location would be just another selector, but this information is not carried explicitly with IP packets, however, it can be obtained using the IP address as a key for looking-up the location in an IP database. For example, the country to which a given IP address is delegated, can be obtained with the common Unix tool whois(1).

whois does an online look-up in the IP databases of the 5 Regional Internet Registries (AFRINIC, APNIC, ARIN, LACNIC, RIPENCC), and this is the most reliable way to obtain the country code for a given IP address, because the RIR's are the authorities for internet number delegations. Unfortunately, online database look-up is by far too slow for even thinking about being utilized on the firewall level, where IP packets need to be processed in a microsecond time scale. Therefore, a locally maintained IP Geo-location database is indispensable in the given respect. The System's own routing and filtering tables can be configured to do these tasks if there is a source of the appropriate data. The ipdbtools(1) are designed to provide this data and to assist managing and using it.

The three tools in the package are:

  ipup

A tool to utilize the IP Geo-location tables to look-up the country code belonging to an IP address or generate sorted lists of CIDR compatible IP address/masklen pairs per country code, formatted as raw CIDR ranges or ipfw(8) table construction directives.

  ipdb

A tool for consolidating the IP address ranges from the RIR delegation statistics files into binary sorted tables of IP ranges + country codes, suitable for direct utilization by the ipup look-up tool. IPv4 and IPv6 ranges are stored in separate files.

  ipdb-update.sh

A shell script to update the IP Geo-location tables by downloading the 5 RIR delegation statistics files from a Regional Internet Registry mirror, and invoking ipdb to generate the binary sorted tables. It is suitable for invocation by cron.

Setting up the local IP Geo-location tables

The authoritative IP Geo-location information must be obtained from the 5 RIR's, and compiled into an optimized format, suitable for quickly looking-up the country codes of given IP addresses. This information is present in so called delegation statistics files on the ftp servers of each RIR, and APNIC, LACNIC and RIPENCC mirror the files of the other RIR's on their servers - as of the date of this writing, ARIN and AFRINIC do not mirror current delegation statistics of the other RIR's.

1) Choose one of the three useful mirror sites, depending on where you are located:

  ftp.ripe.net

RIPENCC -- Europe and Eurasia [default mirror]

  ftp.apnic.net

APNIC -- Asia Pacific

  ftp.lacnic.net

LACNIC -- Latin America and Caribbean

2) As user root execute the shell script ipdb-update.sh with the chosen mirror as the parameter, for example ftp.apnic.net:

# ipdb-update.sh ftp.apnic.net
>>>>
/usr/local/etc/ipdb/IPRanges/afrinic.md5 100% of 74 B 277 kBps 0s
/usr/local/etc/ipdb/IPRanges/afrinic.dat 100% of 397 kB 1330 kBps 0s
/usr/local/etc/ipdb/IPRanges/apnic.md5 100% of 73 B 264 kBps 0s
/usr/local/etc/ipdb/IPRanges/apnic.dat 100% of 4045 kB 1259 kBps 4s
/usr/local/etc/ipdb/IPRanges/arin.md5 100% of 67 B 246 kBps 0s
/usr/local/etc/ipdb/IPRanges/arin.dat 100% of 8160 kB 1270 kBps 7s
/usr/local/etc/ipdb/IPRanges/lacnic.md5 100% of 74 B 274 kBps 0s
/usr/local/etc/ipdb/IPRanges/lacnic.dat 100% of 1870 kB 1271 kBps 2s
/usr/local/etc/ipdb/IPRanges/ripencc.md5 100% of 74 B 270 kBps 0s
/usr/local/etc/ipdb/IPRanges/ripencc.dat 100% of 10 MB 1258 kBps 9s
ipdb v1.1.2 (128), Copyright (C) 2016-2018 Dr. Rolf Jansen
Processing RIR data files ...

afrinic.dat apnic.dat arin.dat lacnic.dat ripencc.dat

Number of processed IP-Ranges = 113267

As shown above, this will download the delegation statistics data together with MD5 hashes for integrity checking into the directory /usr/local/etc/ipdb/IPRanges/. Then the ipdb tool will process the data files and generate two binary sorted table (.bst) files, one for the IPv4 ranges /usr/local/etc/IPRanges/ipcc.bst.v4 and another one for the IPv6 ranges /usr/local/etc/IPRanges/ipcc.bst.v6.

USAGE AND OPTIONS

Quering the local IP Geo-location tables

Use the ipup tool for the various queries:

-h

Show the usage instructions.

[-r bstfiles]

Base path to the binary sorted tables (.v4 and .v6) with the consolidated IP ranges which were generated by the ipdb tool [default: /usr/local/etc/ipdb/IPRanges/ipcc.bst].

First usage form -- CC query:

 

⟨IP_address⟩

IPv4 or IPv6 address for which the country code should be looked-up.

Second usage form -- firewall and routing table generation:

 

-t CC:DD:.. | CC=nnnnn:DD=mmmmm:.. | CC:DD=ooooo:EE;.. | ""

Output all IP address/masklen pairs belonging to the listed countries, given by 2 letter capital country codes, separated by colon. An empty CC list (denoted by "") means any country code. A table value can be assigned per country code in the following manner:
  -t BR=10000:DE=10100:US:CA:AU=10200.
In the case of no assignment, no value [0] or the global value defined by either the -v or the -x option is utilized.

[-n table_number]

The ipfw table number between 0 and 65534 [default: 0].

[-v table_value]

A global 32-bit unsigned value for all ipfw table entries [default: no value -> 0].

[-x offset]

Decimal encode the given CC and add it to the offset for computing the table value:
value = offset + ((C1 - 'A')*26 + (C2 - 'A'))*10.

[-p]

Plain IP table generation, i.e. without ipfw table construction directives, and any -n, -v and -x flags are ignored in this mode.

[-4]

Process only the IPv4 address ranges.

[-6]

Process only the IPv6 address ranges.

Third usage form -- compute the encoded value of a country code:

 

-q CC

The country code to be encoded (see -x flag above).

EXAMPLES

Check whether the IP Geo-location tables are ready by looking-up some addresses using the ipup tool:

$ ipup 62.175.157.33
   62.175.157.33 in 62.174.0.0 - 62.175.255.255 in ES

$ ipup 141.33.17.2
   141.33.17.2 in 141.12.0.0 - 141.80.255.255 in DE

$ ipup 99.67.80.80
   99.67.80.80 in 98.160.0.0 - 99.191.255.255 in US

$ ipup 192.168.1.1
   192.168.1.1 not found

$ ipup 2001:0618:85a3:08d3:1319:8a2e:0370:7344
   2001:0618:85a3:08d3:1319:8a2e:0370:7344 in 2001:618:0:0:0:0:0:0 - 2001:618:ffff:ffff:ffff:ffff:ffff:ffff in CH

Firewall Examples

ipup can be used for Geo-blocking together with ipfw(8). For this purpose, ipup would generate tables of CIDR ranges for the selected country codes, and these tables can be directly piped into ipfw(8). The respective configuration script may contain something like:

...
# Allow only web access from DE, BR, US:
/usr/local/bin/ipup -t DE:BR:US -n 7 | /sbin/ipfw -q /dev/stdin
/sbin/ipfw -q add 70 deny tcp from not table\(7\) to any 80,443 in recv em0 setup
...

OR vice versa:

...
# Deny web access from certain countries we don't like this week:
/usr/local/bin/ipup -t TR:SA:RU:GB -n 66 | /sbin/ipfw -q /dev/stdin
/sbin/ipfw -q add 70 allow tcp from not table\(66\) to any 80,443 in recv em0 setup
...

In the case of a different firewall facility, a plain table (without ipfw directives) can be generated using ipup by specifying the -p flag. The table may be piped into a pre-processing command before being passed to the firewall utility:

# Output data in the format of some other fictional firewall:
/usr/local/bin/ipup -t FR:ES:PT -x0 | awk '{print "add-filter", $4, $5}'

OR

/usr/local/bin/ipup -p -t US:CA | while read TABLE NUM ADD ADDR VAL; do myfirewall add filter $ADDR value $VAL; done

Routing Example

ipup is well suited for manipulating the system's routing table by the way of the route(8) utility:
...
# Force packets to Austria to take a different route:
/usr/local/bin/ipup -p -t AT | while read LINE; do /sbin/route add $LINE $SOMEROUTER; done
...

Cronjob for keeping the IP Geo-location tables updated

ipdb-update.sh may be executed by a weekly (perhaps daily) cronjob, for this you might want to add the following entry to /etc/crontab:

...
# Weekly update of the IP Geo-location tables
  5 4 * * 6 root /usr/local/bin/ipdb-update.sh ftp.apnic.net > /dev/null 2>&1 && /fullpath/to/fw_or_router_reinit_script
...

FILES

/usr/local/etc/IPRanges/

directory for maintaining the IP Geo-location tables

/usr/local/etc/IPRanges/ipcc.bst.v4

binary (uint32_t) sorted table of IPv4 ranges and its country codes

/usr/local/etc/IPRanges/ipcc.bst.v6

binary (uint128t) sorted table of IPv6 ranges and its country codes

SEE ALSO

whois(1), ipfw(8), route(8)

in Ports: ip2cc(1), IP::Country(3)

AUTHOR

Dr. Rolf Jansen - Copyright (c) 2016 - all rights reserved.

IMPORTANT NOTE

Improper use of the ipdb tools may result in erroneous IP tables, and firewalls or routers may be rendered non-functional once configured with incorrect tables.

In NO event shall the author and/or copyright owner be liable for ANY damages resulting from ANY use of this software. Use the ipdb tools at your own risk!

BUGS

The ipdb tools have been carefully developed and tested. Anyway, the tools are provided without any expressed or implied warrantee of being 100 % bug free.