Help to mine BGP data by labeling each BGP update message. See doc/taxonomy.md for the detail of labels.
In addition to BLT, we developed a simple anomaly detector to find significant routing anomalies.
Install py-radix
pip install py-radix
Install BGPStream Core. See official site (https://bgpstream.caida.org/docs/install/bgpstream) for more detail.
After installing dependency run below example command
python blt.py -v 4 -s 20170825 -e 20170826 -c route-views.linx -o /hoge/hoge.blt
Then you can get BLT file in which there are BGP update messages with labels based on the taxonomy (see doc/taxonomy.md for the detail) between 2017/08/25 00:00 and 2017/08/26 00:00.
This is just a text file, so you can grep for what you want to focus on (labels, prefixes, AS etc...) easily.
Detect significant routing anomalies easily using this anomaly detector based on BLT taxonomy.
On August 25th 2017 around 3:22 UTC, Google (AS15169) leaks over 150k routes for small prefixes that were presumably used for their internal traffic engineering. This kind of incident is called "Route-Leak". You can catch this incident using our anomaly-detector.
- Run BLT to get BGP messages and corresponding labels.
python bltReader.py -v 4 -s 20170825 -f 20170826 -c route-views.linx -o ~/test/20170825_GoogleLeak/20170825_GoogleLeak.blt
- Run convert_blt_to_pickle.py to get pickle file.
python convert_blt_to_pickle.py ~/test/20170825_GoogleLeak/20170825_GoogleLeak.blt
- Then you can draw the figure running below command.
python anomaly_detector.py ~/test/20170825_GoogleLeak/20170825_GoogleLeak.pkl
Second example is an outage in Syria that coincide with national examination in that country on June 1st 2017. Because this event is much smaller than that of Google or other big BGP event, we should focus-on the country or other unit of network. So, we prepared two grep tools. One can grep for the prefixes in the country, the other can grep for the AS. In this example, we grep for the prefixes in Syria.
- Run BLT to get BGP messages and corresponding labels.
python bltReader.py -v 4 -s 20170601 -f 20170601 -c route-views.linx -o ~/test/20170601_SyriaOutage/20170601_SyriaOutage.blt
- Grep for Syria using the grep_country.py.
python grep_country.py ~/test/20170601_SyriaOutage/20170601_SyriaOutage.blt SY > ~/test/20170601_SyriaOutage/20170601_SyriaOutage_SY.blt
- Run convert_blt_to_pickle.py to get pickle file.
python convert_blt_to_pickle.py ~/test/20170601_SyriaOutage/20170601_SyriaOutage_SY.blt
- Then you can draw the figure running below command.
python anomaly_detector.py ~/test/20170601_SyriaOutage/20170601_SyriaOutage_SY.pkl
