#gramscidevoted, Final project for BIG DIVE 2 A project by: Giuseppe Futia, Enrico Zimuel, Rocco Corriero This dataset contains data collected by Neubot, a research project on network neutrality. The group visualized the network neutrality, aggregated by countries, providers and Neubot instances. The output is a view of the world where you can explore the neutrality level estimation. Link to Project: http://gramscidevoted.bigdive.eu/ Link to Video Presentation: http://www.youtube.com/watch?v=h_W7zVZXOEg Link to BigDive page: http://www.bigdive.eu/final-projects/

1. Diving into Neubot dataDiving into Neubot data
Team “Gramsci devoted”:
Rocco Corriero, Giuseppe Futia, Enrico Zimuel

2. Team
Rocco Corriero
Data visualizazion using D3.js
Giuseppe Futia
Data visualization using D3.js
Enrico Zimuel
Data analysis (Python + MongoDB)

3. What is network
neutrality?
Network neutrality (NN) is the principle
that the Internet should be neutral with
respect to kinds of applications, senders
and destinations. In other words, a
network is neutral
when ROUTERS forward packets using
a first come, first served strategy.

4. What is Neubot?
Neubot (the network neutrality bot) is
a free software Internet bot,
developed and maintained by the
Nexa Center for Internet and Society,
that gathers network performance
data useful to investigate network
neutrality

5. Dataset
CSV files based on month measurements,
about 110 Mb, 450k entries, contains the
following fields:
"client_address", "client_country", "client_provider",
"uuid", "day", "connect_time“, "download_speed“,
"neubot_version“, "platform“, "remote_address“,
"test_name“, "timestamp“, "upload_speed", "city”,
"asnum", "region", ", "hour", "month", "year", "weekday“

6. How to estimate the NN?
We used the following formula:
NN = 1 - | ST-BT | / ST
where ST is the Speed Test (bytes/sec) and BT
is the BitTorrent Test (bytes/sec)

7. Data analysis
Filter +
geodata
Network
Neutrality
Aggregate
by country
geodata
city and
country
NN
Filtered
data
neubot.csv neubot.json
MongoDB
Python Python Python
MongoDB MongoDB

8. Performance
Filtering, geodata, store in MongoDB (neubot.py)
450K rows in 3 minutes
Network Neutrality (neutrality.py)
450K rows in 8 minutes
Aggregate by country (aggregate_by_country.py)
6204 IPs in less than a minute
We used the INDEX feature of MongoDB!

9. Aggregate by country
We used the weighetd mean to estimate the NN of
a collections of IP:
where Wi
is the number of clients (same IP), and Xi
is the NN

10. Visualization

11. Viz aggregation

12. Viz aggregation
Represent the Network Neutrality with the
aggregation based on :
Country
Client_provider
Neubot instances

13. DEMO

14. Thanks!
Source code of the project:
https://github.com/ezimuel/BigDive2Gramsci
Neubot project:
http://www.neubot.org/