{
"cells": [
{
"cell_type": "markdown",
"id": "41485f29-7287-40d2-a87a-c5daeb84f731",
"metadata": {},
"source": [
"## Analyzing raw OONI data, a case study\n",
"\n",
"The goal of this notebook is to explain some of the common workflows that can be adopted when performing analysis of OONI data. This will be done within the context of a specific case study and will focus on the analysis of [Web Connectivity](https://github.com/ooni/spec/blob/master/nettests/ts-017-web-connectivity.md) data.\n",
"\n",
"We will be focusing on answering the following 2 research questions:\n",
"- What domains present signs of blocking in Russia between the 23rd of February and the 17th of March 2022?\n",
"- How does the blocking vary from ISP to ISP?\n",
"\n",
"It can be useful, before you dive into more extensive analysis, to get a sense for what you are likely to find in the data by using the [Measurement Aggregation Toolkit](https://explorer.ooni.org/experimental/mat). For example you can pick a certain country and plot the [anomalies with a per-domain breakdown](https://explorer.ooni.org/experimental/mat?probe_cc=RU&test_name=web_connectivity&category_code=GRP&since=2022-03-09&until=2022-04-09&axis_x=measurement_start_day&axis_y=domain) (it's often helpful to limit the domains to categories that are most relevant, so as to focus on interesting insight).\n",
"\n",
"In doing so, you will understand if there is something interesting to investigate in the country in question at all and will also help in identifying some examples of interesting sites that you might want to further investigate.\n",
"\n",
"It's also posisble to use the same API the MAT relies on, for downloading the anomaly,confirmed,failure,ok breakdowns to be used in your own analysis or plotting tooling. Depending on the type of analysis you need to do, this might be sufficient, however keep in mind that the anomaly flag is [suscpetible to false positives](https://ooni.org/support/faq/#why-do-false-positives-occur).\n",
"\n",
"It's also useful, while you are performing the analysis, to refer to OONI Explorer to inspect the measurements that present anomalies, so as to be able to identify patterns that you can use to further improve your detection heuristics.\n",
"\n",
"At a high level the workflow we are going to look at is the following:\n",
"\n",
"\n",
"\n",
"### Downloading the data\n",
"\n",
"Once you have gotten a feel for the data, it's time to download the raw dataset.\n",
"\n",
"We offer a tool called oonidata (that's currently in BETA and be sure you have at least v0.2.3), which can be installed by running:\n",
"```\n",
"pip install oonidata\n",
"```\n",
"\n",
"To download all OONI data for this example notebook, run the following command (you should have at least 38GB on disk):\n",
"```\n",
"oonidata sync --start-day 2022-02-23 --end-day 2022-03-17 --probe-cc RU --test-name web_connectivity --output-dir ooni-russia-data\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "a96ac23b-e9cd-482c-b598-ba70184eee58",
"metadata": {},
"outputs": [],
"source": [
"import pandas as pd\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"\n",
"from datetime import datetime, timedelta\n",
"from dateutil.parser import parse as parse_date\n",
"from urllib.parse import urlencode, quote, urlparse\n",
"\n",
"from tqdm import tqdm\n",
"tqdm.pandas()"
]
},
{
"cell_type": "markdown",
"id": "aff8b48e-786d-462f-91ed-881f995a9a5f",
"metadata": {},
"source": [
"### OONI Explorer utility functions\n",
"\n",
"Below are a couple of useful utility functions when dealing with measurements. They take a dataframe row and return (or print) the OONI Explorer URL. This is useful to get a link to OONI explorer to more easily inspect the raw measurement to better understand what is going on."
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "db0a313f-7410-409a-9059-a6e19bd157a9",
"metadata": {},
"outputs": [],
"source": [
"def get_explorer_url(e):\n",
" query = ''\n",
" if 'input' in e.keys() and e['input']:\n",
" query = '?input={}'.format(quote(e['input'], safe=''))\n",
" return 'https://explorer.ooni.org/measurement/{}{}'.format(e['report_id'], query)\n",
" \n",
"def print_explorer_url(e):\n",
" print(get_explorer_url(e))"
]
},
{
"cell_type": "markdown",
"id": "0a2c55b2-f304-4288-92ff-965991ffdea6",
"metadata": {},
"source": [
"### Extracting metadata from raw measurements\n",
"\n",
"The OONI raw data is very rich, but for most analysis use-cases you just need a subset of the fields or some value that is derived from them.\n",
"\n",
"Below are functions that will extract all the metadata we care about from the web_connectivity test."
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "b0ed1fd5-5338-43ec-ae9a-d3dd468a5e2f",
"metadata": {},
"outputs": [],
"source": [
"import requests\n",
"from base64 import b64decode\n",
"import hashlib\n",
"import json\n",
"import re\n",
"\n",
"def get_raw_measurement(row):\n",
" r = requests.get(\"https://api.ooni.io/api/v1/measurement_meta\", params={\n",
" 'report_id':row['report_id'],\n",
" 'input': row['input'],\n",
" 'full': True\n",
" })\n",
" j = r.json()\n",
" return json.loads(j['raw_measurement'])\n",
"\n",
"def get_resolved_ips(msmt):\n",
" queries = msmt['test_keys'].get('queries', [])\n",
" if not queries:\n",
" return ''\n",
" answers = queries[0].get('answers', [])\n",
" if not answers:\n",
" return []\n",
" \n",
" ip_list = []\n",
" for a in answers:\n",
" ip = a.get('ipv4', '')\n",
" if ip:\n",
" ip_list.append(ip)\n",
" return ip_list\n",
"\n",
"def get_control_failure(msmt):\n",
" if 'test_keys' not in msmt:\n",
" return 'missing_test_keys'\n",
" return msmt['test_keys']['control_failure']\n",
"\n",
"def get_test_keys_blocking(msmt):\n",
" return str(msmt['test_keys']['blocking'])\n",
"\n",
"def get_http_experiment_failure(msmt):\n",
" return str(msmt['test_keys']['http_experiment_failure'])\n",
"\n",
"def get_resolver_info(msmt):\n",
" return {\n",
" 'resolver_ip': msmt.get('resolver_ip', ''),\n",
" 'resolver_asn': msmt.get('resolver_asn', ''),\n",
" 'resolver_network_name': msmt.get('resolver_network_name', '')\n",
" }\n",
"\n",
"def get_network_events(msmt):\n",
" return msmt['test_keys'].get('network_events', [])\n",
"\n",
"def get_tcp_connect(msmt):\n",
" return msmt['test_keys'].get('tcp_connect', [])\n",
"\n",
"def decode_body(body):\n",
" if body is None:\n",
" return ''\n",
" if isinstance(body, dict):\n",
" raw_body = b64decode(body['data'])\n",
" try:\n",
" return raw_body.decode('utf-8')\n",
" except:\n",
" return raw_body\n",
" return body\n",
"\n",
"def get_last_response_body(msmt):\n",
" try:\n",
" # The requests/response list sorts them from the newest to the oldest, \n",
" # hence the first item in the list is the last response we received.\n",
" body = msmt['test_keys']['requests'][0]['response']['body']\n",
" return decode_body(body)\n",
" except (KeyError, TypeError, IndexError):\n",
" return ''\n",
"\n",
"TITLE_REGEXP = re.compile(\"
(.*?)\", re.IGNORECASE | re.DOTALL)\n",
"# Doesn't take into account ordering\n",
"META_TITLE_REGEXP = re.compile(\"= ts:\n",
" continue\n",
" if query.get('until') and parse_date(query['until']) <= ts:\n",
" continue\n",
" yield p\n",
" \n",
"def iter_raw_measurements(query):\n",
" path_list = list(iter_jsonl_paths(query))\n",
" print(f\"processing {len(path_list)}\")\n",
" for fp in tqdm(path_list):\n",
" for msmt in iter_msmts(fp):\n",
" if query.get('probe_asn') and msmt['probe_asn'] != query['probe_asn']:\n",
" continue\n",
" if query.get('domain'):\n",
" domain = urlparse(msmt['input']).netloc\n",
" if domain != query['domain']:\n",
" continue\n",
" yield msmt"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "6439fc37-2715-4cc1-9b50-a4754aebf955",
"metadata": {},
"outputs": [],
"source": [
"import csv\n",
"\n",
"def msmt_to_csv(query, output_file=\"output.csv\"):\n",
" with open(output_file, 'w') as output_file:\n",
" csv_writer = None\n",
" for msmt in iter_raw_measurements(query):\n",
" msmt_meta = get_measurement_meta(msmt)\n",
" if csv_writer is None:\n",
" fieldnames = msmt_meta.keys()\n",
" csv_writer = csv.DictWriter(output_file, fieldnames=fieldnames)\n",
" csv_writer.writeheader()\n",
" csv_writer.writerow(msmt_meta)"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "898e8ebf-bce6-4a7e-905a-0563460b539d",
"metadata": {},
"outputs": [],
"source": [
"def get_msmt_df(query):\n",
" msmt_list = []\n",
" for msmt in iter_raw_measurements(query):\n",
" mdf = pd.DataFrame([get_measurement_meta(msmt)])\n",
" msmt_list.append(mdf)\n",
" return pd.concat(msmt_list, ignore_index=True)"
]
},
{
"cell_type": "markdown",
"id": "5dbaf33a-5686-4c16-8c9c-380fa55f0302",
"metadata": {},
"source": [
"Here we do the actual conversion to CSV."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5a5546b9-d164-4f14-a115-ff14cfe675b4",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"processing 14234\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
" 94%|████████████████████████████████████████████████████████▌ | 13411/14234 [1:27:58<05:48, 2.36it/s]"
]
}
],
"source": [
"msmt_to_csv({\n",
" 'since': '2022-02-23',\n",
" 'until': '2022-03-17',\n",
" 'probe_cc': 'RU',\n",
" 'test_name': 'web_connectivity'\n",
"}, output_file=\"ooni-data-russia.csv\")"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "3f784f91-8de0-4595-9559-998ff3d40e23",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"3423845 ooni-data-russia.csv\n"
]
}
],
"source": [
"!wc -l ooni-data-russia.csv"
]
},
{
"cell_type": "markdown",
"id": "0cb1e94e-746b-42cd-898f-6f46122e65d1",
"metadata": {},
"source": [
"We then load the CSV file in memory as a pandas dataframe for more analysis"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "3dbca11f-b14a-4ff8-8a78-dcd0c15db207",
"metadata": {},
"outputs": [],
"source": [
"df_ru = pd.read_csv('ooni-data-russia.csv')"
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "d7c05cfc-c6bc-48d3-b4d0-7ec3f85eecef",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"3152336"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"len(df_ru)"
]
},
{
"cell_type": "markdown",
"id": "9616a5be-8915-45af-93dd-4ad6e0d01c82",
"metadata": {},
"source": [
"When dealing with websites, we generally care to look at data from a domain centric perspective. This allows us to group together URLs that are of the same domain, but that have different paths.\n",
"\n",
"Since the raw dataset doesn't include the `domain` we add this column here."
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "0adbc67f-adf9-4d7b-aa47-93b91b696b66",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|██████████████████████████████████████████████████████| 3152336/3152336 [00:08<00:00, 365955.32it/s]\n"
]
}
],
"source": [
"df_ru['domain'] = df_ru['input'].progress_apply(lambda r: urlparse(r).netloc)"
]
},
{
"cell_type": "code",
"execution_count": 14,
"id": "c15eadb9-8149-4d80-b011-f9c72a59dbb2",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"13.035878223367035"
]
},
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df_ru.memory_usage(deep=True).sum()/1024**3"
]
},
{
"cell_type": "markdown",
"id": "9cf49c12-b6c0-4c0c-af15-9872f28fe712",
"metadata": {},
"source": [
"### Hunting for blocking fingerprints\n",
"\n",
"We can have a very high confidence that the blocking is intentional (and not caused by transient network failures), when it fits in the following classes:\n",
"- DNS level interference\n",
"- HTTP level intereference\n",
"- TLS MITM\n",
"\n",
"\n",
"The first two classes, though, are susceptive to false positives, because sometimes the IP returned in a DNS query can differ based on the geographical location (think CDNs) and sometimes the content of a webpage can also vary from request to request (think the homepage of a news site).\n",
"\n",
"On the other hand, once we find a blocking fingerprint, we can with great confidence claim that access to that particular site is being restricted. For example we might notice that when a site is blocked on a particular network, the DNS query always returns a given IP address or we might know that the HTTP title for a blockpage is always \"Access to this website is denied\".\n",
"\n",
"Our goal now to come up with some heuristics that will allow us to, in a way, hunt for these blockpage fingerprints in the big dataset that we have available."
]
},
{
"cell_type": "markdown",
"id": "7ce9e1ba-6888-44a4-bc74-3daaf697f89e",
"metadata": {},
"source": [
"### Same title, but different page\n",
"\n",
"One heuristic which we can apply to spotting blockpages, is that we can say that a web page that looks exactly the same for many different sites. Based on this fairly simple intuition, we can look for blockpage fingerprints by just counting for the number of domains that share the same HTTP title tag."
]
},
{
"cell_type": "code",
"execution_count": 15,
"id": "1b97c56c-2da8-4987-aabd-191f6fb5003a",
"metadata": {},
"outputs": [],
"source": [
"title_domain_count = df_ru[\n",
" df_ru['blocking'] == 'http-diff'\n",
"].groupby('http_title')['domain'].nunique().sort_values().reset_index()"
]
},
{
"cell_type": "markdown",
"id": "4881770f-2b65-4574-88c4-ac6611ee02cf",
"metadata": {},
"source": [
"As we can see in the breakdown below, all these blockpage fingerprints look fairly suspicious and are quite likely to be an indication of blocking. Some of them, however, might be signs of server-side blocking (ex. Geoblocking or DDOS prevention). This is why it's best, to obtain a high degree of accuracy, to investigate these manually and add them to a fingerprint database.\n",
"\n",
"This is a shared effort amonst censorship research projects, for example you can find a repo of known blocking fingerprints maintained by the CitizenLab here: https://github.com/citizenlab/filtering-annotations "
]
},
{
"cell_type": "code",
"execution_count": 16,
"id": "8bf95466-f879-4187-a670-8f38ab86c95f",
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" http_title | \n",
" domain | \n",
"
\n",
" \n",
" \n",
" \n",
" | 181 | \n",
" ДоÑтуп к информационному реÑурÑу ограничен | \n",
" 10 | \n",
"
\n",
" \n",
" | 182 | \n",
" ДоÑтуп к реÑурÑу ограничен! | \n",
" 10 | \n",
"
\n",
" \n",
" | 183 | \n",
" Just a moment... | \n",
" 10 | \n",
"
\n",
" \n",
" | 184 | \n",
" ERROR: The requested URL could not be retrieved | \n",
" 11 | \n",
"
\n",
" \n",
" | 185 | \n",
" 403 Forbidden | \n",
" 13 | \n",
"
\n",
" \n",
" | 186 | \n",
" Dr.Web не рекомендует поÑещать Ñтот Ñайт | \n",
" 14 | \n",
"
\n",
" \n",
" | 187 | \n",
" Инфолинк | \n",
" 17 | \n",
"
\n",
" \n",
" | 188 | \n",
" ДоÑтуп ограничен | Ðаука-СвÑзь | \n",
" 18 | \n",
"
\n",
" \n",
" | 189 | \n",
" ÐÐ½Ñ‚Ð¸Ð²Ð¸Ñ€ÑƒÑ ESET NOD32 | \n",
" 21 | \n",
"
\n",
" \n",
" | 190 | \n",
" РКРМегаФон | \n",
" 25 | \n",
"
\n",
" \n",
" | 191 | \n",
" ДоÑтуп ограничен! | \n",
" 26 | \n",
"
\n",
" \n",
" | 192 | \n",
" ПодрÑд - Сайт заблокирован по решению Ñуда | \n",
" 26 | \n",
"
\n",
" \n",
" | 193 | \n",
" ДоÑтуп заблокирован / Access blocked | \n",
" 26 | \n",
"
\n",
" \n",
" | 194 | \n",
" Марк | \n",
" 37 | \n",
"
\n",
" \n",
" | 195 | \n",
" RialCom | \n",
" 39 | \n",
"
\n",
" \n",
" | 196 | \n",
" ИнÑÐ¸Ñ | \n",
" 45 | \n",
"
\n",
" \n",
" | 197 | \n",
" МегаФон Kaspersky | \n",
" 50 | \n",
"
\n",
" \n",
" | 198 | \n",
" МегаФон Gameloft | \n",
" 51 | \n",
"
\n",
" \n",
" | 199 | \n",
" Akado | \n",
" 51 | \n",
"
\n",
" \n",
" | 200 | \n",
" МегаФон ПреÑÑа | \n",
" 52 | \n",
"
\n",
" \n",
" | 201 | \n",
" Ð¯Ð½Ð´ÐµÐºÑ Ñ ÐœÐµÐ³Ð°Ð¤Ð¾Ð½Ð¾Ð¼ | \n",
" 53 | \n",
"
\n",
" \n",
" | 202 | \n",
" МегаФон Знаю, кто звонит | \n",
" 53 | \n",
"
\n",
" \n",
" | 203 | \n",
" ПодпиÑка START Ñ ÐœÐµÐ³Ð°Ð¤Ð¾Ð½Ð¾Ð¼ | \n",
" 54 | \n",
"
\n",
" \n",
" | 204 | \n",
" МегаФон SMS-фильтр | \n",
" 54 | \n",
"
\n",
" \n",
" | 205 | \n",
" ДоÑтуп к запрошенному реÑурÑу заблокирован | \n",
" 55 | \n",
"
\n",
" \n",
" | 206 | \n",
" IVI Ñ ÐœÐµÐ³Ð°Ð¤Ð¾Ð½Ð¾Ð¼ | \n",
" 57 | \n",
"
\n",
" \n",
" | 207 | \n",
" МТС | \n",
" 57 | \n",
"
\n",
" \n",
" | 208 | \n",
" МегаФон МегаФон ТВ | \n",
" 59 | \n",
"
\n",
" \n",
" | 209 | \n",
" Запрещено | \n",
" 59 | \n",
"
\n",
" \n",
" | 210 | \n",
" МегаФон Стоп-реклама | \n",
" 63 | \n",
"
\n",
" \n",
" | 211 | \n",
" Планета | \n",
" 65 | \n",
"
\n",
" \n",
" | 212 | \n",
" MTC | \n",
" 69 | \n",
"
\n",
" \n",
" | 213 | \n",
" ДоÑтуп запрещён | \n",
" 72 | \n",
"
\n",
" \n",
" | 214 | \n",
" Домен заблокирован | \n",
" 73 | \n",
"
\n",
" \n",
" | 215 | \n",
" РКР| \n",
" 74 | \n",
"
\n",
" \n",
" | 216 | \n",
" РеÑÑƒÑ€Ñ Ð·Ð°Ð±Ð»Ð¾ÐºÐ¸Ñ€Ð¾Ð²Ð°Ð½ | \n",
" 75 | \n",
"
\n",
" \n",
" | 217 | \n",
" Орион телеком :: БЛОКИРОВКР| \n",
" 75 | \n",
"
\n",
" \n",
" | 218 | \n",
" ДоÑтуп заблокирован | \n",
" 76 | \n",
"
\n",
" \n",
" | 219 | \n",
" Единый рееÑÑ‚Ñ€ доменных имен, указателей Ñтрани... | \n",
" 79 | \n",
"
\n",
" \n",
" | 220 | \n",
" ДоÑтуп к реÑурÑу заблокирован | \n",
" 80 | \n",
"
\n",
" \n",
" | 221 | \n",
" Страница заблокирована | \n",
" 84 | \n",
"
\n",
" \n",
" | 222 | \n",
" ДоÑтуп закрыт | \n",
" 87 | \n",
"
\n",
" \n",
" | 223 | \n",
" Данный реÑÑƒÑ€Ñ Ð·Ð°Ð±Ð»Ð¾ÐºÐ¸Ñ€Ð¾Ð²Ð°Ð½ | \n",
" 87 | \n",
"
\n",
" \n",
" | 224 | \n",
" TTK :: ДоÑтуп к реÑурÑу ограничен | \n",
" 90 | \n",
"
\n",
" \n",
" | 225 | \n",
" ДоÑтуп к реÑурÑу ограничен | \n",
" 90 | \n",
"
\n",
" \n",
" | 226 | \n",
" ДоÑтуп к запрашиваемому реÑурÑу ограничен | \n",
" 92 | \n",
"
\n",
" \n",
" | 227 | \n",
" РеÑÑƒÑ€Ñ Ð·Ð°Ð±Ð»Ð¾ÐºÐ¸Ñ€Ð¾Ð²Ð°Ð½ - Resource is blocked | \n",
" 95 | \n",
"
\n",
" \n",
" | 228 | \n",
" ДоÑтуп ограничен | \n",
" 109 | \n",
"
\n",
" \n",
"
\n",
"
\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" dns_resolved_ips | \n",
" domain | \n",
"
\n",
" \n",
" \n",
" \n",
" | 1737 | \n",
" ['184.85.124.165'] | \n",
" 3 | \n",
"
\n",
" \n",
" | 1738 | \n",
" ['178.248.233.32'] | \n",
" 3 | \n",
"
\n",
" \n",
" | 1739 | \n",
" ['185.107.56.192'] | \n",
" 3 | \n",
"
\n",
" \n",
" | 1740 | \n",
" ['185.107.56.195'] | \n",
" 3 | \n",
"
\n",
" \n",
" | 1741 | \n",
" ['185.107.56.52'] | \n",
" 3 | \n",
"
\n",
" \n",
" | ... | \n",
" ... | \n",
" ... | \n",
"
\n",
" \n",
" | 1862 | \n",
" ['80.76.104.20'] | \n",
" 222 | \n",
"
\n",
" \n",
" | 1863 | \n",
" ['100.64.64.66'] | \n",
" 223 | \n",
"
\n",
" \n",
" | 1864 | \n",
" ['95.213.158.61'] | \n",
" 225 | \n",
"
\n",
" \n",
" | 1865 | \n",
" ['188.186.157.49'] | \n",
" 238 | \n",
"
\n",
" \n",
" | 1866 | \n",
" [] | \n",
" 1590 | \n",
"
\n",
" \n",
"
\n",
"
130 rows × 2 columns
\n",
"
\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" dns_resolved_ips | \n",
" domain | \n",
"
\n",
" \n",
" \n",
" \n",
" | 1819 | \n",
" ['81.200.2.238'] | \n",
" 12 | \n",
"
\n",
" \n",
" | 1820 | \n",
" ['188.114.97.132'] | \n",
" 13 | \n",
"
\n",
" \n",
" | 1821 | \n",
" ['83.69.208.124'] | \n",
" 14 | \n",
"
\n",
" \n",
" | 1822 | \n",
" ['127.0.0.2'] | \n",
" 16 | \n",
"
\n",
" \n",
" | 1823 | \n",
" ['188.114.97.136'] | \n",
" 16 | \n",
"
\n",
" \n",
" | 1824 | \n",
" ['188.114.96.136'] | \n",
" 16 | \n",
"
\n",
" \n",
" | 1825 | \n",
" ['31.28.24.3'] | \n",
" 16 | \n",
"
\n",
" \n",
" | 1826 | \n",
" ['35.168.95.233'] | \n",
" 16 | \n",
"
\n",
" \n",
" | 1827 | \n",
" ['195.128.72.1'] | \n",
" 17 | \n",
"
\n",
" \n",
" | 1828 | \n",
" ['195.128.72.3'] | \n",
" 17 | \n",
"
\n",
" \n",
" | 1829 | \n",
" ['188.65.128.218'] | \n",
" 20 | \n",
"
\n",
" \n",
" | 1830 | \n",
" ['212.109.26.243'] | \n",
" 21 | \n",
"
\n",
" \n",
" | 1831 | \n",
" ['188.114.98.136'] | \n",
" 21 | \n",
"
\n",
" \n",
" | 1832 | \n",
" ['188.114.99.136'] | \n",
" 22 | \n",
"
\n",
" \n",
" | 1833 | \n",
" ['193.58.251.1'] | \n",
" 22 | \n",
"
\n",
" \n",
" | 1834 | \n",
" ['188.114.98.132'] | \n",
" 24 | \n",
"
\n",
" \n",
" | 1835 | \n",
" ['188.114.96.128'] | \n",
" 24 | \n",
"
\n",
" \n",
" | 1836 | \n",
" ['81.88.208.208'] | \n",
" 28 | \n",
"
\n",
" \n",
" | 1837 | \n",
" ['10.1.1.3'] | \n",
" 30 | \n",
"
\n",
" \n",
" | 1838 | \n",
" ['89.21.139.21'] | \n",
" 31 | \n",
"
\n",
" \n",
" | 1839 | \n",
" ['188.114.97.128'] | \n",
" 32 | \n",
"
\n",
" \n",
" | 1840 | \n",
" ['188.114.97.7', '188.114.96.7'] | \n",
" 33 | \n",
"
\n",
" \n",
" | 1841 | \n",
" ['188.114.96.7', '188.114.97.7'] | \n",
" 34 | \n",
"
\n",
" \n",
" | 1842 | \n",
" ['37.252.254.39'] | \n",
" 36 | \n",
"
\n",
" \n",
" | 1843 | \n",
" ['188.114.97.2', '188.114.96.2'] | \n",
" 37 | \n",
"
\n",
" \n",
" | 1844 | \n",
" ['188.114.98.128'] | \n",
" 39 | \n",
"
\n",
" \n",
" | 1845 | \n",
" ['78.29.1.40'] | \n",
" 42 | \n",
"
\n",
" \n",
" | 1846 | \n",
" ['188.43.20.67'] | \n",
" 44 | \n",
"
\n",
" \n",
" | 1847 | \n",
" ['188.114.99.132'] | \n",
" 44 | \n",
"
\n",
" \n",
" | 1848 | \n",
" ['188.114.96.2', '188.114.97.2'] | \n",
" 44 | \n",
"
\n",
" \n",
" | 1849 | \n",
" ['185.77.150.2'] | \n",
" 46 | \n",
"
\n",
" \n",
" | 1850 | \n",
" ['46.175.31.250'] | \n",
" 49 | \n",
"
\n",
" \n",
" | 1851 | \n",
" ['188.114.99.128'] | \n",
" 60 | \n",
"
\n",
" \n",
" | 1852 | \n",
" ['176.103.130.135'] | \n",
" 62 | \n",
"
\n",
" \n",
" | 1853 | \n",
" ['0.0.0.0'] | \n",
" 73 | \n",
"
\n",
" \n",
" | 1854 | \n",
" ['78.24.40.190'] | \n",
" 78 | \n",
"
\n",
" \n",
" | 1855 | \n",
" ['62.140.245.46'] | \n",
" 81 | \n",
"
\n",
" \n",
" | 1856 | \n",
" ['46.175.31.251'] | \n",
" 81 | \n",
"
\n",
" \n",
" | 1857 | \n",
" ['127.0.0.1'] | \n",
" 89 | \n",
"
\n",
" \n",
" | 1858 | \n",
" ['77.238.226.53'] | \n",
" 163 | \n",
"
\n",
" \n",
" | 1859 | \n",
" ['62.33.207.197', '62.33.207.196'] | \n",
" 202 | \n",
"
\n",
" \n",
" | 1860 | \n",
" ['85.142.29.248'] | \n",
" 203 | \n",
"
\n",
" \n",
" | 1861 | \n",
" ['62.33.207.196', '62.33.207.197'] | \n",
" 208 | \n",
"
\n",
" \n",
" | 1862 | \n",
" ['80.76.104.20'] | \n",
" 222 | \n",
"
\n",
" \n",
" | 1863 | \n",
" ['100.64.64.66'] | \n",
" 223 | \n",
"
\n",
" \n",
" | 1864 | \n",
" ['95.213.158.61'] | \n",
" 225 | \n",
"
\n",
" \n",
" | 1865 | \n",
" ['188.186.157.49'] | \n",
" 238 | \n",
"
\n",
" \n",
" | 1866 | \n",
" [] | \n",
" 1590 | \n",
"
\n",
" \n",
"
\n",
"
\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" domain | \n",
" report_id | \n",
"
\n",
" \n",
" \n",
" \n",
" | 119 | \n",
" shajtanshop.com | \n",
" 1 | \n",
"
\n",
" \n",
" | 59 | \n",
" instagram.com | \n",
" 1 | \n",
"
\n",
" \n",
" | 34 | \n",
" facebook.com | \n",
" 1 | \n",
"
\n",
" \n",
" | 36 | \n",
" fapreactor.com | \n",
" 1 | \n",
"
\n",
" \n",
" | 89 | \n",
" nani24.cc | \n",
" 1 | \n",
"
\n",
" \n",
" | ... | \n",
" ... | \n",
" ... | \n",
"
\n",
" \n",
" | 115 | \n",
" rutracker.org | \n",
" 477 | \n",
"
\n",
" \n",
" | 156 | \n",
" www.bbc.com | \n",
" 513 | \n",
"
\n",
" \n",
" | 58 | \n",
" imrussia.org | \n",
" 515 | \n",
"
\n",
" \n",
" | 135 | \n",
" twitter.com | \n",
" 1873 | \n",
"
\n",
" \n",
" | 182 | \n",
" www.facebook.com | \n",
" 1972 | \n",
"
\n",
" \n",
"
\n",
"
269 rows × 2 columns
\n",
"