Yandex-tank prepare phout file with statistics after load testing. tanktools module helps to parse such files and convert to DataFrame. You can use pandas module in manual mode to handle DataFrame or use build-in functions.

Generate Yandex-tank ammo from pcap or har files using pcap2ammo or har2ammo scrips. HTTP requests are extracted completely with headers and body.

So you can:

• calc quantiles
• get information about timings, latency, status codes
• extract requests by timestamp, tag and other columns
• group and analyze specific data like total/partial RPS, average request/response size
• calc statistical metrics
• convert pcap file to ammo
• convert har file to ammo
• filter out and modify requests on ammo generating

pip install tanktools

It is possible to parse a part of staistics by time and overal count

from tanktools import phout
flags = {
    'from_date': '2018-01-18 20:09:50.123',
    'to_date'  : '2018-01-18 20:10:00.456',
    'limit': 100
}
data = phout.parse_phout('phout.log', flags)
print("Total count: %d" % phout.size(data))

Total count: 100

data = phout.parse_phout('phout.log')
phout.print_quantiles(data, 'receive_time')

Percentiles for 150030 requests
    from 2018-01-18 20:09:42.983
    to   2018-01-18 20:10:55.108:
quantile (%)  receive_time (mks)
        10.0                   9
        20.0                   9
        30.0                  10
        40.0                  10
        50.0                  10
        60.0                  10
        70.0                  11
        80.0                  12
        90.0                  13
        95.0                  14
        98.0                  16
        99.0                  17
        100.0                 716

data = phout.parse_phout('phout.log')
# Convert and print timing in milliseconds
print("\n\nLatency median: %d ms" % int(data.latency.median() / 1000))

Latency median: 30 ms

data = phout.parse_phout('phout.log')
rps = phout.get_rps(data)

data = phout.parse_phout('phout.log')
phout.print_http_reponses(data)

HTTP code   count  percent (%)
     500   83429        56.38
     200   61558        41.60
     502    2944         1.99
       0      41         0.03

data = phout.parse_phout('phout.log')
selected_http_responses = data[data.proto_code == 200]
print("Latency median for 200 OK: %d" %
      selected_http_responses.latency.median())

Latency median for 200 OK: 3539

print("Avg. Request / Response: %d / %d bytes" % (
    data.size_in.astype(float).mean(),
    data.size_out.astype(float).mean()
))

Avg. Request / Response: 364 / 26697 bytes

print("RPS at request:")
chunk_size = int(phout.size(data) / 2)
for start in range(0, phout.size(data), chunk_size):
    data_subset = phout.subset(data, start, chunk_size)
    print("\t%s: %.2f" %
          (start + chunk_size, phout.get_rps(data_subset)))

RPS at request:
    73986: 2062.50
    147972: 2530.56

pcap2ammo file.pcap

73
GET https://rambler.ru/ HTTP/1.1\r\n
Host: rambler.ru\r\n
Content-Length: 0\r\n\r\n

pcap2ammo -S file.pcap

Stats:
    total: 1
    complete: 1
    incorrect: 0
    incomplete: 0

pcap2ammo -o out.ammo file.pcap

Applyed for all requests, containing specified headers

pcap2ammo --add-header 'Referer: http://domain.com' --add-header 'X-Ip: 1.1.1.1' file.pcap

pcap2ammo --delete-header 'Content-Length' file.pcap
pcap2ammo --delete-header 'Connection' --add-header 'Connection: keep-alive' file.pcap

pcap2ammo -f 'ip.src==10.10.10.10 and tcp.dport==8080' file.pcap

pcap2ammo -F '"rambler.ru" in http.uri' file.pcap

You can use logical expressions in filters

pcap2ammo -F '"keep-alive" in http.headers["connection"] or "Keep-alive" in http.headers["connection"]' file.pcap

String functions over HTTP headers

pcap2ammo -F '"keep-alive" in http.headers["connection"].lower()' file.pcap

Use excluding filters also

pcap2ammo -F '"rambler.ru" != http.headers["host"]' file.pcap

har2ammo file.har

73
GET https://rambler.ru/ HTTP/1.1\r\n
Host: rambler.ru\r\n
Content-Length: 0\r\n\r\n

har2ammo -S file.har

Stats:
    total: 1
    complete: 1
    incorrect: 0
    incomplete: 0

har2ammo -o out.ammo file.har

Applyed for all requests, containing specified headers

har2ammo --add-header 'Referer: http://domain.com' --add-header 'X-Ip: 1.1.1.1' file.har

har2ammo --delete-header 'Content-Length' file.har
har2ammo --delete-header 'Connection' --add-header 'Connection: keep-alive' file.har

har2ammo -F '"rambler.ru" in http.uri' file.har

You can use logical expressions and python functions in filters

har2ammo -F '"keep-alive" in http.headers["connection"] or "Keep-alive" in http.headers["connection"]' file.har
har2ammo -F '"keep-alive" not in http.headers["connection"].lower()' file.har

Please, see more information about filters in pcaper package description.