人間とウェブの未来

「ウェブの歴史は人類の歴史の繰り返し」という観点から色々勉強しています。

Webサーバのベンチマークツールはh2loadが便利

Webサーバ

Webサーバのベンチマークをとるのが趣味になりつつあるmatsumotoryです。

Webサーバのベンチマークについては、abからはじまりwrk等を使っていたのですが、最近ではほぼh2loadを使っています。

h2loadはnghttp2というHTTP/2ライブラリのアプリケーションに含まれているツールですが、

  • HTTP/2（SPDYも）とHTTP/1.xに両対応している
  • ベンチマーク側の同時スレッド数を増やせる
  • TLS及びSNIもサポートしている
  • 最小、最大、平均、標準偏差あたりもちゃんとでる

ので、色々プロトコルを変えつつ同じベンチマークツールで、値の目安を出すにはとても重宝しています。

Nghttp2: HTTP/2 C Library - nghttp2.org

実行結果のサンプルは例えば以下、

$ h2load -c 100 -n 10000 https://localhost:58082/
starting benchmark...
spawning thread #0: 100 total client(s). 10000 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-RSA-AES128-GCM-SHA256
Server Temp Key: ECDH P-256 256 bits
Application protocol: http/1.1
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 300.79ms, 33245.34 req/s, 4.14MB/s
requests: 10000 total, 10000 started, 10000 done, 10000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 10000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 1306800 bytes total, 859500 bytes headers (space savings 0.00%), 110000 bytes data
                     min         max         mean         sd        +/- sd
time for request:    16.04ms    225.26ms    115.41ms     59.69ms    56.07%
time for connect:    49.75ms    101.39ms     56.42ms      5.38ms    87.00%
time to 1st byte:    67.78ms    279.02ms    100.17ms     23.69ms    78.00%
req/s           :     343.79      850.22      449.69      204.01    79.00%

ちゃんとHTTP/1.1を検出してくれたり、HTTP/2に対しては、

$ h2load -c 100 -n 10000 https://localhost:8080/index.html
starting benchmark...
spawning thread #0: 100 total client(s). 10000 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-RSA-AES128-GCM-SHA256
Server Temp Key: ECDH P-256 256 bits
Application protocol: h2
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 2.12s, 4714.33 req/s, 229.61KB/s
requests: 10000 total, 10000 started, 10000 done, 10000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 10000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 498746 bytes total, 95746 bytes headers (space savings 92.09%), 220000 bytes data
                     min         max         mean         sd        +/- sd
time for request:   167.45ms       1.87s       1.12s    553.00ms    63.00%
time for connect:    47.93ms    313.94ms     89.12ms     69.25ms    90.00%
time to 1st byte:   215.82ms       2.11s       1.21s    587.02ms    63.00%
req/s           :      47.35      443.00      125.54      108.58    87.00%

というように自動で検知してくれて非常に便利です。今後、TLSやHTTP/2、SPDYとか色々と混在した環境では非常に重宝するかと思います。Unix Domain Socketにも対応していると個人的な用途ではさらに便利だと思っています。

今後はHTTP/1についてもこれでやっていこうかなぁと思います。

$ h2load -h
Usage: h2load [OPTIONS]... [URI]...
benchmarking tool for HTTP/2 and SPDY server

  <URI>       Specify URI to access.   Multiple URIs can be specified.
              URIs are used  in this order for each  client.  All URIs
              are used, then  first URI is used and then  2nd URI, and
              so  on.  The  scheme, host  and port  in the  subsequent
              URIs, if present,  are ignored.  Those in  the first URI
              are used solely.  Definition of a base URI overrides all
              scheme, host or port values.
Options:
  -n, --requests=<N>
              Number of  requests across all  clients.  If it  is used
              with --timing-script-file option,  this option specifies
              the number of requests  each client performs rather than
              the number of requests across all clients.
              Default: 1
  -c, --clients=<N>
              Number  of concurrent  clients.   With  -r option,  this
              specifies the maximum number of connections to be made.
              Default: 1
  -t, --threads=<N>
              Number of native threads.
              Default: 1
  -i, --input-file=<PATH>
              Path of a file with multiple URIs are separated by EOLs.
              This option will disable URIs getting from command-line.
              If '-' is given as <PATH>, URIs will be read from stdin.
              URIs are used  in this order for each  client.  All URIs
              are used, then  first URI is used and then  2nd URI, and
              so  on.  The  scheme, host  and port  in the  subsequent
              URIs, if present,  are ignored.  Those in  the first URI
              are used solely.  Definition of a base URI overrides all
              scheme, host or port values.
  -m, --max-concurrent-streams=<N>
              Max  concurrent  streams  to issue  per  session.   When
              http/1.1  is used,  this  specifies the  number of  HTTP
              pipelining requests in-flight.
              Default: 1
  -w, --window-bits=<N>
              Sets the stream level initial window size to (2**<N>)-1.
              For SPDY, 2**<N> is used instead.
              Default: 30
  -W, --connection-window-bits=<N>
              Sets  the  connection  level   initial  window  size  to
              (2**<N>)-1.  For SPDY, if <N>  is strictly less than 16,
              this option  is ignored.   Otherwise 2**<N> is  used for
              SPDY.
              Default: 30
  -H, --header=<HEADER>
              Add/Override a header to the requests.
  --ciphers=<SUITE>
              Set allowed  cipher list.  The  format of the  string is
              described in OpenSSL ciphers(1).
  -p, --no-tls-proto=<PROTOID>
              Specify ALPN identifier of the  protocol to be used when
              accessing http URI without SSL/TLS.
              Available protocols: spdy/2, spdy/3, spdy/3.1, h2c and
              http/1.1
              Default: h2c
  -d, --data=<PATH>
              Post FILE to  server.  The request method  is changed to
              POST.
  -r, --rate=<N>
              Specifies  the  fixed  rate  at  which  connections  are
              created.   The   rate  must   be  a   positive  integer,
              representing the  number of  connections to be  made per
              rate period.   The maximum  number of connections  to be
              made  is  given  in  -c   option.   This  rate  will  be
              distributed among  threads as  evenly as  possible.  For
              example,  with   -t2  and   -r4,  each  thread   gets  2
              connections per period.  When the rate is 0, the program
              will run  as it  normally does, creating  connections at
              whatever variable rate it  wants.  The default value for
              this option is 0.
  --rate-period=<DURATION>
              Specifies the time  period between creating connections.
              The period  must be a positive  number, representing the
              length of the period in time.  This option is ignored if
              the rate option is not used.  The default value for this
              option is 1s.
  -T, --connection-active-timeout=<DURATION>  
              Specifies  the maximum  time that  h2load is  willing to
              keep a  connection open,  regardless of the  activity on
              said connection.  <DURATION> must be a positive integer,
              specifying the amount of time  to wait.  When no timeout
              value is  set (either  active or inactive),  h2load will
              keep  a  connection  open indefinitely,  waiting  for  a
              response.
  -N, --connection-inactivity-timeout=<DURATION>
              Specifies the amount  of time that h2load  is willing to
              wait to see activity  on a given connection.  <DURATION>
              must  be a  positive integer,  specifying the  amount of
              time  to wait.   When no  timeout value  is set  (either
              active or inactive), h2load  will keep a connection open
              indefinitely, waiting for a response.
  --timing-script-file=<PATH>
              Path of a file containing one or more lines separated by
              EOLs.  Each script line is composed of two tab-separated
              fields.  The first field represents the time offset from
              the start of execution, expressed as a positive value of
              milliseconds  with microsecond  resolution.  The  second
              field represents the URI.  This option will disable URIs
              getting from  command-line.  If '-' is  given as <PATH>,
              script lines will be read  from stdin.  Script lines are
              used in order for each client.   If -n is given, it must
              be less  than or  equal to the  number of  script lines,
              larger values are clamped to the number of script lines.
              If -n is not given,  the number of requests will default
              to the  number of  script lines.   The scheme,  host and
              port defined in  the first URI are  used solely.  Values
              contained  in  other  URIs,  if  present,  are  ignored.
              Definition of a  base URI overrides all  scheme, host or
              port values.
  -B, --base-uri=<URI>
              Specify URI from which the scheme, host and port will be
              used  for  all requests.   The  base  URI overrides  all
              values  defined either  at  the command  line or  inside
              input files.
  --npn-list=<LIST>
              Comma delimited list of  ALPN protocol identifier sorted
              in the  order of preference.  That  means most desirable
              protocol comes  first.  This  is used  in both  ALPN and
              NPN.  The parameter must be  delimited by a single comma
              only  and any  white spaces  are  treated as  a part  of
              protocol string.
              Default: h2,h2-16,h2-14,spdy/3.1,spdy/3,spdy/2,http/1.1
  --h1        Short        hand         for        --npn-list=http/1.1
              --no-tls-proto=http/1.1,    which   effectively    force
              http/1.1 for both http and https URI.
  -v, --verbose
              Output debug information.
  --version   Display version information and exit.
  -h, --help  Display this help and exit.

--

  The <DURATION> argument is an integer and an optional unit (e.g., 1s
  is 1 second and 500ms is 500 milliseconds).  Units are h, m, s or ms
  (hours, minutes, seconds and milliseconds, respectively).  If a unit
  is omitted, a second is used as unit.