Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

neqo 0.8.0 #1973

Merged
merged 1 commit into from
Jul 11, 2024
Merged

neqo 0.8.0 #1973

merged 1 commit into from
Jul 11, 2024

Conversation

KershawChang
Copy link
Collaborator

For issue #1971

@KershawChang KershawChang merged commit 9f0a86d into mozilla:main Jul 11, 2024
19 checks passed
@KershawChang KershawChang deleted the neqo_080 branch July 11, 2024 07:20
Copy link

Failed Interop Tests

QUIC Interop Runner, client vs. server

All results

Succeeded Interop Tests

QUIC Interop Runner, client vs. server

Unsupported Interop Tests

QUIC Interop Runner, client vs. server

Copy link

Benchmark results

Performance differences relative to 9f0a86d.

coalesce_acked_from_zero 1+1 entries: No change in performance detected.
       time:   [192.02 ns 192.41 ns 192.85 ns]
       change: [-1.1013% -0.3197% +0.4350%] (p = 0.44 > 0.05)

Found 10 outliers among 100 measurements (10.00%)
5 (5.00%) high mild
5 (5.00%) high severe

coalesce_acked_from_zero 3+1 entries: No change in performance detected.
       time:   [234.55 ns 235.17 ns 235.83 ns]
       change: [-0.3373% +0.0604% +0.4448%] (p = 0.77 > 0.05)

Found 14 outliers among 100 measurements (14.00%)
1 (1.00%) low mild
8 (8.00%) high mild
5 (5.00%) high severe

coalesce_acked_from_zero 10+1 entries: No change in performance detected.
       time:   [235.18 ns 236.06 ns 237.07 ns]
       change: [-0.0140% +0.4274% +0.9218%] (p = 0.08 > 0.05)

Found 10 outliers among 100 measurements (10.00%)
2 (2.00%) low mild
8 (8.00%) high severe

coalesce_acked_from_zero 1000+1 entries: No change in performance detected.
       time:   [214.24 ns 214.43 ns 214.65 ns]
       change: [-0.9793% -0.3426% +0.3393%] (p = 0.33 > 0.05)

Found 12 outliers among 100 measurements (12.00%)
6 (6.00%) high mild
6 (6.00%) high severe

RxStreamOrderer::inbound_frame(): Change within noise threshold.
       time:   [119.23 ms 119.32 ms 119.42 ms]
       change: [+0.0101% +0.2075% +0.3591%] (p = 0.01 < 0.05)

Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mild

transfer/Run multiple transfers with varying seeds: No change in performance detected.
       time:   [54.986 ms 58.451 ms 62.027 ms]
       thrpt:  [64.488 MiB/s 68.434 MiB/s 72.746 MiB/s]
change:
       time:   [-2.3204% +6.8695% +15.745%] (p = 0.13 > 0.05)
       thrpt:  [-13.603% -6.4279% +2.3755%]

Found 7 outliers among 100 measurements (7.00%)
1 (1.00%) low mild
6 (6.00%) high mild

transfer/Run multiple transfers with the same seed: No change in performance detected.
       time:   [65.426 ms 72.207 ms 78.920 ms]
       thrpt:  [50.684 MiB/s 55.396 MiB/s 61.137 MiB/s]
change:
       time:   [-16.736% -4.6967% +7.7891%] (p = 0.47 > 0.05)
       thrpt:  [-7.2262% +4.9281% +20.100%]
1-conn/1-100mb-resp (aka. Download)/client: No change in performance detected.
       time:   [153.02 ms 158.73 ms 165.92 ms]
       thrpt:  [602.71 MiB/s 630.01 MiB/s 653.52 MiB/s]
change:
       time:   [-11.875% -4.7462% +2.8953%] (p = 0.28 > 0.05)
       thrpt:  [-2.8138% +4.9827% +13.475%]

Found 1 outliers among 10 measurements (10.00%)
1 (10.00%) high mild

1-conn/10_000-parallel-1b-resp (aka. RPS)/client: No change in performance detected.
       time:   [436.44 ms 440.00 ms 443.61 ms]
       thrpt:  [22.542 Kelem/s 22.727 Kelem/s 22.913 Kelem/s]
change:
       time:   [-0.6730% +0.4338% +1.5939%] (p = 0.47 > 0.05)
       thrpt:  [-1.5689% -0.4319% +0.6775%]

Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild

1-conn/1-1b-resp (aka. HPS)/client: 💚 Performance has improved.
       time:   [42.130 ms 42.559 ms 43.006 ms]
       thrpt:  [23.253  elem/s 23.497  elem/s 23.736  elem/s]
change:
       time:   [-5.0959% -3.5405% -1.9864%] (p = 0.00 < 0.05)
       thrpt:  [+2.0267% +3.6705% +5.3696%]

Client/server transfer results

Transfer of 33554432 bytes over loopback.

Client Server CC Pacing Mean [ms] Min [ms] Max [ms] Relative
msquic msquic 143.6 ± 63.9 96.7 371.9 1.00
neqo msquic reno on 281.1 ± 41.4 262.6 397.9 1.00
neqo msquic reno 295.8 ± 58.8 259.4 460.5 1.00
neqo msquic cubic on 278.0 ± 39.5 255.6 388.4 1.00
neqo msquic cubic 269.4 ± 10.2 257.1 286.9 1.00
msquic neqo reno on 180.0 ± 68.3 139.3 416.7 1.00
msquic neqo reno 162.8 ± 51.1 118.3 317.7 1.00
msquic neqo cubic on 227.2 ± 149.9 135.1 624.8 1.00
msquic neqo cubic 203.1 ± 105.7 105.7 521.4 1.00
neqo neqo reno on 223.3 ± 73.7 161.1 389.3 1.00
neqo neqo reno 210.2 ± 86.7 160.3 473.1 1.00
neqo neqo cubic on 221.2 ± 132.5 154.9 692.4 1.00
neqo neqo cubic 221.6 ± 108.7 166.4 564.9 1.00

⬇️ Download logs

Copy link

Firefox builds for this PR

The following builds are available for testing. Crossed-out builds did not succeed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant