Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HTTP2 Max Streams aware client pooling #3623

Open
howardjohn opened this issue Apr 5, 2024 · 5 comments
Open

HTTP2 Max Streams aware client pooling #3623

howardjohn opened this issue Apr 5, 2024 · 5 comments
Labels
C-feature Category: feature. This is adding a new feature.

Comments

@howardjohn
Copy link
Contributor

Is your feature request related to a problem? Please describe.
We are dealing with an issue where pooled http2 connections are hanging due max streams limits. This is problematic for us, as all our streams are very long lived, so they end up ~indefinitely blocked.

We were hoping to get around this by tracking stream counts as part of the pool logic.

However, neither current or max stream counts are exposed.

Would it be feasible to expose those (at least Max - current can be tracked in our own code)?

Or is there an alternative solution we should look into?

Previous discussion brought up

No, the client won't start opening new connections when max streams is reached.
This is because of RFC 9113 Section 9.1:

Clients SHOULD NOT open more than one HTTP/2 connection to a given host and port pair, where the host is derived from a URI, a selected alternative service [ALT-SVC], or a configured proxy.

Describe the solution you'd like
A few options are plausible.

  1. Pool can be configured to automatically open up new connections when connections are "full"
  2. Hyper exposes max and current stream counts, so a pooling implementation can easily build this logic themselves. This could look something like fn is_open() { self.current() < self.max() }
  3. Hyper exposes the current max streams (or, possibly, arbitrary http2 connection settings information. A pooling implementation can track the current streams themselves. I think this could be done reliably external to hyper, but there may be edge cases I am not considering.

Additional context
Here is our current pool: https://github.com/istio/ztunnel/blob/a0d539ee6b395a823c31db8f382048dc1947bcb7/src/proxy/pool.rs#L68

Basically we are operating on http2::SendRequests

Some other libraries built on h2 expose similar: https://github.com/cloudflare/pingora/blob/20fd391f3e78c9349149f11fae94da9e4657478e/pingora-core/src/connectors/http/v2.rs#L239

@howardjohn howardjohn added the C-feature Category: feature. This is adding a new feature. label Apr 5, 2024
@edwardwc
Copy link
Contributor

Hey there @howardjohn,

If I understand correctly, you'd like to see how many streams are currently open?

@howardjohn
Copy link
Contributor Author

Ideally the current and the configured max streams on the connection - the max is dynamic based on the client/server SETTINGS.

Without this we end up blocking when we have two many long lived requests to the same host

@edwardwc
Copy link
Contributor

@howardjohn have you considered using an atomic counter that a structure increments or decrements when it is created and dropped to track the count?

@howardjohn
Copy link
Contributor Author

@howardjohn have you considered using an atomic counter that a structure increments or decrements when it is created and dropped to track the count?

Yes, that is our current workaround. However, that only enables us to know how many active streams we have produced as a client. We cannot find the negotiated max streams on the connection because its not exposed by hyper. So today we are guessing no one will set it <100... but there is no guarantee, and it makes us extremely conservative

@dswij
Copy link
Member

dswij commented Apr 27, 2024

That's a fair ask. I'll get into it as soon as I have time to spare.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-feature Category: feature. This is adding a new feature.
Projects
None yet
Development

No branches or pull requests

3 participants