Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Setting a default timeout on all HTTP connections #397

Merged
merged 4 commits into from
Jan 30, 2020

Conversation

hiranya911
Copy link
Contributor

@hiranya911 hiranya911 commented Jan 29, 2020

The google-auth package now sets a default timeout of 120 seconds for all outgoing requests (see googleapis/google-auth-library-python#435). This is causing a couple of test failures in our codebase: https://github.com/firebase/firebase-admin-python/actions/runs/32412728

But more importantly this change leads to unpredictable default behavior in applications that use firebase_admin. Specifically, developers who are on older versions of google-auth won't get any timeout (timeout=None), while developers on new versions will get a 2 minutes timeout (timeout=120).

This PR addresses the above discrepancy by setting a default timeout in our own code. This has the following advantages:

  1. Developers using firebase_admin will see consistent behavior regardless of their google-auth version.
  2. We get more control over the default timeout, and are no longer affected by the changes in google-auth.

Note that httpTimeout option is currently only respected by a subset of our modules. And there's some code duplication in how various modules read the httpTimeout option. We can address this in a separate PR.

RELEASE NOTE: Admin SDK now sets a default timeout of 120 seconds on all outgoing HTTP requests.

@hiranya911 hiranya911 changed the title Setting a default timeout on all HTTP connections fix: Setting a default timeout on all HTTP connections Jan 29, 2020
``serviceAccountId`` and ``httpTimeout``. If ``httpTimeout`` is not set, HTTP
connections initiated by client modules such as ``db`` will not time out.
``serviceAccountId`` and ``httpTimeout``. If ``httpTimeout`` is not set, the SDK
uses a default timeout of 120 seconds.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a behavioural change. If you want to keep the behaviour the same, we could instead explicitly set a None timeout and defer using the new default of 120s until the next major release of firebase-admin-python. (Although defaulting to an indefinite timeout does seem a little broken, so I'd be happy to call this a bug fix instead.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

google-auth shipped the same change as minor version bump. Given our next version is going to be a major version bump this should be ok. And yes, I also prefer to consider this a bug fix than anything else.

self, credential=credential, base_url=base_url, headers={'User-Agent': _USER_AGENT})
self.credential = credential
self.timeout = timeout
self, credential=credential, base_url=base_url, timeout=timeout,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The docs above now seem wrong: "If not set connections will never timeout, which is the default behavior of the underlying requests library."

I think you could probably just remove that sentence entirely since there is no default value for this parameter. Alternatively, you could use "If set to None, the connections will never timeout."

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

def test_default_timeout(self, user_mgt_app):
auth_service = auth._get_auth_service(user_mgt_app)
user_manager = auth_service.user_manager
assert user_manager._client.timeout == _http_client.DEFAULT_TIMEOUT_SECONDS
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm... It seems like all of our components will have an approximately identical test like this one. We could remove them all and just add this into the _http_client tests. Or we could add a parameterized test that verifies that each component sets the timeout appropriately. (No action required; I'm just thinking out loud.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, and I think the existing test cases in _http_client already provides us sufficient coverage. In each of the other modules it's probably enough to check that the Client.timeout is correctly set. We can do that clean up in a future iteration.

@rsgowman rsgowman assigned hiranya911 and unassigned rsgowman Jan 29, 2020
@hiranya911 hiranya911 assigned rsgowman and unassigned hiranya911 Jan 29, 2020
Copy link
Member

@rsgowman rsgowman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm w/ nits; though make sure you take another look at the doc string in db.py.

@@ -900,13 +900,13 @@ def __init__(self, credential, base_url, timeout, params=None):
credential: A Google credential that can be used to authenticate requests.
base_url: A URL prefix to be added to all outgoing requests. This is typically the
Firebase Realtime Database URL.
timeout: HTTP request timeout in seconds. If not set connections will never
timeout: HTTP request timeout in seconds. If set to None connections will never
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: the 'which is the default behaviour ...' part is no longer true.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If set to None connections will never
timeout, which is the default behavior of the underlying requests library.

That is still true. The requests library behavior hasn't yet changed. The default timeout=None behavior still comes from there.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops; yeah I missed the change from 'if unset' to 'if set to none' (despite suggesting exactly that earlier. 😳)

@pytest.mark.parametrize('timeout', [7, None])
def test_timeout(timeout):
@pytest.mark.parametrize('timeout', [7, 0, None])
def test_custom_timeout(timeout):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function looks mostly identical to test_default_timeout. You could optionally refactor to something like this:

def _helper_test_timeout(**kwargs):
    client = _http_client.HttpClient()
    timeout = kwargs.get('timeout', _http_client.DEFAULT_TIMEOUT_SECONDS)
    assert client.timeout == timeout
    ...
    if timeout is None
        assert recorder[0]._extra_kwargs['timeout'] is None
    else:
        assert ... == pytest.approx(timeout, 0.001)

def test_default_timeout():
    _helper_test_timeout()

@parameterize(...)
def test_custom_timeout(timeout)
    _helper_test_timeout(timeout=timeout)

(Or you could parameterize the helper directly.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Refactored with parameterization.

project_management_service = project_management._get_project_management_service(app)
assert project_management_service._client.timeout == timeout
finally:
firebase_admin.delete_app(app)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BaseProjectManagementTest seems to call testutils.cleanup_apps() in it's teardown() method. If you're willing to rely on that, you could drop the try, finally, and delete_app() lines. Presumably you'd nave to not hard-code the app name ("timeout-app") but instead use something based on the test parameter or something like that instead. Optional.

I have no objection to leaving it in either, but it's odd to see the app cleaned up in this test case, but not the one above.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@rsgowman rsgowman assigned hiranya911 and unassigned rsgowman Jan 30, 2020
@lahirumaramba lahirumaramba removed their assignment Jan 30, 2020
@hiranya911 hiranya911 merged commit b5f228f into master Jan 30, 2020
@hiranya911 hiranya911 deleted the hkj-default-timeout branch January 30, 2020 19:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants