Skip to content

Conversation

@garrettheel
Copy link
Member

@garrettheel garrettheel commented Jan 4, 2019

Using botocore's vendored requests library is becoming increasingly problematic. This moves to using urllib3 instead via botocore, rather than managing the requests version directly and further diverging from the botocore implementation.

Breaking changes include:

  • Dropping python 2.6 support
  • No more session_cls or requests_timeout_seconds config
    • Added connect_timeout_seconds, read_timeout_seconds and max_pool_connections config
  • Exceptions raised should remain the same, however they will no longer wrap requests exceptions.
    • For example where the exc.cause was previously a requests.exceptions.ReadTimeout, it would now be a botocore.exceptions.ReadTimeoutError.

Improvements:

  • Implements the before-send botocore hook to support testing via moto

Related: #558, #566, #559, #543, #569

Also related: boto/botocore#1486

Benchmarks

Here are some simple benchmarks using a dynalite server. Seems to be ~10% faster for the read/write path.

# (kill-requests)$ python -m timeit -s 'import benchmark; benchmark.setup()' -n 2000 -r 10 -- "benchmark.test()"
2000 loops, best of 10: 1.63 msec per loop
# (master)$ python -m timeit -s 'import benchmark; benchmark.setup()' -n 2000 -r 10 -- "benchmark.test()"
2000 loops, best of 10: 1.82 msec per loop

from pynamodb.models import Model
from pynamodb.attributes import NumberAttribute, UnicodeAttribute


class UserModel(Model):
    class Meta:
        table_name = 'UserModel'
        host = 'http://localhost:8000'

    user_name = UnicodeAttribute(hash_key=True, attr_name='user_name')
    user_id = UnicodeAttribute(range_key=True)
    zip_code = NumberAttribute(null=True)
    email = UnicodeAttribute()


def setup():
    UserModel.create_table(read_capacity_units=1, write_capacity_units=1, wait=True)
    UserModel("test", user_id="1234", email="[email protected]", zip_code=1234).save()


def test():
    UserModel.get("test", "1234")

@garrettheel garrettheel force-pushed the kill-requests branch 2 times, most recently from aac47c6 to 32108d3 Compare January 4, 2019 18:33
@garrettheel garrettheel force-pushed the kill-requests branch 7 times, most recently from 975db72 to 3e0517b Compare April 11, 2019 02:43
@garrettheel garrettheel force-pushed the kill-requests branch 3 times, most recently from d3864fe to 31324ae Compare April 13, 2019 21:21
@garrettheel garrettheel force-pushed the kill-requests branch 6 times, most recently from 6ed64ae to 69035f5 Compare April 16, 2019 20:51
@garrettheel garrettheel changed the title Kill requests in favour of urllib3 Stop using requests in favour of botocore & urllib3 Apr 16, 2019
@garrettheel garrettheel merged commit 71c33e8 into master Apr 16, 2019
@garrettheel garrettheel deleted the kill-requests branch April 16, 2019 21:54
archlinux-github pushed a commit to archlinux/svntogit-community that referenced this pull request Jan 29, 2021
* Cleanup dependencies
    - python-dateutil: no longer needed after pynamodb/PynamoDB#850
    - python-six & python-mock: no longer needed after pynamodb/PynamoDB#786
    - python-requests: no longer needed after pynamodb/PynamoDB#580
      (This is actually a change in 4.0, but I didn't review dependencies then)
* Update environment variables for testing
* Make tests less verbose

git-svn-id: file:///srv/repos/svn-community/svn@834674 9fca08f4-af9d-4005-b8df-a31f2cc04f65
archlinux-github pushed a commit to archlinux/svntogit-community that referenced this pull request Jan 29, 2021
* Cleanup dependencies
    - python-dateutil: no longer needed after pynamodb/PynamoDB#850
    - python-six & python-mock: no longer needed after pynamodb/PynamoDB#786
    - python-requests: no longer needed after pynamodb/PynamoDB#580
      (This is actually a change in 4.0, but I didn't review dependencies then)
* Update environment variables for testing
* Make tests less verbose


git-svn-id: file:///srv/repos/svn-community/svn@834674 9fca08f4-af9d-4005-b8df-a31f2cc04f65
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants