-
Notifications
You must be signed in to change notification settings - Fork 66
Description
- API: python-ndb
- AppEngine
- Python version: Python 3.8
- google-cloud-ndb version: 1.9.0
Steps to reproduce
On a _post_put_hook(), we have a deferred classmethod, via Cloud Tasks, which will do a few additional processing based on the updated or created entity. We pass self.key to the deferred method, and it will then start by retrieving the entity thru a simple entity = key.get().
It works most of the time, but it sometimes happens that key.get() will return None, although the entity definitely does exist. We tried adding use_cache=False, use_global_cache=False to the get(), but that did not help.
We could circumvent the issue by passing the entity itself (self) to the deferred method, but that is not good practice as it might have been modified before the deferred method runs. We could also add some code to sleep a while, then try again (and this actually does work). But I thought firebase in datastore mode now provided strong consistency, and I thought that meant I was certain to be able to read a freshly put entity.
The elapsed time between the put() and the deferred_post_put_hook() that will do the key.get() may vary, but in one example it was around 100ms, and yet the get() returned None.
Code example
class Thing(ndb.Model):
name = ndb.StringProperty()
def _post_put_hook(self, future):
Thing.deferred_post_put_hook(self.key)
@classmethod
@defer() # The defer() custom decorator handles calling the method via a Cloud Task. It is quite reliable, and is likely not guilty here.
def deferred_post_put_hook(cls, thing_key: ndb.Key):
thing: Thing = thing_key.get()
# Would process thing from here on, but on occasions thing is None