Skip to content

Consistency issue #751

@patb2

Description

@patb2
  1. API: python-ndb
  2. AppEngine
  3. Python version: Python 3.8
  4. google-cloud-ndb version: 1.9.0

Steps to reproduce

On a _post_put_hook(), we have a deferred classmethod, via Cloud Tasks, which will do a few additional processing based on the updated or created entity. We pass self.key to the deferred method, and it will then start by retrieving the entity thru a simple entity = key.get().

It works most of the time, but it sometimes happens that key.get() will return None, although the entity definitely does exist. We tried adding use_cache=False, use_global_cache=False to the get(), but that did not help.
We could circumvent the issue by passing the entity itself (self) to the deferred method, but that is not good practice as it might have been modified before the deferred method runs. We could also add some code to sleep a while, then try again (and this actually does work). But I thought firebase in datastore mode now provided strong consistency, and I thought that meant I was certain to be able to read a freshly put entity.

The elapsed time between the put() and the deferred_post_put_hook() that will do the key.get() may vary, but in one example it was around 100ms, and yet the get() returned None.

Code example

class Thing(ndb.Model):
    name = ndb.StringProperty()

    def _post_put_hook(self, future):
        Thing.deferred_post_put_hook(self.key)

    @classmethod
    @defer()     # The defer() custom decorator handles calling the method via a Cloud Task.  It is quite reliable, and is likely not guilty here.
    def deferred_post_put_hook(cls, thing_key: ndb.Key):

        thing: Thing = thing_key.get()
        
        # Would process thing from here on, but on occasions thing is None
        

Metadata

Metadata

Assignees

No one assigned

    Labels

    🚨This issue needs some love.api: datastoreIssues related to the googleapis/python-ndb API.priority: p2Moderately-important priority. Fix may not be included in next release.type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions