Skip to content

datastore_active is being rewritten due to race condition #3245

Closed
@ZtF

Description

@ZtF

CKAN Version if known (or site URL)

2.5.2

Please describe the expected behaviour

When file resource saved to FileStore, datapusher is notified about it, downloads the resource, process the file, create new table in DataStore and sets datastore_active flag for the particular resource.

Please describe the actual behaviour

It's working as described, though when uploading multiple resources (eg. through API), there is a race condition probably in ckan.lib.dictization.model_save.package_dict_save (or somewhere in the stack trace).
Sometimes the problem can even produce "Resource not found" in logs though it's most certainly there at the time. That's a case where datapusher tries to patch the resource and set datastore_active to True.

What steps can be taken to reproduce the issue?

In httpd conf file, the race condition can be "resolved" with forcing one process and one thread for CKAN.
Something like this in ckan httpd config file:
WSGIDaemonProcess ckan display-name=ckan processes=1 threads=1
If it's eg. 15 threads and 2 processes, the described issue appears.

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions