Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Datapusher error when root_path is set #2866

Closed
AdrianMBarrera opened this issue Feb 5, 2016 · 4 comments · Fixed by #3442
Closed

Datapusher error when root_path is set #2866

AdrianMBarrera opened this issue Feb 5, 2016 · 4 comments · Fixed by #3442
Assignees
@amercader

Comments

@AdrianMBarrera
Copy link

Hi,

After setting root_path variable in CKAN config file, Datapusher no longer works. We always get:

Error: Process completed but unable to post to result_url

Contents of datapusher.error.log:

[Fri Feb 05 09:20:00.905675 2016] [:error] [pid 5997:tid 140341959157632] Exception AttributeError: "'NoneType' object has no attribute 'Error'" in <generator object raw at 0x7fa3d9384410> ignored
[Fri Feb 05 11:16:52.104411 2016] [:error] [pid 1502:tid 140224457529088] /usr/lib/ckan/datapusher/lib/python2.7/site-packages/sqlalchemy/sql/sqltypes.py:185: SAWarning: Unicode type received non-unicode bind param value 'ca8c86c6-ca87-4b36-b6bb-3...'. (this warning may be suppressed after 10 occurrences)
[Fri Feb 05 11:16:52.119255 2016] [:error] [pid 1502:tid 140224457529088]   (util.ellipses_string(value),))
[Fri Feb 05 11:16:57.624008 2016] [:error] [pid 1502:tid 140224457529088] Job "push_to_datastore (trigger: RunTriggerNow, run = True, next run at: None)" raised an exception
[Fri Feb 05 11:16:57.624065 2016] [:error] [pid 1502:tid 140224457529088] Traceback (most recent call last):
[Fri Feb 05 11:16:57.624096 2016] [:error] [pid 1502:tid 140224457529088]   File "/usr/lib/ckan/datapusher/lib/python2.7/site-packages/apscheduler/scheduler.py", line 512, in _run_job
[Fri Feb 05 11:16:57.624106 2016] [:error] [pid 1502:tid 140224457529088]     retval = job.func(*job.args, **job.kwargs)
[Fri Feb 05 11:16:57.624116 2016] [:error] [pid 1502:tid 140224457529088]   File "/usr/lib/ckan/datapusher/src/datapusher/datapusher/jobs.py", line 287, in push_to_datastore
[Fri Feb 05 11:16:57.624125 2016] [:error] [pid 1502:tid 140224457529088]     resource = get_resource(resource_id, ckan_url, api_key)
[Fri Feb 05 11:16:57.624134 2016] [:error] [pid 1502:tid 140224457529088]   File "/usr/lib/ckan/datapusher/src/datapusher/datapusher/jobs.py", line 233, in get_resource
[Fri Feb 05 11:16:57.624143 2016] [:error] [pid 1502:tid 140224457529088]     check_response(r, url, 'CKAN')
[Fri Feb 05 11:16:57.624151 2016] [:error] [pid 1502:tid 140224457529088]   File "/usr/lib/ckan/datapusher/src/datapusher/datapusher/jobs.py", line 144, in check_response
[Fri Feb 05 11:16:57.624160 2016] [:error] [pid 1502:tid 140224457529088]     response=response.text)
[Fri Feb 05 11:16:57.624168 2016] [:error] [pid 1502:tid 140224457529088] HTTPError

If we don't use the root_path variable and set our subpath in site_url then Datapusher works again but some CKAN extensions like archiver/qa get broken as they need the root_path variable to get the subpath.

We think this is related to #2599

Any help to fix this would be appreciated. Thank you in advance.
@amercader amercader self-assigned this Feb 9, 2016
@amercader
Copy link
Member

The result_url is the callback URL that CKAN tells the DataPusher to ping once it has finished its importing. It should point to the /api/3/action/datapusher_hook endpoint. Right now this URL is constructed using ckan.site_url and my gut feeling is that this is the correct approach.

I think there is definitely some confusion regarding ckan.site_url and ckan.root_path.
@wardi here you seem to suggest that ckan.site_url should not have path components but that's not what the docs say and a quick grep over the code shows that is generally used as being the full base url of the CKAN site (eg).

I'm not sure what ckan.root_path was intended for (I guess for some i18n stuff) but I think that if extensions are relying on it to build URLs that's probably wrong

http://docs.ckan.org/en/ckan-2.5.1/maintaining/configuration.html#ckan-site-url

@wardi
Copy link
Contributor

wardi commented Feb 24, 2016

@amercader here's where I learned about root_path: #2599 (comment)
Given that routes supports defining the structure of the complete url from hostname down with this variable, it's incorrect to build urls with site_url alone. We need to use url_for which should take root_path into account. I'd say the docs and other places in the code where urls are constructed manually need to be updated.

@AdrianMBarrera
Copy link
Author

So, which is the best approach? Setting ckan.site_url (then some extensions don't work) or setting ckan.root_path (then DataPusher fails)?

@torfsen torfsen mentioned this issue Jun 2, 2016
Closed
@drnextgis
Copy link

I'm facing with the same issue.

@jheino jheino mentioned this issue Feb 17, 2017
Merged
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@amercader amercader

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Use the url_for() helper for datapusher URLs
4 participants
@wardi @amercader @drnextgis @AdrianMBarrera