Skip to content

Fix Python3 syntax issues in Python tests#5572

Merged
Schpidi merged 2 commits intoMapServer:masterfrom
claudep:py3tests
Apr 23, 2018
Merged

Fix Python3 syntax issues in Python tests#5572
Schpidi merged 2 commits intoMapServer:masterfrom
claudep:py3tests

Conversation

@claudep
Copy link
Copy Markdown
Contributor

@claudep claudep commented Mar 23, 2018

No description provided.

@Schpidi
Copy link
Copy Markdown
Member

Schpidi commented Mar 23, 2018

@geographika you should get notifications for this...

@geographika
Copy link
Copy Markdown
Member

@claudep - thanks for these. I'm working on updating Appveyor to run both p2 and py3 tests.
Could you exclude testConstructorBytesIO (by renaming xtestConstructorBytesIO) so the current Appveyor tests pass?
Even better would be if you can work out why the test is failing, and fix it.

@claudep
Copy link
Copy Markdown
Contributor Author

claudep commented Mar 29, 2018

I found a bunch of other things to update. However, I'm struggling to follow instructions to properly build python mapscript. Instructions are scattered through several files. What instructions should I follow?

@claudep
Copy link
Copy Markdown
Contributor Author

claudep commented Mar 29, 2018

Ha! now some tests run which did not run before, so we have new errors.
"POINT (5 7)" vs "POINT (5.0000000000000000 7.0000000000000000)"
I remember GEOS has a trim option for WKT output (http://geos.osgeo.org/doxygen/classgeos_1_1io_1_1WKTWriter.html#a53f742121d0757cd4ca83795ffab1c3f). Don't know if mapserver can use this API or not, or if we should simply account for this possible difference in the tests.

@geographika
Copy link
Copy Markdown
Member

There should be rounding applied to the tests from here - https://github.com/mapserver/mapserver/blob/branch-7-0/mapscript/python/tests/cases/testing.py#L118

Are you running on Linux or Windows when building MapScript? The .travis.yaml and AppVeyor.yaml files show working builds for each OS.

@claudep
Copy link
Copy Markdown
Contributor Author

claudep commented Mar 30, 2018

Linux-only here 😄

@claudep
Copy link
Copy Markdown
Contributor Author

claudep commented Mar 30, 2018

The test utilities you suggested are in the mapscript python tests, while the errors are in the mapserver/msautotest tests. Should we copy the same sort of utilities in msautotest?

@claudep
Copy link
Copy Markdown
Contributor Author

claudep commented Mar 31, 2018

I'm still getting errors when running Python mapscript tests, even if they are not visible on Travis. Specifically, hash table tests are failing, because since 10fe3d7, hashTableObj in Python is a dict instead of a swig proxy of hashTableObj (@rouault should know why this was done). However, this should also affect Python 2 and has nothing to do with this patch.

@geographika
Copy link
Copy Markdown
Member

@claudep - the conversion of the hashTableObj to a Python dict creates a much nicer Python API.
I believe the issue is in the pymodule.i file, and even when compiling with Python3 the Python2 dictionary creation is used:

#if PY_VERSION_HEX >= 0x03000000
        PyObject *py_key = PyUnicode_FromString(key);
        PyObject *py_val = PyUnicode_FromString(val);
#else
        PyObject *py_key = PyString_FromString(key);
        PyObject *py_val = PyString_FromString(val);
#endif

According to this link the above syntax is incorrect as "the macro PY_VERSION_HEX is only defined when compiling the code (and including the python header), but not when SWIG parses the code."

Could you try building with the following?

%#if PY_VERSION_HEX >= 0x03000000
        PyObject *py_key = PyUnicode_FromString(key);
        PyObject *py_val = PyUnicode_FromString(val);
%#else
        PyObject *py_key = PyString_FromString(key);
        PyObject *py_val = PyString_FromString(val);
%#endif

This is the approach/syntax used by @rouault in the GDAL bindings as can be seen here.

@claudep
Copy link
Copy Markdown
Contributor Author

claudep commented Apr 5, 2018

Thanks for these findings. I'm not sure I'll be able to work on this before the start of next week.

the conversion of the hashTableObj to a Python dict creates a much nicer Python API.

Yes I admit, but it is backwards incompatible as it contradicts the API documentation stating that keys are case-insensitive and several API disappeared (clear, nextkey, remove, set). The docs should explain that at least.

@claudep
Copy link
Copy Markdown
Contributor Author

claudep commented Apr 10, 2018

Just pushed with the suggested modifications.

@geographika
Copy link
Copy Markdown
Member

@claudep - excellent work, and an big step forward in getting the test suite matching the current API and ready for Python3.
Not sure if you are on the MapServer dev mailing lists, but I started a discussion on other major changes relating to Python MapScript - http://lists.osgeo.org/pipermail/mapserver-dev/2018-April/thread.html#15384 - would be great to have a few devs on this.

@Schpidi - I've reviewed the above changes. They all relate to Python MapScript files, and all enabled tests are passing, so I'd be for merging to master.

@Schpidi
Copy link
Copy Markdown
Member

Schpidi commented Apr 23, 2018

Sorry, that slipped through. Thanks for taking care!

@claudep claudep deleted the py3tests branch April 23, 2018 12:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants