Closed
Description
With the release of XGBoost 1.0.x (i.e xgboost-1.0.1-py3-none-manylinux1_x86_64.whl), it seems that installing TVM from scratch (rebuilding Docker containers) makes tests/python/unittest/test_autotvm_xgboost_model.py to fail with a segfault.
Investigating it a bit further, if I manually revert it to xgboost-0.90 it works fine. Using xgboost-1.0.1, this is the message I see:
tests/python/unittest/test_autotvm_xgboost_model.py::test_fit Fatal Python error: Segmentation fault
Thread 0x00007f4f98de4700 (most recent call first):
File "/usr/lib/python3.6/multiprocessing/connection.py", line 379 in _recv
File "/usr/lib/python3.6/multiprocessing/connection.py", line 407 in _recv_bytes
File "/usr/lib/python3.6/multiprocessing/connection.py", line 250 in recv
File "/usr/lib/python3.6/multiprocessing/pool.py", line 463 in _handle_results
File "/usr/lib/python3.6/threading.py", line 864 in run
File "/usr/lib/python3.6/threading.py", line 916 in _bootstrap_inner
File "/usr/lib/python3.6/threading.py", line 884 in _bootstrap
Thread 0x00007f4f905e3700 (most recent call first):
File "/usr/lib/python3.6/threading.py", line 295 in wait
File "/usr/lib/python3.6/queue.py", line 164 in get
File "/usr/lib/python3.6/multiprocessing/pool.py", line 415 in _handle_tasks
File "/usr/lib/python3.6/threading.py", line 864 in run
File "/usr/lib/python3.6/threading.py", line 916 in _bootstrap_inner
File "/usr/lib/python3.6/threading.py", line 884 in _bootstrap
Thread 0x00007f4f8fde2700 (most recent call first):
File "/usr/lib/python3.6/multiprocessing/pool.py", line 406 in _handle_workers
File "/usr/lib/python3.6/threading.py", line 864 in run
File "/usr/lib/python3.6/threading.py", line 916 in _bootstrap_inner
File "/usr/lib/python3.6/threading.py", line 884 in _bootstrap
Current thread 0x00007f4fb514c700 (most recent call first):
File "/usr/local/lib/python3.6/dist-packages/xgboost/core.py", line 1248 in update
File "/usr/local/lib/python3.6/dist-packages/xgboost/training.py", line 74 in _train_internal
File "/usr/local/lib/python3.6/dist-packages/xgboost/training.py", line 209 in train
File "/workspace/python/tvm/autotvm/tuner/xgboost_cost_model.py", line 272 in fit_log
File "/workspace/tests/python/unittest/test_autotvm_xgboost_model.py", line 35 in test_fit
File "/usr/local/lib/python3.6/dist-packages/_pytest/python.py", line 167 in pytest_pyfunc_call
File "/usr/local/lib/python3.6/dist-packages/pluggy/callers.py", line 187 in _multicall
File "/usr/local/lib/python3.6/dist-packages/pluggy/manager.py", line 87 in <lambda>
File "/usr/local/lib/python3.6/dist-packages/pluggy/manager.py", line 93 in _hookexec
File "/usr/local/lib/python3.6/dist-packages/pluggy/hooks.py", line 286 in __call__
File "/usr/local/lib/python3.6/dist-packages/_pytest/python.py", line 1445 in runtest
File "/usr/local/lib/python3.6/dist-packages/_pytest/runner.py", line 134 in pytest_runtest_call
File "/usr/local/lib/python3.6/dist-packages/pluggy/callers.py", line 187 in _multicall
File "/usr/local/lib/python3.6/dist-packages/pluggy/manager.py", line 87 in <lambda>
File "/usr/local/lib/python3.6/dist-packages/pluggy/manager.py", line 93 in _hookexec
File "/usr/local/lib/python3.6/dist-packages/pluggy/hooks.py", line 286 in __call__
File "/usr/local/lib/python3.6/dist-packages/_pytest/runner.py", line 210 in <lambda>
File "/usr/local/lib/python3.6/dist-packages/_pytest/runner.py", line 237 in from_call
File "/usr/local/lib/python3.6/dist-packages/_pytest/runner.py", line 210 in call_runtest_hook
File "/usr/local/lib/python3.6/dist-packages/_pytest/runner.py", line 185 in call_and_report
File "/usr/local/lib/python3.6/dist-packages/_pytest/runner.py", line 99 in runtestprotocol
File "/usr/local/lib/python3.6/dist-packages/_pytest/runner.py", line 84 in pytest_runtest_protocol
File "/usr/local/lib/python3.6/dist-packages/pluggy/callers.py", line 187 in _multicall
File "/usr/local/lib/python3.6/dist-packages/pluggy/manager.py", line 87 in <lambda>
File "/usr/local/lib/python3.6/dist-packages/pluggy/manager.py", line 93 in _hookexec
File "/usr/local/lib/python3.6/dist-packages/pluggy/hooks.py", line 286 in __call__
File "/usr/local/lib/python3.6/dist-packages/_pytest/main.py", line 271 in pytest_runtestloop
File "/usr/local/lib/python3.6/dist-packages/pluggy/callers.py", line 187 in _multicall
File "/usr/local/lib/python3.6/dist-packages/pluggy/manager.py", line 87 in <lambda>
File "/usr/local/lib/python3.6/dist-packages/pluggy/manager.py", line 93 in _hookexec
File "/usr/local/lib/python3.6/dist-packages/pluggy/hooks.py", line 286 in __call__
File "/usr/local/lib/python3.6/dist-packages/_pytest/main.py", line 247 in _main
File "/usr/local/lib/python3.6/dist-packages/_pytest/main.py", line 197 in wrap_session
File "/usr/local/lib/python3.6/dist-packages/_pytest/main.py", line 240 in pytest_cmdline_main
File "/usr/local/lib/python3.6/dist-packages/pluggy/callers.py", line 187 in _multicall
File "/usr/local/lib/python3.6/dist-packages/pluggy/manager.py", line 87 in <lambda>
File "/usr/local/lib/python3.6/dist-packages/pluggy/manager.py", line 93 in _hookexec
File "/usr/local/lib/python3.6/dist-packages/pluggy/hooks.py", line 286 in __call__
File "/usr/local/lib/python3.6/dist-packages/_pytest/config/__init__.py", line 93 in main
File "/usr/local/lib/python3.6/dist-packages/pytest/__main__.py", line 7 in <module>
File "/usr/lib/python3.6/runpy.py", line 85 in _run_code
File "/usr/lib/python3.6/runpy.py", line 193 in _run_module_as_main
./tests/scripts/task_python_unittest.sh: line 27: 24582 Segmentation fault (core dumped) TVM_FFI=ctypes python3 -m pytest -v tests/python/unittest
@tqchen, I didn't see any PR or discussion about it, but are you aware about any ongoing initiative to move TVM to XGBoost 1.0.x, or shall we pin xgboost to be 0.90, to prevent the error to happen? (note: I'm happy to send a patch to pin the version)
Metadata
Metadata
Assignees
Labels
No labels