Skip to content

Fixed parser of OpenCL 3.0 for NVIDIA GPU #2001

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

redradist
Copy link
Contributor

No description provided.

@redradist redradist changed the title Fixed parser of OpenCL for NVIDIA GPU Fixed parser of OpenCL 3.0 for NVIDIA GPU Jul 20, 2024
@vpirogov vpirogov added this to the v3.6 milestone Jul 22, 2024
Copy link
Contributor

@densamoilov densamoilov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please elaborate on the reasoning behind this PR?

@vpirogov
Copy link
Contributor

@redradist,

Adding 3.0 to the list of OpenCL version makes sense. We are still looking for your comments on the rationale for remaining changes.

@vpirogov vpirogov added the platform:gpu-nvidia Codeowner: @oneapi-src/onednn-gpu-nvidia label Jul 26, 2024
@redradist
Copy link
Contributor Author

redradist commented Jul 28, 2024

@vpirogov @densamoilov Because this code successfully detect version of OpenCL library, but this one with CMAKE_CHECK_SYMBOLS cannot, even with defined variables to include dirs and link libraries:

...

function(_FIND_OPENCL_VERSION)
  include(CheckSymbolExists)
  include(CMakePushCheckState)
  set(CMAKE_REQUIRED_QUIET ${OpenCL_FIND_QUIETLY})
  set(CMAKE_REQUIRED_INCLUDES ${OpenCL_INCLUDE_DIR})
  set(CMAKE_REQUIRED_LIBRARIES ${OpenCL_LIBRARY})

  CMAKE_PUSH_CHECK_STATE()
  foreach(VERSION "3_0" "2_2" "2_1" "2_0" "1_2" "1_1" "1_0")
    if(APPLE)
      CHECK_SYMBOL_EXISTS(
        CL_VERSION_${VERSION}
        "${OpenCL_INCLUDE_DIR}/Headers/cl.h"
        OPENCL_VERSION_${VERSION})
    else()
      CHECK_SYMBOL_EXISTS(
        CL_VERSION_${VERSION}
        "${OpenCL_INCLUDE_DIR}/CL/cl.h"
        OPENCL_VERSION_${VERSION})
    endif()

    message(STATUS "OPENCL_VERSION_${VERSION} = ${OPENCL_VERSION_${VERSION}}")

    if(OPENCL_VERSION_${VERSION})
      string(REPLACE "_" "." VERSION "${VERSION}")
      set(OpenCL_VERSION_STRING ${VERSION} PARENT_SCOPE)
      string(REGEX MATCHALL "[0-9]+" version_components "${VERSION}")
      list(GET version_components 0 major_version)
      list(GET version_components 1 minor_version)
      set(OpenCL_VERSION_MAJOR ${major_version} PARENT_SCOPE)
      set(OpenCL_VERSION_MINOR ${minor_version} PARENT_SCOPE)
      break()
    endif()
  endforeach()
  CMAKE_POP_CHECK_STATE()
endfunction()

...

For CL_VERSION_${VERSION} inside file /usr/include/CL/cl.h (not even in /usr/include/CL/cl_version.h), defines are empty ...
OPENCL_VERSION_${VERSION} is always empty, due to possible missed defines. There are ifdef instead of define with number.
Maybe due to this it cannot detection library version

CL_VERSION_${VERSION}
"${OpenCL_INCLUDE_DIR}/Headers/cl.h"
OPENCL_VERSION_${VERSION})
foreach(VERSION "3_0" "2_2" "2_1" "2_0" "1_2" "1_1" "1_0")
Copy link
Contributor

@densamoilov densamoilov Jul 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(random place)

There are a few things that look suspicious to me:

  • We didn't implement FindOpenCL.cmake module, we've taken it from CMake (see here) and it still uses CHECK_SYMBOL_EXISTS and the community seems to be fine with it
  • The version macros are standardized by the OpenCL specification so if the macros are missing then it likely means that there is something wrong with the OpenCL implementation you use
  • It should be fine to check cl.h header as it includes cl_version.h header and CHECK_SYMBOL_EXISTS does perform a preprocessing

Given all this, I think the issue you encounter is somewhere on your end.

Copy link
Contributor Author

@redradist redradist Aug 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@densamoilov Here is my OpenCL files for NVIDIA GPU:
cl_files.zip

It looks fine for me

@vpirogov vpirogov modified the milestone: v3.6 Sep 19, 2024
@vpirogov
Copy link
Contributor

Considering the discussion the best way forward is updating FindOpenCL.cmake from upstream.

@vpirogov vpirogov requested a review from a team as a code owner November 1, 2024 23:27
@redradist
Copy link
Contributor Author

redradist commented Dec 7, 2024

@densamoilov @vpirogov The CMake version says that it supports (Detection of OpenCL 2.1 and 2.2), also I've found that already someone implemented the similar version of FindOpenCL by trying to compile example project

@vpirogov
Copy link
Contributor

vpirogov commented Dec 9, 2024

@redradist, if FindOpenCL.cmake from upstream CMake works in your scenario please update oneDNN copy accordingly. Be mindful of SYCL related changes that are local to oneDNN copy though.

@redradist
Copy link
Contributor Author

@redradist, if FindOpenCL.cmake from upstream CMake works in your scenario please update oneDNN copy accordingly. Be mindful of SYCL related changes that are local to oneDNN copy though.

@vpirogov
No, it does not work. It supports only OpenCL 2.2 as I mentioned.
Test project compilation looks for me more reliable

@vpirogov
Copy link
Contributor

No, it does not work. It supports only OpenCL 2.2 as I mentioned.

Is this conclusion based on the comment in the file or actual testing? I see the comment comment that says 'Detection of OpenCL 2.1 and 2.2.', however commit history for the file includes patch indicating 3.0 support: Kitware/CMake@587d194.

We would like to avoid creating and maintaining completely new FindOpenCL.cmake.

@redradist
Copy link
Contributor Author

No, it does not work. It supports only OpenCL 2.2 as I mentioned.

Is this conclusion based on the comment in the file or actual testing?

It is based on actual testing. You could verify it by yourself, just install OpenCL driver for NVIDIA GPU on Ubuntu and you'll see what the problem.
My solution resolve issue completly

@redradist redradist force-pushed the fix/opencl-nvidia branch 2 times, most recently from a8904c7 to b048e70 Compare December 14, 2024 09:51
@redradist
Copy link
Contributor Author

No, it does not work. It supports only OpenCL 2.2 as I mentioned.

Is this conclusion based on the comment in the file or actual testing? I see the comment comment that says 'Detection of OpenCL 2.1 and 2.2.', however commit history for the file includes patch indicating 3.0 support: Kitware/CMake@587d194.

We would like to avoid creating and maintaining completely new FindOpenCL.cmake.

@vpirogov One additional comment, previously this cmake/FindOpenCL.cmake worked properly, but since NVIDIA update they driver with support of OpenCL 3.0, it is broken

@vpirogov
Copy link
Contributor

One additional comment, previously this cmake/FindOpenCL.cmake worked properly, but since NVIDIA update they driver with support of OpenCL 3.0, it is broken

Could you please indicate which NVIDIA OpenCL driver package you are testing with? I'm running Ubuntu 22.04 with the latest NVIDIA driver, but FindOpenCL.cmake from this PR fails:

$ apt list --installed | grep opencl

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

cuda-opencl-12-6/unknown,unknown,now 12.6.68-1 amd64 [installed,upgradable to: 12.6.77-1]
cuda-opencl-dev-12-6/unknown,unknown,now 12.6.68-1 amd64 [installed,upgradable to: 12.6.77-1]
ocl-icd-libopencl1/jammy,now 2.2.14-3 amd64 [installed,automatic]
(base) vpirogov@jfldnn043:~/work/dnnl/build$ ls
CheckOpenCLSymbol  CMakeCache.txt  CMakeFiles
(base) vpirogov@jfldnn043:~/work/dnnl/build$ rm -rf *
(base) vpirogov@jfldnn043:~/work/dnnl/build$ cmake .. -DONEDNN_GPU_RUNTIME=OCL
CMake Deprecation Warning at CMakeLists.txt:17 (cmake_minimum_required):
  Compatibility with CMake < 3.5 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.


-- CMAKE_BUILD_TYPE is unset, defaulting to Release
-- The C compiler identification is GNU 11.3.0
-- The CXX compiler identification is GNU 11.3.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- DNNL_TARGET_ARCH: X64
-- DNNL compat: set DNNL_GPU_RUNTIME to ONEDNN_GPU_RUNTIME with value `OCL`
-- DNNL_LIBRARY_NAME: dnnl
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")
-- GPU support is enabled (OpenCL)
-- Found OpenCL headers: OpenCL_INCLUDE_DIR-NOTFOUND
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
/nfs/pdx/home/vpirogov/work/dnnl/build/CheckOpenCLSymbol/OpenCL_INCLUDE_DIR
   used as include directory in directory /nfs/pdx/home/vpirogov/work/dnnl/build/CheckOpenCLSymbol
OpenCL_LIBRARY
    linked by target "TestOpenCL" in directory /nfs/pdx/home/vpirogov/work/dnnl/build/CheckOpenCLSymbol

CMake Error at cmake/FindOpenCL.cmake:78 (try_compile):
  Failed to generate test project build system.
Call Stack (most recent call first):
  cmake/FindOpenCL.cmake:189 (_FIND_OPENCL_VERSION)
  cmake/OpenCL.cmake:35 (find_package)
  CMakeLists.txt:130 (include)


-- Configuring incomplete, errors occurred!

Environment details:

$ uname -a
Linux jfldnn043 5.15.0-56-generic #62-Ubuntu SMP Tue Nov 22 19:54:14 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

$ apt list --installed | grep nvidia-driver

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

nvidia-driver-560-open/unknown,now 560.35.03-0ubuntu1 amd64 [installed,upgradable to: 560.35.05-0ubuntu1]

$ apt list --installed | grep opencl

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

cuda-opencl-12-6/unknown,unknown,now 12.6.68-1 amd64 [installed,upgradable to: 12.6.77-1]
cuda-opencl-dev-12-6/unknown,unknown,now 12.6.68-1 amd64 [installed,upgradable to: 12.6.77-1]
ocl-icd-libopencl1/jammy,now 2.2.14-3 amd64 [installed,automatic]

FindOpenCL.cmake from upstream works fine in this environment.

@redradist
Copy link
Contributor Author

One additional comment, previously this cmake/FindOpenCL.cmake worked properly, but since NVIDIA update they driver with support of OpenCL 3.0, it is broken

Could you please indicate which NVIDIA OpenCL driver package you are testing with? I'm running Ubuntu 22.04 with the latest NVIDIA driver, but FindOpenCL.cmake from this PR fails:

$ apt list --installed | grep opencl

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

cuda-opencl-12-6/unknown,unknown,now 12.6.68-1 amd64 [installed,upgradable to: 12.6.77-1]
cuda-opencl-dev-12-6/unknown,unknown,now 12.6.68-1 amd64 [installed,upgradable to: 12.6.77-1]
ocl-icd-libopencl1/jammy,now 2.2.14-3 amd64 [installed,automatic]
(base) vpirogov@jfldnn043:~/work/dnnl/build$ ls
CheckOpenCLSymbol  CMakeCache.txt  CMakeFiles
(base) vpirogov@jfldnn043:~/work/dnnl/build$ rm -rf *
(base) vpirogov@jfldnn043:~/work/dnnl/build$ cmake .. -DONEDNN_GPU_RUNTIME=OCL
CMake Deprecation Warning at CMakeLists.txt:17 (cmake_minimum_required):
  Compatibility with CMake < 3.5 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.


-- CMAKE_BUILD_TYPE is unset, defaulting to Release
-- The C compiler identification is GNU 11.3.0
-- The CXX compiler identification is GNU 11.3.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- DNNL_TARGET_ARCH: X64
-- DNNL compat: set DNNL_GPU_RUNTIME to ONEDNN_GPU_RUNTIME with value `OCL`
-- DNNL_LIBRARY_NAME: dnnl
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")
-- GPU support is enabled (OpenCL)
-- Found OpenCL headers: OpenCL_INCLUDE_DIR-NOTFOUND
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
/nfs/pdx/home/vpirogov/work/dnnl/build/CheckOpenCLSymbol/OpenCL_INCLUDE_DIR
   used as include directory in directory /nfs/pdx/home/vpirogov/work/dnnl/build/CheckOpenCLSymbol
OpenCL_LIBRARY
    linked by target "TestOpenCL" in directory /nfs/pdx/home/vpirogov/work/dnnl/build/CheckOpenCLSymbol

CMake Error at cmake/FindOpenCL.cmake:78 (try_compile):
  Failed to generate test project build system.
Call Stack (most recent call first):
  cmake/FindOpenCL.cmake:189 (_FIND_OPENCL_VERSION)
  cmake/OpenCL.cmake:35 (find_package)
  CMakeLists.txt:130 (include)


-- Configuring incomplete, errors occurred!

Environment details:

$ uname -a
Linux jfldnn043 5.15.0-56-generic #62-Ubuntu SMP Tue Nov 22 19:54:14 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

$ apt list --installed | grep nvidia-driver

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

nvidia-driver-560-open/unknown,now 560.35.03-0ubuntu1 amd64 [installed,upgradable to: 560.35.05-0ubuntu1]

$ apt list --installed | grep opencl

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

cuda-opencl-12-6/unknown,unknown,now 12.6.68-1 amd64 [installed,upgradable to: 12.6.77-1]
cuda-opencl-dev-12-6/unknown,unknown,now 12.6.68-1 amd64 [installed,upgradable to: 12.6.77-1]
ocl-icd-libopencl1/jammy,now 2.2.14-3 amd64 [installed,automatic]

FindOpenCL.cmake from upstream works fine in this environment.

I'm using the following OpenCL packages:

$ apt list --installed | grep opencl

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

ocl-icd-libopencl1/jammy,now 2.2.14-3 amd64 [installed,automatic]
ocl-icd-libopencl1/jammy,now 2.2.14-3 i386 [installed,automatic]
ocl-icd-opencl-dev/jammy,now 2.2.14-3 amd64 [installed]
opencl-c-headers/jammy,jammy,now 3.0~2022.01.04-1 all [installed,automatic]
opencl-clhpp-headers/jammy,jammy,now 3.0~2.0.15-1ubuntu1 all [installed,automatic]
opencl-headers/jammy,jammy,now 3.0~2022.01.04-1 all [installed]

It is very strange that your version failed to compile ...

I use CUDA 11.8 and there are not packages like cuda-opencl-11-8 in repository

@vpirogov
Copy link
Contributor

vpirogov commented Dec 16, 2024

I'm using the following OpenCL packages:

This is Kronos standard OpenCL loader and headers. It's vendor neutral and works perfectly fine with current FindOpenCL.cmake as tested on clean Ubuntu 22.04 image. This PR fails though:

$ cmake .. -DONEDNN_GPU_RUNTIME=OCL
-- CMAKE_BUILD_TYPE is unset, defaulting to Release
-- The C compiler identification is GNU 11.4.0
-- The CXX compiler identification is GNU 11.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- DNNL_TARGET_ARCH: X64
-- DNNL compat: set DNNL_GPU_RUNTIME to ONEDNN_GPU_RUNTIME with value `OCL`
-- DNNL_LIBRARY_NAME: dnnl
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")
-- GPU support is enabled (OpenCL)
-- Found OpenCL headers: /usr/include
CMake Warning (dev) at cmake/FindOpenCL.cmake:78 (try_compile):
  try_compile given unknown argument "BINARY_DIR".
Call Stack (most recent call first):
  cmake/FindOpenCL.cmake:189 (_FIND_OPENCL_VERSION)
  cmake/OpenCL.cmake:35 (find_package)
  CMakeLists.txt:130 (include)
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) at cmake/FindOpenCL.cmake:78 (try_compile):
  try_compile given unknown argument
  "/nfs/pdx/home/vpirogov/work/dnnl/build/CheckOpenCLSymbol".
Call Stack (most recent call first):
  cmake/FindOpenCL.cmake:189 (_FIND_OPENCL_VERSION)
  cmake/OpenCL.cmake:35 (find_package)
  CMakeLists.txt:130 (include)
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Error: The source directory "CheckOpenCLSymbol" does not exist.
Specify --help for usage, or press the help button on the CMake GUI.
CMake Error at cmake/FindOpenCL.cmake:78 (try_compile):
  Failed to configure test project build system.
Call Stack (most recent call first):
  cmake/FindOpenCL.cmake:189 (_FIND_OPENCL_VERSION)
  cmake/OpenCL.cmake:35 (find_package)
  CMakeLists.txt:130 (include)


-- Configuring incomplete, errors occurred!
See also "/nfs/pdx/home/vpirogov/work/dnnl/build/CMakeFiles/CMakeOutput.log".

Looks like the problem you are trying to address is somewhere else.

@redradist
Copy link
Contributor Author

@vpirogov Issue was more deeply investigated and new PR was opened with smaller fix, see #2378

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
platform:gpu-nvidia Codeowner: @oneapi-src/onednn-gpu-nvidia
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants