Notify CMake on Windows about CUDA

[Haydnspass]

Hey,

Thanks for your awesome Github action! My question might be a bit out of scope of this repo, but I wonder whether you could still help me.
I am trying to build something on windows and ubuntu and need nvcc for that. The build process is like:
conda-build --> ptyhon setup.py --> CMake --> nvcc

Locally, the build runs fine, but I wonder how I can tell CMake the CUDACXX on a windows runner?
If you want to have a look, you can find the workflow below. I tried setting the CUDACXX via powershell but CMake does not find it.
Any ideas?

https://github.com/TuragaLab/SplinePSF/blob/9aa4e4569fce8550a7b22295191447f88ee592e9/.github/workflows/build_upload_test.yaml#L54-L80

[DStrelak]

Hi @Haydnspass
for cmake, one should be able to set variables via cmake -DVARIABLE1=value1. Otherwise I've seen using following syntax:

- name: SomeName
        run: myCommand
        env:
          VARIABLE1: 1

or

echo "VARIABLE1=1" >> $GITHUB_ENV

[Jimver]

Hi @Haydnspass
for cmake, one should be able to set variables via cmake -DVARIABLE1=value1. Otherwise I've seen using following syntax:

- name: SomeName
        run: myCommand
        env:
          VARIABLE1: 1

or

echo "VARIABLE1=1" >> $GITHUB_ENV

Yes this is the proper way to pass environment variables around. Unfortunattely setting an environment variable in the shell in one step isn't saved to other steps.

[DStrelak]

Unfortunattely setting an environment variable in the shell in one step isn't saved to other steps.

For that, one can use global variables (e.g. GITHUB_PATH). Mind though that changes to these do not apply in the current action :D

[Haydnspass]

Thanks for your answers! :) However, how would I set CUDACXX to your CUDA_Path? I think they are equivalent

This, naive, idea does not work (ignore the intendations)

- name: "Install CUDA Toolkit"
        if: ${{ matrix.os_base == 'windows' || matrix.os_base == 'ubuntu'}}
        uses: Jimver/cuda-toolkit@v0.1.0
        id: cuda-toolkit
        with:
          cuda: '11.2.2'

- name: try to echo
  if: ${{ matrix.os_base == 'windows' }}
  env:
    CUDACXX: $env:CUDA_PATH
  run: |
    echo $Env:CUDACXX

[Jimver]

You could use something along these lines:
https://docs.github.com/en/actions/reference/workflow-commands-for-github-actions#setting-an-environment-variable

So

- name: Set CUDACXX
    run: |
        echo "CUDACXX=$CUDA_PATH" >> $GITHUB_ENV

Not sure how that translates to powershell though.

This way it will be saved across different steps.

[Haydnspass]

Hm Okay,
so I managed to set and save the environment variable (by the way it needs to be CUDACXX=${{ env.CUDA_PATH }}\bin\nvcc.exe) when I use it directly.
Strangly, cmake

• finds the cuda toolkit
• warns me that is has not used the CUDACXX variable ?!
• does not find the CUDA compiler

I don't quite get it 😅 If you want to have a direct look, you can see the workflow here: https://github.com/TuragaLab/SplinePSF/actions/runs/682310612/workflow (only windows relevant for now)

Here is what it says

# first I queried where nvcc is
Run Get-Command nvcc | Select-Object -ExpandProperty Definition
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2\bin\nvcc.exe

    Directory: D:\a\SplinePSF\SplinePSF\cpp_cuda_c


Mode                LastWriteTime         Length Name                                                                  
----                -------------         ------ ----                                                                  
d-----        3/24/2021   8:13 AM                build                                                                 
-- The CXX compiler identification is GNU 8.1.0
-- The C compiler identification is GNU 8.1.0
-- Check for working CXX compiler: C:/ProgramData/chocolatey/bin/c++.exe
-- Check for working CXX compiler: C:/ProgramData/chocolatey/bin/c++.exe -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Check for working C compiler: C:/ProgramData/chocolatey/bin/gcc.exe
-- Check for working C compiler: C:/ProgramData/chocolatey/bin/gcc.exe -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Looking for a CUDA compiler
-- Looking for a CUDA compiler - NOTFOUND
-- Found Python: C:/hostedtoolcache/windows/Python/3.8.8/x64/python3.exe (found version "3.8.8") found components:  Interpreter Development 
-- Performing Test HAS_FLTO
-- Performing Test HAS_FLTO - Success
-- Found pybind11: C:/Miniconda/envs/spline_dev/Library/include (found version "2.6.2" )
-- Found CUDA: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.2 (found version "11.2") 
CMake Warning at CMakeLists.txt:20 (message):
  No CUDA compiler found or disabled by user defined command line variable
  '-DCUDA_DISABLED=True'.

     You may need to specify its path like 	'export CUDACXX=/usr/local/cuda/bin/nvcc'


-- Configuring done
-- Generating done
CMake Warning:
  Manually-specified variables were not used by the project:

    CUDACXX


-- Build files have been written to: D:/a/SplinePSF/SplinePSF/cpp_cuda_c/build
[1/4] Building C object CMakeFiles/spline_psf_cpu_impl.dir/src/spline_psf.c.obj
[2/4] Linking C static library libspline_psf_cpu_impl.a
[3/4] Building CXX object CMakeFiles/spline.dir/src/pybind_spline.cpp.obj
[4/4] Linking CXX shared module spline.cp38-win_amd64.pyd

[Jimver]

I think cmake sometimes has trouble with backslash paths, maybe you can try something like this:

run: |
  $CUDA_PATH_FORWARD_SLASH = "$env:CUDA_PATH" -replace "\\", "/"
  cmake -GNinja -DCUDACXX="${{ env.CUDA_PATH_FORWARD_SLASH }}/bin/nvcc.exe" ..

Edit: You should also use quotes for filepaths with spaces (thanks Program Files (x86) ...):

[Haydnspass]

Very good comment. Still don't manage... Found somewhere that it actually needs to be just CUDA instead of CUDACXX in "new CMAKE". This lead to CMAKE at least not complaining about an unused env variable (which I believe is good?)
I also hard-coded the path with UNIX slashes, still doesn't work... Very strange.

https://github.com/TuragaLab/SplinePSF/actions/runs/683102570/workflow

[Jimver]

I think I made it work using an updated CMake plus configuration change: see my PR: TuragaLab/SplinePSF#12

[anders-wind]

Works for ubuntu-20.04, fails on windows-2019. I'm using find_package(CUDAToolkit) in my cmake file and it finds the root just fine(Found CUDAToolkit: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.2/include (found version "11.2.152") ).
Unfortunately it fails with The CUDA compiler identification is unknown and then later: No CMAKE_CUDA_COMPILER could be found.

I've tried to see what you did to help @Haydnspass (such as replacing backslashes with forward slashes - setting CMAKE_CUDA_COMPILER) but it does not seem to have helped

nvcc --version is also available when running as a command so it should be "findable" in path

[anders-wind]

Print of the contents in CMakeFiles/CMakeError.log after failing:

C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Microsoft\VC\v160\BuildCustomizations\CUDA 11.2.targets(606,9): error : The CUDA Toolkit v11.2 directory '' does not exist.  Please verify the CUDA Toolkit is installed properly or define the CudaToolkitDir property to resolve this error. [D:\a\build\CMakeFiles\3.20.3\CompilerIdCUDA\CompilerIdCUDA.vcxproj]

I seem to recall there being some problems with some missing VS toolchain files for express CUDA installations - might be related.

EDIT: Looks like the right files are located in the VS folder:

     Directory: C:\Program Files (x86)\Microsoft Visual 
Studio\2019\Enterprise\MSBuild\Microsoft\VC\v160\BuildCustomizations

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a---           2/15/2021 12:39 PM          14544 CUDA 11.2.props
-a---           2/15/2021 12:39 PM          51464 CUDA 11.2.targets
-a---           2/15/2021 12:39 PM          32298 CUDA 11.2.xml
-a---           5/31/2021  4:02 AM            710 ImageContentTask.props
-a---           5/31/2021  4:02 AM           3271 ImageContentTask.targets
-a---           5/31/2021  4:02 AM           4710 ImageContentTask.xml
-a---           5/31/2021  4:02 AM           1186 lc.props
-a---           5/31/2021  4:02 AM           4091 lc.targets
-a---           5/31/2021  4:02 AM           6503 lc.xml
-a---           5/31/2021  4:02 AM           1276 marmasm.props
-a---           5/31/2021  4:02 AM           3533 marmasm.targets
-a---           5/31/2021  4:02 AM           8302 marmasm.xml
-a---           5/31/2021  4:02 AM           1450 masm.props
-a---           5/31/2021  4:02 AM           4716 masm.targets
-a---           5/31/2021  4:02 AM          14652 masm.xml
-a---           5/31/2021  4:02 AM            701 MeshContentTask.props
-a---           5/31/2021  4:02 AM           3156 MeshContentTask.targets
-a---           5/31/2021  4:02 AM           1490 MeshContentTask.xml
-a---           2/15/2021 12:39 PM         265728 Nvda.Build.CudaTasks.v11.2.dll
-a---           5/31/2021  4:02 AM            764 ShaderGraphContentTask.props
-a---           5/31/2021  4:02 AM           3277 ShaderGraphContentTask.targets
-a---           5/31/2021  4:02 AM           1490 ShaderGraphContentTask.xml

[anders-wind]

According to: https://gitlab.kitware.com/cmake/cmake/-/issues/18059 it could be a missing CUDA_PATH_VX_Y

Could it be that the code:

core.exportVariable(
    'CUDA_PATH_VX_Y',
    `CUDA_PATH_V${version.major}_${version.minor}`
)

Should be changed to (or in addition have):

core.exportVariable(
    `CUDA_PATH_V${version.major}_${version.minor}`,
    cudaPath
)

EDIT: Yep that solves it :) Creating a PR.