Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add tensorflow with gpu support for windows #567

Merged
merged 3 commits into from
Jun 2, 2018
Merged

Add tensorflow with gpu support for windows #567

merged 3 commits into from
Jun 2, 2018

Conversation

Neiko2002
Copy link
Member

Tested with cuda 9.0 + cudnn 7.0 and cuda 9.2 + cudnn 7.1

Is it possible to merge attributes from other platform annotation? Right now I had to copy the "link" attributes from the "windows" platform to the "windows-x86_64" with "-gpu" extension.

Furthermore it might be better to exchange the "-gpu" extension with "-cuda90" or "-cuda92". On the other hand all the different configurations are horrible (e.g. cuda version, cudnn version, cpu instruction set) to maintain.

@saudet
Copy link
Member

saudet commented May 30, 2018

Unfortunately, it's not currently possible to "merge" annotation values as it would make it hard to override them. If you have a good idea to enhance JavaCPP that would allow both concatenating and overriding results easily with annotations like this, let me know!

Other than that, we'll need to leave -DPYTHON_EXECUTABLE="C:/Python27/python.exe" in there because it won't build with the version of Python that comes with MSYS2.

@Neiko2002
Copy link
Member Author

Added the PYTHON_EXECUTABLE again.

@saudet
Copy link
Member

saudet commented May 31, 2018

I can't get it to build here. The Python script don't seem to be running properly. Which distribution of Python are you using?

@Neiko2002
Copy link
Member Author

Neiko2002 commented May 31, 2018

Anaconda with python 3.6 was in the path of the system. Which python script has problems?

@saudet
Copy link
Member

saudet commented May 31, 2018

No, it doesn't seem to be because of the version of Python, it doesn't work with 3.6.5 either. I'm getting this kind of error:

CMake Error: Could not open file for write in copy operation C:/msys64/tmp/javacpp-presets/tensorflow/cppbuild/windows-x86_64-gpu/tensorflow-1.8.0/tensorflow/contrib/cmake/build/CMakeFiles/tf_core_gpu_kernels.dir/__/__/core/kernels/tf_core_gpu_kernels_generated_cwise_op_gpu_squared_difference.cu.cc.obj.cmake.pre-gen.tmp
CMake Error: : System Error: No such file or directory
CMake Error at C:/msys64/mingw64/share/cmake-3.8/Modules/FindCUDA.cmake:1511 (configure_file):
  configure_file Problem configuring file
Call Stack (most recent call first):
  C:/msys64/mingw64/share/cmake-3.8/Modules/FindCUDA.cmake:1723 (CUDA_WRAP_SRCS)
  tf_core_kernels.cmake:217 (cuda_add_library)
  CMakeLists.txt:448 (include)

A Python script somewhere is supposed to create those .cmake.pre-gen.tmp files, but they don't get (all) created for some reason here on my installation of Windows 10.

@saudet
Copy link
Member

saudet commented May 31, 2018

All the failing paths are longer than 260 characters, so this has something to do with the MAX_PATH limit:
https://bugs.python.org/issue18199
https://bugs.python.org/issue27731
But enabling support for it in the registry does not help, hum...

@Neiko2002
Copy link
Member Author

Neiko2002 commented May 31, 2018

I had path length problems somewhere else, so my directory is just called: f:/G/JP/tensorflow/. But if this is really an issue we should check it somewhere and tell the user beforehand.

Instead of building tensorflow in contrib/cmake/build, we could build it somewhere else. The CI scripts of tensorflow for example, create a "cmake_build" directory at the top level directory.

@saudet saudet merged commit e6c8c08 into bytedeco:master Jun 2, 2018
@saudet
Copy link
Member

saudet commented Jun 4, 2018

This is working well for me, thanks! I've uploaded snapshots manually and will keep doing that until we figure out what to do about the CI service (issue #568).

Let's hope they get Bazel working before long; a build time of 4 hours is pretty ridiculous. As far as I know, we're only missing support for static libraries: bazelbuild/bazel#1920

BTW, keep an eye on tensorflow/tensorflow#17390. Google is expending effort to build a community around support for Java.

@Neiko2002
Copy link
Member Author

@saudet Thanks for pointing me to #17390. There are too many Tensorflow Github tickets to follow. I will be at a conference in Yokohama (Japan) next week and have same holidays afterwards. When I'm back in office I could start working on examples for feature.pb.h, example.pb.h, record_reader.h, record_writer.h.

The other day I was building Tensorflow with Bazel but had the feeling it takes much longer on Windows and uses less available resources as cmake.

@saudet
Copy link
Member

saudet commented Jun 5, 2018

Which conference is that BTW? I live in Japan, not too far from Yokohama/Tokyo...

@Neiko2002
Copy link
Member Author

Neiko2002 commented Jun 5, 2018

It is the ICMR 2018 from 11th to 14th of June. After some traveling around the country I will be back in Yokohama from 25th-30th and will be in Tokyo from 30th-7th. If you want to meet we better discuss the details via email.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants