Dev setup conflicts -- Update developer guide?

[ErichKramer]

Hey I was trying to run the test cases for foolbox and I ran into some issues. Did I overlook something in the development guide? Should I create a PR to update dependencies and have explicit virtualenv steps?

• I used virtualenv to make a virtualenvironment and I installed the dev-py3 reqs. I got an issue telling me that python-coveralls coverage requirement was incompatible so I manually removed coverage and installed version 4.0.3, which should have been compatible with coveralls.

• I ran the test cases and got issues telling me tensorflow was not installed. This repeated so that I intsalled the python3 modules for theano, torch, and tensorflow.

• Did an editable install of foolbox as directed in docs at root of repo

This run of pytest tells me that it cannot find modules, lasagne, mxnet, keras.

• After installing these I am given downsample issues, for this I have to downgrade to theano 0.7 as it and lasagne have compatibility problems. Futher discussed here: ImportError: cannot import name downsample aigamedev/scikit-neuralnetwork#235

Only after all of these fixes was I able to properly run the testing. Even then I have theano failures and I'm not sure what module versions are correct, I am getting assertion failures.

My pytest output is available here. http://web.engr.oregonstate.edu/~kramerer/theano_failure

I think that the requirements should be updated for development, and the guide should be fleshed out.

[jonasrauber]

Please check the travis.yml file that configures our automated tests. Dependencies such as TensorFlow are not installed as part of the package requirements, because we cannot know whether the user wants tensorflow-gpu or tensorflow and because users often just want one the frameworks, e.g. pytorch, not all of them.
Regarding all the other things, I didn't really get the problem. Again, I hope that looking at the travis.yml file will help you understand what we do to run the tests. If you still encounter problems, please describe the steps to reproduce them in detail and let me know where the documentation is wrong. It would indeed be great to have a better development documentation.

[ErichKramer]

I think I misunderstand how testing works.
On the top level I read the following from the docs:

To run the tests, you need to have pytest and pytest-cov installed. Afterwards, you can simply run pytest in the root folder of the project.

https://foolbox.readthedocs.io/en/latest/user/development.html

And tried to run pytest. I ran into some dependency and module issues and my issue talks through how I resolved them to run the tests on my machine.

I'm not very familiar with Travis so that is a factor into my misunderstanding. Are the tests meant to only run when changes are pushed to github?

[jonasrauber]

No, you didn't misunderstand it. I just pointed you to the travis.yml config file, because it's one way to figure out how our automated testing works and thus to see what you need to run the tests locally.

We should certainly add a statement to the documentation that one needs to install all the frameworks (TensorFlow, PyTorch, …) to run all the tests. If you want it would be great if would make a pull-request adding this clarification to the documentation, based on the dependencies we list in the travis config.