Trying to get Travis-CI builds working #194

rwest · 2014-03-28T21:19:50Z

This is where we get Travis-CI continuous integration working!

For some reason Github think my rwest/RMG-Py is no longer the same project as GreenGroup/RMG-Py and I cannot send pull requests. While I wait for them to fix this, I have pushed this topic branch to GreenGroup and am sending the pull request from there.
I wanted to do it via a pull request to see how the Travis-CI thing handles it, and because it may take several commits to get right…
Let's not merge until it seems to be working
https://travis-ci.org/GreenGroup/RMG-Py/

Current status:

rwest · 2014-03-28T22:17:44Z

Partial success with the last commit.
The Python 2.6 build went ahead and built Scipy from source anyway, but at least
the Python 2.7 build used the apt-get version, with the line Requirement already satisfied (use --upgrade to upgrade): scipy in /usr/lib/python2.7/dist-packages (from -r requirements.txt (line 8))

As for RDKit, it got past the Checking you have rdkit… line, but now fails on
Checking rdkit has InChI support... Traceback (most recent call last): File "<string>", line 1, in <module> AssertionError: RDKit installed without InChI Support

Until we get MOPAC etc running we're not going to be able to do QM things, so I'm tempted to try a build config that doesn't try QM (and thus won't need the InChI support, I think…?)

wking · 2014-03-28T22:28:35Z

On Fri, Mar 28, 2014 at 03:17:44PM -0700, Richard West wrote:

Partial success with the last commit.
The Python 2.6 build went ahead and built Scipy from source anyway, but at least

You can (apparently) cache these builds to save time in later runs 1.

the Python 2.7 build used the apt-get version, with the line
Requirement already satisfied (use --upgrade to upgrade): scipy in /usr/lib/python2.7/dist-packages (from -r requirements.txt (line 8))

That's nice, but surprising. Travis claims to use isolated
virtualenvs, which should make system-wide Python modules inaccessible
2.

As for RDKit, it got past the Checking you have rdkit… line, but now fails on
Checking rdkit has InChI support... Traceback (most recent call last): File "<string>", line 1, in <module> AssertionError: RDKit installed without InChI Support

Until we get MOPAC etc running we're not going to be able to do QM
things, so I'm tempted to try a build config that doesn't try QM
(and thus won't need the InChI support, I think…?)

And it probably failed (if it even got that far) for Python 2.6, since
we're not building RDKit. It may be better to remove Python 2.6 from
testing until we work the kinks out with 2.7.

rwest · 2014-03-28T22:39:06Z

I think the cacheing is only for private repositories so we'd have to pay $129/month.
The system_site_packages option I got from this blog post
I have removed Python 2.6, and the InChI support check for now, until we get it working

wking · 2014-03-28T22:44:17Z

On Fri, Mar 28, 2014 at 03:39:06PM -0700, Richard West wrote:

I think the cacheing is only for private repositories so we'd have
to pay $129/month.

Ah, that explains why I hadn't seen it in other projects using Travis
;).

The system_site_packages option I got from this blog
post

Lovely. That is much nicer than building them for each test.

rwest · 2014-03-28T23:14:22Z

Making progress. Couple of problems.

Failure: ImportError (No module named pydqed) ... ERROR
Should check this is in our installation instructions, and add to the Travis script (or Makefile).
a segfault :-( this is reminiscent of when we don't build RDKit from source using exactly the same compilers and libraries as everything else. This could be a pain, as RDKit takes for ever to build from source...

wking · 2014-03-28T23:31:06Z

There are also warnings like:

$ pip install numpy --use-mirrors
--use-mirrors has been deprecated and will be removed in the future. Explicit uses of --index-url and/or --extra-index-url is suggested.

which are probably due to my copy-pasting from an out-of-date section
in the Travis docs.

You can probably drop the 'pip install .' too, since everything
important it probably built by 'make noQM'.

rwest · 2014-04-01T12:18:42Z

Yay. No more segfaults. Some unit tests fail (we expect that) but it runs to completion.

Still to fix:

ERROR: Failure: ImportError (No module named pydas)
failing tests report broken build
QM not tested
output if test jobs not checked
build takes half an hour!

rwest · 2014-04-01T14:15:12Z

PyDAS and PyDQED now fixed.
Next problem, it's looking in the wrong place for the database:
IOError: [Errno 2] No such file or directory: '/home/travis/build/GreenGroup/RMG-database/input/....

connie · 2014-04-01T15:14:16Z

It would have been nice if PyDAS and PyDQED were not forked, because now they will always refer to jwallen's repository as master even though he will not longer be actively updating.

rwest · 2014-04-02T12:46:43Z

It works!
I suggest we merge (after #200 and #201) and do any further optimization (speed up or checking the sample job results) later

The builds got this far then stopped for 10 minutes and were killed: $ git clone git@github.com:GreenGroup/RMG-database.git Cloning into 'RMG-database'... The authenticity of host 'github.com (192.30.252.128)' can't be established. RSA key fingerprint is 16:27:ac:a5:76:28:2d:36:63:1b:56:4d:eb:df:a6:48. Are you sure you want to continue connecting (yes/no)? Hopefully the https version can continue without the RSA fingerprint check...

Currently it spends thousands of lines of log file (and quite some time) compiling scipy for each build. The Ubuntu apt-get install of python-scipy should save this, but you need to enable the system site packages (instead of the isolated virtual environment) to use it. [1,2] Hopefully this (using the system site packages) also fixes the apparent failure of rdkit: current build failed with: echo "Checking you have rdkit..." Checking you have rdkit... Traceback (most recent call last): File "<string>", line 1, in <module> ImportError: No module named rdkit [1] http://danielnouri.org/notes/2012/11/23/use-apt-get-to-install-python-dependencies-for-travis-ci/ [2] https://github.com/dnouri/nolearn/blob/master/.travis.yml

Trying to get the simplest possible configuration working first, then will add complexity.

--use-mirrors has been deprecated, and pip install . likely isn't needed

This should speed up and simplify the build, if it works.

These lines in the build script are based off the RDKit travis config https://github.com/rdkit/rdkit/blob/master/.travis.yml But modified to turn on InChI support Once this is working we can worry about cacheing the finished build to speed it up.

Unlike the instructions for users in our installation guide, don't have Travis sudo the make install for PyDAS and PyDQED. 'sudo' on Ubuntu makes it discover the wrong python, which then doesn't have Cython installed, and it all goes wrong. Hopefully this clears it up. I also split into several lines, to make debugging the Travis log easier.

- RMG-database - RPyDAS - RPyDQED - Rrdkit etc. should all live alongside, not inside, the RMG-Py directory. Hopefully this works solves the database problem

Getting Travis-CI builds working

rwest changed the title ~~Use the https not git@ protocol to get the RMG-database in .travis.yml~~ Trying to get Travis-CI builds working Mar 28, 2014

rwest closed this Mar 28, 2014

rwest reopened this Mar 28, 2014

rwest mentioned this pull request Mar 28, 2014

.travis.yml: Configure Travis CI for automated testing #193

Closed

rwest mentioned this pull request Apr 1, 2014

Progress towards fixing unit tests #199

Merged

rwest added 9 commits April 2, 2014 17:20

Simplify travis config: no QM and no Python 2.6

ae8980a

Trying to get the simplest possible configuration working first, then will add complexity.

Remove redundant parts from Travis config file.

6196867

--use-mirrors has been deprecated, and pip install . likely isn't needed

Try using the apt-get version of matplotlib too, in travis build.

bbf1342

This should speed up and simplify the build, if it works.

Trying to get Travis things in the right directories.

7b3f3b6

- RMG-database - RPyDAS - RPyDQED - Rrdkit etc. should all live alongside, not inside, the RMG-Py directory. Hopefully this works solves the database problem

Try running eg1 as well as make test in travis testing.

48b06e9

connie added a commit that referenced this pull request Apr 2, 2014

Merge pull request #194 from GreenGroup/travis

cf1eb31

Getting Travis-CI builds working

connie merged commit cf1eb31 into master Apr 2, 2014

rwest pushed a commit to rwest/RMG-Py that referenced this pull request Apr 2, 2014

Merge pull request ReactionMechanismGenerator#194 from GreenGroup/travis

1cd40f8

Getting Travis-CI builds working

rwest deleted the travis branch April 2, 2014 22:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Trying to get Travis-CI builds working #194

Trying to get Travis-CI builds working #194

rwest commented Mar 28, 2014

rwest commented Mar 28, 2014

wking commented Mar 28, 2014

rwest commented Mar 28, 2014

wking commented Mar 28, 2014

rwest commented Mar 28, 2014

wking commented Mar 28, 2014

rwest commented Apr 1, 2014

rwest commented Apr 1, 2014

connie commented Apr 1, 2014

rwest commented Apr 2, 2014

Trying to get Travis-CI builds working #194

Trying to get Travis-CI builds working #194

Conversation

rwest commented Mar 28, 2014

rwest commented Mar 28, 2014

wking commented Mar 28, 2014

rwest commented Mar 28, 2014

wking commented Mar 28, 2014

rwest commented Mar 28, 2014

wking commented Mar 28, 2014

rwest commented Apr 1, 2014

rwest commented Apr 1, 2014

connie commented Apr 1, 2014

rwest commented Apr 2, 2014