Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Spinnaker to v.1.19.0.22 #660

Merged
merged 13 commits into from
Dec 24, 2018
Merged

Conversation

jpsacha
Copy link
Member

@jpsacha jpsacha commented Dec 17, 2018

  • Added QuickSpinC module
  • Added SpinVideoC module
  • Added TransportLayer*C modules
  • Change Linux CI build to use Ubuntu to ease setup, Spinnaker is distributed for Ubuntu. Centos would require hacking to fix linking issues.

* Added QuickSpinC module
* Added SpinVideoC module
* Added TransportLayer*C modules
* Change Linux CI build to use Ubuntu to ease setup, Spinnaker is distributed for Ubuntu.
@saudet
Copy link
Member

saudet commented Dec 17, 2018

@jpsacha
Copy link
Member Author

jpsacha commented Dec 17, 2018

I was trying with Centos6, was not able to install alien due to missing dependencies. May work with Centos7, but it requires additional configuration, compared to Ubuntu.

@jpsacha
Copy link
Member Author

jpsacha commented Dec 17, 2018

I was able to install alien on Centos 7 and convert Spinnaker *.deb packages to *.rpm, but cannot install them due to numerous missing dependencies (manual editing using rpmrebuild is not a viable option).

At this point I will make sure that current setup (with Ubuntu) works with full 'javacpp-presets' build (it does work if I just build Spinnaker). Hopefully last commit [a476f38] corrects the Travis configuration issue. Need to wait till regular CI build is finished.

@saudet
Copy link
Member

saudet commented Dec 18, 2018

Since we're not actually using FlyCapture or Spinnaker during the builds, we don't need to install any of the dependencies. Anyway, I'll leave @vb216 decide what to do about this. If you're able to maintain the build with that version of Ubuntu, that's fine tough, but please get it working! I won't merge this until the builds pass.

@vb216
Copy link
Member

vb216 commented Dec 18, 2018

I've not got any strong opinion really.. I remember having similar problems with alien last time I tried it, so I guess maybe its logical to have a more 'reliable' setup on ubuntu than something pushed into centos, only downside I can see is the ci .sh files growing. But, I wouldn't be surprised if we have to revisit those at some point anyway if they grow to the point of becoming difficult to maintain.

@jpsacha
Copy link
Member Author

jpsacha commented Dec 18, 2018

Looks to me that it will be easier to maintain Ubuntu build for Spinnaker, it may require less changes in the future, building on Centos may potentially require tweaks with each release.

It may also make sense to move Ubuntu/Spinnaker part from install-travis.sh to a separate script. This should simplify logic and amount of if/then switches. In travis.yml, Spinnker job will use that new script instead of install-travis.sh. Some of the common code (first 42 lines, last 3 lines, and "run install" logic lines 80-110 and 333-400) can be put into functions and shared between scripts.

I can do, in this PR, the basic split, that is, just move the parts needed to build Spinnaker to a separate script. install-travis.sh will look like before this PR, but with Spinnaker section removed. Adding the functions to share code between the two scripts should be a separate PR though, as it will require additional testing.

@saudet
Copy link
Member

saudet commented Dec 19, 2018

Sounds alright sure, builds for linux-armhf are already using Ubuntu so let's do it that way. Could you make Spinnaker use the same environment as those builds to make this more maintainable?

@jpsacha
Copy link
Member Author

jpsacha commented Dec 19, 2018

I changed Travis configuration to build without docker, similar to linux-armhf. Tested on its own fine. Need to wait for full CI build to see if everything is fine.

On other hand, I do not understand how FlyCapture is build on linux-armhf, the SDK archive is downloaded, copied, but never extracted.

@saudet
Copy link
Member

saudet commented Dec 19, 2018

Thanks, but let's not do Bionic. Stay with Trusty and harmonize the build environment with linux-armhf.

For ARM, it gets extracted in cppbuild.sh because that's how they distribute those versions:
https://github.com/bytedeco/javacpp-presets/blob/master/flycapture/cppbuild.sh#L19
The x86 versions are bundled in DEB packages and less easy to install.

@saudet
Copy link
Member

saudet commented Dec 19, 2018

I've just tried it on a machine running Trusty and it installs just fine. The only thing that doesn't work is the GUI, so we just need to comment out the relevant packages from install_flycapture.sh:

sudo dpkg -i libflycapture-2*
#sudo dpkg -i libflycapturegui-2*
sudo dpkg -i libflycapturevideo-2*
sudo dpkg -i libflycapture-c-2*
#sudo dpkg -i libflycapturegui-c-2*
sudo dpkg -i libflycapturevideo-c-2*
sudo dpkg -i libmultisync-2*
sudo dpkg -i libmultisync-c-2*
#sudo dpkg -i flycap-2*
#sudo dpkg -i flycapture-doc-2*
#sudo dpkg -i updatorgui*

@jpsacha
Copy link
Member Author

jpsacha commented Dec 19, 2018

The current Spinnaker build setup is simple, easy to maintain, and builds really fast, only 2 min 22 seconds. Faster than any other job.

I have put a lot of work in it trying to satisfy changing demands, first moving to Centos docker, then getting rid of docker, now back-porting to an obsolete distro. I am done tweaking it. Back-porting to an Trusty would make configuration more complex and more difficult to maintain.

I rather move it to a separate repo, that is simpler to maintain, faster to release, and does not require days to complete CI build. Similar with FlyCapture (the PR #613 is waiting there for over 3 months to be merged). They are not needed by anything in javacpp-presets, and do not depend on anything in javacpp-presets.

@saudet
Copy link
Member

saudet commented Dec 20, 2018

Your fork is already on a separate repo! You can comment out everything you don't need in .appveyor.yml and .travis.yml, and the builds will happen fast, either as part of this PR, or with your own accounts on AppVeyor and Travis CI.

However, what I am offering is to maintain the builds you create as part of Bytedeco. Whenever something breaks with the builds, I'm the one that has to fix them, so I'm merely asking to make it easy to maintain. If you do not wish to have your work maintained and integrated into the distribution, that is fine. You don't need to do anything else than what you're doing now.

@saudet
Copy link
Member

saudet commented Dec 20, 2018

If you simply do not have the time to work on this though, let's leave @vb216 do his job. :) We'll eventually get it working.

@jpsacha
Copy link
Member Author

jpsacha commented Dec 21, 2018

@saudet I know you are doing a tremendous work maintaining javacpp projects. I really appreciate that. However it is quite difficult to contribute to javacpp-presets due to its complex build setup and extremely long CI builds. Overtime this thing become very large.

Spinnaker (and FlyCapture) are only small pieces of javacpp-presets. Going forward, after this release, it really will be easier to keep them in a separate small repo, so they are easier to maintain and faster to release. I would prefer to keep it under Bytedeco. We can figure out details when there is a need for next release for Spinnaker/FlyCapture, after 1.19.0.22.

I can fix FlyCapture, if there are issues with wrapper, but will leave CI build setup to @vb216.

@saudet
Copy link
Member

saudet commented Dec 21, 2018

You don't need to keep everything as part of your fork or pull requests. Like I said, you can remove all other libraries from .appveyor.yml and .travis.yml, and CI will be fast. As to why this and everything else related to builds isn't all automatic and fully transparent, well someone needs to work on it! But in the meantime, it shouldn't prevent anyone from asking questions to figure out how to work best as a team. The build system is not in such a bad shape.

Anyway, as part of pull #613 we managed to extract without installing the DEB packages for FlyCapture on CentOS and msiexec seems to work fine for Windows as well, so let's do this for Spinnaker too. I see no reason why it couldn't work the same. @vb216 If you have some time to help with this, please do!

@vb216
Copy link
Member

vb216 commented Dec 21, 2018

Might be a quiet spell coming up from me again I'm afraid - job change on the horizon and usually takes a good few months to get up and running and back to having any spare capacity

@saudet
Copy link
Member

saudet commented Dec 24, 2018

Ok, so with that we should be able to maintain with minimal effort until FlyCapture3, Spinnaker2, or something comes out.

@saudet saudet merged commit 438d249 into bytedeco:master Dec 24, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants