Nightly server build setup remains unreliable

[jnweiger]

https://rotor.int.owncloud.com/job/owncloud-server-nightly does not succeed to push new tar-archives into obs. I am blocked working on fixing packaging configuration in ce:nightly so that we can produce packages for https://owncloud.org/install/#testing-development

During the last week I've opened tickets for sysadmin, @danimo was helpful, but the list of issues does not get shorter. Some of them have been fixed multiple times and keep re-occuring.

Error
/usr/bin/git tag -a -f -m 'Jenkins Build #1043 jenkins-owncloud-server-nightly-1043 # timeout=10'
FATAL: Could not apply tag jenkins-owncloud-server-nightly-1043

Workaround

ssh root@rotor.int.owncloud.com
docker exec -ti ocobsclient_occlient_1 bash
su - jenkins
git config --global user.email jenkins@owncloud.com
git config --global user.name "Sir Jenkins of ownCloud"

Error
/tmp/hudson6363963229645362926.sh: line 7: wget: command not found

Workaround

ssh root@rotor.int.owncloud.com
docker exec -ti ocobsclient_occlient_1 bash
zypper in wget

Error
the apiurl 'https://obs.int.owncloud.com' does not exist in the config file. Please enter your credentials for this apiurl.

Workaround

ssh root@rotor.int.owncloud.com
docker exec -ti ocobsclient_occlient_1 bash
su - jenkins
sed -i -e 's@s2@obs.int@' .ocoscrc

Error
./buildpackage.pl -p -n owncloud-9.1.0prealpha.20160304.tar.bz2
*** certificate verify failed at depth 1
Reason: unable to get local issuer certificat

Workaround

ssh root@rotor.int.owncloud.com
docker exec -ti ocobsclient_occlient_1 bash
su - jenkins
cd workspace/owncloud-server-nightly/jenkins/obs_integration/
export 'OBS_INTEGRATION_OSC=/usr/bin/osc -c ~/.ocoscrc'
./buildpackage.pl -p -n owncloud-9.1.0prealpha.20160304.tar.bz2
 Would you like to
 0 - quit (default)
 1 - continue anyways
 2 - trust the server certificate permanently
 9 - review the server certificate

 Enter choice [0129]: 2

Especially the last one -- running commands interactively -- feels so wrong.

The toolchain is long and complex. This results in multiple sources of errors. Note that as long as one single error occurs, we don't see a nightly build at all.

@dragotin @danimo

[jnweiger]

This one re-occured tonight. Reapplied the workaround.

Error
the apiurl 'https://obs.int.owncloud.com' does not exist in the config file. Please enter your credentials for this apiurl.

Workaround

ssh root@rotor.int.owncloud.com
docker exec -ti ocobsclient_occlient_1 bash
su - jenkins
sed -i -e 's@s2@obs.int@' .ocoscrc