Get snapcraft build approved

[LyzardKing]

Can somebody add the snapcraft secrets to the admin repo for the github actions?
Then we can uncomment the line in the build script and push the jabref snap to the store

as described here: https://forum.snapcraft.io/t/building-and-pushing-snap-packages-from-gitlab/9537
The tldr is:
snapcraft login
snapcraft export-login snapcraft.login
base64 snapcraft.login | xsel --clipboard

paste to the SNAPCRAFT_LOGIN_FILE in secrets

[LyzardKing]

I wonder if (to simplify things) the snap should not be built in the github actions after all..
it seems to complicate things, and it might instead just automatically trigger (build.snapcraft can do that) once the url for the latest build.jabref file is updated.
To enable this it would have to live in a separate repo (under the jabref organization, like JabRef/jabref-snap)
So without having to change the url a succesful build upstream would trigger a snapcraft build...without the docker addition..
@koppor @tobiasdiez what do you think?

[koppor]

This refs #5379.

[koppor]

I think, we should stick building it here. That way, we can ship daily builds. - Otherweise, we had to change our built files. Refs #5399

[LyzardKing]

It would ship daily builds anyway, ending up in the --edge channel.
But there would only ever be one daily build available.
That's the tradeoff..
A simple and automated build system, with 4 levels of "available" images: edge (continuously built, beta, candidate and stable).
Anyway, I think the snap is coming along nicely, with the new browser integration :)

[tobiasdiez]

I'm also in favor of build it here on github. This way we are more flexible (what triggers a build or prepare files before building). Moreover, we have everything in one place and treat different app stores in a similar way. As building the snap is working now, I also don't see any advantages in using build.snapcraft.io.

Mr @koppor just needs to login in snapcraft and provide github with the necessary secrets ;-)

[LyzardKing]

Building on builds.jabref is good.
It works and you can build different branches easily.
Just as a note, #5399 is not an issue with snapcraft, since it would refresh from master even if the name/version are the same.
In the previous snap the version was always set to 5.0.0-dev

[koppor]

I followed #5379 (comment) and provided SNAPCRAFT_LOGIN_FILE as secret in this repository.

[koppor]

Build upload activated at fd2d32b in the master branch. Added || true to allow failures of this step.

[LyzardKing]

Great! Now let's see if it builds.. I noticed in the script the other day it was set to push to beta...did we set it to edge?
(I'm not on my computer now..)

It should say push --release=edge for the daily build

jabref/.github/workflows/deployment.yml

Line 92 in fd2d32b

cd build/distribution/ && mdkir .snapcraft && echo ${SNAPCRAFT_LOGIN_FILE} | base64 --decode --ignore-garbage > .snapcraft/snapcraft.cfg && snapcraft push --release=beta *.snap || true

[tobiasdiez]

Done: 372c3ec

[LyzardKing]

I'm sorry...I can't type...in the same line there's mdkir that should be mkdir

[koppor]

And we cannot read either 🤣🤣. Fixed at 2e796e9

[LyzardKing]

The upload works now, the only thing missing is to ask for the review.
Since we used a special permission to add the json file needed for the native-messaging-hosts part,
you need to ask on the snapcraft forum for permission.
It only has to be done one time, and if we link the discussion I had related to getting native-messaging-hosts to work in the snap it should be fairly straight forward

EDIT: It's better if the request is official (it starts from one you) but if you want to tag me and the discussion I had on the forum I'm happy to intervene if it's needed :)

[koppor]

@LyzardKing Please provide me the link to the discussion so that I can comment on the release review (https://dashboard.snapcraft.io/snaps/jabref/revisions/551/feedback/) accordingly. Link to the discussion: https://forum.snapcraft.io/t/personal-file-under-mozilla-native-messaging-hosts/13627

[LyzardKing]

I'm not sure what the procedure is, but once the request is on the forum they can redirect the proper people to look at it.
Thanks!

[LyzardKing]

@koppor
You could try opening an explicit request like this one: https://forum.snapcraft.io/t/personal-files-request-for-kontena-lens/13504
The permission is system-files, with access to /usr/lib/mozilla/native-messaging-hosts/org.jabref.jabref.json

[Siedlerchr]

Btw, the snapcraft build fails at build.snapcraft.io:
Do we need to remove the snapcraft.yml file?

Pulling jabref 
Failed to pull source: 'build/distribution/JabRef-portable_linux.tar.gz'.
Please ensure the source path is correct and that it is accessible.
See `snapcraft help sources` for more information.
Build failed
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/lpbuildd/target/build_snap.py", line 267, in run
    self.pull()
  File "/usr/lib/python2.7/dist-packages/lpbuildd/target/build_snap.py", line 233, in pull
    env=env)
  File "/usr/lib/python2.7/dist-packages/lpbuildd/target/build_snap.py", line 104, in run_build_command
    return self.backend.run(args, env=full_env, **kwargs)
  File "/usr/lib/python2.7/dist-packages/lpbuildd/target/lxd.py", line 502, in run
    subprocess.check_call(cmd, **kwargs)
  File "/usr/lib/python2.7/subprocess.py", line 541, in check_call
    raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '['lxc', 'exec', 'lp-bionic-amd64', '--env', 'LANG=C.UTF-8', '--env', 'SHELL=/bin/sh', '--env', 'SNAPCRAFT_LOCAL_SOURCES=1', '--env', 'SNAPCRAFT_SETUP_CORE=1', '--env', 'SNAPCRAFT_BUILD_INFO=1', '--env', 'SNAPCRAFT_IMAGE_INFO={"build-request-id": "lp-51911051", "build-request-timestamp": "2019-10-13T08:47:39Z", "build_url": "https://launchpad.net/~build.snapcraft.io/+snap/800385119bd8f8b023fe0dab7fbce0bc/+build/702124"}', '--env', 'SNAPCRAFT_BUILD_ENVIRONMENT=host', '--env', 'http_proxy=http://10.10.10.1:8222/', '--env', 'https_proxy=http://10.10.10.1:8222/', '--env', 'GIT_PROXY_COMMAND=/usr/local/bin/snap-git-proxy', '--', '/bin/sh', '-c', 'cd /build/jabref && linux64 snapcraft pull']' returned non-zero exit status 2
Revoking proxy token...

RUN: /usr/share/launchpad-buildd/bin/in-target scan-for-processes --backend=lxd --series=bionic --arch=amd64 SNAPBUILD-702124
Scanning for processes to kill in build SNAPBUILD-702124

[koppor]

@Siedlerchr Can you provide the link to the build? I disabled the build 12 hours ago so that we build using GitHub workflows (as decided above).

[koppor]

Reading https://forum.snapcraft.io/t/personal-files-request-for-kontena-lens/13504/4?u=koppor it seems to be required that we are patient. Since I asked at the thread our supporter for an approval, I would suggest that we wait until Sunday with an explicit request. - Then I would ask @LyzardKing to write down the request since I am not aware of the details and cannot properly join the discussion.

[Siedlerchr]

https://build.snapcraft.io/user/JabRef/jabref/702124
(a day ago ) Did not know that you disabled the build there.

[LyzardKing]

It takes some time for the request to be approved.
I'll write a request, then link it here, so you can keep an eye on it.
Then I'll link the previous forum post with the methodology followed in developing the feature (https://forum.snapcraft.io/t/personal-file-under-mozilla-native-messaging-hosts/13627) in the new request.
Is this acceptable?

[koppor]

Sure!

[LyzardKing]

Great! So I'll open the request now :)
https://forum.snapcraft.io/t/system-files-permission-for-jabref-native-messaging-hosts-integration/13702

[LyzardKing]

By the way.. the terminal fix in the docker image worked... the apt log is still very long, but the snapcraft stuff is very limited and readable

 Pulling jabref 
Building gnome-extension 
make -j2
gcc -Wall -O2 -o bindtextdomain.so -fPIC -shared ./../src/bindtextdomain.c -ldl
make install DESTDIR=/home/runner/work/jabref/jabref/parts/gnome-extension/install
install -d /home/runner/work/jabref/jabref/parts/gnome-extension/install/gnome-platform
install -d /home/runner/work/jabref/jabref/parts/gnome-extension/install/data-dir
install -d /home/runner/work/jabref/jabref/parts/gnome-extension/install/data-dir/icons
install -d /home/runner/work/jabref/jabref/parts/gnome-extension/install/data-dir/sounds
install -d /home/runner/work/jabref/jabref/parts/gnome-extension/install/data-dir/themes
install -D -m755 desktop-launch "/home/runner/work/jabref/jabref/parts/gnome-extension/install/snap/command-chain"/desktop-launch
install -D -m644 bindtextdomain.so "/home/runner/work/jabref/jabref/parts/gnome-extension/install/lib"/bindtextdomain.so

Building jabref 
Staging gnome-extension 
Staging jabref 
Priming gnome-extension 
Priming jabref 
Snapping 'jabref' ...

Snapped jabref_5.0.0_amd64.snap
Preparing to push 'jabref_5.0.0_amd64.snap'.
After pushing, an attempt will be made to release to 'edge'

[LyzardKing]

There's still other parts of the log that can be cut down (regarding the upload), but that's up to the snapcraft tool to remove on "dumb" terminals :)

[LyzardKing]

Accepted! The vetting has gone through.
I updated the snap in a branch and tested it (with firefox/jabfox) locally

[AEgit]

I can confirm, that the JabRef snap is now available again. I installed the --edge version (sudo snap install --edge jabref). As far as I can tell, however, this does not represent the current developer version (according to snapcraft.io/jabref this version is from 6th October 2019) and, indeed, it does not work at the moment, but gives the following error message, when started from the terminal:

/snap/jabref/550/bin/desktop-launch: line 574: /snap/jabref/550/jabref/bin/JabRefMain: No such file or directory

I reckon the available snap version only needs to be updated and then things will work again (?).

[AEgit]

JabRef 5.0--2020-03-08--93de138
Linux 5.3.0-40-generic amd64
Java 13.0.2

I updated the snap version today using snap refresh --devmode jabref. The snap version appears to be working now (it still throws some errors, but at least the snap version starts now). Well done to everyone involved!

Can this issue now be closed?

[LyzardKing]

The snap works.
@koppor @tobiasdiez Do you think it could be promoted to stable?
That would give more visibility in the app stores

[koppor]

Current issue: all releases show the same JabRef version:

This should be fixed for 5.1. I think, we can use the same version as JabRef uses internally in the buildinfo file. See

jabref/.github/workflows/deployment.yml

Line 73 in c2bd898

run: ./gradlew -PprojVersion="${{ steps.gitversion.outputs.AssemblySemVer }}" -PprojVersionInfo="${{ steps.gitversion.outputs.InformationalVersion }}" jlinkZip

for the embedding in the GitHub action thing.

I am working on the snapcraft branch to build and publish the 5.0 release of JabRef. Still shows 5.0.44 as version, but is OK... (Currently installing Ubuntu 20.04 to test the image).

[koppor]

Not sure how I can release a version as full release.

Do we need to change the grade to something else?

jabref/snap/snapcraft.yaml

Line 8 in c2bd898

grade: devel

Feel free to submit a PR to the snapcraft branch.

[LyzardKing]

Ah yes, the grade needs to be set to stable

[koppor]

beta channel runs fine:

[LyzardKing]

I am using it daily on Ubuntu Mate 20.04, so I can confirm it works.
But if we change to grade: stable the new versions can be automatically shown in the app store and be installed without adding --beta, --edge (which can be reserved for actual beta versions)

[koppor]

Some command line parameters are passed to JabRef instead of the JVM:

[koppor]

But if we change to grade: stable the new versions can be automatically shown in the app store and be installed without adding --beta, --edge (which can be reserved for actual beta versions)

I think, we should keep --edge for the daily builds and use the release channel only for real releases. To keep the releases in Ubuntu conistent with our releases. --beta should also point to JabRef beta releases. -- (Decision from our devcall: https://github.com/JabRef/jabref/wiki/Minutes#2020-03-17-8oo)

[koppor]

Released as stable version. I will close the issue. Regarding the parameters, we could still discuss here. If a user finds it strange, they can open a new issue ^^.

[tobiasdiez]

Right now grade is set to stable in the master branch. Is this correct in view of #5417 (comment) ?

[LyzardKing]

Yes, now jabref can be searched in the snap store without issues!

[koppor]

@tobiasdiez Yes. I think, I understood it: grade: stable enables me to release that specific version to stable. If not, I do not have this button (as indicated at #5417 (comment)). Thus, I "just" have to take care at a release which verson to promote as stable. - No other side effects known.

[tobiasdiez]

Ok, I just wanted to make sure that our daily builds don't end up as stable builds...

[LyzardKing]

@tobiasdiez If it's not automated (and it has to be done manually)
The daily uploads should only go in the edge channel.

[koppor]

Yes, they automatically are added to the edge channel.