Distributing an app depending on node-sqlite3 in a Debian/Ubuntu package across platforms

[astorije]

First of all, apologies if this issue appears unclear as I am still trying to wrap my head around this.

We develop an app that is meant to be installed and run on a server (AWS instance, Raspberry Pi, etc.).
We also ship a .deb package for this app to make streamline installation: instead of installing with npm/yarn + having to manage a user for it + running pm2 or the like for daemonization, the .deb package takes care of all that by the way of a simple apt install ./thelounge.deb.

.deb packages are made by Travis CI by installing the npm package before building the .deb file.
At this point however, the .deb is pretty much tailored to the platform and arch that Travis is building against. We recently started to use node-sqlite3 and right away got reports that the .deb couldn't be installed anymore (thelounge/thelounge-deb#26).

We are trying to fix this, and have some pending work (thelounge/thelounge-deb#31) but even that would prevent the .deb to be installed on non-amd64 arch so, say, Raspbian / Raspberry Pis.

Are there any known strategies to deal with this kind of situation?
I'm assuming we are not the only one trying to ship a .deb package with node-sqlite3 (or anything depending on a package that uses node-gyp / node-pre-gyp really) but I failed to find examples to follow that would solve our case.
Thanks in advance! 🙏

[thom-nic]

I might not 100% understand your use case but, but hopefully this will help... I build sqlite3 cross-architecture -not platform- e.g. Debian on x86-64 build host -> targeting Debian on ARM.

First - node-pre-gyp is your (or my) friend. It already tries to find a prebuilt binary for the target platform during install, and usually doesn't find one for ARM so then falls back to build-from-source which requires the correct toolchain. So I break the problem into two steps:

• pre-build a binary for my target platform/ arch, upload to my own mirror
• do an npm install, with correct target flags, and point to my prebuilt mirror

Again, my build host and target are both Debian-based: I've used combinations of Stretch, Jessie, and Ubuntu Xenial host which, I'm sure matters when it comes to gcc/ glibc versions so YMMV.

Also worth noting this same process has worked for other binary packages like bcrypt.

Step 1: Cross-architecture node-pre-gyp

I do this part in a cross-architecture chroot with qemu-user, similar to what you would do if you were building a cross-arch distro e.g. with debootstrap. So here I'm not using a cross toolchain, I'm using the 'native' ARM tool chain in a chroot under ARM emulation. See this documentation for further details. I'd like to think it should be possible to cross-build a node module given the correct toolchain but for whatever reason I never went down that path.

Once you build the native binary, you're going to upload to a mirror that you own. node-pre-gyp conveniently supports AWS S3 as a mirror so that's what I'm using. Export appropriate keys or set values in ~/.aws/credentials or .node_pre_gyprc. See doc

# export node_pre_gyp_accessKeyId="xxx"
# export node_pre_gyp_secretAccessKey="yyy"
# or
# export AWS_ACCESS_KEY_ID="xxx"
# export AWS_SECRET_ACCESS_KEY="yyy"

export node_pre_gyp_bucket=myorg-gyp-mirror
export node_pre_gyp_mirror="https://${node_pre_gyp_bucket}.s3.amazonaws.com"

pushd node_modules/sqlite3
# build and upload:
npg="../.bin/node-pre-gyp"
$npg rebuild --build-from-source      # we're in emulated ARM so platform and arch will be inferred
$npg package publish                  # <-- this will upload to your mirror: you should see appropriate arch in the filename

Repeat as needed if you have other native dependencies.

Note also binaries are specific to V8 ABI version so if you change major nodejs version or support multiple versions you may also need to do this more than once for each version you support.

Step 2: npm install for your target platform

This part can be done on any host platform/ architecture because we're specifying the target arch and all it needs to do is download the correct prebuilt binary:

# export the same $node_pre_gyp_mirror as above...

npm i --production --no-optional \
    --target_arch=arm --target_platform=linux --target_libc=glibc \
    --node_sqlite3_binary_host_mirror=$node_pre_gyp_mirror \

And you should see the npm install output say it found a prebuit binary rather than falling back to build-from-source.

Once this is done, you're free to package the entire node_modules/ into your .deb as you've been doing before - but you should put the correct Architecture in your DEBIAN/control as your .deb is now architecture-dependent.

Hope this helps!

[springmeyer]

@thom-nic - nice write up. It would be really cool to automate the creation of ARM binaries on travis. Would you be up for helping with that? See #418

Are there any known strategies to deal with this kind of situation?

@astorije - I gather you are asking this both in general (how to make portable binaries) and specifically in relation to the original error at thelounge/thelounge-deb#26?

As far as the original error at thelounge/thelounge-deb#26:

Error: Cannot find module '/usr/lib/node_modules/thelounge/node_modules/sqlite3/lib/binding/node-v59-linux-x64/node_sqlite3.node'

This is happening because you packaged the module on travis with node v8 (ABI 57) but the user is running node v9 (ABI 59). Binaries for node c++ addons like node-sqlite3 (Unless they use N-API, which we don't yet here, refs #979) only only compatible with each major version, so they are versioned that way and the error shows you are not installing the expected version. Not that node-pre-gyp has a --target open (https://github.com/mapbox/node-pre-gyp#options) which can be used to build a binary for a specific node version you want to support that is different from the one running on travis.

As far as other considerations on portable binaries (not related to node.js or sqlite) you might be interested to follow mapbox/cpp#4

[springmeyer]

I'll also add that the scripts inside https://github.com/mapbox/mapbox-studio-classic/tree/mb-pages/scripts (while not up to date and probably partially broken) illustrate of way of packaging a node app (including node c++ addon) that can be portable to linux and osx systems without requiring users to install node, npm, etc. So what you are trying to do is totally doable, just takes some care and effort.

[thom-nic]

@springmeyer I've never used Travis and actually I don't think the debootstrap/chroot envrionment is exactly right for a CI build environment (it's just convenient for me since my end-goal is an armhf distro.) However this comment might be prove helpful: travis-ci/travis-ci#3376 (comment)