Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

automated arm/v7 and arm64 builds for alpine and linux built using docker buildx and github actions #1362

Closed
wants to merge 5 commits into from

Conversation

n1ru4l
Copy link

@n1ru4l n1ru4l commented Jul 21, 2020

This pull request adds a GitHub action that uses docker buildx for generating binaries for the following platforms.

  • node16-linux-armv7
  • node16-linux-arm64
  • node16-alpine-armv7
  • node16-alpine-arm64
  • node15-linux-armv7
  • node15-linux-arm64
  • node15-alpine-armv7
  • node15-alpine-arm64
  • node14-linux-armv7
  • node14-linux-arm64
  • node14-alpine-armv7
  • node14-alpine-arm64
  • node13-linux-armv7
  • node13-linux-arm64
  • node13-alpine-armv7
  • node13-alpine-arm64
  • node12-linux-armv7
  • node12-linux-arm64
  • node12-alpine-armv7
  • node12-alpine-arm64
  • node11-linux-armv7
  • node11-linux-arm64
  • node11-alpine-armv7
  • node11-alpine-arm64
  • node10-linux-armv7
  • node10-linux-arm64
  • node10-alpine-armv7
  • node10-alpine-arm64

The binaries will also be deployed by using node-pre-gyp publish.

An example build can be seen here: https://github.com/n1ru4l/node-sqlite3/actions

TODO:

  • Only trigger GitHub action upon pushing a tag (e.g. v5.0.0)
  • Enable GitHub Actions on this Repository (repository maintainer)
  • Set AWS_SECRET_ACCESS_KEY and AWS_SECRET_ACCESS_KEY github secrets (repository maintainer)

@tkurki
Copy link

tkurki commented Jul 22, 2020

Nice! Node 10 is still LTS maintenance, would you consider including it?

@n1ru4l
Copy link
Author

n1ru4l commented Jul 22, 2020

@tkurki done 😊

@MehediH
Copy link

MehediH commented Jul 24, 2020

Nice work! Would it be difficult to add Windows ARM64 support here?

@n1ru4l
Copy link
Author

n1ru4l commented Jul 25, 2020

@MehediH i think that would be something for a different pull request. AFAIK it is not possible to emulate arm64 Windows with docker so it would require us relying on some other method.

@n1ru4l
Copy link
Author

n1ru4l commented Aug 18, 2020

@kewde Could you please take a look at this and provide some feedback 😊

@kewde kewde self-assigned this Sep 6, 2020
@aostrun
Copy link

aostrun commented Sep 18, 2020

Any updates on this PR? When can we expect to get the prebuilt binaries for ARM?

@Codelica
Copy link

Would be sad to see @n1ru4l's work not make it in. 😢 Sqlite is perfect for ARM! 👍 @kewde is this being considered?

@n1ru4l
Copy link
Author

n1ru4l commented Oct 22, 2020

Repository owners only need to setup the credentials as github action secrets and merge this PR.

Ideally I would also prefer using Github releases for hosting the built binaries instead of a S3 bucket that cannot be browsed 😅.

So if a maintainer is reading this pls also consider checking out mapbox/node-pre-gyp#509 I would also volunteer for implementing that Github mirror feature.

But right now TBH I am not willing to put in the work if it never gets reviewed...

Mapbox please consider allocating more time for maintaining your open source projects...

@sudhirj
Copy link

sudhirj commented Dec 12, 2020

@kewde Is there anything we can do to help with this? Not having it is causing a lot of pain when deploying to AWS on the Graviton systems.

@gpetrov
Copy link

gpetrov commented Dec 13, 2020

Any progress in merging this?

Note linux/arm64 binaries are very important for the upcoming Docker Desktop for Apple Silicon M1 as those will run natively there.

@sudhirj
Copy link

sudhirj commented Dec 14, 2020

In the meantime does anyone have any examples of how to self-host these binaries? I'd assume one could run the build, post the binaries on S3/local folder and pass in some kind of config or override to use the mirror/cache?

@gpetrov
Copy link

gpetrov commented Mar 29, 2021

Really hope this will be merged soon, there is a huge need for arm64 binaries.

@gpetrov
Copy link

gpetrov commented May 23, 2021

Can anybody accept this PR and finalise the arm64 binary publishing?
it is really hard needed #1413 (comment)

@nickrum, @kewde

@tom-boothman
Copy link

travis supports arm64 architectures as well now, if keeping everything in travisci is the sticking point
https://docs.travis-ci.com/user/multi-cpu-architectures/

@ppedziwiatr
Copy link

Hi, is there any chance to merge this any time soon?

@alinjie
Copy link

alinjie commented Jul 8, 2021

ARM binaries would be amazing! Looking forward to this being merged

@lukidoescode
Copy link

@n1ru4l Thanks soooo much!

@micaelmbagira
Copy link

@kewde can you please review/merge this PR?

@jiuxiaxixi
Copy link

Can anybody accept this PR and finalise the arm64 binary publishing?
it is really hard needed #1413 (comment)

@nickrum, @kewde

@jiuxiaxixi
Copy link

So, what's the progress now?

@gaby
Copy link

gaby commented Sep 7, 2021

After 14 months, Can someone PLEASE merge this? @kewde @kkaefer @orlandov @springmeyer

@kewde
Copy link
Collaborator

kewde commented Oct 10, 2021

@springmeyer can you please enable github actions on this repo? It seems that it doesn't work in branchs (cfr branch ci/github)

Travis CI has quit so our build system is impacted, no new builds can be done, I also don't have the AWS S3 access keys to publish builds either so we're pretty stuck here.

RUN npx node-pre-gyp install --build-from-source
RUN npm run test
RUN npx node-pre-gyp package
RUN if [ "$AWS_ACCESS_KEY_ID" = "SKIP" ] || [ "$AWS_SECRET_ACCESS_KEY" = "SKIP" ] ; then echo "SKIP S3 PUBLISH" ; else npx node-pre-gyp publish ; fi
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I will change this if it eventually gets merged, we have a different mechanism to detect whether to publish or not.

on:
push:
tags:
- "v*"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: will change to every push.

@springmeyer
Copy link
Contributor

@springmeyer can you please enable github actions on this repo? It seems that it doesn't work in branchs (cfr branch ci/github)

@kewde can you get in touch with me by email to discuss?

@super-ax
Copy link

super-ax commented Dec 2, 2021

@n1ru4l How to solve this error when compiling to the end
版本:Ubuntu x86_64
image

@JessJNielsen
Copy link

Any updates? 🙏

@jiuxiaxixi
Copy link

jiuxiaxixi commented Feb 14, 2022 via email

@95gabor
Copy link

95gabor commented Feb 14, 2022

Do you need extra contribution to merge this? Can I help somehow?

daniellockyer added a commit that referenced this pull request Apr 14, 2022
refs #1362

- the referenced PR added Linux ARM64 and ARMv7 prebuilds to CI so we can
  get prebuilt binaries upon releasing
- CI has recently been reworked so I've rebased the PR and fixed it up
- unfortunately, `node-pre-gyp` seems to ignore the difference between
  armv6 and armv7, and treats them both as `arm` so I've had to disable
  prebuilds for them for now
- building via QEMU is INCREDIBLY slow so I've configured
  it to only run during a release or upon manual execution
- full credit to @n1ru4l for the original PR
@daniellockyer
Copy link
Member

Hey all! 👋🏻 Ghost recently took over maintenance of node-sqlite3 and I've spent some time updating CI.

Prebuilds are now completely done in GitHub Actions so this PR was perfect to add support for ARM. I've rebased the PR, merged it with what we've currently got, fixed up a few issues and I've just pushed it to master! 🎉 It'll be released soon 🙂

One problem - I ran into this issue with node-pre-gyp. In short, armv6 and armv7 are not differentiated, so building on armv7 will generate an {arch} of arm. For now, I've disabled armv* until we can sort that out.

💯 credit to @n1ru4l for the PR - thanks a lot 🕺🏻

PS. if anyone knows of a magical way to speed up cross-compilation in GitHub Actions, let me know! The ARM builds are currently taking ~40mins so I've had to disable them for every push and only run them on releases 🐌

@n1ru4l n1ru4l deleted the feat-automated-arm-build branch April 14, 2022 11:17
@daniellockyer
Copy link
Member

Released in v5.0.4 🙂

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.