Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Node.js to v16 and update frontend dependencies #519

Merged
merged 29 commits into from
Jun 26, 2023

Conversation

Splines
Copy link
Member

@Splines Splines commented Jun 21, 2023

Build in production suddenly failed with

Error: error:0308010C:digital envelope routines::unsupported
opensslErrorStack: [ 'error:03000086:digital envelope routines::initialization error' ],
library: 'digital envelope routines',
reason: 'unsupported',
 code: 'ERR_OSSL_EVP_UNSUPPORTED'

and Node.js version was Node.js v18.13.0 although we explicitly specified v14. This might be related to this issue. We now install Node.js v16 as the EOL of v14 was already reached in April this year. v16 is only maintained until the end of September 2023 (see here), so we should upgrade to v18 eventually, which is currently the LTS. We switched to nvm to install Node.js as v18 was somehow installed although we specified v16, see the already mentioned issue.

image
from here

also changed shell to bash
We use nvm to install Node.js as nodesource distributions somehow
always install Node.js 18 instead
(see nodesource/distributions#1583)
Also alias the default version
If you want to see the full output during Docker build,
you can use the "--progress=plain" option.
@Splines Splines added the dependencies Pull requests that update a dependency file label Jun 21, 2023
@Splines Splines self-assigned this Jun 21, 2023
@fosterfarrell9
Copy link
Collaborator

fosterfarrell9 commented Jun 21, 2023

With this, I am getting (after I nuked all existing volumes and images and rebuilt everything) the following error from webpacker:

development-webpacker-1    | ERROR in multi ./app/javascript/packs/application.js ./app/javascript/packs/application.sass
development-webpacker-1    | Module not found: Error: Can't resolve 'sass-loader' in '/usr/src/app'
development-webpacker-1    |  @ multi ./app/javascript/packs/application.js ./app/javascript/packs/application.sass application[1]
development-webpacker-1    | 
development-webpacker-1    | ERROR in multi (webpack)-dev-server/client?protocol=ws%3A&hostname=0.0.0.0&port=8080&pathname=%2Fws&logging=info&overlay=true&reconnect=10&hot=true&live-reload=true (webpack)/hot/dev-server.js ./app/javascript/packs/application.js ./app/javascript/packs/application.sass
development-webpacker-1    | Module not found: Error: Can't resolve 'sass-loader' in '/usr/src/app'

A similar message is also in the logs for experimental.

@Splines
Copy link
Member Author

Splines commented Jun 21, 2023

Thanks, just stumbled on this too. We still need sass-loader for webpacker to translate .scss (or .sass) to .css, the sass npm package is "just" the engine for sass used during the translation. Will fix.

@Splines
Copy link
Member Author

Splines commented Jun 21, 2023

After updating sass-loader, I got this error. Then I figured out that @rails/webpacker somehow still has webpacker v4 as dependency in yarn.lock even though they have a v5 specified in their package.json.

This is probably related to rails/webpacker#2969

This is because we still use webpacker v4 currently.
We will upgrade this depenedency when we switch from @rails/webpacker
to webpack in the future.
@codecov

This comment was marked as off-topic.

This is so that Node.js is not installed again, which would result in v18,
which we don't want at the moment.
This ensures that the app user can access node that was previously
installed with nvm in /root/.nvm
@Splines Splines marked this pull request as ready for review June 21, 2023 22:42
@Splines
Copy link
Member Author

Splines commented Jun 22, 2023

Hey @christian-heusel, it'd be great if you could take a look at these changes and give a review as you have created the original Dockerfile ☺

Copy link
Member

@christian-heusel christian-heusel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @christian-heusel, it'd be great if you could take a look at these changes and give a review as you have created the original Dockerfile ☺️

That is not correct, I have just heavily modified the setup!
4b5ca69 adds the dockerfile and was not done by me 😊

Anyways, I have left some comments about stuff that could be improved in my opinion, although I fear I am of no much help here 😄

Review is based on source code only, no actual testing has been done by me (yet).

docker/development/Dockerfile Outdated Show resolved Hide resolved
docker/development/Dockerfile Outdated Show resolved Hide resolved
docker/production/Dockerfile Outdated Show resolved Hide resolved
docker/run_cypress_tests/Dockerfile Outdated Show resolved Hide resolved
@Splines
Copy link
Member Author

Splines commented Jun 23, 2023

That is not correct, I have just heavily modified the setup!

Oh, sorry ;)

@christian-heusel Thanks a lot for your feedback!

Although I fear I am of no much help here

Definitely not the case

- Replaced Yarn installation with "corepack", a new binary shipped with
Node.js starting with v16. This also means that Yarn is upgraded from v1 to v2
with a different CLI, e.g. we now use "yarn workspaces focus"
instead of "yarn install" to install the dependencies for the workspace
in the current working directory.
- Update GPG key handling during PostgreSQL installation
- Added some more comments to the Dockerfile
- Moved copying of Node.js up in the Dockerfile

More yarn-related changed in subsequent commits.
- Added a fix so that corepack really installs the version we want
and not some other old version
- with Yarn2+, building curerntly fails, so we switch back to Yarn1 and
deal with an upgrade of Yarn in the future
Yarn1 does not generate this folder
"unnecessary" means dependencies that were added when trying out Yarn2+
Copy link
Member

@christian-heusel christian-heusel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are still some problems with this as the development image will not build currently, for fixes see the comments 😊

Also great work on applying the previous suggestions! 🙌🏻 🥳

docker/development/Dockerfile Outdated Show resolved Hide resolved
docker/run_cypress_tests/Dockerfile Outdated Show resolved Hide resolved
docker/production/Dockerfile Outdated Show resolved Hide resolved
docker/production/Dockerfile Show resolved Hide resolved
docker/development/Dockerfile Show resolved Hide resolved
docker/development/Dockerfile Outdated Show resolved Hide resolved
docker/production/Dockerfile Outdated Show resolved Hide resolved
@Splines Splines merged commit f63ec6f into mampf-next Jun 26, 2023
@Splines Splines deleted the security/update-dependencies branch June 26, 2023 19:04
@Splines
Copy link
Member Author

Splines commented Jun 26, 2023

@christian-heusel Thanks a lot for reviewing this PR 👍

Splines added a commit that referenced this pull request Jun 27, 2023
* Use npm audit to update npm packages

* Use Node.js version 16 in Docker development

also changed shell to bash
We use nvm to install Node.js as nodesource distributions somehow
always install Node.js 18 instead
(see nodesource/distributions#1583)

* Use node after installation

* Split installation of Node.js to multiple RUN commands

Also alias the default version
If you want to see the full output during Docker build,
you can use the "--progress=plain" option.

* Uniy sass packages and update yarn lock

* Reflect changes in production and test Dockerfiles

* Add comments to Dockerfile and tighten run commands

* Add sass-loader back as dependency

* Switch back to old version of sass-loader

This is because we still use webpacker v4 currently.
We will upgrade this depenedency when we switch from @rails/webpacker
to webpack in the future.

* Use "-no-install-recommends" for yarn install in prod

This is so that Node.js is not installed again, which would result in v18,
which we don't want at the moment.

* Use consistent ${} syntax in Dockerfile & fix Node path

* Copy Node.js over to /usr/local

This ensures that the app user can access node that was previously
installed with nvm in /root/.nvm

* Apply quote arguments suggestion (in code review)

Co-authored-by: Christian Heusel <christian@heusel.eu>

* Quote arguments in other Dockerfiles as well

* Replace deprecated apt-key and improve Dockerfiles

- Replaced Yarn installation with "corepack", a new binary shipped with
Node.js starting with v16. This also means that Yarn is upgraded from v1 to v2
with a different CLI, e.g. we now use "yarn workspaces focus"
instead of "yarn install" to install the dependencies for the workspace
in the current working directory.
- Update GPG key handling during PostgreSQL installation
- Added some more comments to the Dockerfile
- Moved copying of Node.js up in the Dockerfile

More yarn-related changed in subsequent commits.

* Upgrade to Yarn 2 and add packages for non-error webpack build

* Copy Yarn to usr folder alongside other Node.js tooling

* Activate Yarn for app user

* Only copy node over to usr folder

* Go back to Yarn1 and explicitly set Yarn version

- Added a fix so that corepack really installs the version we want
and not some other old version
- with Yarn2+, building curerntly fails, so we switch back to Yarn1 and
deal with an upgrade of Yarn in the future

* Remove ".yarn" from .gitignore

Yarn1 does not generate this folder

* Remove unnecessary dependencies from package.json

"unnecessary" means dependencies that were added when trying out Yarn2+

* Add back missing "--production=false" to "yarn install"

* Use "ruby:3.1.4-bullseye" to have PostgreSQL available

* Add missing `apt update` in dev and test Dockerfiles

Co-authored-by: Christian Heusel <christian@heusel.eu>

* Group `update` and `install` in one RUN statement

(in prod Dockerfile)

* Remove superfluous newline

* Make Node.js version logging one RUN statement

* Explicitly set `--production=true` during `yarn install`

---------

Co-authored-by: Christian Heusel <christian@heusel.eu>
@Splines Splines mentioned this pull request Aug 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Pull requests that update a dependency file
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants