Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tracking delayed dep updates because of old Node support #2760

Closed
trentm opened this issue Jun 6, 2022 · 6 comments · Fixed by #3611
Closed

tracking delayed dep updates because of old Node support #2760

trentm opened this issue Jun 6, 2022 · 6 comments · Fixed by #3611
Assignees
Labels
agent-nodejs Make available for APM Agents project planning.
Milestone

Comments

@trentm
Copy link
Member

trentm commented Jun 6, 2022

I'm using this issue to track dependencies that we are not updating while we still support old major versions of node. Sometimes it is because the newer dependency version actually breaks with the older node, and sometimes it is just because that dep dropped support for the older node (but might not yet actually break).

node v8

node v10

node v12

  • dep pino@8 dropped support for node <v14

ESM-only

  • devDep get-port@6 is ESM-only and dropped support for node <v12.20
@trentm trentm added this to the next-major milestone Jun 6, 2022
@github-actions github-actions bot added the agent-nodejs Make available for APM Agents project planning. label Jun 6, 2022
@trentm trentm mentioned this issue Aug 30, 2023
24 tasks
@trentm
Copy link
Member Author

trentm commented Sep 1, 2023

Okay, changes leading to 4.x releases are on main now, so we can move on this. The new current base is v14.5.0.

Full npm outdated output. THis is the starting point for (immediate) deps we can consider upgrading now:

% npm outdated
Package                               Current          Wanted          Latest
@aws-sdk/client-s3                    3.379.1         3.405.0         3.405.0
@aws-sdk/s3-request-presigner         3.379.1         3.405.0         3.405.0
@babel/cli                             7.22.6         7.22.10         7.22.10
@babel/core                            7.22.8         7.22.11         7.22.11
@babel/preset-env                      7.22.7         7.22.14         7.22.14
@elastic/elasticsearch-canary  8.8.0-canary.2  8.8.0-canary.2  8.9.1-canary.1
@types/node                            20.5.0          20.5.8          20.5.8
@types/pino                            6.3.12          6.3.12           7.0.5
ajv                                     7.2.4           7.2.4          8.12.0
apollo-server-core                     3.11.1          3.12.1          3.12.1
apollo-server-express                  3.11.1          3.12.1          3.12.1
aws-sdk                              2.1400.0        2.1450.0        2.1450.0
backport                                5.6.6           5.6.6           9.2.3
escape-string-regexp                    4.0.0           4.0.0           5.0.0
eslint                                 8.42.0          8.48.0          8.48.0
fastify                                4.22.0          4.22.2          4.22.2
get-port                                5.1.1           5.1.1           7.0.0
glob                                    7.2.3           7.2.3          10.3.4
got                                    11.8.6          11.8.6          13.0.0
graphql                                16.6.0          16.8.0          16.8.0
ioredis                                 5.3.0           5.3.2           5.3.2
knex                                    2.4.2           2.5.1           2.5.1
lru-cache                               6.0.0           6.0.0          10.0.1
measured-reporting                     1.51.1          1.51.1           2.0.0
mimic-response                          1.0.0           1.0.0           4.0.0
mkdirp                                  0.5.6           0.5.6           3.0.1
mysql2                                  3.2.4           3.6.0           3.6.0
ndjson                                  1.5.0           1.5.0           2.0.0
pino                                   6.14.0          6.14.0          8.15.0
prettier                                3.0.2           3.0.3           3.0.3
readable-stream                         3.6.2           3.6.2           4.4.2
redis                                   4.3.1           4.6.8           4.6.8
rimraf                                  3.0.2           3.0.2           5.0.1
semver                                  6.3.1           6.3.1           7.5.4
source-map                       0.8.0-beta.0    0.8.0-beta.0           0.7.4
tape                                    5.6.5           5.6.6           5.6.6
test-all-versions                       4.1.1           4.1.1           5.0.1
ws                                      7.5.9           7.5.9          8.13.0

Trimmed of things that aren't relevant to update for this, and with notes
on deps to update.

Package                               Current          Wanted          Latest
mkdirp                                  0.5.6           0.5.6           3.0.1
rimraf                                  3.0.2           3.0.2           5.0.1
semver                                  6.3.1           6.3.1           7.5.4
    Should be no issues updating the set above here.

pino                                   6.14.0          6.14.0          8.15.0
    Tests with 14. No "engines", so not sure if dependent on a particular
    14.x. It uses readable-stream@4, which deps on 14.17 for AbortController
    usage -- which it polyfills if not available. In node 14.x it is only available
    with `--experimental-abortcontroller`, so I think this is fine.
    v14.17 adding `<AbortSignal>.reason`, so the fail case is a user using
    v14 less than 14.17, with `--experimental-abortcontroller` and something
    in the APM agent's code path tries to use AbortSignal.reason.
    We could consider setting min to >14.17, but not sure it is necessary.
readable-stream                         3.6.2           3.6.2           4.4.2
    Requires >14.17 for AbortController. See comment for `pino` above. I think
    it is fine to upgrade to this. More possibly impactful would be if there
    are subtle behaviour changes that impact stream-chopper.

@types/pino                            6.3.12          6.3.12           7.0.5
    We should drop this. See https://www.npmjs.com/package/@types/pino
    Perhaps that only applied once we update pino dep.

ajv                                     7.2.4           7.2.4          8.12.0
    All good. Notes on ajv@8 here: https://github.com/elastic/apm-agent-nodejs/pull/2755#issuecomment-1147725249

lru-cache                               6.0.0           6.0.0          10.0.1
    Yes, but look. Replace our async-cache usage with this.

ndjson                                  1.5.0           1.5.0           2.0.0
    No real value in upgrading, but could. It was just modernized, and
    dropped the 'serialize' export name in favour of 'stringify'.

glob                                    7.2.3           7.2.3          10.3.4
    glob@10 14.17 for AbortSignal, but we don't use that and only use `glob`
    in our tests, so all good.

@trentm
Copy link
Member Author

trentm commented Sep 5, 2023

lru-cache / async-cache update handling is being done in #3610.

@trentm
Copy link
Member Author

trentm commented Sep 5, 2023

Actually we are not ready to update to readable-stream because the curious (questionable?) usage in http-apm-client results in:

/Users/trentm/el/apm-agent-nodejs13/node_modules/readable-stream/lib/internal/streams/writable.js:251
    else if (encoding !== 'buffer' && !Buffer.isEncoding(encoding)) throw new ERR_UNKNOWN_ENCODING(encoding)
                                                                    ^

TypeError [ERR_UNKNOWN_ENCODING]: Unknown encoding: Symbol(transaction)
    at _write (/Users/trentm/el/apm-agent-nodejs13/node_modules/readable-stream/lib/internal/streams/writable.js:251:75)
    at Client.Writable.write (/Users/trentm/el/apm-agent-nodejs13/node_modules/readable-stream/lib/internal/streams/writable.js:286:10)
    at Client.sendTransaction (/Users/trentm/el/apm-agent-nodejs13/lib/apm-client/http-apm-client/index.js:856:15)
    at Instrumentation.addEndedTransaction (/Users/trentm/el/apm-agent-nodejs13/lib/instrumentation/index.js:562:20)
    at Transaction.end (/Users/trentm/el/apm-agent-nodejs13/lib/instrumentation/transaction.js:420:32)
    at Instrumentation.endTransaction (/Users/trentm/el/apm-agent-nodejs13/lib/instrumentation/index.js:756:9)
    at Agent.endTransaction (/Users/trentm/el/apm-agent-nodejs13/lib/agent.js:208:47)
    at Server.<anonymous> (/Users/trentm/el/apm-agent-nodejs13/test/integration/api-schema/basic.test.js:141:13)
    at Object.onceWrapper (node:events:627:28)
    at Server.emit (node:events:513:28)
    at emitListeningNT (node:net:1497:10)
    at processTicksAndRejections (node:internal/process/task_queues:82:21)

trentm added a commit that referenced this issue Sep 5, 2023
This updates a number of dependencies now that our base Node.js version
is v14.

There is some subtlety in the 'pino' and 'glob' deps. They both use
AbortController (pino via its usage of readable-stream@4) or
<AbortSignal>.reason to support aborting async operations. Full support
for these were not added until Node.js 14.17 -- yet this APM agent
currently supports back to 14.5.0.

I feel this is not an issue. In node 14.x `AbortController` et al are only available
behind a flag (`node --experimental-abortcontroller`).  Node.js v14.17
added `<AbortSignal>.reason`, so the fail case with Pino is a user using
(a) a Node.js version in the range [v14.5, v14.17), and (b) with the
`--experimental-abortcontroller` flag, and (c) something in the APM
agent's code path tries to use `AbortSignal.reason`. When the flag is
not used both readable-stream@4 and glob@10 will use a polyfill.

We could consider setting min to >14.17, but I don't currently think it
is necessary.

Closes: #2760
@trentm
Copy link
Member Author

trentm commented Sep 5, 2023

glob@10 is out as well because it uses "private instance methods", which break in v14.5 at least:

% node test/test.js
/Users/trentm/el/apm-agent-nodejs13/node_modules/glob/node_modules/minimatch/dist/cjs/ast.js:86
    #fillNegs() {
             ^

SyntaxError: Unexpected token '('
    at wrapSafe (internal/modules/cjs/loader.js:1117:16)
    at Module._compile (internal/modules/cjs/loader.js:1165:27)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1221:10)
    at Module.load (internal/modules/cjs/loader.js:1050:32)
    at Function.Module._load (internal/modules/cjs/loader.js:938:14)
    at Module.require (internal/modules/cjs/loader.js:1090:19)
    at require (internal/modules/cjs/helpers.js:75:18)
    at Object.<anonymous> (/Users/trentm/el/apm-agent-nodejs13/node_modules/glob/node_modules/minimatch/dist/cjs/index.js:9:18)
    at Module._compile (internal/modules/cjs/loader.js:1201:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1221:10)

Node v14.6.0 is fine.


Need to go back to glob@8 which supports Node >=12.

@trentm
Copy link
Member Author

trentm commented Sep 6, 2023

rimraf@5 is out. It at rimraf@4 dep on glob@10. See previous comment.

@trentm
Copy link
Member Author

trentm commented Sep 6, 2023

^^^ we bumped our min to Node.js v14.17, so glob@10 and rimraf@5 are back in.

trentm added a commit that referenced this issue Sep 6, 2023
This bumps the mininum supported Node.js from v14.5 to v14.17;
and bumps a number of dependencies.

- v14.5 was for AsyncLocalStorage
- v14.6 brings a V8 with spport for private methods syntax, FinalizationRegistry (used by pino@8)
- v14.8 brings AsyncResource.bind
- v14.17 brings AbortController, AbortSignal.reason (used by readable-stream@4, glob@10, etc.)

Closes: #2760
trentm added a commit that referenced this issue Sep 7, 2023
lru-cache@7 added support for async fetching, and async-cache
was deprecated

See https://github.com/isaacs/node-lru-cache#fetchmethod-read-only
for the best docs for using lru-cache for async caching.

Refs: #2760
fpm-peter pushed a commit to fpm-git/apm-agent-nodejs that referenced this issue Aug 20, 2024
This bumps the mininum supported Node.js from v14.5 to v14.17;
and bumps a number of dependencies.

- v14.5 was for AsyncLocalStorage
- v14.6 brings a V8 with spport for private methods syntax, FinalizationRegistry (used by pino@8)
- v14.8 brings AsyncResource.bind
- v14.17 brings AbortController, AbortSignal.reason (used by readable-stream@4, glob@10, etc.)

Closes: elastic#2760
fpm-peter pushed a commit to fpm-git/apm-agent-nodejs that referenced this issue Aug 20, 2024
lru-cache@7 added support for async fetching, and async-cache
was deprecated

See https://github.com/isaacs/node-lru-cache#fetchmethod-read-only
for the best docs for using lru-cache for async caching.

Refs: elastic#2760
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
agent-nodejs Make available for APM Agents project planning.
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

1 participant