Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

slow performance when creating multiple zip files. #495

Open
JamesKyburz opened this issue Sep 24, 2023 · 1 comment
Open

slow performance when creating multiple zip files. #495

JamesKyburz opened this issue Sep 24, 2023 · 1 comment
Labels
bug Something isn't working

Comments

@JamesKyburz
Copy link

Slow zip performance

When creating multiple zip files there are performance issues, some blocking the event loop.

I have a private repository with a serverless stack containing 8 lambdas which takes 70s to package (and alot more in GitHub actions...)

To Reproduce

Create a serverless.yml with multiple functions using similar dependencies.

Using the config

'use strict'

module.exports = function esmConfig() {
  const nodeMajorVersion = process.version.match(/v([\d]+)/)[1]
  return {
    bundle: true,
    minify: true,
    format: 'esm',
    outputFileExtension: '.mjs',
    sourcemap: true,
    target: `node${nodeMajorVersion}`,
    exclude: ['@aws-sdk/*'],
    define: {
      'require.resolve': 'undefined'
    },
    platform: 'node',
    banner: {
      js: `
import { createRequire } from 'node:module'
import { dirname } from 'node:path'
import { fileURLToPath } from 'node:url'

const require = createRequire(import.meta.url)
const __filename = fileURLToPath(import.meta.url)
const __dirname = dirname(__filename)
`,
    },
    watch: {
      pattern: './functions/**',
      ignore: ['.serverless/**/*', '.build']
    }
  }
}

Expected behavior
A clear and concise description of what you expected to happen.

A large list of exclude files (9K+), blocking fs operations including the zip function causes slowdown.

Screenshots or Logs
Adding lots of console logs, and a timer interval with 3s delay to show event loop blockages.

After a few tweaks of making sync fs async the time was 70s zipping 8 functions.

When loop is output is shows the delay since the last output (this should be around 3000ms depending on of the event loop was blocked or not)

Packaging something for stage local (us-east-1)
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
-bundle
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
+bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/create-project/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/create-project"}
+bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/initial-bom/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/initial-bom"}
+bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/project/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/project"}
+bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/projects/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/projects"}
+bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/update-row/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/update-row"}
+bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/select-product/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/select-product"}
+bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/update-project/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/update-project"}
+bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/add-material/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/add-material"}
-bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/add-material/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/add-material"}
+bundle# pkg.rebuild
-bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/update-row/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/update-row"}
+bundle# pkg.rebuild
-bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/create-project/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/create-project"}
+bundle# pkg.rebuild
-bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/project/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/project"}
+bundle# pkg.rebuild
-bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/select-product/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/select-product"}
+bundle# pkg.rebuild
-bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/initial-bom/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/initial-bom"}
+bundle# pkg.rebuild
-bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/projects/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/projects"}
+bundle# pkg.rebuild
-bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/update-project/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/update-project"}
+bundle# pkg.rebuild
-bundle# pkg.rebuild
-bundle# pkg.rebuild
-bundle# pkg.rebuild
-bundle# pkg.rebuild
-bundle# pkg.rebuild
-bundle# pkg.rebuild
-bundle# pkg.rebuild
-bundle# pkg.rebuild
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
+bundle
-packExternalModules
pack external [9be10826-915b-4379-94fb-d155330fddf4]
findUpIO {"name":"package.json","directory":"/Users/x/Documents/src/private-stack/packages/project"}
findUpIO {"name":"package.json","directory":"/Users/x/Documents/src/private-stack/packages"}
findUpIO {"name":"package.json","directory":"/Users/x/Documents/src/private-stack"}
external options {
  "exclude": [],
  "external": [],
  "upperPackageJson": "/Users/x/Documents/src/private-stack"
}
pack external [9be10826-915b-4379-94fb-d155330fddf4] took 0 ms
+packExternalModules
-copyExtras
loop! [3000]ms {"handles":4,"requests":10}
loop! [3000]ms {"handles":4,"requests":1}
+copyExtras
-pack
pack[2fde070a-92f8-4e0a-874e-686efab601d3]
pack[2fde070a-92f8-4e0a-874e-686efab601d3] step 1 took 64 ms
pack[2fde070a-92f8-4e0a-874e-686efab601d3] build options {
  "concurrency": null,
  "zipConcurrency": null,
  "bundle": true,
  "target": "node18",
  "external": [],
  "exclude": [
    "@aws-sdk/*"
  ],
  "nativeZip": false,
  "packager": "npm",
  "packagerOptions": {
    "noInstall": false,
    "ignoreLockfile": false
  },
  "installExtraArgs": [],
  "watch": {
    "pattern": "./functions/**",
    "ignore": [
      ".serverless/**/*",
      ".build"
    ],
    "chokidar": {
      "ignoreInitial": true
    }
  },
  "keepOutputDirectory": false,
  "platform": "node",
  "outputFileExtension": ".mjs",
  "skipBuild": false,
  "skipBuildExcludeFns": [],
  "minify": true,
  "format": "esm",
  "sourcemap": true,
  "define": {
    "require.resolve": "undefined"
  },
  "banner": {
    "js": "\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"
  }
}
pack[2fde070a-92f8-4e0a-874e-686efab601d3] step 2 took 65 ms
pack[2fde070a-92f8-4e0a-874e-686efab601d3] step 3 took 65 ms
zip mapper[d942aec8-c067-4ea7-98eb-524c5c079db0]
zip mapper[df95d021-7c20-4a3d-a818-17ccf7d71ac0]
zip mapper[80bd9f0e-9765-484b-a3d4-37db70f96d24]
zip mapper[d2272b7d-3d98-4583-be29-80cae63c8707]
zip mapper[049b790e-1c5a-49e2-8b8c-e36e5e1a753a]
zip mapper[dce2c865-bcd0-414f-9693-6c9a39f6d9bf]
zip mapper[a90cbd59-796e-4871-9e29-e01aab0a968b]
zip mapper[eeaaa25d-4194-44eb-b264-76db7d6cd9d7]
zipMapper[80bd9f0e-9765-484b-a3d4-37db70f96d24] 164 so far took 252 ms
zipMapper[df95d021-7c20-4a3d-a818-17ccf7d71ac0] 164 so far took 255 ms
zipMapper[049b790e-1c5a-49e2-8b8c-e36e5e1a753a] 164 so far took 257 ms
zipMapper[d2272b7d-3d98-4583-be29-80cae63c8707] 164 so far took 259 ms
zipMapper[d942aec8-c067-4ea7-98eb-524c5c079db0] 164 so far took 262 ms
zipMapper[dce2c865-bcd0-414f-9693-6c9a39f6d9bf] 164 so far took 263 ms
zipMapper[a90cbd59-796e-4871-9e29-e01aab0a968b] 164 so far took 265 ms
zipMapper[eeaaa25d-4194-44eb-b264-76db7d6cd9d7] 164 so far took 272 ms
loop! [3006]ms {"handles":4,"requests":9280}
loop! [3123]ms {"handles":4,"requests":9280}
loop! [3029]ms {"handles":4,"requests":9280}
loop! [3015]ms {"handles":4,"requests":9280}
loop! [3078]ms {"handles":4,"requests":9280}
loop! [3050]ms {"handles":4,"requests":9280}
loop! [3031]ms {"handles":4,"requests":9280}
loop! [3053]ms {"handles":4,"requests":9280}
exclude calculation [80bd9f0e-9765-484b-a3d4-37db70f96d24] took 25 seconds
exclude file count [80bd9f0e-9765-484b-a3d4-37db70f96d24] 9576
zipMapper[80bd9f0e-9765-484b-a3d4-37db70f96d24] 175 so far took 25 seconds
zipMapper[80bd9f0e-9765-484b-a3d4-37db70f96d24] 182 so far took 25 seconds
zipMapper[80bd9f0e-9765-484b-a3d4-37db70f96d24] 185 so far took 25 seconds
zipMapper[80bd9f0e-9765-484b-a3d4-37db70f96d24] 203 so far took 30 seconds
zipping[80bd9f0e-9765-484b-a3d4-37db70f96d24]
about to zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/project.zip
exclude calculation [049b790e-1c5a-49e2-8b8c-e36e5e1a753a] took 30 seconds
exclude file count [049b790e-1c5a-49e2-8b8c-e36e5e1a753a] 9576
zipMapper[049b790e-1c5a-49e2-8b8c-e36e5e1a753a] 175 so far took 30 seconds
zipMapper[049b790e-1c5a-49e2-8b8c-e36e5e1a753a] 182 so far took 30 seconds
zipMapper[049b790e-1c5a-49e2-8b8c-e36e5e1a753a] 185 so far took 30 seconds
zipMapper[049b790e-1c5a-49e2-8b8c-e36e5e1a753a] 203 so far took 35 seconds
zipping[049b790e-1c5a-49e2-8b8c-e36e5e1a753a]
about to zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/update-row.zip
exclude calculation [d2272b7d-3d98-4583-be29-80cae63c8707] took 35 seconds
exclude file count [d2272b7d-3d98-4583-be29-80cae63c8707] 9576
zipMapper[d2272b7d-3d98-4583-be29-80cae63c8707] 175 so far took 35 seconds
zipMapper[d2272b7d-3d98-4583-be29-80cae63c8707] 182 so far took 35 seconds
zipMapper[d2272b7d-3d98-4583-be29-80cae63c8707] 185 so far took 35 seconds
zipMapper[d2272b7d-3d98-4583-be29-80cae63c8707] 203 so far took 40 seconds
zipping[d2272b7d-3d98-4583-be29-80cae63c8707]
about to zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/projects.zip
exclude calculation [d942aec8-c067-4ea7-98eb-524c5c079db0] took 40 seconds
exclude file count [d942aec8-c067-4ea7-98eb-524c5c079db0] 9576
zipMapper[d942aec8-c067-4ea7-98eb-524c5c079db0] 175 so far took 40 seconds
zipMapper[d942aec8-c067-4ea7-98eb-524c5c079db0] 182 so far took 40 seconds
zipMapper[d942aec8-c067-4ea7-98eb-524c5c079db0] 185 so far took 40 seconds
zipMapper[d942aec8-c067-4ea7-98eb-524c5c079db0] 203 so far took 45 seconds
zipping[d942aec8-c067-4ea7-98eb-524c5c079db0]
about to zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/create-project.zip
exclude calculation [dce2c865-bcd0-414f-9693-6c9a39f6d9bf] took 44 seconds
exclude file count [dce2c865-bcd0-414f-9693-6c9a39f6d9bf] 9576
zipMapper[dce2c865-bcd0-414f-9693-6c9a39f6d9bf] 175 so far took 45 seconds
zipMapper[dce2c865-bcd0-414f-9693-6c9a39f6d9bf] 182 so far took 45 seconds
zipMapper[dce2c865-bcd0-414f-9693-6c9a39f6d9bf] 185 so far took 45 seconds
zipMapper[dce2c865-bcd0-414f-9693-6c9a39f6d9bf] 203 so far took 49 seconds
zipping[dce2c865-bcd0-414f-9693-6c9a39f6d9bf]
about to zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/select-product.zip
loop! [25451]ms {"handles":4,"requests":1030}
exclude calculation [df95d021-7c20-4a3d-a818-17ccf7d71ac0] took 49 seconds
exclude file count [df95d021-7c20-4a3d-a818-17ccf7d71ac0] 9576
zipMapper[df95d021-7c20-4a3d-a818-17ccf7d71ac0] 175 so far took 49 seconds
zipMapper[df95d021-7c20-4a3d-a818-17ccf7d71ac0] 182 so far took 49 seconds
zipMapper[df95d021-7c20-4a3d-a818-17ccf7d71ac0] 185 so far took 49 seconds
zipMapper[df95d021-7c20-4a3d-a818-17ccf7d71ac0] 203 so far took 54 seconds
zipping[df95d021-7c20-4a3d-a818-17ccf7d71ac0]
about to zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/initial-bom.zip
exclude calculation [eeaaa25d-4194-44eb-b264-76db7d6cd9d7] took 54 seconds
exclude file count [eeaaa25d-4194-44eb-b264-76db7d6cd9d7] 9576
zipMapper[eeaaa25d-4194-44eb-b264-76db7d6cd9d7] 175 so far took 54 seconds
zipMapper[eeaaa25d-4194-44eb-b264-76db7d6cd9d7] 182 so far took 54 seconds
zipMapper[eeaaa25d-4194-44eb-b264-76db7d6cd9d7] 185 so far took 54 seconds
zipMapper[eeaaa25d-4194-44eb-b264-76db7d6cd9d7] 203 so far took 59 seconds
zipping[eeaaa25d-4194-44eb-b264-76db7d6cd9d7]
about to zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/add-material.zip
loop! [9521]ms {"handles":4,"requests":14}
exclude calculation [a90cbd59-796e-4871-9e29-e01aab0a968b] took 58 seconds
exclude file count [a90cbd59-796e-4871-9e29-e01aab0a968b] 9576
zipMapper[a90cbd59-796e-4871-9e29-e01aab0a968b] 175 so far took 59 seconds
zipMapper[a90cbd59-796e-4871-9e29-e01aab0a968b] 182 so far took 59 seconds
zipMapper[a90cbd59-796e-4871-9e29-e01aab0a968b] 185 so far took 59 seconds
zipMapper[a90cbd59-796e-4871-9e29-e01aab0a968b] 203 so far took 1 minute
zipping[a90cbd59-796e-4871-9e29-e01aab0a968b]
about to zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/update-project.zip
loop! [4697]ms {"handles":4,"requests":10}
zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/update-row.zip: 28.427s
zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/project.zip: 33.363s
zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/create-project.zip: 18.937s
zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/select-product.zip: 14.258s
zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/projects.zip: 23.702s
zipped[049b790e-1c5a-49e2-8b8c-e36e5e1a753a] took 28 seconds
zip mapper[049b790e-1c5a-49e2-8b8c-e36e5e1a753a] took 1 minute
zipped[80bd9f0e-9765-484b-a3d4-37db70f96d24] took 33 seconds
zip mapper[80bd9f0e-9765-484b-a3d4-37db70f96d24] took 1 minute
zipped[d942aec8-c067-4ea7-98eb-524c5c079db0] took 19 seconds
zip mapper[d942aec8-c067-4ea7-98eb-524c5c079db0] took 1 minute
zipped[dce2c865-bcd0-414f-9693-6c9a39f6d9bf] took 14 seconds
zip mapper[dce2c865-bcd0-414f-9693-6c9a39f6d9bf] took 1 minute
zipped[d2272b7d-3d98-4583-be29-80cae63c8707] took 24 seconds
zip mapper[d2272b7d-3d98-4583-be29-80cae63c8707] took 1 minute
zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/update-project.zip: 33.987ms
zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/add-material.zip: 4.731s
zipped[a90cbd59-796e-4871-9e29-e01aab0a968b] took 34 ms
zip mapper[a90cbd59-796e-4871-9e29-e01aab0a968b] took 1 minute
zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/initial-bom.zip: 9.425s
zipped[eeaaa25d-4194-44eb-b264-76db7d6cd9d7] took 5 seconds
zip mapper[eeaaa25d-4194-44eb-b264-76db7d6cd9d7] took 1 minute
zipped[df95d021-7c20-4a3d-a818-17ccf7d71ac0] took 9 seconds
zip mapper[df95d021-7c20-4a3d-a818-17ccf7d71ac0] took 1 minute
+pack

✔ Service packaged (70s)
loop! [3001]ms {"handles":3,"requests":0}

Changing the exclude to be an empty array instead of 9k+ files made a massive difference, this time sls package took under 10s.

The changed line was this line

Packaging something for stage local (us-east-1)
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
-bundle
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
+bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/create-project/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/create-project"}
+bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/initial-bom/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/initial-bom"}
+bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/project/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/project"}
+bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/projects/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/projects"}
+bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/update-row/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/update-row"}
+bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/select-product/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/select-product"}
+bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/update-project/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/update-project"}
+bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/add-material/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/add-material"}
-bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/add-material/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/add-material"}
+bundle# pkg.rebuild
-bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/create-project/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/create-project"}
+bundle# pkg.rebuild
-bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/update-row/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/update-row"}
+bundle# pkg.rebuild
-bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/initial-bom/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/initial-bom"}
+bundle# pkg.rebuild
-bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/projects/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/projects"}
+bundle# pkg.rebuild
-bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/project/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/project"}
+bundle# pkg.rebuild
-bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/select-product/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/select-product"}
+bundle# pkg.rebuild
-bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/update-project/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/update-project"}
+bundle# pkg.rebuild
-bundle# pkg.rebuild
-bundle# pkg.rebuild
-bundle# pkg.rebuild
-bundle# pkg.rebuild
-bundle# pkg.rebuild
-bundle# pkg.rebuild
-bundle# pkg.rebuild
-bundle# pkg.rebuild
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
+bundle
-packExternalModules
pack external [7edb1fe2-d90b-4989-b5e3-8402c7b7ebde]
findUpIO {"name":"package.json","directory":"/Users/x/Documents/src/private-stack/packages/project"}
findUpIO {"name":"package.json","directory":"/Users/x/Documents/src/private-stack/packages"}
findUpIO {"name":"package.json","directory":"/Users/x/Documents/src/private-stack"}
external options {
  "exclude": [],
  "external": [],
  "upperPackageJson": "/Users/x/Documents/src/private-stack"
}
pack external [7edb1fe2-d90b-4989-b5e3-8402c7b7ebde] took 1 ms
+packExternalModules
-copyExtras
loop! [3000]ms {"handles":4,"requests":10}
loop! [3000]ms {"handles":4,"requests":1}
+copyExtras
-pack
pack[25373c50-1cb7-4914-adf5-a8ab8ff77065]
pack[25373c50-1cb7-4914-adf5-a8ab8ff77065] step 1 took 66 ms
pack[25373c50-1cb7-4914-adf5-a8ab8ff77065] build options {
  "concurrency": null,
  "zipConcurrency": null,
  "bundle": true,
  "target": "node18",
  "external": [],
  "exclude": [
    "@aws-sdk/*"
  ],
  "nativeZip": false,
  "packager": "npm",
  "packagerOptions": {
    "noInstall": false,
    "ignoreLockfile": false
  },
  "installExtraArgs": [],
  "watch": {
    "pattern": "./functions/**",
    "ignore": [
      ".serverless/**/*",
      ".build"
    ],
    "chokidar": {
      "ignoreInitial": true
    }
  },
  "keepOutputDirectory": false,
  "platform": "node",
  "outputFileExtension": ".mjs",
  "skipBuild": false,
  "skipBuildExcludeFns": [],
  "minify": true,
  "format": "esm",
  "sourcemap": true,
  "define": {
    "require.resolve": "undefined"
  },
  "banner": {
    "js": "\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"
  }
}
pack[25373c50-1cb7-4914-adf5-a8ab8ff77065] step 2 took 67 ms
pack[25373c50-1cb7-4914-adf5-a8ab8ff77065] step 3 took 67 ms
zip mapper[2fea2fa1-1e0d-4096-b242-bf4bc925397f]
zip mapper[4750e4dd-584f-4fcb-8896-8b0bf0d8aa5c]
zip mapper[4909a123-683e-4c6c-ac32-e479cd80aea4]
zip mapper[5e64fb4c-7668-4e7b-9ef9-06d17591ee30]
zip mapper[8f27abb4-4b27-4a8c-a286-dd71d6a1b423]
zip mapper[0082936c-58d4-4d4d-809c-d7a289ad91d4]
zip mapper[8f8f9792-f8f1-4aa1-99d7-960bcd397f9b]
zip mapper[1620f390-5a22-483b-8454-de6a4cccda46]
zipMapper[4750e4dd-584f-4fcb-8896-8b0bf0d8aa5c] 164 so far took 251 ms
exclude calculation [4750e4dd-584f-4fcb-8896-8b0bf0d8aa5c] took 0 ms
exclude file count [4750e4dd-584f-4fcb-8896-8b0bf0d8aa5c] 0
zipMapper[4750e4dd-584f-4fcb-8896-8b0bf0d8aa5c] 175 so far took 251 ms
zipMapper[4750e4dd-584f-4fcb-8896-8b0bf0d8aa5c] 182 so far took 251 ms
zipMapper[4750e4dd-584f-4fcb-8896-8b0bf0d8aa5c] 185 so far took 251 ms
zipMapper[4750e4dd-584f-4fcb-8896-8b0bf0d8aa5c] 203 so far took 256 ms
zipping[4750e4dd-584f-4fcb-8896-8b0bf0d8aa5c]
about to zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/initial-bom.zip
zipMapper[5e64fb4c-7668-4e7b-9ef9-06d17591ee30] 164 so far took 256 ms
exclude calculation [5e64fb4c-7668-4e7b-9ef9-06d17591ee30] took 0 ms
exclude file count [5e64fb4c-7668-4e7b-9ef9-06d17591ee30] 0
zipMapper[5e64fb4c-7668-4e7b-9ef9-06d17591ee30] 175 so far took 257 ms
zipMapper[5e64fb4c-7668-4e7b-9ef9-06d17591ee30] 182 so far took 257 ms
zipMapper[5e64fb4c-7668-4e7b-9ef9-06d17591ee30] 185 so far took 257 ms
zipMapper[5e64fb4c-7668-4e7b-9ef9-06d17591ee30] 203 so far took 261 ms
zipping[5e64fb4c-7668-4e7b-9ef9-06d17591ee30]
about to zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/projects.zip
zipMapper[4909a123-683e-4c6c-ac32-e479cd80aea4] 164 so far took 261 ms
exclude calculation [4909a123-683e-4c6c-ac32-e479cd80aea4] took 0 ms
exclude file count [4909a123-683e-4c6c-ac32-e479cd80aea4] 0
zipMapper[4909a123-683e-4c6c-ac32-e479cd80aea4] 175 so far took 261 ms
zipMapper[4909a123-683e-4c6c-ac32-e479cd80aea4] 182 so far took 261 ms
zipMapper[4909a123-683e-4c6c-ac32-e479cd80aea4] 185 so far took 261 ms
zipMapper[4909a123-683e-4c6c-ac32-e479cd80aea4] 203 so far took 265 ms
zipping[4909a123-683e-4c6c-ac32-e479cd80aea4]
about to zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/project.zip
zipMapper[0082936c-58d4-4d4d-809c-d7a289ad91d4] 164 so far took 266 ms
exclude calculation [0082936c-58d4-4d4d-809c-d7a289ad91d4] took 0 ms
exclude file count [0082936c-58d4-4d4d-809c-d7a289ad91d4] 0
zipMapper[0082936c-58d4-4d4d-809c-d7a289ad91d4] 175 so far took 266 ms
zipMapper[0082936c-58d4-4d4d-809c-d7a289ad91d4] 182 so far took 266 ms
zipMapper[0082936c-58d4-4d4d-809c-d7a289ad91d4] 185 so far took 266 ms
zipMapper[0082936c-58d4-4d4d-809c-d7a289ad91d4] 203 so far took 269 ms
zipping[0082936c-58d4-4d4d-809c-d7a289ad91d4]
about to zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/select-product.zip
zipMapper[8f27abb4-4b27-4a8c-a286-dd71d6a1b423] 164 so far took 270 ms
exclude calculation [8f27abb4-4b27-4a8c-a286-dd71d6a1b423] took 0 ms
exclude file count [8f27abb4-4b27-4a8c-a286-dd71d6a1b423] 0
zipMapper[8f27abb4-4b27-4a8c-a286-dd71d6a1b423] 175 so far took 270 ms
zipMapper[8f27abb4-4b27-4a8c-a286-dd71d6a1b423] 182 so far took 270 ms
zipMapper[8f27abb4-4b27-4a8c-a286-dd71d6a1b423] 185 so far took 270 ms
zipMapper[8f27abb4-4b27-4a8c-a286-dd71d6a1b423] 203 so far took 273 ms
zipping[8f27abb4-4b27-4a8c-a286-dd71d6a1b423]
about to zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/update-row.zip
zipMapper[2fea2fa1-1e0d-4096-b242-bf4bc925397f] 164 so far took 275 ms
exclude calculation [2fea2fa1-1e0d-4096-b242-bf4bc925397f] took 0 ms
exclude file count [2fea2fa1-1e0d-4096-b242-bf4bc925397f] 0
zipMapper[2fea2fa1-1e0d-4096-b242-bf4bc925397f] 175 so far took 275 ms
zipMapper[2fea2fa1-1e0d-4096-b242-bf4bc925397f] 182 so far took 275 ms
zipMapper[2fea2fa1-1e0d-4096-b242-bf4bc925397f] 185 so far took 275 ms
zipMapper[2fea2fa1-1e0d-4096-b242-bf4bc925397f] 203 so far took 278 ms
zipping[2fea2fa1-1e0d-4096-b242-bf4bc925397f]
about to zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/create-project.zip
zipMapper[1620f390-5a22-483b-8454-de6a4cccda46] 164 so far took 278 ms
exclude calculation [1620f390-5a22-483b-8454-de6a4cccda46] took 0 ms
exclude file count [1620f390-5a22-483b-8454-de6a4cccda46] 0
zipMapper[1620f390-5a22-483b-8454-de6a4cccda46] 175 so far took 278 ms
zipMapper[1620f390-5a22-483b-8454-de6a4cccda46] 182 so far took 279 ms
zipMapper[1620f390-5a22-483b-8454-de6a4cccda46] 185 so far took 279 ms
zipMapper[1620f390-5a22-483b-8454-de6a4cccda46] 203 so far took 282 ms
zipping[1620f390-5a22-483b-8454-de6a4cccda46]
about to zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/add-material.zip
zipMapper[8f8f9792-f8f1-4aa1-99d7-960bcd397f9b] 164 so far took 282 ms
exclude calculation [8f8f9792-f8f1-4aa1-99d7-960bcd397f9b] took 0 ms
exclude file count [8f8f9792-f8f1-4aa1-99d7-960bcd397f9b] 0
zipMapper[8f8f9792-f8f1-4aa1-99d7-960bcd397f9b] 175 so far took 282 ms
zipMapper[8f8f9792-f8f1-4aa1-99d7-960bcd397f9b] 182 so far took 282 ms
zipMapper[8f8f9792-f8f1-4aa1-99d7-960bcd397f9b] 185 so far took 282 ms
zipMapper[8f8f9792-f8f1-4aa1-99d7-960bcd397f9b] 203 so far took 286 ms
zipping[8f8f9792-f8f1-4aa1-99d7-960bcd397f9b]
about to zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/update-project.zip
zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/projects.zip: 2.062s
zipped[5e64fb4c-7668-4e7b-9ef9-06d17591ee30] took 2 seconds
zip mapper[5e64fb4c-7668-4e7b-9ef9-06d17591ee30] took 2 seconds
zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/update-row.zip: 2.051s
zipped[8f27abb4-4b27-4a8c-a286-dd71d6a1b423] took 2 seconds
zip mapper[8f27abb4-4b27-4a8c-a286-dd71d6a1b423] took 2 seconds
loop! [3000]ms {"handles":4,"requests":8}
zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/project.zip: 2.293s
zipped[4909a123-683e-4c6c-ac32-e479cd80aea4] took 2 seconds
zip mapper[4909a123-683e-4c6c-ac32-e479cd80aea4] took 3 seconds
zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/select-product.zip: 2.738s
zipped[0082936c-58d4-4d4d-809c-d7a289ad91d4] took 3 seconds
zip mapper[0082936c-58d4-4d4d-809c-d7a289ad91d4] took 3 seconds
zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/add-material.zip: 2.727s
zipped[1620f390-5a22-483b-8454-de6a4cccda46] took 3 seconds
zip mapper[1620f390-5a22-483b-8454-de6a4cccda46] took 3 seconds
zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/initial-bom.zip: 2.754s
zipped[4750e4dd-584f-4fcb-8896-8b0bf0d8aa5c] took 3 seconds
zip mapper[4750e4dd-584f-4fcb-8896-8b0bf0d8aa5c] took 3 seconds
zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/create-project.zip: 2.733s
zipped[2fea2fa1-1e0d-4096-b242-bf4bc925397f] took 3 seconds
zip mapper[2fea2fa1-1e0d-4096-b242-bf4bc925397f] took 3 seconds
zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/update-project.zip: 2.726s
zipped[8f8f9792-f8f1-4aa1-99d7-960bcd397f9b] took 3 seconds
zip mapper[8f8f9792-f8f1-4aa1-99d7-960bcd397f9b] took 3 seconds
+pack

✔ Service packaged (9s)
loop! [3001]ms {"handles":3,"requests":0}

Versions (please complete the following information):

  • OS: macOS 13.5.2
  • Serverless Framework Version: 3.35.2
  • Plugin Version: 1.48.3

Additional context

I can send a pull request changing the fs operations, and speeding up zip too. However I have some thoughts on why and how exclude is used, if the config has bundle true then only a handful of files should be added to the zip (maybe just one).

So here we could handle exclude differently?
@JamesKyburz JamesKyburz added the bug Something isn't working label Sep 24, 2023
@fredrik
Copy link
Contributor

fredrik commented Sep 26, 2023

@JamesKyburz Not sure if it's useful for you, but the slowdowns and lockups I experienced when zipping all the files at the same time were relieved by setting zipConcurrency.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@fredrik @JamesKyburz