Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable E2E deploy tests on publish #37019

Merged
merged 4 commits into from
May 21, 2022
Merged

Conversation

ijjk
Copy link
Member

@ijjk ijjk commented May 18, 2022

This enables testing against Vercel deployments whenever we publish a new canary to ensure we aren't regressing there. These tests can also be run locally with a Vercel token with VERCEL_TEST_TOKEN=<token> NEXT_TEST_MODE=deploy node run-tests.js --type e2e

@ijjk ijjk added the created-by: Next.js team PRs by the Next.js team. label May 18, 2022
@ijjk
Copy link
Member Author

ijjk commented May 18, 2022

Stats from current PR

Default Build (Decrease detected ✓)
General
vercel/next.js canary ijjk/next.js tests/enable-deploy-e2e Change
buildDuration 16.4s 16.1s -343ms
buildDurationCached 6.4s 6.4s -19ms
nodeModulesSize 479 MB 479 MB
Page Load Tests Overall decrease ⚠️
vercel/next.js canary ijjk/next.js tests/enable-deploy-e2e Change
/ failed reqs 0 0
/ total time (seconds) 3.939 3.979 ⚠️ +0.04
/ avg req/sec 634.66 628.3 ⚠️ -6.36
/error-in-render failed reqs 0 0
/error-in-render total time (seconds) 1.352 1.396 ⚠️ +0.04
/error-in-render avg req/sec 1849.25 1791.19 ⚠️ -58.06
Client Bundles (main, webpack)
vercel/next.js canary ijjk/next.js tests/enable-deploy-e2e Change
925.HASH.js gzip 179 B 179 B
framework-HASH.js gzip 42 kB 42 kB
main-HASH.js gzip 29 kB 29 kB
webpack-HASH.js gzip 1.44 kB 1.44 kB
Overall change 72.6 kB 72.6 kB
Legacy Client Bundles (polyfills)
vercel/next.js canary ijjk/next.js tests/enable-deploy-e2e Change
polyfills-HASH.js gzip 31 kB 31 kB
Overall change 31 kB 31 kB
Client Pages
vercel/next.js canary ijjk/next.js tests/enable-deploy-e2e Change
_app-HASH.js gzip 1.36 kB 1.36 kB
_error-HASH.js gzip 193 B 193 B
amp-HASH.js gzip 308 B 308 B
css-HASH.js gzip 327 B 327 B
dynamic-HASH.js gzip 2.71 kB 2.71 kB
head-HASH.js gzip 359 B 359 B
hooks-HASH.js gzip 920 B 920 B
image-HASH.js gzip 5.73 kB 5.73 kB
index-HASH.js gzip 263 B 263 B
link-HASH.js gzip 2.65 kB 2.65 kB
routerDirect..HASH.js gzip 320 B 320 B
script-HASH.js gzip 391 B 391 B
withRouter-HASH.js gzip 318 B 318 B
85e02e95b279..7e3.css gzip 107 B 107 B
Overall change 16 kB 16 kB
Client Build Manifests
vercel/next.js canary ijjk/next.js tests/enable-deploy-e2e Change
_buildManifest.js gzip 458 B 458 B
Overall change 458 B 458 B
Rendered Page Sizes
vercel/next.js canary ijjk/next.js tests/enable-deploy-e2e Change
index.html gzip 532 B 532 B
link.html gzip 546 B 546 B
withRouter.html gzip 527 B 527 B
Overall change 1.6 kB 1.6 kB

Default Build with SWC (Decrease detected ✓)
General
vercel/next.js canary ijjk/next.js tests/enable-deploy-e2e Change
buildDuration 18.3s 18.4s ⚠️ +110ms
buildDurationCached 6.5s 6.3s -153ms
nodeModulesSize 479 MB 479 MB
Page Load Tests Overall decrease ⚠️
vercel/next.js canary ijjk/next.js tests/enable-deploy-e2e Change
/ failed reqs 0 0
/ total time (seconds) 3.926 3.913 -0.01
/ avg req/sec 636.83 638.95 +2.12
/error-in-render failed reqs 0 0
/error-in-render total time (seconds) 1.38 1.387 ⚠️ +0.01
/error-in-render avg req/sec 1811.13 1801.81 ⚠️ -9.32
Client Bundles (main, webpack)
vercel/next.js canary ijjk/next.js tests/enable-deploy-e2e Change
925.HASH.js gzip 178 B 178 B
framework-HASH.js gzip 42.7 kB 42.7 kB
main-HASH.js gzip 29.4 kB 29.4 kB
webpack-HASH.js gzip 1.45 kB 1.45 kB
Overall change 73.7 kB 73.7 kB
Legacy Client Bundles (polyfills)
vercel/next.js canary ijjk/next.js tests/enable-deploy-e2e Change
polyfills-HASH.js gzip 31 kB 31 kB
Overall change 31 kB 31 kB
Client Pages
vercel/next.js canary ijjk/next.js tests/enable-deploy-e2e Change
_app-HASH.js gzip 1.35 kB 1.35 kB
_error-HASH.js gzip 179 B 179 B
amp-HASH.js gzip 311 B 311 B
css-HASH.js gzip 324 B 324 B
dynamic-HASH.js gzip 2.89 kB 2.89 kB
head-HASH.js gzip 357 B 357 B
hooks-HASH.js gzip 920 B 920 B
image-HASH.js gzip 5.82 kB 5.82 kB
index-HASH.js gzip 261 B 261 B
link-HASH.js gzip 2.78 kB 2.78 kB
routerDirect..HASH.js gzip 322 B 322 B
script-HASH.js gzip 392 B 392 B
withRouter-HASH.js gzip 317 B 317 B
85e02e95b279..7e3.css gzip 107 B 107 B
Overall change 16.3 kB 16.3 kB
Client Build Manifests
vercel/next.js canary ijjk/next.js tests/enable-deploy-e2e Change
_buildManifest.js gzip 457 B 457 B
Overall change 457 B 457 B
Rendered Page Sizes
vercel/next.js canary ijjk/next.js tests/enable-deploy-e2e Change
index.html gzip 532 B 532 B
link.html gzip 547 B 547 B
withRouter.html gzip 528 B 528 B
Overall change 1.61 kB 1.61 kB
Commit: 1ae9f9f

@ijjk ijjk force-pushed the tests/enable-deploy-e2e branch from f3ed673 to 1ae9f9f Compare May 21, 2022 03:51
@ijjk ijjk marked this pull request as ready for review May 21, 2022 03:56
styfle
styfle previously approved these changes May 21, 2022
@ijjk ijjk merged commit 2411d36 into vercel:canary May 21, 2022
@ijjk ijjk deleted the tests/enable-deploy-e2e branch May 21, 2022 09:46
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jun 20, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
created-by: Next.js team PRs by the Next.js team.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants